Skip to content

Guide

CSV to JSON: escaping, types, and the pitfalls that corrupt data

The conversion itself is trivial. The corruption happens in the two questions converters answer silently: where does this field end, and what type is this value?

By Published

Converting CSV to JSON looks like the easiest job in data plumbing: rows in, objects out. It is also one of the most reliable ways to corrupt a dataset without a single error message. The corruption never comes from the conversion step itself — it comes from two questions every converter has to answer, usually silently: where does this field end, and what type is this value.

CSV looks simple. It isn’t.

JSON has a real grammar (ECMA-404). CSV has RFC 4180, which is less a specification than a description of common behaviour — and plenty of real-world files ignore even that. The RFC’s rules that naive code gets wrong:

  • Fields may be quoted, and a quoted field may contain commas, so "Smith, John" is one field. Splitting a line on commas is wrong the moment any field is quoted.
  • Quotes are escaped by doubling: "5'10""" encodes 5'10". Backslash escaping, which many hand-rolled parsers assume, is not RFC 4180.
  • Quoted fields may contain line breaks. A multi-line address is legal CSV, which means you cannot even read a CSV file line by line. This single rule is why a correct parser must be stateful.

Then reality adds what the RFC doesn’t cover: files “delimited” by semicolons or tabs (European locales, where the comma is the decimal separator), a UTF-8 byte-order mark that silently glues three invisible bytes onto the first header name (so your code looks up id and the key is actually id), and legacy encodings like Windows-1252 that turn every accented name into mojibake when read as UTF-8. A converter that doesn’t detect the delimiter, strip the BOM, and get the encoding right corrupts the file before type handling even starts.

Header detection

RFC 4180 makes the header row optional, and nothing in the file says whether it’s present. Guess wrong in one direction and your first data record becomes the key names for everything else; guess wrong in the other and you get keys like field1. Good converters use a heuristic — a first row that is all non-numeric, non-repeating strings is probably a header — but heuristics fail on all-text data, so any converter worth using lets you override the guess. Watch for duplicate header names too: JSON object keys must be unique, so two amount columns means one silently wins or gets renamed.

The type problem: everything is a string

CSV has no types. 42, TRUE, and 2024-01-05 are all just character sequences, and the converter has to decide what they become in JSON. Every choice loses something:

  • Leading zeros. Infer 02134 as a number and the ZIP code becomes 2134. Same for phone numbers, product codes, and bank sort codes: identifiers that look numeric aren’t numbers.
  • Big integers.JSON’s grammar allows numbers of any size, but JavaScript (and most JSON parsers) store them as IEEE 754 doubles, exact only up to 253 − 1. A 19-digit database ID or credit-card-length number gets rounded to a nearby value — silently. IDs belong in strings.
  • Booleans. Is TRUE the JSON value true or the string a user typed? What about Yes, Y, 1? There is no right answer, only a documented one.
  • Dates. JSON has no date type at all, and 03/04/2024 is March 4th or April 3rd depending on locale. Keeping dates as strings is the only safe default.
  • Empty fields. CSV cannot distinguish null from an empty string from a missing value; JSON distinguishes all three.

Excel deserves its own warning here, because much CSV corruption happens before any converter sees the file. Excel applies aggressive type inference on open: leading zeros are stripped, integers beyond 15 digits are rounded, and date-like strings are converted — famously including human gene names like MARCH1 and SEPT2, which appeared as dates in thousands of published genetics papers until the genes were officially renamed in 2020. If a CSV has ever been opened in Excel and saved, treat every numeric-looking identifier in it as suspect.

The conservative rule: let the converter infer types for obvious measures, but keep anything that functions as an identifier — ZIP codes, IDs, account numbers — as strings, and spot-check the output before trusting it.

Flat rows vs nested JSON

A CSV row is flat; a JSON object need not be. When the target shape is nested, the nesting has to be encoded in the column names, and the de facto convention is path headers: a column named user.address.city (dot notation) or user[address][city] (bracket notation) expands into nested objects, and items[0].sku, items[1].sku into an array of objects. Many converters and libraries support this expansion; none of it is standardised, so dots that are literally part of a column name (price.usd as a flat key) will be mis-expanded unless the tool lets you switch expansion off. Pick one convention, apply it in both directions, and write it down.

Round-tripping back: JSON to CSV loses more

The reverse conversion is lossier. A uniform array of flat objects maps cleanly onto a table; anything else forces decisions. Nested objects flatten into path columns. Arrays are worse: an order with three line items either explodes into three rows (duplicating the order fields and changing the row count) or gets serialized as a JSON string inside a CSV cell — technically valid, practically unreadable in a spreadsheet. Records with different keys union into a wide, sparse table where null, empty string, and “key absent” all collapse into the same empty cell. If your data survives CSV→JSON→CSV byte-identically, it was genuinely tabular; if not, the diff shows you exactly what structure CSV cannot carry. Inspecting the intermediate JSON in a formatter before converting back makes those losses visible instead of silent.

When JSONL beats a JSON array

A conventional conversion wraps every row-object in one big JSON array — which means a consumer must parse the entire file before touching the first record, and a producer must hold it all to write the closing bracket. JSON Lines(JSONL/NDJSON) drops the array: one complete JSON object per line, newline-separated. Each line parses independently, so the format streams, appends, splits, and greps like CSV while keeping JSON’s types and nesting. It is the native bulk format for tools like BigQuery and Elasticsearch, and the right target whenever the row count is large or unbounded. Rule of thumb: JSON array for a config-sized payload an API returns whole; JSONL for datasets.

Size and streaming

JSON repeats every key name in every record, so converted output typically runs two to four times the size of the source CSV — worth remembering before converting a multi-gigabyte export. Conversely, CSV’s quoted-newline rule means correct streaming still requires a stateful parser, not a line splitter. For files up to the tens of megabytes, an in-browser converter handles the whole job locally; beyond that, stream with a library (Python’s csv module, PapaParse, and their equivalents all parse incrementally) and emit JSONL so neither side needs the dataset in memory. And if the data is heading for a config file rather than a pipeline, JSON may not be the final stop anyway — see our comparison of JSON and YAML for configuration.

Short version: parse CSV with a real parser, never a comma split; strip the BOM and confirm the delimiter and encoding; treat identifiers as strings no matter how numeric they look; never round-trip data files through Excel; encode nesting with one documented path convention; and reach for JSONL the moment the data stops fitting comfortably in memory.

Frequently asked questions

How do I convert CSV to JSON?
Parse the CSV with a real RFC 4180 parser (never a naive split on commas), treat the first row as field names, and emit one JSON object per data row with those names as keys. A client-side converter does this in the browser without uploading the file; for code, use an established CSV library in your language rather than hand-rolling the quoting rules. The two decisions to make deliberately are header handling and whether to infer types or keep everything as strings.
Why does splitting a CSV line on commas break?
Because RFC 4180 allows fields to be quoted, and a quoted field may contain commas, double quotes (escaped by doubling them), and even line breaks. A value like "Smith, John" is one field, not two, and a quoted address can legally span multiple lines. Splitting on commas — or reading line by line — mis-parses all of these. Any correct converter uses a stateful parser that tracks whether it is inside quotes.
Why does Excel change my CSV values?
Excel applies type inference on open: it strips leading zeros from anything that looks like a number (ZIP codes, phone numbers, product codes), rounds integers longer than 15 digits to fit its floating-point cells, and reinterprets date-like strings — the reason thousands of genetics papers contained gene names like MARCH1 converted to dates, until the genes were renamed in 2020. The damage happens on open and is saved back if you re-export. Import CSV via Excel's data-import dialog with columns set to Text, or avoid round-tripping data files through Excel entirely.
How do I convert CSV to JSON with nested objects?
CSV is flat, so nesting must be encoded in the column names. The common convention is dot or bracket paths: columns named user.name and user.address.city (or user[address][city]) expand into nested objects, and items[0], items[1] into arrays. Converters and libraries that support 'header path expansion' apply this automatically; without it, you get one flat object per row and must restructure in code afterwards. There is no standard — pick one convention and use it consistently in both directions.
When should I use JSON instead of CSV — and vice versa?
Use CSV when the data is genuinely tabular (same columns every row, scalar values), needs to open in spreadsheets, or is large enough that JSON's repeated key names hurt. Use JSON when records are nested, fields vary between records, or types matter — CSV has no types, so numbers, booleans, and nulls only survive in JSON. For large record streams, JSON Lines (one object per line) combines JSON's types with CSV's line-at-a-time processing.
What data is lost when converting JSON to CSV?
Anything that isn't a flat table of scalars. Nested objects must be flattened into path-named columns; arrays of objects either explode into multiple rows or get serialized as strings; the distinction between null, empty string, and a missing key collapses, because CSV represents all three as an empty field; and every type annotation disappears, since CSV values are just text. If the JSON is a uniform array of flat objects, the conversion is lossless — otherwise expect to make (and document) flattening decisions.

Sources & references

Authoritative references cited by this piece. Verified by Buğra Sözeri on the dates shown and re-checked at every deploy.

Related

Published July 15, 2026