Handle large JSON (performance tips)
A few megabytes of JSON is easy for any modern tool. Hundreds of megabytes is a different story. Parsing needs memory several times the file size, and rendering every node in a tree view can freeze a browser tab. This guide covers practical ways to open, inspect, reduce and compare large JSON files.
How big is "large"?
As a rough guide on a typical laptop:
- Under 10 MB: opens instantly in a browser editor, including tree and table views.
- 10–100 MB: usually fine in text mode. Tree views become slower because every node is a UI element.
- Over 100 MB: better handled with command-line tools or streaming parsers, or reduced first.
Memory is the real limit. Parsing a 200 MB document can need 1 GB or more of RAM, because every string, number and object becomes a separate value in memory.
1) Use the right view in the browser
In the JSON Formatter, large files work best in text mode, which displays the document without building a node for every value. Switch to tree or table only once you have narrowed the data down. The editor runs entirely in your browser, so speed depends on your device's memory rather than an upload.
2) Look at the shape before the details
Before scrolling through thousands of lines, find out what the document contains:
bash# Top-level keys jq 'keys' big.json # How many items an array has jq '.events | length' big.json # The first item, to see its shape jq '.events[0]' big.json
The JSONPath panel in the formatter gives a similar overview of every path in the document.
3) Extract only what you need
Most of the time you only need part of a large file. Pull it out with jq and work with the smaller result:
bash# Only failed events jq '[.events[] | select(.status == "failed")]' big.json > failed.json # Only a few fields per item jq '[.events[] | {id, status, createdAt}]' big.json > slim.json
4) Split large arrays
If the document is one huge array, convert it to NDJSON / JSON Lines, one object per line. Standard text tools can then split, count and search it:
bashjq -c '.events[]' big.json > events.ndjson split -l 100000 events.ndjson chunk_ wc -l events.ndjson
5) Stream instead of loading everything
When a file is too large to load at all, use a streaming parser that reads it piece by piece:
jq --streamprocesses path/value pairs without building the whole document.- In Python,
ijsoniterates over items in a large array with constant memory. - In Node.js, libraries such as
stream-jsondo the same.
If you control the producer, write NDJSON in the first place. It is designed for streaming and appending.
6) Make diffs manageable
Comparing two large files line by line produces enormous, unreadable diffs. Reduce the noise first:
- Remove volatile fields such as timestamps, request IDs and random tokens.
- Sort keys so ordering differences disappear.
- Compare a filtered subset, such as one customer or one day of events, rather than everything.
Then compare the two JSON files.
7) Shrink files you store or send
- Minify JSON that machines read; see How to minify JSON.
- Compress with gzip or Brotli. JSON is highly repetitive, so it often compresses to 10–20% of its original size.
- Drop fields nobody uses. The biggest savings usually come from removing data rather than whitespace.
8) Be careful when sharing large payloads
Large exports often contain more personal data than you realize. Before attaching one to a ticket or sharing it, read JSON privacy & safety tips. Share links built from the data itself also have URL length limits; see How to share JSON data.