Parsing Partial JSON From a Streaming LLM Response Without Guessing

Parsing Partial JSON From a Streaming LLM Response Without Guessing

Streaming feels obviously better for a chat response: tokens appear as they're generated instead of the user staring at a spinner. It's a much harder sell when the response is structured output that downstream code needs to parse — a tool call, a form fill, a classification with fields. JSON.parse on {"status": "pendi throws, every time, because it's not looking at broken JSON, it's looking at JSON that doesn't exist yet. Most teams solve this by waiting for the stream to finish before parsing anything, which throws away the latency benefit that streaming was supposed to buy in the first place.

Why the quick fixes break

The obvious workaround — try/except around JSON.parse on every partial chunk, and just show nothing until it stops throwing — technically works, but it means the UI sits blank until the entire object closes, which is the exact non-streaming behavior you started with. A more common hack is trimming the buffer back to the last complete-looking brace, but "complete-looking" is doing a lot of work in that sentence: a string value can legitimately contain an unescaped-looking sequence of braces and quotes that fools a naive trimmer, and a key can be mid-token when the buffer gets cut. These approaches don't fail loudly — they fail by occasionally showing a value that looks complete but isn't, which is worse than showing nothing.

Use a tolerant parser, not the strict one

The actual fix is a parser built for partial input: a small state machine that tracks bracket depth and whether it's currently inside a string, and — instead of throwing on an unterminated string or an open bracket — closes them speculatively to produce a valid-but-incomplete document. Fields that are structurally closed (a key whose value has a terminating quote, brace, or bracket) are safe to read and render immediately. Fields still open should stay hidden, not partially rendered, because a half-written string is not a preview of the final value, it's just wrong. This is a genuinely small amount of code — a handful of state transitions — and it's worth writing once as a shared utility rather than re-deriving the "trim to last brace" hack in every project that streams structured output.

Validate per field, then validate the whole document again

Incremental parsing tells you a field is structurally complete; it doesn't tell you the field is correct. Run schema validation on each field as it closes so you can catch obviously wrong types early and give faster feedback, but treat that as provisional. Once the stream ends, validate the assembled object against the full schema before you let anything downstream act on it — call a tool, write to a database, trigger a side effect. Streaming is a UI and latency optimization; it should never be the reason a malformed or incomplete object reaches code that assumes it's valid. Show the user progress as it arrives, but only commit to an action once you have the whole, validated thing.