When a document is too long for a single request, the usual answer is map-reduce: split it into pieces, summarize each piece, then summarize the summaries. It works well enough in a demo. In production it fails quietly, because each stage compresses, and compression is where detail goes to die.

This piece covers the failure modes that matter and the small design choices that prevent them.

Where map-reduce goes wrong

Chunk boundaries cut ideas in half. A fixed-size split can separate a claim from its qualification, or a table from its caption. Split on structure first (headings, sections, speaker turns) and only fall back to size when a section is itself too large. Overlapping a few sentences between neighbors helps, but structure helps more.

The reduce step sees summaries, not evidence. By the time the final pass runs, the model cannot check anything against the source. If a map-stage summary overstated a point, the reduce stage will repeat it with more confidence. Treat every intermediate summary as untrusted input to the next stage.

Position and emphasis get invented. The merge pass tends to give equal weight to every chunk, so a footnote-level detail from section 9 can end up next to the main finding from section 2. If importance matters, have the map stage return a structured record instead of prose.

Contradictions get smoothed away. Two chunks that disagree often merge into a blended sentence that is true of neither. Ask explicitly for conflicts to be listed rather than resolved.

A pattern that holds up

Make the map stage emit structured notes, not paragraphs. For each chunk, ask for:

  • the key claims, each as one sentence,
  • a pointer back to where it came from (section name or chunk index),
  • any numbers, names, or dates quoted verbatim,
  • anything in the chunk that looks like a caveat or exception.

Then the reduce stage works on a list of attributed claims. It can group, deduplicate, and order them, and the final summary can carry the pointers forward so a reader can click through to the source passage.

Two cheap checks make this much safer:

  1. Spot-verify against source. Sample a handful of claims from the final summary and confirm each one appears in the chunk it points to. A mismatch rate you can measure beats a general feeling that the summary looks fine.
  2. Compare to a single-pass baseline on short documents. For documents that do fit in one request, run both approaches and diff the results. Differences show you exactly what the chunked pipeline loses.

Also decide up front what the summary is for. A summary for deciding whether to read the document differs from one meant to replace reading it, and the second needs far more preserved detail and much stricter attribution.

The closing point

Map-reduce is a lossy compression pipeline, so design it like one: decide what must survive, carry that through every stage in a structured form, and keep a path back to the source. A summary without that path is a claim you have no way to audit.

AI ToolsLarge Language Models