Structured extraction looks simple: give the model a document and a schema, get back JSON. The failure mode that bites teams is not malformed JSON. It is well-formed JSON containing a confident value for a field the document never mentioned.

If your schema has a required due_date and the invoice has no due date, the model is under pressure to produce something. It may pick the invoice date, a date from a footer, or a plausible-looking guess. The output validates perfectly and is wrong.

Give absence a first-class representation

The fix starts in the schema, not the prompt. Every field that can legitimately be missing should allow it explicitly, and you should distinguish between different kinds of "no value":

  • Not present: the document does not say.
  • Ambiguous: the document says something, but it could mean more than one thing.
  • Not applicable: the field does not make sense for this kind of document.

Even a simple pair, a nullable value plus a short reason string, is far better than forcing a value. Then say so plainly in the prompt: "If the document does not state a field, return null and explain in one phrase. A null is a correct answer. Do not infer values from other fields."

Ask for evidence

For any field where an invented value would be costly, ask for a supporting quote from the source next to the value. Then verify mechanically that the quote appears in the document. A value with no verifiable quote can be downgraded to null or routed to review. This turns a fuzzy trust problem into a string check.

Test the absence cases on purpose

Your eval set probably over-represents complete documents, because those are easiest to label. Deliberately build cases with each field removed in turn, and measure the rate at which the model returns null versus invents something. This "fabrication rate per field" is a number you can track across prompt and model changes, and it usually differs wildly between fields.

Handle nulls downstream

None of this helps if the consuming code coerces null to an empty string or zero. A missing quantity treated as 0 is a different bug from a missing quantity treated as unknown. Decide per field what a null triggers: a default, a review queue, or a hard stop, and write that down next to the schema.

The closing point

An extraction system is only as honest as its schema allows. If the only way to satisfy the contract is to produce a value, you have built a machine for producing plausible guesses. Make "I don't know" a valid, tested, and handled output, and your accuracy numbers will start to mean something.

Developer ToolsLarge Language Models