Most LLM integrations check whether the call returned. Far fewer check why it stopped. A response cut off by a token limit comes back as a perfectly healthy HTTP 200 with plausible-looking text, which makes truncation one of the quietest ways for an AI feature to be wrong.

The details here follow Anthropic's documentation on handling stop reasons. Other providers expose the same idea under different field names, so check yours.

Every response says why it stopped

Anthropic's docs list these values for stop_reason, with the action each implies:

| Value | What it means | What to do | |---|---|---| | end_turn | The model finished naturally. | Use the response. | | max_tokens | It hit your max_tokens limit. | Raise the limit, or continue the response. | | stop_sequence | It emitted one of your custom stop sequences. | Check which one. | | tool_use | It's calling a tool. | Run the tool and return the result. | | pause_turn | A server-tool loop reached its iteration limit. | Send the content back to continue. | | refusal | The model declined to respond. | Read the stop details, and consider a fallback model. | | model_context_window_exceeded | The response filled the model's context window. | Treat it as truncated. |

If your code reads only the text, all of these look the same. Two of them are truncations in disguise: max_tokens and model_context_window_exceeded.

How truncation hurts, by output format

  • Free text ends mid-sentence. Users notice but usually blame the model rather than report it.
  • JSON becomes invalid and the parser fails. This is the loud case, and it tends to get retried blindly, which costs money and often truncates in the same place.
  • A tool call can be cut off partway through. The docs specifically cover an incomplete tool_use block and say to retry with a higher max_tokens.
  • Lists and reports can end cleanly at item seven of ten. Nothing looks broken and nobody learns that items eight to ten existed.

That last one is the dangerous case, because no error is ever raised.

Make the stop reason part of your contract

Don't make it an optional detail. A few rules:

  1. Return it from your wrapper. Something like { text, stopReason, truncated }, where callers can't ignore truncated without making a deliberate choice.
  2. Branch on it explicitly. max_tokens and model_context_window_exceeded should never flow into the same path as end_turn.
  3. Size the limit from data. Look at the distribution of real output lengths and set max_tokens above the high percentiles, not at a round number. Alert when the truncation rate rises above near zero.
  4. Don't retry the identical request. If the cap caused the problem, the same request will fail the same way. Raise the limit, ask for a shorter format, or split the task.
  5. Validate shape. If you asked for ten items, check that you got ten. A length check also catches the model that simply stopped early.
  6. Continue deliberately. The docs describe continuing by appending the partial assistant response to the conversation and calling again. That can work for prose. For structured output it's fragile, because you have to stitch two pieces into one valid document. Splitting the work into smaller calls is usually more reliable.

The docs also describe empty responses that arrive with end_turn. They occur when text is added directly after tool results, or when a completed response is sent back unchanged. The prevention is to send only tool_result blocks after a tool call, with no extra text, and if empty replies persist, to add a continuation prompt as a new user message. It's another case where "the call succeeded" and "the feature worked" are different statements.

Test the edge on purpose

Write a test that forces a tiny output limit and asserts your system surfaces the truncation instead of rendering partial text. It's one of the cheapest tests you can add, and it exercises a path nobody visits until production data gets long. Model changes make this more likely, because a newer model may write longer answers than the one your limit was tuned for. See Silent Model Upgrades Are a Production Risk for how to catch that kind of shift.

The takeaway

Output length is a resource limit, and resource limits deserve the same treatment as timeouts and rate limits: explicit handling, a metric, and an alert. If you aren't tracking your truncation rate, assume it isn't zero.

Sources

AI IntegrationLarge Language Models