Most prompt advice is about what to tell the model. Stop sequences are about something quieter: where the output is allowed to end. They are a small, deterministic feature of the API, and they solve a handful of annoying problems that people otherwise try to fix with more prose in the prompt.

What a stop sequence actually does

A stop sequence is a string you give the API. When the model generates it, generation halts, and the string itself is normally not included in the returned text. Because it is applied by the serving layer rather than learned behavior, it is reliable in a way an instruction like "stop after the answer" is not.

That makes it a good fit for three situations:

  • Delimited sections. If you ask the model to produce an answer between markers, such as <answer> and </answer>, set the closing tag as a stop sequence. You get the content without trailing commentary, and you pay for fewer output tokens.
  • Simulated dialogue. If a prompt contains a transcript with speaker labels, the model may continue by inventing the other speaker's next line. Setting that label (for example \nUser:) as a stop sequence cuts the output off before the invention.
  • Single-item outputs. When you want one line, a label, or one record, a newline can act as the stop sequence, which also bounds cost and latency.

Pick sequences that cannot appear in legitimate output

The main way this goes wrong is choosing a stop string that sometimes belongs in the answer. A newline stops a multi-line answer after its first line. } truncates JSON at the first closing brace of a nested object. Check your stop strings against a sample of real outputs, and prefer unusual markers (a tag the model would not otherwise write) over punctuation that occurs naturally.

Also remember that the stop string can match inside a code block, a quoted passage, or content echoed from the input. If you are summarizing a document that itself contains your marker, generation will end early. Escaping or choosing a marker that cannot appear in the source is part of the design.

Check how the output ended

Stopping early is not free of ambiguity. The API reports why generation ended, and you should read it. A response that ended because it hit your stop sequence is expected. One that ended because it reached the maximum token limit is probably truncated. A simple rule in your client code:

  1. If the reason is the stop sequence, accept the output and parse it.
  2. If the reason is the length limit, treat the output as incomplete and handle it as an error or retry with a larger limit.
  3. If the stop sequence never fired when you expected it to, log that case, because it means the model deviated from your format.

Where your provider supports starting the response with text you supply, the same boundary thinking applies to the beginning of the output: you can fix the first characters and let the model continue from them. Support for that varies by model and provider, so verify it before relying on it.

The closing point

If you find yourself writing "do not add any explanation after the JSON" for the third time, consider whether a boundary enforced by the API would be better than a request made in prose. Use stop sequences for the edges you can specify mechanically, keep the prompt for the judgment calls, and always inspect why a generation ended before trusting what it returned.

Large Language Models