Model output is not just something you read. In most AI features it becomes something your system *does*: it gets rendered into a page, interpolated into a query, written to a file, passed to a shell, or fed to the next tool in an agent loop. That makes it input to your own code, and input to your own code deserves the same suspicion as anything a stranger could send.
Teams are often careful about what goes *into* the model and relaxed about what comes *out*, because the output feels like it came from "our" component. But the model can reproduce attacker-supplied text from a document it retrieved, echo markup from a user's message, or simply produce something malformed on a bad day. Whether the cause is malice or noise, the failure lands in your downstream code.
A few habits that cost little:
- Escape on render. If output goes into HTML, treat it as untrusted text exactly as you would a user comment. Don't turn on raw HTML or unvetted Markdown links because the content "came from the AI."
- Parameterize, never concatenate. If a model produces a value used in a query or command, pass it as a parameter or argument, and validate it against an allowlist where you can.
- Validate structure before acting. Parse into a schema, reject anything that doesn't fit, and decide in advance what the system does on rejection. A clean failure beats a partial action.
- Give tool calls the least authority. Whatever a model-produced argument can reach, assume it eventually will. Scope paths, hosts, and permissions accordingly.
None of this is exotic. It's the ordinary discipline of input handling, applied to a source that happens to feel friendly. The next time you review an AI feature, trace each place model text leaves the model and ask what would happen if that text were hostile. The answer usually points straight at the next fix.