Letting a model write and run code is one of the most useful things an agent can do: data analysis, quick transformations, verifying its own arithmetic. It is also the moment your system starts executing text that nobody on your team reviewed. The model is not malicious, but its input may be (a document it summarized, a web page it fetched), and even a well-behaved model writes infinite loops and rm -rf typos. Treat the generated code as untrusted from the first line.

The boundary is the sandbox, not the prompt

"Only write safe code" in a system prompt is a preference, not a control. A control is something that still holds when the model is wrong or manipulated. For code execution that means constraints enforced by the runtime, outside the model's reach:

  • Process isolation. Run in a container, microVM, or similar isolated environment with no shared state between runs. A fresh environment per task avoids one run poisoning the next; if you reuse sessions for speed, be explicit about what persists.
  • Resource ceilings. Set hard limits on CPU time, wall-clock time, memory, process count, and output size. A while True loop should end as a timeout error, not a pager alert. Cap how much stdout you feed back to the model too, or one noisy script will flood its context.
  • Filesystem scope. Mount only the inputs the task needs, read-only where possible, and give it a scratch directory that is destroyed afterward. Never mount your home directory, credentials, or the application's own source.
  • Network egress. Default to no network. If a task truly needs it, allow-list specific hosts. Open egress turns "ran some code" into "sent your data anywhere", which is exactly what an injected instruction would ask for.
  • No inherited secrets. The sandbox's environment variables should be empty unless you put something there on purpose. Pass scoped, short-lived credentials if a call is unavoidable.

Make failures useful, and log everything

When a limit is hit, return a clear message to the model ("killed after 10s CPU limit") so it can simplify instead of retrying the same thing. Log the code, the limits applied, exit status, and truncated output for each run. When something strange happens, you will want to replay exactly what was executed.

Also decide up front what happens to files the code produces. Treat them like any other untrusted upload: scan or type-check before you serve them to a user, and never render generated HTML on your own origin.

A closing test

Ask one question of your design: if the generated code were written by an attacker, what is the worst thing it could do from inside this environment? If the answer involves your network, your secrets, or other users' data, the fix belongs in the sandbox configuration, not in a better prompt. Build the box first, then decide how much freedom to put inside it.

AI AgentsAI SecurityDeveloper Tools