Issue #81

TechInsights Notes — The Rollback You're Missing Is the One for Your Prompts

sent to 25 subscribers

Every team with an on-call rotation has a rollback plan for code. Fewer have one for prompts, even though a one-line system-prompt edit can change a model's behavior as much as a real code change — and typically ships with a fraction of the review.

The gap shows up at the worst time: a prompt change quietly makes a support bot terser, or a summarizer starts dropping a caveat it used to include, and the only fix available is redeploying the previous build. That's slower than it needs to be, and it means nobody reverts fast, so a bad prompt sits in production longer than a bad code change ever would.

The fix is to treat prompts like the release artifact they actually are: give every version an immutable ID the moment it's written, put the rollback behind a flag your on-call engineer can flip without redeploying anything, and log which prompt version served every single request — not which version was active that week, but the actual version behind each call. That last part matters most. Without per-request logging, you can't tell whether a spike in bad outputs started before or after your last prompt change, and "roll back and see if it helps" becomes your only diagnostic tool.

None of this requires new infrastructure — most teams already have a feature-flag system that can hold a string as easily as a boolean. The missing piece is usually the habit, not the tooling. Prompts changed the product; they should get the same release discipline code already has.

← Back to archive