Why an AI Feature Needs Its Own Error Budget, Separate From Uptime
Most teams already track an error budget for uptime: some acceptable amount of downtime or 5xx responses before it triggers a slowdown on new work. AI features need a second, separate budget — and conflating the two hides the failures that actually matter most.
Uptime error budgets catch the loud failures: the service is down, the request timed out, the API returned an error. An LLM-backed feature's most damaging failures are usually quiet ones — a confidently wrong answer, a subtly malformed structured output that downstream code accepts anyway, a tool call with the wrong arguments that "succeeds" without doing the right thing. None of these show up as a 5xx. All of them show up as a user getting bad output from a system that reported itself healthy the whole time.
A model-quality error budget tracks a different question: out of the outputs your feature produced, how many were actually correct or acceptable, as judged against your evals or a sampled human review — not whether the request completed. Once that number exists as its own tracked metric, it changes what triggers a response. Uptime stays green while quality error budget burns down; that's a real signal, not a contradiction, and it should be able to independently halt a rollout the same way an uptime budget would.
The practical version of this is smaller than it sounds: pick a quality metric you can measure cheaply and repeatably, set a threshold below which you pause and investigate rather than ship the next change, and review it on a cadence separate from your uptime dashboard. It won't catch everything. But it catches the category of failure that "the service is up" was never designed to see.