The case for a designated skeptic on every AI feature team
Every team building an AI feature develops a house optimism — the demo worked, the eval numbers went up, the model is "pretty good now." That optimism is necessary to ship anything, but it's also exactly the bias that lets a real failure mode slip past three rounds of review because everyone in the room already believes the feature works.
The fix that keeps showing up in teams that ship reliable AI features isn't a longer checklist — it's a person. One reviewer, rotated or fixed, whose explicit job during review is to argue the feature is worse than the team thinks: to bring the weirdest input they can construct, to ask what happens when the tool call fails halfway through, to push back on an eval score by asking what it doesn't measure. Not a naysayer for its own sake — a skeptic with the standing to be taken seriously, and a job description that makes skepticism their contribution rather than their personality.
This works for a boring reason: skepticism is effortful, and without an assigned owner, effort gets rationed toward whoever is loudest, not whoever is right. Naming the role means the hard questions get asked on schedule instead of by chance.
It costs almost nothing to try. Pick your next AI feature review, assign one person to argue against shipping, and see what they find that the rest of the room missed. If the answer is "nothing," you've lost twenty minutes. Most teams find something.