Many AI features are shipped with a reassuring line in the design doc: "a human reviews the output before it goes out." That sentence quietly assumes something that is rarely measured: that the reviewer has the time and attention to catch the errors.
Consider the arithmetic. If a model drafts 200 items a day and one person reviews them, and each careful review takes three minutes, that is ten hours of work. In practice the reviewer spends thirty seconds per item, skims, and approves. The review step still exists on paper. Its error-catching power has collapsed.
There are a few ways to make review capacity a real design input instead of an assumption.
Count it. Estimate items per day, realistic seconds per careful review, and reviewers available. If the product exceeds capacity, either more people or less volume is required. Pretending otherwise just hides the gap.
Make errors easy to see. Reviewers catch problems faster when the interface highlights what changed, shows the source passage next to a claim, or flags the fields the system is least sure about. A wall of fluent text is the hardest thing to proofread, because fluency reads as correctness.
Review by risk, not by order. Route high-stakes or low-confidence items to careful review, spot-check the rest, and track how often spot-checks find problems. If they find none for months, that is useful evidence. If they find some, you now know the real error rate.
Seed known errors. Occasionally insert items with a deliberately planted mistake and see whether they are caught. It feels uncomfortable and is the most direct measurement of whether the review step works at all.
The takeaway: a human in the loop is a capacity-limited component, like a database or a rate limit. Size it, measure it, and watch it. Otherwise it becomes a checkbox that makes everyone feel safer while catching very little.