Ask two engineers to look at the same model output and decide if it's "good enough to ship," and you'll often get two different answers — not because one of them is wrong, but because "good enough" was never actually defined. One person is grading against correctness, the other against tone, and a third silently weighs how confident the answer sounds. None of that disagreement shows up in a dashboard; it just shows up later as inconsistent judgment calls that nobody can point to a rule for.
This is a bigger problem than it looks, because most teams treat manual review of model output as ground truth: if a human says it's fine, it's fine. But human judgment on unstructured text is only as reliable as the rubric behind it, and most teams don't have one — they have a vague shared sense of quality that two people apply differently on a bad day, a rushed day, or a day when they're reviewing the fiftieth output in a row.
The fix is unglamorous but cheap: write down the two or three things a "good" output actually has to do — not vibes, but checkable claims like "cites the right document" or "doesn't invent a number that isn't in the source" — and have reviewers grade against that list, not their gut. Run a short calibration pass where two people grade the same ten outputs independently and compare, before trusting either one alone on the next hundred. It won't remove disagreement entirely, but it turns disagreement into information — a sign your rubric is missing something — instead of noise you can't act on.