When a user reports a bug in a deterministic feature, the report usually points at something close to the actual cause: a button doesn't work, a page 500s, a field saves the wrong value. When a user reports a bug in an AI feature, the report almost always describes a symptom one or two layers removed from the cause, because the user has no way to see the layer where the bug actually lives.
"It gave a weird answer" could mean the retrieval step returned the wrong documents, the prompt template broke on an edge case in the user's input, the model itself produced an off-distribution response, or the output was fine and the rendering layer mangled it. All four look identical from the user's side of the screen: unexpected text. Treating the report as if it already named the cause — and going straight to "let's improve the prompt" — means you'll sometimes fix the wrong layer, watch the report recur, and use that recurrence to blame the model when the model was never the issue.
The practical fix is boring but effective: before touching anything, reconstruct what the system actually saw and returned for that specific interaction — the retrieved context, the exact prompt sent, the raw model output, and the rendered output — and only then ask which layer diverged from what you'd expect. Most "the AI got it wrong" reports resolve into a much more ordinary bug once you can see all four layers side by side. The AI is often the layer that was working exactly as designed.