When you debug a normal software bug, you lean on an assumption so basic you rarely state it: the same input, run again, produces the same output. That assumption is what makes bisecting, adding a breakpoint, or re-running a failing test meaningful — the bug is a fixed property of the code, waiting to be found.
That assumption quietly breaks once an LLM call sits in the path. The same prompt, same input, same model version can produce a passing result nine times and a failing one on the tenth, and neither the pass nor the fail is "the bug" in the traditional sense — both are samples from a distribution. Debugging here means asking a different question than usual: not "what line of code produces this output," but "what shifts the distribution of outputs enough that the failure rate becomes visible." A single failing transcript is evidence, not proof, and a single passing re-run after a fix is close to no evidence at all.
Practically, this changes what a reasonable debugging session looks like. Instead of reproducing once and reading a stack trace, you're running a case — or a small batch of related cases — enough times to estimate a failure rate before and after a change, and treating anything smaller than that sample size as noise. It also changes what counts as a fix: a change that makes a failure a little less frequent is real progress, even though it wouldn't count as "fixed" in a deterministic system.
Bring statistical thinking to the debugging session, not just code-reading — the bug you're chasing may not be a single point of failure at all.