Redact at the Edge: Keeping Raw PII Out of LLM Prompts and Logs Without Losing the Ability to Debug

Most teams don't decide to send raw customer PII to a third-party model provider or store it unredacted in an observability tool — it happens by accretion. A prompt template gets a customer's name and email interpolated in because it made the output read naturally. A few-shot example gets built from a real support ticket because it was the clearest one on hand. Six months later, an audit finds unredacted addresses and phone numbers sitting in prompt logs inside a logging vendor's infrastructure, and nobody can point to the moment that became true.

The instinct after that discovery is usually to overcorrect: strip anything that looks like PII before it touches a log, full stop. That solves the compliance problem and creates a new one — when a customer reports that the AI feature gave a wrong answer, the engineer investigating sees a prompt full of [REDACTED] tokens and has no way to tell whether the model reasoned correctly about the (now invisible) input. Debugging an LLM feature already means reconstructing what the model saw; redaction that destroys the shape of the input makes that reconstruction impossible.

Redact before construction, not after logging

The fix is to move redaction earlier than most teams put it. Instead of redacting a finished prompt or a captured log line — which requires re-detecting PII in free text after the fact, an error-prone pattern-matching problem — redact at the point where the prompt is assembled from structured fields you already know are sensitive. If a field is a customer's name, replace it with a stable pseudonymous token before it ever reaches the prompt template, the model call, or the log line. The model never sees the real value, the log never stores it, and the redaction is a lookup on a known field rather than a regex hoping to catch every phone number format.

Keep the mapping, not the data, and keep it separate

The reason this doesn't sacrifice debuggability is that the pseudonymous tokens are consistent within a request: PERSON_1, EMAIL_1, ORDER_87421 instead of the real values. An engineer reading a logged prompt can still see that the model correctly referenced EMAIL_1 in its reply, or wrongly conflated PERSON_1 with PERSON_2 — the reasoning is fully visible even though the underlying data isn't. The reversible mapping from token back to real value lives in a separate, access-controlled store with its own short retention window, not alongside the logs. That way a leak of the logging system — the far more likely failure, since dozens of engineers can query it — doesn't leak the PII itself, and the handful of people with a legitimate reason to unmask a token for an active investigation go through a path that's audited on its own.

The part that actually determines whether this works long-term isn't the redaction technique — it's making it non-optional. If redaction is a helper function individual engineers remember to call, it will be skipped under deadline pressure by exactly the person who most needed it enforced. Put it in the one library every prompt has to pass through to reach the model, and make bypassing it require a visible, reviewable exception rather than a missed step.