Every prompt that survives a few months of maintenance grows rules. "Be concise." "Always include the reasoning." "Never ask the user clarifying questions." "If information is missing, ask the user for it." Each rule was added to fix a real problem. Each is reasonable alone. Together, some of them contradict each other in specific situations, and the prompt says nothing about which one wins.
The model will still produce an answer. It resolves the conflict silently, and often differently from one run to the next. That inconsistency is easy to mistake for randomness when it is really an unspecified decision in your own spec.
Find the collisions
You rarely find these by rereading the prompt, because each rule looks fine in isolation. Find them by asking, for each pair of rules, "is there an input where obeying both is impossible?" Common collision patterns:
- Brevity versus completeness. "Keep answers under three sentences" against "list every applicable caveat."
- Autonomy versus confirmation. "Complete the task without interrupting the user" against "confirm before any destructive step."
- Format versus content. "Always respond in JSON" against "if you cannot answer, explain why in plain language."
- User instructions versus operator instructions. A user message asks for something the system prompt forbids, or loosely implies an exception.
A fast technique: take your ten most awkward real inputs and, for each, list which rules *fire*. Any input where two rules fire in opposite directions is a collision.
State precedence, and give the loser a fallback
Resolve each collision in the text, in one place, near the top or in a dedicated section. Make the ordering explicit and, where you can, give the losing rule a way to be partly honored:
Priority order when instructions conflict:
1. Safety and data-handling rules above.
2. The output format contract.
3. The user's explicit request.
4. Style guidance (length, tone).
If you cannot satisfy the format and also explain a problem, put the
explanation in the "notes" field of the JSON rather than breaking the format.The fallback line matters. A bare ranking tells the model what to drop; a fallback tells it how to keep the spirit of the lower-priority rule. Without it, you get a response that obeys rule 2 and leaves the user with no explanation at all.
Test the collision, not just the happy path
Once the precedence is written down, add one eval case per collision, designed so that the two rules pull apart. These cases are cheap and they pin the behavior: if a later edit reintroduces the ambiguity or someone adds a new rule that cuts across the ordering, the case fails instead of production drifting.
Treat precedence like any other contract. A prompt without one is making hundreds of small, unrecorded decisions on your behalf, and you will only learn what they were by reading user complaints.
Comments (0)
No comments yet — be the first.
Sign in to leave a comment.