A surprising number of prompts contain a quiet input nobody listed: the current date. A system prompt that says "Today is {{today}}" is helpful in production and corrosive in testing.
Here is how it bites. You build an eval set in March. Some cases involve relative time: "what's due next week", "is this invoice overdue", "how old is this account". Each of those has a correct answer that depends on the date the test runs. In April the same case produces a different expected result, and your pass rate moves for reasons that have nothing to do with the model or your prompt. Worse, it moves slowly, so it looks like gradual regression rather than a bug in the harness.
The date is not the only culprit. The same pattern shows up with:
- Locale and time zone when formatting dates, currency, or numbers.
- Randomized ordering of retrieved documents or few-shot examples.
- Live lookups such as "latest price" or "current user count" that a test accidentally reaches.
The fix is cheap and boring, which is why it gets skipped:
- Make the clock an explicit input to prompt rendering, passed in rather than read from the system. Production passes the real date; tests pass a fixed one.
- Store the pinned date alongside each eval case, so expected answers stay correct forever.
- Seed any randomness, and record the seed with the run.
- Add one test that deliberately moves the date and confirms the rendered prompt changes only where you expect.
There is a bonus. Once the date is a parameter, you can test the cases that matter most and break most often: month boundaries, leap days, daylight-saving changes, and the end of the year.
If your pass rate changes and you can't explain it, check what the prompt knew about *when* it was running before you blame the model.