Most prompt engineering happens in English, gets evaluated against an English test set, and ships worldwide the moment it clears that bar. The prompt usually keeps working in other languages in the sense that it produces fluent, plausible-looking output. Whether it's doing the same job it does in English is a separate question that almost nobody checks, and the gap between those two things is where a lot of quietly bad international experiences come from.
Where the Assumptions Hide
Length instructions are the easiest to get wrong. "Answer in under 100 words" behaves very differently once you're in a language where the tokenizer needs two or three times as many tokens to represent the same sentence — the model either truncates content that would have fit comfortably in English, or ignores the constraint entirely because honoring it would make the answer useless. Formatting cues have the same problem: examples built around English punctuation, quotation conventions, or markdown bullet habits don't necessarily map onto languages with different orthographic norms, and a model will sometimes force the target language into an English-shaped template rather than a natural one.
Register is subtler and easier to miss in a spot check. An instruction like "be direct and confident" reads as professional in English business writing and can read as brusque or inappropriately informal in a language with more built-in politeness marking — the model complies with the instruction and still produces something that feels wrong to a native speaker, for reasons that don't show up unless you specifically ask one. And few-shot examples written in English don't teach the underlying rule as cleanly in another language, because a lot of what the model is generalizing from is example-specific phrasing, not just the abstract instruction above it.
Testing for It Instead of Assuming It
The fix isn't a more careful English prompt — it's treating other languages as a real slice of your eval set instead of an implicit assumption. Build a small set of real queries and known-good answers in each target language, rather than machine-translating your English eval set and grading against translated expectations, since that only tests translation fidelity, not whether the task itself is being done correctly. Check length constraints in tokens for that language's tokenizer, not in words. And get a native speaker to review a sample of outputs specifically for register and naturalness, not just factual correctness, since that's the failure mode automated grading is least likely to catch.
None of this needs to be exhaustive before a first international launch. Three languages tested deliberately, with real native-speaker review of a small sample, will surface more actual problems than zero languages tested at all — and it's far cheaper to fix a register problem in a prompt than to explain it after users have already noticed.
Comments (0)
No comments yet — be the first.
Sign in to leave a comment.