If your product depends on a specific hosted model version, that version will eventually be retired. Providers publish deprecation schedules because they have to; running old models forever is not sustainable. The date is rarely a surprise, yet teams routinely treat it as one, because nobody owned the question "what happens when this model goes away?"

Treating deprecation as an ordinary migration project, planned in advance, turns it from an emergency into a routine task.

Know what you depend on

Start with an inventory. Most teams discover they depend on more model identifiers than they thought: a main feature, a background summarization job, an internal tool someone built last year, an eval harness using a judge model. Each is a separate migration.

For every call site, record:

  • The model identifier, and whether it is pinned to a version or a floating alias.
  • The prompt it uses and where that prompt lives.
  • What a failure looks like to the user.
  • Who owns it.

Keep this inventory in the repository, not a wiki, so that it changes when the code changes. A single config module that holds every model identifier makes the inventory almost free, and makes the eventual swap a one-line change instead of a search across the codebase.

Make the migration testable

The hard part of a model swap is not the code. It is knowing whether behavior changed in ways that matter. That is only answerable if you already have an evaluation set that reflects real usage.

A reasonable sequence:

  1. Run your existing eval set against the candidate model with the current prompt, and record the baseline difference.
  2. Read the failures rather than only the aggregate. Prompts tuned to one model often lean on its quirks: a formatting habit, a particular reaction to terse instructions.
  3. Adjust the prompt only where failures show a real regression, and re-run. Resist rewriting everything at once.
  4. Roll out behind a flag or in shadow mode, comparing outputs on live traffic before moving users.
  5. Keep the old path available until the new one has handled real load for a while, up to the retirement date.

Floating aliases are a trade, not a shortcut

Aliases that track the latest version save you migration work and cost you control: behavior can change under you without a deploy. Pinned versions give you reproducibility and a deadline. Pick deliberately, and if you use an alias in production, keep your canary checks running so a silent change gets noticed.

The takeaway

Put the retirement date of each model you use on a calendar, assign an owner, and schedule the eval run at least a month before the deadline. The migration itself is usually modest. What turns it into a crisis is discovering, with a week left, that nobody can say whether the replacement is good enough.

Artificial IntelligenceLarge Language ModelsSoftware Engineering