Embeddings Are Versioned Data: Planning for the Day You Change the Embedding Model

Most teams treat a vector store like a cache: fill it once, query it forever. That works until someone wants a better embedding model, a different chunking scheme, or a new text-cleaning step. At that point you discover that the index was never just data. It was data *plus* the exact function that produced it.

Vectors from different models are not comparable

Two embedding models can both emit 1024-dimensional vectors and still live in unrelated spaces. Cosine similarity between a query embedded by model B and documents embedded by model A returns numbers, but the numbers are noise. Nothing throws an error, which is the dangerous part: retrieval quality simply degrades and the symptom shows up weeks later as "search feels worse."

The same applies to quieter changes. Altering the lowercasing rule, stripping HTML differently, adding a title prefix to each chunk, or changing the chunk size all change what gets embedded. Any of them can shift the space enough to matter when old and new vectors are mixed.

Stamp every vector with its recipe

Store a small embedding_recipe alongside each record or collection:

• model identifier and version
• output dimensions and any truncation
• preprocessing/normalisation version
• chunker version and parameters

At query time, assert that the query-side recipe matches the collection's recipe, and fail loudly if not. This single check turns a silent quality regression into an immediate, debuggable error.

Migrate with two indexes, not one in-place rewrite

When you do change the recipe, avoid re-embedding in place:

1. Build a second index under the new recipe, backfilling in the background and writing new documents to both.
2. Run your retrieval eval set against both indexes and compare recall at the cutoff you actually use. Do not assume the newer model wins on your data.
3. Shift a slice of traffic (or run in shadow mode and diff results) before cutting over.
4. Keep the old index until the rollback window passes, then delete it.

Budget for this up front. Re-embedding a large corpus costs real money and time, and the dual-write period means roughly double storage for a while.

The takeaway

Write the embedding recipe into your schema on day one, enforce it at query time, and treat any change to it as a migration with an eval gate and a rollback path. The cost is a few extra fields now; the alternative is debugging a retrieval regression that nothing in your logs can explain.