LLM vendor lock-in: how to switch models without rewriting your product
Model pricing and quality shift monthly. Architect so swapping providers is a config change — abstraction layers, portable prompts, and evaluation harnesses.
The best model for your use case this quarter is unlikely to be the best model next quarter. Prices drop, new releases leapfrog incumbents, and providers deprecate versions on their schedule, not yours. If switching means rewriting your product, you are locked in — and lock-in is a cost even when the invoice looks fine.
Where lock-in actually hides
It is rarely the API call. Lock-in lives in provider-specific features baked into your product: proprietary function-calling schemas, prompt templates tuned to one model's quirks, embeddings tied to one vendor's dimensions, and SDK conveniences that skip the abstraction you will need later.
The abstraction that pays for itself
Route every model call through a single internal interface — a gateway, a thin client wrapper, anything with one seam. That seam gives you three things immediately: a single place to add retries and fallbacks, one place to measure cost and latency per model, and the ability to route a percentage of traffic to a candidate model before committing.
Keep prompts and evals portable
Store prompts as versioned assets, not string literals scattered through code. Maintain an evaluation set — a few hundred representative inputs with expected qualities — that runs against any candidate model. When a new model launches, the question is it better for us? becomes an afternoon of measurement instead of a month of debate.
Watch the embedding trap
Embeddings are the stickiest dependency: switching embedding models means re-indexing your entire vector store. If retrieval is core to your product, design the index so re-embedding is a background job you can run, not a migration you fear.
The takeaway
Portability is not about planning to switch — it is about keeping the option cheap. One abstraction seam, versioned prompts, and a standing evaluation set turn model choice from an architecture decision into a configuration change.