One AI Model or Several? Routing and Fallbacks Between Models
One AI model for everything is the easiest setup, and also the most fragile. Simple routing keeps an AI feature answering during provider outages and stops you paying top price for light tasks.
When building a first AI feature, almost everyone starts with one model from one provider. That's reasonable, and fine for a trial. The trouble starts once the feature is used for real: the provider is occasionally slow or errors out, light tasks run on an expensive model, and switching models feels risky because everything depends on it. This article covers how to design routing, meaning the rules for which model handles which request, along with its fallbacks.
Different models for different tasks
Not every request is equally heavy. Explaining a short error message or summarising one paragraph is fine for a fast, cheap light model. Working through a long problem, reading an image, or writing a careful step-by-step explanation suits a stronger model better. A practical pattern is to use a light model by default and keep a strong model for specific cases.
Write the selection rules down so they can be checked, rather than leaving the choice to chance or to another AI's guess.
Conditional escalation to a stronger model
Escalation means a request goes to the stronger model only when it meets certain conditions. Examples of conditions a system can easily check:
- The user explicitly asks for a full explanation, not just a hint.
- The request includes an image or a long document.
- The user has failed several times or keeps repeating the same question.
- The light model's answer was rejected by the automatic checker.
Conditions like these can be logged and counted, so you know what share of requests go to the strong model and whether the rules need loosening or tightening.
Fallbacks when a provider fails
Every AI provider has outages. So that users never see a blank screen, set a clear fallback order:
- A timeout per call. If the model doesn't answer within a few seconds, treat it as failed and move to the next option.
- A backup model or provider. The second choice is tried automatically when the first fails.
- Rest a misbehaving model. If one model fails repeatedly, skip it for a short while so other users don't wait on it too.
- An answer without AI. If every model fails, show official material, an FAQ, or an honest message that the service is having trouble.
Answer without AI when you can
The cheapest and fastest option is often no model at all. If the answer already exists in your system, such as an official hint, an FAQ, an order status, or a schedule, show it directly. Only call AI for free-form questions that can't be answered the ordinary way. This no-AI path also becomes the last fallback when the daily spending cap is reached.
Evaluate before switching models
New models keep arriving. To keep a switch from becoming a gamble, prepare a few things from the start:
- Models set through configuration, not hard-coded, so switching or rolling back needs no code change.
- The same checker for every model. Rules such as "don't cite outside the sources" apply whatever the model.
- A list of test cases from real questions, with expected answers. Run it on the old and new model, then compare results and cost.
- Automated tests for the routing rules, so a change to the fallback order doesn't quietly break escalation.
A real example: the Numekara AI tutor
The AI tutor at Numekara, the education business we run and which is live, uses the pattern above. Its default model is a light DeepSeek model through OpenRouter, with a stronger version set up as the escalation model. The strong model goes first only in specific situations, for example when a student asks for a full worked solution, the question includes a figure, the student asks for a deeper hint, or keeps making the same mistake. For coding exercises the light model is considered enough, and escalation only happens if the student is still stuck after asking for hints several times.
If the first model fails or takes too long, the system tries the next one. A model that fails three times in a row is skipped for a while, and this is tracked per model. After that comes a rule-based responder that builds hints from the official material without calling AI. That responder is also the main path for ordinary hint requests, and the only path once the day's quota or spending cap is used up. Every AI answer, from whichever model, is checked automatically before it appears, and rejected answers are replaced with the official hint.
Provider and model are set through configuration. Until mid-October 2026 the tutor ran on Gemini, and going back to Gemini only takes a settings change. At deploy time, if the OpenRouter key isn't available, the system switches to Gemini automatically. The escalation and fallback order is also guarded by automated tests.
When one model is enough
Routing adds parts you have to maintain. If your AI feature is still a trial, used by few people, or not critical to customers, one model with a timeout and an honest fallback message is enough. Add routing when real usage starts, costs become noticeable, or provider outages start to hurt.
A good AI feature doesn't depend on a single model. It knows when to use a light model, when to step up to a strong one, and when the official material is answer enough.
