Without routing, every request goes to the same place: whichever model is most capable, because nobody wants to be the one who under-sized it.
This is the AI-era version of over-provisioning cloud compute — pick the biggest instance for every workload, because guessing wrong in the other direction is the visible failure. The cost of over-provisioning a model is just as real; it's just spread across thousands of calls instead of one line item, which is exactly why it goes unnoticed for so long. Routing is the fix, and it isn't one decision — it's three, evaluated together, every time a request comes in.
Dimension one: cost
Not every task needs the frontier model. A low-complexity extraction task and a genuinely hard reasoning task look identical in a prompt box and cost wildly different amounts to run on the same model. Classify by complexity, and route the former to a cheaper, faster model — reserve the frontier model for tasks that actually need it.
Dimension two: capability
Cost-only routing breaks the moment a request genuinely needs what the expensive model provides — a long context window, multi-step reasoning, a capability the cheap model doesn't have. Routing on cost alone just moves the failure from "too expensive" to "wrong answer, cheaply." Capability has to be evaluated first; cost is the tiebreaker among models that can actually do the job, not the sole criterion.
Routing on cost alone just moves the failure from "too expensive" to "wrong answer, cheaply."
Dimension three: compliance
The dimension most routing conversations skip entirely: some requests can only go to certain providers or regions regardless of cost or capability — a data-residency requirement, a contractual restriction, a regulatory constraint on where personal data can be processed. A router that doesn't know this will happily send a request to the cheapest, most capable option and violate a compliance requirement nobody thought to check for at request time — because compliance requirements don't show up in a prompt.
What a real router does with all three at once
Three example requests, each decided by a different dimension:
None of these decisions should be made by a person, per request, by hand. That's the actual point of a router: the same judgment an engineer would apply manually, applied automatically, every time — instead of only when someone remembers to.
Where this leaves you
Routing isn't a cost-optimization feature bolted onto a model API. It's the mechanism that makes "outcomes, not tokens" operationally real — the thing that turns three separate judgment calls into one system that makes them consistently, at the volume a real enterprise actually runs at.