On this page
01
What you'll set up
Not every request needs your strongest model. Adaptive routing lets you define tiers of models and lets Cloptima pick the right one per request. In about eleven minutes you will define your tiers, turn routing on in Observe, and read what it would have done.
- Three tiers of candidate models: cheap, balanced, and strong
- A starting tier that fits your risk appetite
- Observe mode, so nothing changes while you learn
- A read on what each request would have cost
You need an owner or admin role and a policy bound to your traffic. You decide the mode, the tiers, and how far to go.
02
How tiers and candidates work
You list candidate models in each tier. For every request, Cloptima starts at the tier you chose and takes the first candidate that can do the job.
1Request arrives
With its needs
2Starting tier
Your choice
3First capable candidate
Moves up a tier if none
4Recommended route
Used by the mode you chose
In Observe the recommendation is recorded and the request goes where it always did. In Canary and Enforce it is used.
03
Define your tiers
Pick the models you trust for each level of work. Any active model in the catalog can be a candidate.
| Tier | A good fit | Example work |
|---|---|---|
| Cheap | Short, simple, high-volume requests | Order status, classification, extraction |
| Balanced | Everyday reasoning | Summaries, drafts, routine analysis |
| Strong | Hard or high-stakes requests | Complex reasoning, long documents |
04
Choose where routing starts
The starting tier is your appetite for trading quality for cost.
| Starting tier | What it means |
|---|---|
| Cheap | Try the cheap tier first, then move up if no candidate qualifies |
| Balanced | Start in the middle and move up if needed |
| Strong | Use the strongest tier, and keep the other tiers for reference |
Start cautious. You can lower the starting tier as the results prove out.
05
What a candidate has to satisfy
A cheap model that cannot do the job is not a saving. Cloptima checks each candidate against the request.
| Check | Why it matters |
|---|---|
| Tool calling | A request that offers tools needs a model that can call them |
| Structured output | JSON mode and JSON schema need support in the model |
| Images and other media | A vision request needs a vision model |
| Streaming | A streamed request needs a model that streams |
| Context room | The prompt and the answer must fit comfortably in the context window |
| A known price | So the cost can be estimated and budgeted |
| A usable credential | You must be able to reach the provider |
| Healthy route | A route that is failing is skipped |
| Your policy lists | Allowed and denied providers and models still apply |
06
Turn it on in Observe
Adaptive routing is on the Expert step of the policy form.
- 1
Open the policy
Go to AI → Policies and open the policy that governs the traffic.
- 2
Open Adaptive routing
On the Expert step, switch on the Adaptive routing card.
- 3
Choose Mode: Observe
Nothing changes for your users.
- 4
Pick candidates for each tier
Use the pickers for Cheap, Balanced, and Strong.
- 5
Name the candidate set
Give it a version label, such as 2026-10-a. Change it whenever you change the candidates, so you can tell results apart.
- 6
Save
Routing starts recording at once.
- 2Adaptive routing
- Choose cheap/balanced/strong candidate models per app so the gateway can route each request to the right tier.On
- 3Mode
- Observe
- 4Cheap tier
- Claude Haiku 4.5
- 4Balanced tier
- Claude Sonnet 5.5
- 4Strong tier
- Claude Opus 5.5
- 5Candidate set version
- 2026-10-a
07
Read what it would have done
Observe records, for each request, the route Cloptima would have chosen and what it would have cost.
| You see | Meaning |
|---|---|
| Recommended route | The provider and model Cloptima would have used |
| Estimated cost change | The difference from the route the request actually took |
| Tier | The tier the recommendation came from |
| Why not | The reason when no candidate qualified |
Look at a few days of traffic. If the recommended routes look right and the estimated saving is worth it, you are ready for a canary.
08
A worked example
Take an assistant that handles 10,000 requests a day. The prices below are an illustration.
| Plan | Requests | Cost per request | Daily cost |
|---|---|---|---|
| Everything on the strong model | 10,000 | $0.0100 | $100.00 |
| Tiered: cheap for simple requests | 7,000 | $0.0004 | $2.80 |
| Tiered: balanced for everyday requests | 2,000 | $0.0030 | $6.00 |
| Tiered: strong for hard requests | 1,000 | $0.0100 | $10.00 |
| Tiered total | 10,000 | $18.80 |
In this illustration tiering cuts the daily cost by about 81 percent. Observe mode tells you your own numbers before you commit.
09
Compare with real spend
Use the Explorer next to the Observe results.
- Group by Model to see where the strong model carries easy work
- Group by App to find the apps worth tiering first
- Check Blocked to be sure tiering is not hiding policy issues
10
Keep it as code
Adaptive routing can live in Terraform.
resource "cloptima_llm_gateway_policy" "support_production" {
name = "support-production"
mode = "enforce"
routing_adaptive_enabled = true
routing_adaptive_mode = "observe"
routing_adaptive_candidate_models_cheap = ["gemini/gemini-2.5-flash-lite"]
routing_adaptive_candidate_models_balanced = ["openai/gpt-4o-mini"]
routing_adaptive_candidate_models_strong = ["anthropic/claude-opus-5-5"]
routing_adaptive_candidate_set_version = "2026-10-a"
}11
If something goes wrong
Most questions are about candidates and credentials.
| What you see | Likely cause | Fix |
|---|---|---|
| No recommendation for a request | No candidate met the request's needs | Add a candidate that supports tools, structured output, or the media the request uses |
| Recommendations never use a tier | Its models fail a check, often price or credentials | Check the credentials and that the models have a known price |
| Recommendations look too aggressive | The starting tier is low | Raise the starting tier |
| Nothing is recorded | The policy is not bound to the traffic | Check the bindings |