On this page
01
What you'll set up
You have watched Observe and you like what you see. In about ten minutes you will send a small share of real traffic through the recommended routes, set the thresholds that stop a bad candidate, and plan how to widen.
- A canary share of live traffic
- Rollback thresholds that you choose
- A schedule for widening
- A switch to Enforce when you are ready
You need a policy with adaptive routing already in Observe. If you do not have one, start with the guide on routing to cheaper models.
02
Choose your mode
Three modes cover the path from looking to doing. You choose when to move.
| Mode | What happens | Use it when |
|---|---|---|
| Observe | Recommendations are recorded, traffic is unchanged | You are learning what routing would do |
| Canary | A share of traffic, which you set, takes the recommended route | You want proof on live traffic |
| Enforce | Every eligible request takes the recommended route | You trust the results |
04
How the canary group is picked
Cloptima groups requests the same way every time, by what the request is, not by chance.
The same request always lands in the same group. A request is either in the canary or it is not, so you can compare like with like and a repeated request does not bounce between routes.
05
Link evidence if your process wants it
Some teams want an eval run and a sign-off behind a routing change. You can link both, and they are yours to use or skip.
- 1
Run an eval
Build a dataset of real requests and run an eval of the routing change. See the prompt registry guides for how evals work.
- 2
Get a sign-off
Ask a reviewer to approve the change in the approval queue under AI → Audit.
- 3
Link them
Add the eval run and the approval to the policy.
Linked evidence is kept with the policy, so you can answer later why a route was changed.
06
Set rollback thresholds
Thresholds are the rules for stopping a candidate that is not working. You choose them.
| Threshold | What it means | A good start |
|---|---|---|
| Max error rate | Stop a candidate when its share of failed requests is above this | 0.05 to 0.1 |
| Max fallback rate | Stop a candidate when too many requests need a fallback | 0.2 |
| Min sample size | Do not judge a candidate until it has served this many requests | 100 or more |
| Evaluation window | How far back to look, in minutes | 30 |
| Cooldown | How long a stopped candidate stays off before it can return, in minutes | 30 to 60 |
07
What automatic rollback does
When a candidate crosses a threshold, Cloptima stops routing to that candidate. The rest of your routing carries on.
1Candidate misbehaves
Errors or fallbacks rise
2Threshold crossed
After the minimum sample
3Candidate stopped
Traffic goes back to the original route
4You re-enable it
When the cause is fixed, or after the cooldown
No deploy is needed. Requests that would have gone to the stopped candidate take the route they would have taken without routing.
08
Widen on your schedule
Widen only when the last step looks good.
| Step | Share | Look at before moving on |
|---|---|---|
| 1 | 1% | Errors, latency, and answer quality on the canary group |
| 2 | 5% | Estimated against actual cost per request |
| 3 | 25% | Fallback rate and provider spend |
| 4 | 50% | Complaints, if your users can report quality |
| 5 | 100%, then Enforce | A week of stable numbers |
Spend a few days at each step. A longer step costs little and shows more.
09
Switch to Enforce
When the canary has run at the share you want, change the mode.
- 1
Choose Mode: Enforce
Every eligible request now takes the recommended route.
- 2
Keep the thresholds
They keep protecting you after the switch.
- 3
Keep watching
Compare spend by Model and Provider in the Explorer over the next week.
10
Pause or undo
You are never locked in.
| To | Do this |
|---|---|
| Stop all routing now | Set Mode back to Observe |
| Take a share away | Lower Canary basis points |
| Stop one candidate | Remove it from its tier |
| Bring a stopped candidate back | Re-enable it from the policy's routing results |
11
If something goes wrong
Most surprises are about shares and thresholds.
| What you see | Likely cause | Fix |
|---|---|---|
| No traffic takes the new route | The share is zero, or requests are outside the group | Check Canary basis points and widen |
| A candidate stopped on its own | It crossed a rollback threshold | Read the reason, fix the cause, and re-enable it |
| A candidate never stops despite errors | The minimum sample size is high | Lower Min sample size, or widen the window |
| Costs did not fall | The recommended routes are close in price | Compare estimated against actual cost and adjust the tiers |