Full AI Governance. A Latency Cost That Rounds To Zero.
Cloptima governs and executes every AI request — policy, live pricing, atomic budget enforcement, guardrails, and durable accounting, plus making the call itself — end to end. We publish, and keep re-publishing, exactly what that costs in added latency, because a number you can check beats a number you have to take on faith.
Why we publish this
Every AI gateway vendor claims to be fast. Almost none isolate governance overhead from provider variance, or say what happens to the number under guardrails and budget enforcement instead of a bare proxy pass-through. That makes "fast" a marketing word, not a comparable fact.
- Vendor benchmarks usually run every policy off
- Provider response time (which no gateway controls) gets blended into the headline number
- Guardrails, budget enforcement, and caching are rarely measured as their own scenario
How we measure it
Measured against a deterministic, fixed-latency backend, not a live model provider, so the number reflects what Cloptima adds, not provider variance. For governed and guardrails, that figure covers the whole request — every governance check plus actually executing the call, start to finish. Timing is sourced from server-side metrics, so client network latency and provider response time are excluded from the published figure.
- Sustained fixed-rate load against a deterministic backend, not a single cold request and not a live model provider
- Every rate has to hold: near-full target traffic achieved, error rate near zero, no dropped iterations
- For blocked-input and model-denied, provider request counts are verified at zero before we publish
- We publish p95, not a cherry-picked tail figure that swings from run to run
What we actually publish
Governed, guardrails, and exact-cache are performance claims: what the full request — governance, safety checks, and making the call — costs in added milliseconds. Blocked-input, model-denied, and budget-race are correctness claims: what governance guarantees, verified by checking that zero provider calls or zero overspend actually happened — not just that the number looked fast.
Full Governance, Executed — Not Just Routed
Portkey, Bifrost, LiteLLM, and similar tools are purpose-built proxy engines. Cloptima is a full governance and execution platform: virtual-key auth, attribution, policy, pricing, atomic budget enforcement, guardrails, and durable accounting, plus making the call itself — all at a cost close to nothing in latency, so governance is never the reason a team skips it.
What each scenario proves
Governed, guardrails, and exact-cache are latency claims. Blocked-input, model-denied, and budget-race are correctness guarantees, verified rather than timed.
| Capability | What it measures | Result |
|---|---|---|
| Governed request | Virtual-key auth, attribution, policy, live pricing, atomic budget enforcement, and execution of the call itself — plus a durable, reconciled accounting record after. | p95 37ms |
| Guardrails | The same governed request with input and output safety detectors enabled (prompt injection, jailbreak, toxicity, PII, secrets) against benign traffic. | p95 38ms |
| Exact response cache | Governed cache hits versus misses on identical requests. | Cache-check overhead: the same ~37ms as a plain governed request. On a hit, the rest of the request and 100% of that request's provider spend are skipped entirely. |
| Prompt-injection enforcement | Enforcement decision latency for requests carrying a known prompt-injection pattern. | Blocked before provider egress in p95 18ms — zero provider calls made, on every run. |
| Model / provider allow-list enforcement | Enforcement decision latency for a request targeting a model or provider outside the bound policy. | Blocked before provider egress in p95 5ms — zero provider calls made, on every run. |
| Atomic budget enforcement | A concurrent burst of requests racing against the same budget ceiling at the same instant. | A concurrent burst racing the same budget ceiling stops admitting the instant the cap is hit — zero overspend, every admitted request reconciled exactly against the durable ledger. |
Launch path
Each scenario below states what it measures and why it matters for a governance buyer. Numbers update as we re-run and re-publish the suite; this page is the single place they live.
FAQ
Operationalize LLM FinOps Across Your Apps
Start with telemetry, gateway governance, or provider bill matching workflows. Keep model spend connected to engineering ownership and finance reporting.