Tune the Exact Cache for Hit Rate and Safety

• Cache key scope that matches how you want to share
• Content classes that separate kinds of traffic
• A choice for requests that contain sensitive data
• A choice for streamed responses
• Limits, compression, and a lookup timeout

You need an exact cache already running, at least in Observe. If not, start with the guide on caching identical requests.

1. 1

   Open the card

   Go to AI → Policies, open the policy, and find Exact response cache on the Expert step.

2. 2

   Open Cache key scope dimensions

   Leave everything selected to keep entries as narrow as possible.

3. 3

   Remove what you want to share across

   Remove App to share between apps, for example.

4. 4

   Save and watch

   Compare the hit rate in Observe before you enforce the broader scope.

{
  "model": "gpt-4o-mini",
  "messages": [{ "role": "user", "content": "What are your opening hours?" }],
  "metadata": { "content_class": "faq" }
}

1. 1

   Label your requests

   Send metadata.content_class on requests, such as faq, docs_qa, or classification.

2. 2

   List the classes to cache

   In Content classes, enter the labels separated by commas.

3. 3

   Leave it blank to cache any class

   Blank means every class is eligible.

Bypass is the gentle choice and the usual one. Block suits workloads where a request with sensitive data should never be sent at all.

• Cacheable models: list models by plain name to admit every provider's offering, or provider/model to pin one
• Routes: the cache covers chat completions, completions, embeddings, Responses, and Messages requests
• Requests that use tools the provider runs for you are never cached

A slow or unavailable cache never blocks a request. It simply behaves as a miss.

• Changing the policy starts the cache fresh for that policy
• Changing a guardrail profile does the same, because the profile is part of the key
• The time to live always applies, so old answers expire on their own
• Invalidate cache, on the policy's row in AI → Policies, clears every stored answer for that policy at once

1. 1

   Note the hit estimate

   Read the preview for the current scope.

2. 2

   Broaden one thing

   Drop App from the scope, or raise the time to live.

3. 3

   Wait a few days

   Compare the estimate.

4. 4

   Keep what helps

   Undo what did not.