OpenAI has enhanced the prompt caching system for the GPT‑6 family to increase default cache hit rates, reduce response latency, and help developers save on costs. The improvements are aimed at persistent agents that run for hours on complex tasks — such as refactoring large codebases or producing thoroughly researched documents and presentations.
Why prompt caching matters and what changed
Applications that operate persistent agents send a series of API requests that frequently reuse the same instructions, tool definitions, and context from earlier turns. OpenAI caches these shared parts to reuse previous computation, which shortens response times and can give developers discounts of up to 90% on cached input tokens.
With GPT‑6, OpenAI updated the caching behavior so that eligible shared prefixes reused within a 30‑minute window now receive cache discounts by default, improving cache hit rates out of the box.
New tools and controls for developers
-
Prompt Caching Dashboard: a dashboard that shows how much of an application’s input is served from cache. It lets you track hit rates over time and compare cached versus uncached tokens using an input composition chart, helping spot drops in caching performance and evaluate the impact of application changes.
-
Prompt caching diagnostics tool: when you encounter an unexpected cache miss, this tool compares a request with a recent response to reveal what prevented reuse — such as changes to the model, tools, settings, or input. It also estimates the number of affected tokens so you can judge the size of the impact and prioritize fixes.
-
Explicit cache breakpoints: developers can mark which prompt prefixes should be reusable. The refreshed prompt caching guide explains how to use these breakpoints, how long cached prefixes remain eligible, and how tool or input changes affect reuse.
Adjusting reasoning and handling tool changes without breaking cache
-
Change reasoning effort without breaking cache: on GPT‑6 models, you can now modify the reasoning effort between responses by appending a configuration_update, while leaving the request‑level reasoning effort unchanged. This allows you to increase effort for harder tasks or lower it for routine follow‑ups while preserving reusable context.
-
Preserve cache as tools and instructions evolve: keep tool definitions, schemas, and ordering stable so earlier context remains reusable. Use allowed_tools to limit callable tools or set tool_choice to none when no tools are required, rather than removing definitions. New developer messages can be appended near the end of the context to override older instructions without deleting them.
Prewarm the cache to cut latency
Prewarming prepares known context ahead of time so the model can begin responding faster when a request arrives. For example, an application can prewarm shared instructions, tool definitions, or reference material during startup before the user’s first query. This moves some processing outside of the user’s wait time.
How to apply these options in your integration
These optional controls build on the engine’s default performance and let developers tailor caching to their workload. OpenAI recommends:
- Monitor cache hit rates in the Prompt Caching Dashboard.
- Investigate unexpected misses with the diagnostics tool.
- Follow the prompt caching guide to improve your setup, or use Codex to review your code, apply improvements, and measure results.
The new dashboard, diagnostics, explicit cache controls, reasoning configuration updates, and prewarming options are intended to help developers maximize the benefits of prompt caching: lower costs, faster responses, and more stable behavior for long‑running, complex agent tasks.
