Observe cache reuse and diagnose misses
OpenAI announced GPT‑6 prompt-caching improvements on September 22. Eligible shared prefixes reused within 30 minutes receive cache discounts, helping long-running agents reuse instructions, tool schemas and conversation context. A new dashboard shows the share of input served from cache, while diagnostics identify whether a model, tool, setting or input change caused a miss and estimate affected tokens.
Developers can define explicit cache breakpoints, adjust reasoning with `configuration_update` without invalidating earlier context, and use `allowed_tools` rather than removing tool definitions. Prewarming can process known shared context before the first user request to reduce initial latency.
Maximum token discount is not total workload savings
Cached input reads can receive discounts of up to 90%, but that does not mean an application's total bill falls 90%. Output tokens, cache writes, retries and tools remain. Customer examples citing 20–36% cost reductions reflect their particular traffic and prompt structure, not a universal result.
Teams should place stable instructions and schemas before changing user data, then compare cache hit rate alongside total cost per successful task, time to first token and answer quality.