How prompt caching works, from the KV cache to your bill

Take turn 11 of a 20-turn coding-agent session on Claude Opus 5.5 (claude-opus-5-5). If the conversation has only ever been appended to, that turn's input costs $0.0334 at Anthropic's list prices as of 27 September 2026. Change one line of the system prompt before sending it, with everything else identical, and the same turn costs $0.2950, 8.8 times as much. Over the whole session the input bill goes from $0.70 to $0.97. The session is illustrative and nothing in it was measured: every figure about it is our arithmetic, done with a short script, on stated token counts and list prices dated 27 September 2026.

The jump has a mechanical cause. A prompt cache holds the model's computed state for the start of a request, and that state stays valid only while every token before the new content is identical, so an edit near the top forces at least everything after it, and with Anthropic's automatic caching alone the whole request, to be processed and written again. This piece follows that mechanism to the invoice: how to read a cached turn on the Anthropic and OpenAI APIs, why a missed token now costs 25 to 50 times a cached one on Claude Opus 5.5 and Fable 5.1, which edits restart the cache and which also void those models' earlier reasoning, and why Anthropic and OpenAI now want settings changed by appending rather than editing. For the practical upshot, skip to the rule for changing a running session at the end.

The example: a 20-turn coding agent

The session used throughout this piece is illustrative: a 20-turn coding agent whose requests grow from 14,000 to 99,500 input tokens, because every turn resends the whole conversation so far. We chose the inputs to resemble an agent with a sizeable toolset; none of them comes from a real trace.

• A fixed prefix of 12,000 tokens: 8,000 tokens of tool definitions and a 4,000-token system prompt.
• A 2,000-token first user message.
• 3,000 tokens of tool results appended on each later turn.
• 1,500 output tokens per turn, of which 1,000 are thinking.

Every earlier output, thinking included, goes back unchanged as input on the next turn. Anthropic's extended thinking documentation says prior-turn thinking blocks are "billed as input tokens like the rest of the conversation history". So each request is the previous one plus 4,500 new tokens: the previous turn's 1,500 output tokens and 3,000 tokens of fresh tool results. Turn 11 sends 59,000 tokens and turn 20 sends 99,500. Over 20 turns that adds up to 1,135,000 input tokens and 30,000 output tokens.

One assumption needs stating. On both Opus 5.5 and Fable 5.1 thinking blocks come back with an empty thinking field by default, the reasoning carried in an encrypted signature, and the same page says "Either way the block is billed the same and passed back the same in multi-turn conversations". No page gives a resent block's input-token count as a number, so the calculation assumes it equals the original thinking tokens: consistent with the documentation, but not stated by it.

Caching is priced as Anthropic's automatic caching, where a single top-level cache_control field puts the breakpoint on the last cacheable block and moves it forward as the conversation grows, with no other breakpoints unless a scenario says so, and as OpenAI's implicit mode, described in the section on the bill. Each turn reads the previous request from cache and writes only its 4,500 new tokens; turn 1 writes everything. Every request is longer than every model's minimum cacheable length and shorter than GPT-6 Sol's 272K-token long-context tier.

What a prompt cache stores, and why it is positional

A prompt cache keeps the key and value tensors a model computed for the start of a request, and Anthropic and OpenAI both say in their documentation that this KV state is what they hold. OpenAI's prompt caching guide puts it plainly: "The prompt cache stores key-value (KV) tensors, not the tokens themselves." Anthropic's API and data retention page says that for prompt caching, "KV cache representations and cryptographic hashes are held in memory for the cache TTL and promptly deleted after expiry."

Keys and values are what attention works with. As a transformer reads a prompt, each layer computes a key vector and a value vector for every token, and each later token looks back over them to decide what to draw on. Within a single request, models already keep these vectors so that generating a new token does not mean recomputing all the earlier ones; Sebastian Raschka's Understanding and Coding the KV Cache in LLMs from Scratch (17 June 2025) walks through that with code. A prompt cache stretches the same idea across requests. Reusing stored state takes a different arithmetic path from computing the whole prompt in one pass, so a cache hit can shift a response's scores in their last bits, one of the reasons temperature 0 does not give the same answer twice.

The reason it has to be a prefix is that the state the model stores for a token is computed from that token and everything before it. In our session, token 5,000 sits in the middle of the tool definitions. Its stored state is reusable on the next request only if tokens 1 to 4,999 are exactly what they were. Change one character at token 300 and the state for every position from 300 onward is different, even if all the text after it is unchanged. The provider can reuse state up to the first difference and nothing after it.

Open serving engines make this concrete. vLLM's automatic prefix caching design splits the KV cache into fixed-size blocks and says "we hash each kv-cache block by the tokens in the block and the tokens in the prefix before the block". Each block's hash includes its parent's hash, and "We only cache full blocks." A changed token changes its block's hash, which changes the next block's, and so on to the end of the request.

The hosted vendors state the same rule in their own terms. Anthropic's prompt caching documentation says: "Because the hash is cumulative, covering everything up to and including the breakpoint, changing any block at or before the breakpoint produces a different hash on the next request." OpenAI's guide says: "Cache reuse requires the entire rendered prefix to match." Google's Gemini caching page says "Cached content is a prefix to the prompt" and does not mention KV state. None of the three publishes how its serving stack stores or finds cached state. The block-and-hash picture is vLLM's, used here because it is documented, not because the vendors are known to work the same way.

Why cached state expires and why writing it costs extra

KV state is large: on the open Qwen2.5-72B-Instruct model it takes 320 KiB per token, so a 99,500-token request, the largest in this piece's example session, would occupy about 32.6 GB of memory. The arithmetic uses the model's published configuration: 80 layers, 8 key-value heads of dimension 128, one key and one value each, and 2 bytes per number in bfloat16, which comes to 327,680 bytes per token. The 12,000-token fixed prefix alone is 3.93 GB. (That request exceeds the model's native 32,768-token context and needs YaRN scaling, which by our reading of the config leaves the size per token unchanged.) Proprietary architectures are not public, so treat this as an illustration of scale, not a figure for any vendor's model. The same arithmetic decides how much context fits on your own GPU, which our guide to how much GPU memory a local model needs works through for popular open models.

State that size cannot be kept indefinitely, so cached entries are evicted. vLLM evicts the least recently used block first. The hosted vendors publish a lifetime instead, and each runs its own clock:

• Anthropic: 5 minutes by default (also the default for automatic caching) or 1 hour at a higher write price. "The lifetime is measured from the start of the request that writes or reads the cache entry, not from the end of its response." The documentation's own example: if a response takes 4 minutes to stream, the next request must start within about 1 minute of it finishing.
• OpenAI, for GPT-5.6 and later models: 30m is the only supported value, and a cached prefix "remains eligible for reuse for 30 minutes after its most recent write or reuse, though OpenAI may retain it longer". The guide lists earlier models as having no cache-write charge and in_memory or 24h retention, which is what many older explainers describe.
• Google: implicit caching "is enabled by default for all Gemini 2.5 and newer models", according to the context caching page, with no documented lifetime; the Generate Content API caching page adds that it carries "no cost saving guarantee". Explicit caches, which you create and pay to store, default to a 1-hour TTL with no minimum or maximum.

Writing to the cache costs 1.25 times the ordinary input price on Anthropic's 5-minute cache and on OpenAI's GPT-5.6 and later, and twice the input price for Anthropic's 1-hour cache. The vendors do not say why; holding many gigabytes per conversation in memory for minutes at a time is our guess.

Four smaller rules explain some common surprises:

• Minimum length. Prompts shorter than 512 tokens on Opus 5.5 and Fable 5.1, 1,024 on OpenAI's GPT-5.6 and later, and 4,096 on Gemini 3.8 Flash are not cached. On Anthropic, "no error is returned".
• Concurrency. On Anthropic "a cache entry only becomes available after the first response begins", so parallel requests sent at the same moment cannot hit each other's entry. Anthropic's advice is to wait for the first response before sending the rest.
• Isolation. Anthropic isolates caches per workspace on the Claude API, and per organisation on Amazon Bedrock and Google Cloud.
• Rate limits. Anthropic says "cache hits are not deducted against your rate limit". OpenAI says "Cached input tokens still count toward tokens-per-minute limits." How each platform counts cached input and max_tokens against its limits is compared in our guide to max_tokens and rate limits.

Reading a cached turn on the bill

On Anthropic's API a cached request reports three input counts, cache_read_input_tokens, cache_creation_input_tokens and input_tokens, and each is billed at its own rate. Together they make up the whole input: total_input_tokens = cache_read_input_tokens + cache_creation_input_tokens + input_tokens. input_tokens counts only the tokens after the last cache breakpoint, and a cache_creation object splits the writes into ephemeral_5m_input_tokens and ephemeral_1h_input_tokens.

Turn 12 of the session on Opus 5.5 would report cache_read_input_tokens of 59,000 (everything turn 11 sent), cache_creation_input_tokens of 4,500 (turn 11's output plus the new tool results) and input_tokens of 0. At $0.20 per million for reads and $5 per million for 5-minute writes, the turn's input costs 59,000 x $0.20 + 4,500 x $5, per million tokens, which is $0.0343. input_tokens is 0 here only because automatic caching puts the breakpoint on the last block; in real traffic a few tokens after the breakpoint are normal, billed at the base input price.

OpenAI reports the same turn differently. Its input_tokens is inclusive, and usage.input_tokens_details carries cached_tokens and cache_write_tokens; the guide's own example code computes ordinary input as input_tokens - cached_tokens - cache_write_tokens. On GPT-6 Sol, turn 12 would show input_tokens of 63,500, cached_tokens of 59,000 and cache_write_tokens of 4,500, leaving 0 ordinary tokens, and would cost 59,000 x $0.20 + 4,500 x $2.50, per million tokens, which is $0.02305. The field called input_tokens excludes cached and written tokens on Anthropic and includes them on OpenAI, so a cost dashboard that reads both APIs the same way will misreport one of them.

Where the writes happen differs too. OpenAI says "Prompt caching is enabled by default". Its guide describes an implicit mode, which places a breakpoint at the end of the latest eligible message (a user message, the last tool response in a group, or the last initial developer message), and an explicit mode with breakpoints you place yourself and "up to four cache writes" per request, in which a request with no breakpoints is not cached at all. The guide refers to a default breakpoint but does not say outright which mode is the default. Google's Interactions API reports Gemini cache hits in usage.total_cached_tokens; Google does not document how implicit hits are discounted, so there is no worked Gemini turn here.

These are the list prices every figure in this piece uses, in US dollars per million tokens as of 27 September 2026:

Model	Input	Cache write	Cache read (share of input)	Output	Minimum cacheable	Cache lifetime
Claude Opus 5.5 (claude-opus-5-5)	$4	$5 (5 min), $8 (1 h)	$0.20 (0.05x)	$20	512 tokens	5 min or 1 h, from request start
Claude Fable 5.1	$10	$12.50 (5 min), $20 (1 h)	$0.25 (0.025x)	$50	512 tokens	5 min or 1 h, from request start
GPT-6 Sol	$2	$2.50 (30 min)	$0.20 (0.1x)	$10	1,024 tokens	at least 30 min after last write or reuse
Gemini 3.8 Flash, implicit caching	$0.75	none	$0.075 (0.1x)	$3.75	4,096 tokens	not documented
Claude Opus 5 (reference)	$5	$6.25 (5 min), $10 (1 h)	$0.50 (0.1x)	$25	512 tokens	5 min or 1 h, from request start

Sources: Anthropic's pricing page, its What's new in Claude Opus 5.5 page and release notes (Opus 5.5 released 22 September 2026); OpenAI's GPT-6 Sol model page (released 22 September 2026, per OpenAI's changelog); Google's Gemini API pricing page, last updated 24 September 2026, Standard paid tier. Three conditions attach:

• GPT-6 Sol prompts with more than 272K input tokens are "priced at 2x input and cache rates and 1.5x output for the full request". Claude 4.6 and later models get the "full 1M token context window at standard pricing".
• Gemini 3.8 Flash's prices are introductory: input is "$0.75 through December 31, 2026. $1.50 starting January 1, 2027", and the output and cached rates double on the same date. Explicit-cache storage costs $0.50 per million tokens per hour. Google's pricing page lists a "context caching" rate without saying that implicit hits are billed at it; pricing implicit hits at that rate is our inference.
• The GPT-6 Sol model page prices cached input "at 10% of the uncached input token rate" and cache writes "at 1.25x the uncached input token rate"; OpenAI's caching guide states the same rules for "GPT-5.6 and later" models without naming Sol.

The cost of a miss, relative to a hit, is the write price divided by the read price, because a token that misses is written at the write rate instead of read at the read rate. On the 5-minute cache that ratio is 25 on Opus 5.5 and 50 on Fable 5.1. On GPT-6 Sol and on Opus 5, whose reads cost a tenth of input, it is 12.5. With Anthropic's 1-hour write it rises to 40 on Opus 5.5, 80 on Fable 5.1 and 20 on Opus 5. Gemini's implicit caching has no write fee, so the ratio does not apply there. Anthropic's pricing page states "A cache hit costs 10% of the standard input price" and immediately gives 2.5% for Fable 5.1 and Mythos 5.1 and 5% for Opus 5.5; the lower read prices are why keeping the cache warm matters more on these models than it did on earlier ones.

Writing still pays off quickly. Anthropic says caching "pays off after one cache read for the 5-minute duration (1.25x write), or after two cache reads for the 1-hour duration (2x write)". That statement is framed at the 0.1x read price; our calculation gives the same one and two reads at 0.05x and 0.025x, and one read for OpenAI's 30-minute write.

Both vendors bill each input token at exactly one of three rates. Anthropic lists the write as its own price; OpenAI says "Cache-write pricing is not an additive fee: input tokens use the uncached-input, cached-input, or cache-write rate."

Twenty turns: uncached, cached and cached with one edit

Over the 20-turn illustrative session on Claude Opus 5.5, input costs $4.54 without caching, $0.70 with append-only caching and $0.97 if the system prompt is edited once at turn 11, at list prices as of 27 September 2026.

Illustrative session, input cost only, Claude Opus 5.5 list prices as of 27 September 2026, all turns within the 5-minute cache lifetime. Calculated, not measured; no API was called.

The two shapes come from what each turn pays for. Without caching, every turn pays full price for the whole history, so the cost of a single turn climbs from $0.074 at turn 2 to $0.398 at turn 20 and the cumulative line curves upward. With caching, each turn pays the cheap read price for the history and the write price only for its 4,500 new tokens, so a turn costs $0.0253 at turn 2 and $0.0415 at turn 20, and the cumulative line is nearly straight. Caching starts behind: turn 1 costs $0.070 cached against $0.056 uncached, because the first write carries the premium.

The system-prompt edit at turn 11 rewrites the whole 59,000-token request once, tools included (the section on edits explains why), a step of $0.26, and from turn 12 the line climbs at the cached slope again.

With reads this cheap, most of a cached bill is the writing of new tokens. On Opus 5.5, 70.6% of the append-only session's cached input spend goes on writes and 29.4% on reads; on Fable 5.1 writes are 82.8%.

The same session on the other models, with the same token counts:

Model	Input, uncached	Input, append-only cached	Saving	Input with one edit at turn 11	Output (either way)
Claude Opus 5.5	$4.5400	$0.7046	84.5%	$0.9662 (+37.1%)	$0.6000
Claude Fable 5.1	$11.3500	$1.5026	86.8%	$2.1703 (+44.4%)	$1.5000
GPT-6 Sol	$2.2700	$0.4559	79.9%	$0.5812 (+27.5%)	$0.3000
Gemini 3.8 Flash	$0.8513	$0.1523	82.1%	$0.1891 (+24.2%)	$0.1125
Claude Opus 5 (reference)	$5.6750	$1.1396	79.9%	$1.4530 (+27.5%)	$0.7500

This table compares billing structures, not vendors. The token counts are identical by construction, but real ones are not: Anthropic says the tokeniser in Claude 4.7 and later models "produces approximately 30% more tokens for the same text", and token counts differ between vendors' tokenisers too, by 5% on English prose and far more on numbers and non-Latin scripts. Each vendor also carries earlier reasoning into the next request differently, which identical counts ignore. The models differ in capability, so the same task would not take the same turns. OpenAI and Google hits are best effort, and the Gemini figures assume every implicit hit lands, which Google does not guarantee.

What the table does show is structural. The edit hurts most on Fable 5.1 (+44.4%) because its reads are the cheapest relative to its own input price, so a rewrite stands out most against a warm turn; on Fable 5.1 the edited turn 11 costs 10.55 times a warm one. Output is untouched by caching and becomes a large share once input is cheap: 11.7% of the uncached bill, against 39.7% to 50.0% of the cached bill across Opus 5.5, Fable 5.1, GPT-6 Sol and Gemini 3.8 Flash. Nor is the saving an artefact of our session shape: varying the fixed prefix from 4,000 to 30,000 tokens and the tokens appended per turn from 1,000 to 10,000 keeps the Opus 5.5 saving between about 83% and 86%, as long as every turn arrives within the cache lifetime.

Anthropic publishes one current reference point for real hit rates, and it is vendor-reported. Its guide to optimising for cost and intelligence says "Over a full day of real traffic, agent loops read a median 84% of their input from the cache, and the top 10% of harnesses, coding or not, read 94% or more", and "Below about 80%, look for something breaking the cache". The figures aggregate first-party API traffic by organisation-day over the 14 days ending 23 August 2026, across Claude models and before Opus 5.5 was released. The carried example reads 91.2% of its input from cache (1,035,500 of 1,135,000 tokens), between that median and the top decile: plausible, not validated. In the terms of our ladder for grading AI claims, Anthropic's hit rates are measured by the vendor on its own traffic, not independently reproduced.

Independent measurements are older. Lumer and colleagues' preprint "Don't Break the Cache" (arXiv 2601.06007 v2, 31 January 2026) ran 40 sessions per model per condition on DeepResearch Bench with a 10,000-token system prompt, at prices as of early January 2026. In each model's best caching mode they report cost savings of 79.6% for GPT-5.2, 78.5% for Claude Sonnet 4.5, 45.9% for GPT-4o and 41.4% for Gemini 2.5 Pro, and time-to-first-token improvements of 6.1% to 30.9%. The authors' own caution applies to our numbers too: "Practitioners should conduct representative experiments for their workloads rather than relying on published benchmarks."

A miss is also slower, because whatever is not read from cache has to be processed before the first output token. We did not measure latency. Anthropic says its 5-minute and 1-hour caches "behave the same with respect to latency".

Which edits restart the cache, and which also void earlier reasoning

On Claude's API the cached prefix runs in a fixed order, tools, then system, then messages, and a change stops the prefix matching at the first changed position. On Claude Opus 5.5 and Fable 5.1 a second, separate check on the same prefix decides whether earlier thinking blocks remain valid.

The first check follows from the position of the edit. Anthropic's table of what invalidates the cache says modifying tool definitions "invalidates the entire cache", because, as its page on mid-conversation system messages puts it, "The tools array sits even earlier in the hashed request prefix than the top-level system field". Editing the system prompt invalidates the system and message levels, and editing, reordering or deleting an earlier message invalidates everything from that message on.

Some settings sit outside that ordered strip but are rendered into it. Anthropic's thinking documentation says "The thinking configuration and the resolved effort level are rendered into the prompt itself", and advises: "Treat any thinking or top-level effort change as starting the cache over." A top-level effort or thinking change always loses the message cache and, depending on the model, the system and tools levels too. Changing tool_choice or disable_parallel_tool_use loses the message level. Switching fast mode on or off loses the system and message levels. Switching model loses everything (in the words of Anthropic's cache diagnostics page, "The cache is per-model"), and so does an idle gap longer than the cache lifetime.

Where the next request picks the cache back up depends, on Anthropic, on where entries were written. The prompt caching page says "Cache writes happen only at your breakpoint" and "The system does not write entries for any earlier position"; its lookback of at most 20 positions "is looking for prior writes, not for stable content." With automatic caching alone, every entry sits at the end of an earlier request and so includes the tools and system prompt. Editing either, or anything that invalidates the whole message level, leaves nothing to read: the whole request is written again, as in the example's $0.2950 turn 11. "Invalidates", above, marks where matching stops; in the diagram below, the bars labelled "lost: rewritten at the write price" mark what is written again, assuming breakpoints after tools and system.

Explicit breakpoints recover part of that, and Anthropic says "Adding more cache_control breakpoints doesn't increase your costs". Its documentation pairs an explicit breakpoint on the system prompt with automatic caching; adding one at the end of tools uses three of the four slots, and each level then survives edits below it. The turn-11 system edit then reads the 8,000 tool tokens and writes 51,000, adding $0.2232 to the Opus 5.5 session instead of $0.2616. A tools edit still loses everything.

The case readers most often hit is trimming an old tool result on the client to save tokens. Say turn 11 cuts the 3,000-token tool result from turn 2 to 500 tokens (an illustrative trim). Uncached, that saves $0.10 over the remaining turns. Cached, it adds $0.2446 to the Opus 5.5 bill with automatic caching alone, or $0.1870 with breakpoints at the end of tools and system. Trimming the previous turn's result instead adds only $0.0046, because the entry written at the end of request 9 should still be live and within the lookback; that is our inference from the documented rules, not a case Anthropic works through. On Opus 5.5 and Fable 5.1 a trim also invalidates every later thinking block; these figures assume those blocks are still sent, as on an older account where that check is not enforced.

The second check is newer. On Claude Opus 5.5 and Fable 5.1, Anthropic's preserved thinking documentation says, "a thinking block stays valid only while everything you sent before it is unchanged on later requests." The checked prefix is the top-level system prompt, the set of tools, and every message before the block. Thinking cannot be disabled on Opus 5.5, so an agent running on it can accumulate a thinking block on every turn. The API enforces this check by default "for accounts created on or after August 31, 2026, 00:00 UTC", returning a 400 invalid_request_error; older accounts are checked only if they set prefix_mismatch_behavior. Anthropic's advice is to make integrations append-only regardless of account age. The alternative setting, drop_block (beta header thinking-binding-controls-2026-08-01 as of 27 September 2026), drops failing blocks instead: "Dropped blocks aren't billed", and "the prompt cache restarts at the edit". Our companion piece on why new model APIs reject old parameters covers the migration steps for that 400 and for Opus 5.5's refusal of forced tool_choice (only auto and none are accepted).

In the carried example, the turn-11 system edit under drop_block drops the 10,000 tokens of thinking from turns 1 to 10. Because those tokens are no longer billed, the edit adds $0.1936 to the Opus 5.5 session instead of $0.2616, with automatic caching alone. That is not a saving. The model has lost that reasoning, and Anthropic warns that "Claude can sometimes think more to re-create the dropped thinking"; the calculation does not model that extra thinking. With the default setting the request fails instead.

Many changes pass the thinking check while still costing the cache. Anthropic lists "Request parameters outside those three fields, such as effort, max_tokens, output_config, tool_choice, and metadata" as outside the check, along with cache_control markers. Removing thinking blocks from the start of the history, from the end or all of them is allowed; in Anthropic's words, "What fails is a gap", so removing one from the middle invalidates the thinking blocks after it. Anthropic's context editing documentation says "server-side context management never invalidates thinking blocks" on these two models, though clearing content "Invalidates cached prompt prefixes".

Two other edits look harmless and are not. Re-rendering context into the first user message with a changed value, such as today's date, a memory block or environment details, invalidates every thinking block and the cache from that message on. So does an image or document URL that returns different bytes on the next request.

Model switches have their own reading rules. Fable 5.1 reads Opus 5.5's thinking blocks on the Claude API; Opus 5.5 does not read Fable or Mythos blocks, which are dropped and "aren't billed and don't count toward input_tokens". Either way the cache starts again: switching the example session from Opus 5.5 to Fable 5.1 at turn 11 costs $0.6676 more than continuing on a warm Fable 5.1 cache would.

Assumes explicit cache breakpoints at the end of tools and system; with automatic caching alone, rows 2 to 4 can rewrite the whole request. Based on Anthropic's prompt caching, preserved thinking and context editing documentation, accessed 27 and 28 September 2026.

Put together, the two checks nest. Anthropic's preserved thinking page says "the edits that invalidate thinking are the edits that restart the cache". We tabulated 25 kinds of edit from Anthropic's documentation: nine cost both the cache and earlier thinking, ten cost the cache only (one of them partly), six cost neither, and we found none that voids thinking while keeping the cache. A few cells are our inference rather than documented; the cache effect of removing the oldest thinking blocks is one. So Anthropic's sentence holds in one direction: every edit that invalidates thinking also restarts the cache at or before that point. It is not an equivalence, and the same page says so in its own row, "Changing top-level output_config.effort (restarts the cache, doesn't affect thinking)".

OpenAI and Google document no equivalent binding as of 27 September 2026. OpenAI's guide lists the settings that reset reuse: model, tools, parallel_tool_calls, text.format, reasoning.effort, text.verbosity and compaction through context_management. Its reasoning guide says "Persisted reasoning can be reused only within the same model family" but describes no prefix check. Google's Interactions API thinking page says thought blocks must be resent "exactly as they were received from the model", and its Generate Content API thinking page says thought signatures "will increase the input tokens you are charged when sent back as part of the request"; neither describes a prefix check.

Changing settings by appending instead of editing

Every mechanism Anthropic and OpenAI now document for changing instructions, tools or reasoning effort in the middle of a session works by adding content to the end of the conversation, so the cached prefix never changes. On Anthropic's API, as of 27 September 2026:

• Instructions: a mid-conversation role: "system" message instead of editing the top-level system field. No beta header is needed. It works on Fable 5.1, Mythos 5.1, Fable 5, Mythos 5, Opus 5.5, Opus 4.8 and Opus 5, but not Sonnet 5.
• Tools: a tool_addition or tool_removal block instead of editing tools, under the beta header inline-tools-2026-09-15 (with mcp-client-2026-09-15 for MCP tools). Anthropic says "The cached prefix still matches, so only the appended message is processed as new input." The exception: unless tools keeps at least one tool not marked defer_loading: true, the first tool defined by value in a tool_addition block costs one full cache miss on that request.
• Effort: an output_config effort change carried in a system message, under the beta header mid-conversation-output-config-2026-07-01, on Fable 5.1, Mythos 5.1, Opus 5.5 and Opus 5 (Fable 5 returns a 400). Anthropic's effort documentation recommends it: "A top-level change restarts the cache and also steers the model less reliably".
• Temporary instructions: turn-scoped system messages, under the beta header mid-conversation-system-clear-at-2026-08-21. Once cleared, a turn-scoped message should be left in place; Anthropic says to "Re-send cleared messages verbatim", and otherwise "The prompt cache misses from that point".

In the example session, a top-level effort change at turn 11 on Opus 5.5 adds between $0.2040 and $0.2616 to the input bill: the lower figure if an explicit breakpoint at the end of system survives the change, the higher with automatic caching alone, where the whole request is rewritten. The per-message change adds nothing beyond the few tokens of the appended message itself, which the calculation does not count. What effort does to the output and the bill is a separate question, covered in what reasoning effort controls.

Anthropic's advice for turning tools off shows the difference between the two checks: "To turn tool use off for a request, send tool_choice: {"type": "none"}. Don't remove tools." That keeps earlier thinking valid but still costs the message-level cache, because tool_choice is rendered into the prompt. Only appending costs neither.

Shrinking a long history is the one case where the prefix has to change. Anthropic's sanctioned route is server-side: context editing (beta header context-management-2025-06-27) and compaction (on demand under compact-2026-09-04; threshold compaction is also in beta). On Opus 5.5 and Fable 5.1 context editing never invalidates earlier thinking, and compaction keeps the thinking of the turns it keeps valid only under conditions Anthropic lists, among them that those turns, system and any tools not marked defer_loading: true are sent unchanged. Both restart the cache at the clear point, and Anthropic warns "You'll incur cache write costs each time content is cleared".

OpenAI's equivalent is narrower. On GPT-6 models, its caching guide says to "append a configuration_update input item to change reasoning effort during a conversation while preserving the earlier cached prefix". The reasoning guide sets the limits: "Configuration updates are supported by the GPT-6 model family in standard, single-agent mode. They change only reasoning effort." The API rejects adjacent updates, the guide says not to combine them with automatic compaction or truncation, and /responses/compact rejects histories that contain them. For the other settings on OpenAI's reset list, no appended replacement is documented.

Finding the cause of a miss

Anthropic and OpenAI both offer cache diagnostics that compare a request with an earlier one and name the reason its prefix did not match.

Anthropic's cache diagnostics became generally available on the Claude API on 23 September 2026, with no beta header; they are not available on Amazon Bedrock, Vertex AI, Microsoft Foundry or Claude Platform on AWS. You pass diagnostics with the previous_message_id of the earlier response. The reasons returned are model_changed, system_changed, tools_changed, messages_changed, previous_message_not_found and unavailable, where unavailable covers differences in tool_choice, thinking, context_management, output_config, output_format or the set of beta headers. The accompanying cache_missed_input_tokens figure is an estimate.

OpenAI's prompt cache diagnostics became generally available in the Responses API for GPT-5.6 and later on 8 September 2026. You pass prompt_cache_options.comparison_response_id. The reasons are model_changed, prompt_cache_key_changed, service_tier_changed, tools_changed, text_format_changed, reasoning_effort_changed, verbosity_changed, context_compacted and input_changed; on these models prompt_cache_key is an optional field "for separate cache accounting".

Mapped onto the symptoms in this piece, and going by the documented reason lists rather than any response we have seen: a system prompt that embeds today's date should surface as system_changed on Anthropic once the date changes, and a memory block re-rendered into the first user message as messages_changed. A top-level effort change should surface as unavailable on Anthropic and reasoning_effort_changed on OpenAI.

When caching costs more than it saves

Prompt caching loses money when turns arrive further apart than the cache lifetime: in the illustrative session, if every turn misses, the cached bill is 1.25 times the uncached one on Anthropic's 5-minute cache and on OpenAI. On Opus 5.5 that is $5.68 against $4.54, because every token is written at the premium and never read. Gemini's implicit caching has no write fee, so the same pattern costs the same as not caching. The same thing happens without any pause if a value near the top changes on every turn, such as a date or memory block re-rendered into the system prompt or the first message: each turn rewrites the whole request.

A single long pause costs one miss, and the longer lifetime does not automatically fix it. A 20-minute pause before turn 11 misses Anthropic's 5-minute cache, adding $0.26 on Opus 5.5, but not OpenAI's 30-minute lifetime. A 45-minute pause misses on GPT-6 Sol too, adding $0.13. Buying Anthropic's 1-hour cache for every write in the session survives the pause but adds $0.30 to the no-pause baseline, more than the $0.26 miss it avoids; the same pause before turn 13 or later, where the rewritten request is larger, would tip the balance the other way. That is close to the rule of thumb in Anthropic's cost guide: use the 1-hour lifetime "once about 1 turn in 20 follows a pause between 5 minutes and an hour and few gaps run over an hour". Our session has 19 gaps, so one pause is 1 in 19.

A keep-alive is a third option, and it is cheapest where reads are cheapest. A cache read refreshes the lifetime, so the same guide says to "send the previous request again with max_tokens set to 0 within 4 minutes of the previous request's start, and every 4 minutes after that, dropping stream if it was set", with the prefix and headers unchanged; such a request "refreshes the cache's lifetime, generates nothing, and bills only the cache read". The API rejects a max_tokens: 0 request that sets thinking.type: "enabled", structured outputs or a forced tool choice, and for those workloads the guide says to "buy the 1-hour duration instead". It adds that "the default adaptive thinking on Claude Fable 5.1 is fine", and Opus 5.5 rejects both thinking.type: "enabled" and a forced tool choice on every request anyway, so of the three only structured outputs can rule out a keep-alive there. A max_tokens: 0 request carrying the top-level compaction parameter is rejected too, so a compaction request should not be re-sent as a keep-alive. Separately, Anthropic's prompt caching page says Message Batches reject max_tokens: 0 outright.

Each keep-alive bills a read of the whole prefix, and a pause needs one for roughly every 4 minutes it lasts, so the cost grows with both the length of the pause and the size of the prefix. By our arithmetic, bridging the example's 20-minute pause before turn 11, counted from request 10's start, takes four keep-alives that each re-read request 10's 54,500 tokens: $0.0436 on Opus 5.5, against the $0.2616 miss and the $0.2985 1-hour premium. A 45-minute pause takes 11, for $0.1199. One pause is cheap to bridge this way. Long pauses before many turns are not, because each adds its own reads of a growing prefix, while the 1-hour premium depends only on the tokens written.

Anthropic measured the trade-off on one job of its own, a 20-issue triage task with three runs per setting. On Opus 5.5, at the default medium effort, keep-alive cost 8% to 13% less than the 1-hour lifetime when 5% or 10% of turns followed a pause of 6 to 32 minutes, but more with a pause before every turn. On Fable 5.1 it cost 13% to 20% less whenever pauses ran for minutes and lost only near 45 minutes; on Opus 5, whose reads cost 0.1x input, it saved nothing measurable. For Opus 5.5 the guide's advice is to "keep the 5-minute cache warm instead when only a turn or two in 20 follow a pause of up to about half an hour", which describes the example session. The Opus 5.5 runs, made on 19 and 20 September 2026 before release, sent keep-alives with max_tokens: 1, and Anthropic says whether the max_tokens: 0 request refreshes an existing entry "was not measured on Opus 5.5".

Besides latency and differences in tokeniser and capability, the calculation leaves out misses from eviction or routing (every warm turn is assumed to hit), batch discounts, Gemini's storage fee for explicit caches and any extra thinking after dropped blocks. It also rests on read prices as of 27 September 2026. If Anthropic moved Opus 5.5 back to a 0.1x read price, its miss-to-hit ratio would halve to 12.5, and a workload of one-off requests with no shared prefix gets nothing from caching at any price.

A rule for changing a running session

Before changing anything in a running session, ask where the change lands. If it lands at the end of the conversation (a new message, a mid-conversation system message, a tool_addition, a per-message effort change, or OpenAI's configuration_update), it costs only its own tokens. If it lands anywhere earlier, or is a top-level setting rendered into the prompt, the next request rewrites everything after the last cache entry that survives it, at the write price: on the 5-minute cache, 25 times the read price on Opus 5.5 and 50 times on Fable 5.1. With automatic caching alone, a tools or system edit, a top-level setting change or a deep history edit leaves no surviving entry, so the whole request is rewritten; explicit breakpoints at the end of tools and system, set before the session starts, keep those levels readable. On Opus 5.5 and Fable 5.1 the change also voids earlier thinking if it touches system, tools or an earlier message. If such a change has to happen, make it once rather than on every turn, and check the next response's diagnostics to confirm the cache came back.

Two gaps in Anthropic's documentation would change the arithmetic: no page gives the input-token count of a resent omitted thinking block, which we assume equals its original thinking tokens, and no example shows a recent history edit resuming from an earlier turn's cache entry, as the documented rules imply and the cheap recent-trim figure assumes. OpenAI and Google do not document whether earlier reasoning is bound to the prefix at all.