Q.What is prompt caching?

The per-token prices on model launch pages are the headline. For anyone running agents, the cache hit rate is the real price, and it is the part providers do not put in the press release.

Key numbers

How it works

A language model reads your prompt before it writes anything, and that reading is real compute. When request two starts with the same 200,000 tokens as request one, the work is identical, so the provider keeps it.

The rules are strict and simple:

1. Prefix only. The cache covers everything from the start of the request up to a marked point. Anthropic builds it in a fixed order: tools, then system, then messages.
2. Exact match. One changed character before the cache point (a timestamp in the system prompt is the classic) and the prefix no longer matches. You pay for a new write and get no read.
3. Short life. Anthropic's cache lives 5 minutes by default and the clock resets on every hit; a 1-hour cache costs more to write. OpenAI's default on GPT-5.6 and later is 30 minutes, with a 12-hour option.
4. A minimum size. Prefixes under the model's minimum (512 tokens on Opus 5.5, 1,024 on OpenAI's GPT-5.6 and later) are not cached at all, and no error is returned.

On Anthropic you turn it on with one field. Automatic caching moves the cache point forward as the conversation grows:

{
  "model": "claude-opus-5-5",
  "max_tokens": 1024,
  "cache_control": { "type": "ephemeral" },
  "system": "...your long, stable instructions...",
  "messages": [{ "role": "user", "content": "..." }]
}

Add "ttl": "1h" inside cache_control for the 1-hour cache. On OpenAI caching is on by default for supported models. Either way, check the usage block of the response: Anthropic reports cache_creation_input_tokens and cache_read_input_tokens, OpenAI reports cached_tokens.

What it did to our bill

We run a lot of agents (the setup is in how we run 20 coding agents on one PC). Every Claude Code session writes a log with the usage block of every API call, so we added them up: 1,427 log files, 68,935 unique calls, September 24 to October 25, 2026.

Of 16.6 billion input tokens, 16.3 billion were cache reads and only 149,222 were sent uncached. Tally of the usage fields in our session logs, Oct 25 2026.

Priced at Anthropic's list rates on October 25, 2026 (our estimate; it ignores any plan or discount and assumes today's prices for the whole month):

• Without caching, all 16.6 billion input tokens at base price plus output: about $66,800.
• With caching, the same calls: about $5,420. Almost all of it is Opus 5.5: $3,199 for 16.0 billion cache reads, $1,489 for 278 million cache writes and $554 for output.

The surprise is the writes. They were 1.7% of Opus input tokens and 32% of its input bill. When a cache expires or a prefix changes, the full context gets written again at 1.25x, and on a 240,000-token context that is the expensive moment.

Our hit rate never dropped below 96% after the first day. Every new session starts with a full write, so a day of many short sessions sits lower than a day of long ones. Our session logs.

When it pays off

The math is simple enough to do in your head. On Opus 5.5, writing a prefix costs 1.25x and each read costs 0.05x, so:

• Used twice: 1.25 + 0.05 = 1.30x, against 2x uncached.
• Used ten times: 1.25 + 9 x 0.05 = 1.70x, against 10x. That is 83% off.
• Used once: 1.25x, against 1x. A cache nobody reads is a 25% surcharge.

So caching wins whenever a prefix gets reused within its lifetime: agents, chat with long system prompts, the same document asked many questions. Our LLM API cost calculator and token counter help size the prefix first.

Common misconceptions

Related

• Stop paying twice for the same LLM answer: the semantic cache, the layer above this one.
• Qwen3 vs Fable 5 on price: the open-model side of the same bill.
• Claude Code vs Codex outages.
• LLM API cost calculator.

FAQ

What is prompt caching in simple terms?

Prompt caching lets an AI provider keep the processed start of your prompt (system prompt, tools, documents, earlier conversation) for a few minutes. When the next request starts with exactly the same text, the provider reuses that work and bills those tokens at a fraction of the normal input price: 0.05x to 0.1x on Anthropic, 0.1x on most current OpenAI models.

How much does prompt caching save?

It depends on how much of each request repeats. Coding agents resend their whole context on every step, so the share is huge: across 68,935 of our Claude Code API calls from Sep 24 to Oct 25 2026, 98.1% of input tokens were cache reads, and the list-price bill came to about $5,420 instead of $66,800 (our estimate at Anthropic's Oct 25 2026 prices).

Is prompt caching the same as KV caching?

It is built on the same idea. A KV cache holds the attention keys and values a model computed for earlier tokens inside one request. Prompt caching keeps that computed prefix across requests, so a later call with the same prefix can skip recomputing it, and the provider passes part of the saving to you as a discount.

Does prompt caching change the answer?

No. The provider reuses the work for the exact same prefix, so the model sees the same input. If any byte before the cache point changes (a timestamp in the system prompt, reordered tools), the prefix no longer matches and you pay for a fresh write instead.

How long does a prompt cache last?

On Anthropic the default is 5 minutes, refreshed for free every time the cache is read, with a 1-hour option that costs 2x base input to write. On OpenAI's GPT-5.6 and later the default minimum lifetime is 30 minutes, with a 12-hour option at 2.5x.

Sources

All checked October 25, 2026.

Provider docs

• Anthropic, Pricing: Claude Opus 5.5 at $4 input, $5 5-minute cache write, $8 1-hour cache write, $0.20 cache hit, $20 output per million tokens; Sonnet 5.5, Fable 5.1, Opus 4.8 and Haiku 5.5 rates used for the small share of our calls on those models; cache hits at 0.05x base input on Opus 5.5 and Sonnet 5.5.
• Anthropic, Prompt caching: automatic caching and explicit breakpoints, up to 4 breakpoints, tools then system then messages, 5-minute default lifetime refreshed on each read, 1-hour option at 2x, 512-token minimum on Opus 5.5, the timestamp example.
• OpenAI, Prompt caching guide: on by default, 1,024-token minimum for GPT-5.6 and later, 30-minute default at a 1.25x write rate, 12-hour option at 2.5x, reads at 0.1x on most of these models and 0.05x on GPT-6.1 Sol, "discounted up to 95%", the cached_tokens usage field.

Our data

• Tensorust Claude Code session logs on our studio PC: the usage block (input_tokens, cache_creation_input_tokens with its 5-minute / 1-hour split, cache_read_input_tokens, output_tokens) of 68,935 unique API calls in 1,427 log files, September 24 to October 25, 2026, deduplicated by message id. Only token counts were read; no prompt or response text.
• Cost estimate: each model's tokens priced at the list rates above, with cache writes split 88.3% 5-minute / 11.7% 1-hour as in the logs overall. It assumes October 25, 2026 prices for the whole period and ignores plans, credits and discounts; it is our estimate, not an invoice.

Images

• All charts by Tensorust from the data above.