How Prompt Caching's Prefix Match Actually Works
Prompt caching lets Claude reuse a previously-processed prefix of a prompt instead of reprocessing it from scratch on every request.
That sounds simple, but the mechanism behind it - an exact-prefix match - has sharp edges that trip up almost every team the first time they wire it up.
Understanding the prefix match model is the difference between reliably getting cache hits worth up to a 90% cost reduction and 85% latency reduction, and silently paying full price while believing caching is working.
Summary
- Core Idea: Claude caches a prompt as an ordered sequence of bytes, and a cache hit only happens when a new request's prefix is byte-for-byte identical to a previously cached prefix, up to a marked breakpoint.
- Why It Matters: Any change earlier in that prefix, even one character, invalidates the cache for everything after it, silently turning a cheap cached request back into a full-price one.
- Key Concepts: prefix, cache breakpoint, cache_control, exact match, cache write vs cache read.
- When to Use: Any workload that resends a large, mostly-stable block of content - a long system prompt, tool definitions, or reference documents - across many requests.
- Limitations / Trade-offs: The cache is not semantic; it does not know two prompts "mean" the same thing, only whether their bytes match.
- Related Topics: cache TTL and pricing, cache breakpoint placement, verifying cache hits via usage fields.
Foundations
A prefix is the leading portion of a request's content, read in the order Claude receives it: tool definitions first, then the system prompt, then the messages array.
A cache breakpoint is a marker you place on a specific block of content using a cache_control field.
Everything from the start of the request up to and including that breakpoint becomes eligible to be stored and reused as a cached prefix.
Think of it like a photocopier that remembers the first N pages of the last document it copied.
If you hand it a new document whose first N pages are identical, it reuses the memorized pages and only processes the new pages after that point.
If even one word on page 3 differs, the photocopier has to start over from page 3, memorizing nothing from the copy it made before.
Here is the smallest example of marking a breakpoint on a system prompt:
import anthropic
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-sonnet-5",
max_tokens=1024,
system=[
{
"type": "text",
"text": "You are a support assistant for Acme Corp. " * 200,
"cache_control": {"type": "ephemeral"},
}
],
messages=[{"role": "user", "content": "What is your return policy?"}],
)The cache_control block on that system text tells Claude: everything up to and including this block is a candidate for caching.
Mechanics & Interactions
The match is exact and positional, not fuzzy and not content-aware.
Claude compares the incoming request's prefix, byte for byte, against what it has cached from a prior request within the cache's TTL window.
If the bytes match all the way up to the breakpoint, that's a cache hit: those tokens are read from cache instead of being reprocessed, which is both cheaper and faster.
If the bytes diverge anywhere before the breakpoint, that's a cache miss for the entire prefix, not just the part that changed - the whole segment up to the breakpoint gets reprocessed and rewritten to the cache.
This is the single most important consequence of the model: caching is all-or-nothing for the prefix, not a partial-credit diff.
Cached prefix (from a prior request):
[tool defs] [system prompt] [breakpoint]
| |
identical identical -> CACHE HIT, everything up to breakpoint read from cache
New request, one field in tool defs changed:
[tool defs*] [system prompt] [breakpoint]
|
differs here -> CACHE MISS for the whole prefix, reprocessed and rewrittenTwo response fields tell you which of these happened: usage.cache_read_input_tokens is populated on a hit, and usage.cache_creation_input_tokens is populated when the request writes a new (or refreshed) entry to the cache.
A single request can show both nonzero, if part of the prefix matched an existing cache entry and a later, larger breakpoint wrote a new one.
Request ordering also matters mechanically: the API builds the effective prefix from tools, then system, then messages, in that fixed order, so a breakpoint on tool definitions covers a shorter, earlier prefix than a breakpoint placed on the system prompt.
Advanced Considerations & Applications
The exact-match requirement means the content before any breakpoint must be deterministic across requests for caching to help at all.
Common culprits that quietly break the match: a timestamp embedded in the system prompt, a request-scoped UUID stitched into instructions, or a JSON object serialized with key order that isn't guaranteed stable.
None of these produce an error - the request still succeeds - but every single one of them silently reprocesses the full prefix as if caching were never configured, because the bytes no longer match.
| Approach | Strength | Weakness | Best Fit |
|---|---|---|---|
| Single breakpoint at end of system prompt | Simple, one line to add | Any change anywhere in system or tools invalidates everything | Stable system prompt with no volatile content |
| Breakpoints on tools and system separately | Isolates tool-schema churn from system-prompt churn | Slightly more bookkeeping in the request builder | Large tool catalogs that change independently of system copy |
| No cache_control at all | Zero risk of stale-content bugs | No cost or latency benefit ever | Prompts that are already short or change every call |
Because the match is positional, moving a stable block of content after a volatile one (instead of before it) is often the fix: put reference documents, tool schemas, and instructions first, and put anything that changes per request - the actual user question, a session ID, "current time" - after the breakpoint, in the messages array where it belongs.
That reordering alone resolves the majority of real-world cache-invalidation bugs without removing any content from the prompt.
Common Misconceptions
- "Prompt caching understands what my prompt means." It doesn't - the match is purely byte-level. Two prompts that are semantically identical but differ in whitespace, key order, or a single character are treated as completely different prefixes.
- "A cache miss means something is broken." A miss just means the bytes didn't match this time - it could be the first request ever, an expired TTL, or genuinely different content. It's expected behavior, not an error state.
- "Only my system prompt affects the cache." Tool definitions come before the system prompt in the effective prefix; a tool schema that changes between requests invalidates the cache just as much as editing the system prompt would.
- "If part of my prompt changed, I still get credit for the part that didn't." You get credit up to the breakpoint immediately before the change, but nothing changes the fact that everything from the divergence point through that breakpoint is reprocessed as new.
FAQs
What exactly counts as "the prefix" in a Claude API request?
The content up to and including a marked cache_control breakpoint, built in a fixed order: tool definitions first, then the system prompt, then the messages array.
Does prompt caching use any kind of semantic or fuzzy matching?
No. It's an exact byte-for-byte match against a previously cached prefix. There is no similarity scoring or normalization step.
If one word changes in my system prompt, do I lose the whole cache?
Yes, for everything from that word through the next breakpoint. The prefix up to the change point is reprocessed and rewritten to cache; nothing after the change point can hit the old entry.
Do tool definitions affect prompt caching?
Yes. Tools are part of the effective prefix and come before the system prompt, so a changing tool schema invalidates the cache even if your system prompt text never changes.
How do I know whether a request was a cache hit or a cache miss?
Check the response's usage object. cache_read_input_tokens is nonzero on a hit; cache_creation_input_tokens is nonzero when the request writes a new cache entry.
Can a single request be both a cache hit and a cache write?
Yes. If part of the prefix matched an existing entry and a later breakpoint covers content that wasn't cached yet, you can see both cache_read_input_tokens and cache_creation_input_tokens populated on the same response.
What are common things that silently break the prefix match?
- A timestamp or "current date" string embedded in the system prompt
- A per-request UUID or session ID placed before the breakpoint
- JSON content serialized with non-deterministic key order
- Any dynamic user-specific text placed ahead of the cached block instead of after it
Does a cache miss produce an error?
No. The request still succeeds normally - it's just processed and billed as if caching weren't in play for that prefix. That's what makes silent invalidation dangerous: nothing tells you it happened unless you check the usage fields.
Where should volatile, per-request content go relative to the breakpoint?
After the breakpoint, typically in the messages array - never before it. Anything that changes every call (current time, a request ID, the live user question) belongs downstream of the stable, cached prefix.
Is the cache shared across different API keys or organizations?
No, caching is scoped to your own requests; it isn't a shared global cache across customers. Treat it as a per-account optimization, not a public resource.
Does reordering my prompt content ever fix a caching problem?
Often, yes. If a stable block sits after a volatile one, moving the stable block earlier (before the breakpoint) and the volatile block later (after it) is usually enough to restore consistent cache hits.
Related
- Prompt Caching Basics - add your first cache_control breakpoint and confirm a hit.
- Placing cache_control Breakpoints on System Prompts and Tools - get the tools/system/messages order right.
- Debugging Silent Cache Invalidators in Long Prompts - find the timestamps and UUIDs breaking your prefix.
- Verifying Cache Hits with cache_read_input_tokens - confirm hits on every response.
- How Claude Token Pricing Actually Works - see where cache reads and writes fit into billing.
Stack versions: Written against the Claude model lineup current as of ~June 2026 - Claude Fable 5, Claude Opus 4.8, Claude Sonnet 5 (the default), and Claude Haiku 4.5 - and the official
anthropicPython SDK (latest 0.x release). Model names, SDK versions, and pricing move quickly - verify current specifics at platform.claude.com/docs before relying on them.