Independent buyer side advisory · Anthropic onlyNew York · London
Prompt Caching

What workloads benefit most from caching.

Prompt caching cuts the cost of repeated context by up to 90 percent, but only on the work that actually repeats. Here is which Claude workloads gain the most, how to spot them in your own traffic, and why some of your biggest line items are caching candidates hiding in plain sight.

Buyer side analysis · 9 min read
34%
Average reduction in Claude spend
$40M+
Anthropic commitments advised
100%
Anthropic focus, no other vendor

Prompt caching is one of the largest cost levers Claude offers, taking up to 90 percent off the repeated portion of your input. But unlike a rate discount, caching does not apply evenly to everything. It rewards a specific pattern: sending the same large block of context over and over. Some workloads are built almost entirely on that pattern and gain enormously from caching, while others have nothing stable to cache and gain nothing. Knowing which is which is the difference between a saving that transforms your bill and an optimization that does nothing. This is how to recognize the workloads where caching pays, and why several of the most expensive things enterprises run are exactly those workloads.

The pattern caching rewards

Caching works by storing a portion of your prompt so that on subsequent calls you pay a steeply reduced rate for that portion instead of the full price every time. The economics are simple: you pay a little more to write something into the cache once, and then far less to read it on every call that reuses it. The lever only pays off when the same large context is reused enough times to earn back the write and then save on the reads. So the question for any workload is not whether caching exists, it is whether the workload sends a large, stable block of context repeatedly. Where it does, the saving is dramatic. Where every call is unique, there is nothing to cache.

The workloads that gain the most

Once you know the pattern, the high value candidates become easy to recognize. These are the workloads where a large, stable context rides along on call after call.

Long system prompts and instructions

Many applications prepend a large, fixed system prompt to every call: detailed instructions, formatting rules, examples, tone guidance, policy. That block is identical on every request and is often a substantial share of the input. Caching it means you pay full price for it essentially once and the reduced rate on every call after, which on a high volume endpoint is a large and continuous saving.

Document question answering

When users ask multiple questions against the same document, contract, report, manual, or knowledge base, the document is the bulk of the input and it does not change between questions. Caching the document and varying only the question turns a workload that repays the full document on every question into one that pays it once and answers cheaply thereafter. This is one of the clearest and most valuable caching patterns there is.

Retrieval and knowledge grounded applications

Applications that ground answers in a stable corpus, a product catalog, a documentation set, a policy library, reuse large reference blocks across many queries. To the extent the grounded context is stable across requests, it is a strong caching candidate, and these applications often move enormous volumes of repeated reference text.

Conversational agents with fixed context

Chat assistants and agents that carry a large, stable preamble, the system instructions, the tool definitions, the persona, into every turn are reusing that preamble continuously. Caching the stable part of the conversation context, while letting the variable turn content pass at full price, captures a saving on the largest fixed component of every exchange.

Few shot prompting

Workloads that include a set of examples in the prompt to steer the model carry those examples on every call. Examples are stable by definition, so they are natural to cache, and few shot prompts can be long enough that caching them matters a great deal.

How to spot them in your own traffic

You do not have to guess. The way to find your caching opportunities is to look at your prompts and measure how much of each one is stable across calls. Instrument your traffic and for each workload ask two questions: how large is the fixed portion of the prompt, and how many times is that fixed portion reused. The workloads to prioritize are the ones where both numbers are high, a large stable block reused many times. A short prompt reused often is a small prize, and a long prompt used only once is no prize at all. The big wins sit where a heavy, stable context meets high call volume, and a simple audit of your prompts surfaces them quickly.

  • Measure the size of the stable, repeated portion of each prompt.
  • Measure how many times that stable portion is reused across calls.
  • Rank workloads by the product of the two, because that is where the saving lives.
  • Start with the high volume endpoints carrying a large fixed system prompt or document.

Structure prompts so the stable part is cacheable

A workload can be a great caching candidate and still capture nothing if the prompt is built so the stable and variable parts are tangled together. To get the saving, structure the prompt so the stable block, the system instructions, the document, the examples, comes first and unchanged, and the variable content, the user question, comes after. When the stable portion is consistent and positioned to be cached, the reduced rate applies cleanly to it. When the stable and variable content are interleaved, the cache cannot do its job. Often the work of capturing the caching saving is simply reorganizing the prompt so the reusable part is cleanly separable.

How caching stacks with the other levers

Caching is powerful alone and more powerful combined. Route a workload to the cheapest capable model and then cache its stable context, and both savings apply to the same work. Run an asynchronous caching workload on the batch path and the batch discount applies to the variable remainder on top of the cached stable portion. The levers compound, and a workload that is routed, cached, and batched can cost a small fraction of the same work sent naively at full price on the top model. Caching is the lever that specifically attacks the repeated context, which is why it pairs so well with the others that attack the model and the latency.

The commercial angle

Caching does not only lower your monthly bill, it lowers the commit you negotiate. When the repeated context across your heaviest workloads drops by up to 90 percent, your aggregate consumption falls, and the committed spend you sign with Anthropic should fall with it. A buyer who has cached the stable context out of their largest workloads is committing to a smaller, optimized number, which reduces exposure to unused commitment and strengthens the position on the rate. We size the deal against the cached workload, not the uncached one, because that is the honest baseline and the cheaper one.

Estimate the saving before you build

You do not have to implement caching to know whether it will pay. The saving on a workload is a simple function of three numbers you can measure today: the size of the stable block you would cache, the number of times that block is reused while a cache entry stays live, and the call volume of the workload. Multiply the size of the stable block by how often it is reused, and you have a rough picture of the tokens caching would move from full price to the reduced rate. Do this for each candidate workload and you get a ranked list of where caching is worth the engineering and where it is not. The workloads at the top, a large stable block reused heavily on a high volume endpoint, are where you start, because that is where the saving is largest and the payback fastest. The exercise takes an afternoon of measurement and saves you from building caching into workloads that will never repay it.

The workloads where the saving is largest in practice

Across the enterprises we work with, a few patterns reliably produce the biggest caching wins, and they are worth naming because teams often overlook them. A customer support assistant that loads the same long policy and product context into every conversation is usually a top candidate, because the context is large, stable, and hit on enormous volume. A document analysis product where users ask many questions against the same uploaded file is another, because the file dominates the input and never changes between questions. A coding assistant that includes the same large instruction set and style guide on every request reuses that block continuously. And any retrieval grounded application with a stable reference corpus reuses large reference blocks across queries. What these share is the same shape: a heavy fixed context meeting high call volume, which is precisely the shape caching is built to reward. If your most expensive workload looks like one of these, caching is likely the single largest lever you have.

Your Anthropic number is negotiable.

Get a quote for a bounded engagement. Fixed fee or gainshare, no risk to you.

Get a Quote

The Counteroffer

Weekly intelligence on Anthropic pricing moves and the buyer side counters that work.

Get a Quote · Book a Strategy Call · The Counteroffer · Blog · New York · London Not affiliated with Anthropic PBC. Independent buyer side advisory only.