Independent buyer side advisory · Anthropic onlyNew York · London
Batch Processing

The latency cost of batch and how to plan it.

Batch runs Claude work at roughly half the real time rate, and the only thing you give up is immediacy. The discount is easy to want. The discipline is planning around the longer return window so the saving never turns into a missed deadline. Here is how to model batch latency, where it bites, and how to design pipelines that capture the fifty percent without ever leaving a user or a report waiting.

Buyer side guide · 12 min read
34%
Average reduction in Claude spend
$40M+
Anthropic commitments advised
100%
Anthropic focus, no other vendor

Batch processing is the simplest discount in the entire Claude token budget. You submit a set of requests as a job, you accept that the results return inside a longer window rather than the instant you ask, and in exchange the rate falls to roughly half the real time price. There is no quality tradeoff, no prompt rewrite, and no model downgrade. The output is identical. The only variable that changes is time. That is exactly why the discount is so easy to leave unclaimed and so easy to misuse: teams either never move work to batch because they fear the wait, or they move work to batch without modeling the wait and then discover that a job they assumed would finish in twenty minutes landed hours later, after the report it fed had already gone out blank. Both failures come from the same root cause, which is treating batch latency as a single fixed number rather than the planning variable it actually is.

The point of this guide is to make batch latency legible. Once you can predict the window, you can schedule around it, and once you can schedule around it the fifty percent saving becomes free money on every workload that no person is actively waiting on. The buyer side question is never whether batch is cheaper. It always is. The question is whether your pipeline is designed so the longer return window costs you nothing, and that is a planning problem you can solve in advance rather than a risk you have to accept.

What the batch window actually is

Real time requests return in seconds because you are paying for priority on live capacity. Batch requests return inside a service window measured in minutes to hours because they run on spare capacity whenever it is available, and that flexibility is what funds the discount. The window is an upper bound, not a fixed delay. Many batch jobs finish well inside it, especially smaller ones submitted at quieter times, but you cannot plan a deadline against the lucky case. You plan against the bound. If the committed window is up to a day, then every pipeline that depends on a batch result must assume the result could legitimately arrive near the end of that window and still be on time. Design for the worst case the service promises, not the average you usually see, and batch never surprises you.

This matters because batch latency is not random, it is bounded and therefore plannable. A delivery that arrives anywhere inside the promised window is the service working correctly, not a failure. Teams that treat a result arriving in the back half of the window as a problem have simply set their internal expectations against the average rather than the bound, and that mismatch, not the batch system, is what breaks their schedule.

Where the latency actually bites

The cost of batch latency is never the latency itself. It is the dependency that sits downstream of the batch result and assumes the result is already there. A nightly enrichment job that feeds a morning dashboard is a perfect batch candidate, because the gap between when the job could finish and when anyone reads the dashboard is many hours of pure slack. The same enrichment feeding a report that goes out fifteen minutes after the job is triggered is a trap, because there is no slack to absorb a window that runs long. The workload is identical. What differs is the size of the buffer between completion and consumption, and that buffer is the only thing that determines whether batch latency costs you anything at all.

So the planning exercise is not workload by workload, it is dependency by dependency. For every batch candidate, find the moment its output is actually consumed and measure the gap between the latest possible batch completion and that consumption moment. If the gap comfortably exceeds the committed window, the work belongs in batch and the latency costs nothing. If the gap is tight or negative, either the consumption moment has to move later or the work has to stay real time. Most teams find that the majority of their asynchronous work has enormous downstream slack they were never using, which is precisely why the saving is so large and so safe once it is planned.

Modeling the window into your schedule

The practical way to plan is to work backwards from the deadline rather than forwards from the trigger. Start with the moment the result must be ready, subtract the committed batch window, and the answer is the latest moment you can submit the job. Submit before that line and the deadline is safe even if the window runs to its bound. This single piece of arithmetic removes almost all batch risk, because it converts a vague worry about lateness into a hard submission time you can schedule a job against. A nightly job whose output is needed by eight in the morning, against a window of up to several hours, simply has to be submitted before the early hours, and a scheduler enforces that without anyone thinking about it again.

  • Identify the deadline: the exact moment the batch output must be ready for its consumer.
  • Subtract the committed window: use the service upper bound, never the average you usually see.
  • The result is your latest safe submission time: schedule the job to start before it.
  • Add a margin: leave a buffer so a window that runs long still clears the deadline.
  • Monitor completions against the window so you learn your real distribution over time.

Once these submission times are wired into a scheduler, batch stops being a judgement call made under pressure and becomes a property of the pipeline. The job runs early, the result lands inside the window, the deadline is met, and the bill is half what it would have been on the real time path. That is the entire mechanic, and it is durable because it does not depend on anyone remembering to be careful.

Designing pipelines that absorb the window

The most robust pattern is to put deliberate slack between batch completion and consumption by decoupling the two. Rather than triggering a report the instant a batch job is requested, write batch outputs to a store and have the consumer read from that store on its own schedule. The store absorbs the window. If the job finishes early the result waits in the store. If it finishes late but inside the window the consumer still finds it ready at read time. This decoupling is what lets a pipeline run almost entirely on batch rates while never exposing a user or a downstream system to the wait, and it is the single most valuable design move for a team trying to maximize the batch share of its spend.

The opposite pattern, chaining a real time consumer directly onto a batch producer with no buffer, is what gives batch a bad reputation inside engineering teams that tried it once and got burned. The fix is never to abandon batch, it is to insert the buffer. A queue, a store, a scheduled read, any of these converts a fragile direct dependency into a resilient decoupled one, and the saving comes back without the risk.

How batch latency interacts with the other levers

Batch is one of three compounding levers, alongside prompt caching at up to ninety percent on repeated input and model routing across Opus, Sonnet, and Haiku. Latency planning is what lets batch stack cleanly with the other two. A bulk job that reuses a large shared prefix captures the caching discount on the input and the batch discount on the rate at the same time, and routing that same job to Sonnet or Haiku instead of Opus adds the routing saving on top. The three levers multiply rather than add, and the reason teams that deploy all three together see aggregate reductions in the forty to seventy percent range is precisely this stacking. Latency planning is the enabler, because it is what makes you comfortable moving the largest possible share of volume onto the batch path where the other two levers can also apply.

Why this belongs in the contract conversation

Latency planning is not only an engineering exercise, it changes the number you should commit to Anthropic. Your committed spend should reflect the real optimized cost of your workloads, and a workload that runs most of its volume on the batch path at half the rate costs materially less than the same volume run entirely real time. A buyer who commits before planning the batch window locks the real time premium into the commitment for the length of the term and pays it whether or not the work ever needed to be real time. A buyer who plans the window first commits to a leaner, truer number and negotiates from demonstrated efficiency rather than habit. The sequence is the whole game: model the latency, move the work, then size the commit, never the reverse.

We sit on the buyer side of this entirely. We help teams map every workload against the waiting test, model the batch window into their schedules, design the buffers that absorb it, and then carry that optimized cost base into the commitment conversation with Anthropic so the saving is captured in the contract rather than stranded after it. If you want help planning the window before you plan the commit, book a strategy call and we will walk your specific pipelines with you.

Reading the window distribution over time

The committed window is your planning bound, but the distribution of actual completion times is your operating intelligence, and the two should both inform how you run batch. Once you have been running batch jobs for a while, you can see where your completions actually land inside the window: clustered early, spread evenly, or pushing toward the bound. That distribution is shaped by job size, submission time, and overall demand on the service, and learning it lets you tighten your scheduling without taking risk. If your jobs reliably finish in the first third of the window, you can place deadlines closer to submission with confidence, while still keeping the bound as the line you never cross. The discipline is to plan against the bound and operate against the distribution, using the bound for safety and the distribution for efficiency.

Crucially, you should keep watching the distribution rather than measuring it once and assuming it holds. The mix of work you submit changes, the times you submit change, and the service load changes, so a distribution measured six months ago may not describe today. Build the monitoring into the pipeline so completion times against the window are a standing metric, not a one off study, and you will catch any drift toward the bound before it threatens a deadline rather than after a job lands late.

The cost of getting latency planning wrong

It is worth being concrete about what poor latency planning actually costs, because the failure modes are asymmetric. The cost of being too conservative is opportunity: you leave asynchronous work on the real time path because you were unsure the window would clear, and you pay the full premium on volume that could have run at half price. That cost is real but quiet, a saving never captured rather than a loss incurred, and it compounds invisibly over the term. The cost of being too aggressive is sharp: a job that runs to the bound when you planned against the average lands after its deadline, and a report goes out incomplete or a downstream process runs on stale data. The first failure wastes money, the second breaks a commitment, and good planning avoids both by sizing the buffer to the bound rather than the hope.

The reason latency planning is a buyer side concern and not only an engineering one is that both failure modes show up in the contract. The conservative failure inflates the commitment because the unoptimized real time volume gets baked into the committed number. The aggressive failure undermines confidence in batch and pushes teams back toward real time, which inflates the commitment the same way. Sound latency planning is what lets you move the maximum safe share of volume to batch, which is what lets you commit to the leanest honest number. The engineering discipline and the commercial outcome are the same thing viewed from two ends.

Plan the batch window properly.

Book a strategy call and we will map which of your Claude workloads can move to batch and how to schedule them so latency never costs you a deadline.

Book a Strategy Call

The Counteroffer

Weekly intelligence on Anthropic pricing moves and the buyer side counters that work.

Get a Quote · Book a Strategy Call · The Counteroffer · New York · London Not affiliated with Anthropic PBC. Independent buyer side advisory only.