Batch processing for evaluation runs.

Evaluation runs are one of the largest hidden line items in a mature Claude deployment, and they are also the easiest place to cut cost in half without changing a single result. If your team runs regression suites against prompts, grades model outputs at scale, or sweeps across configurations to pick a model, you are almost certainly paying the full synchronous rate for work that has no business being synchronous. Batch processing exists for exactly this pattern, and most buyers leave the saving on the table because nobody told them the eval pipeline was the cheapest thing to move.

This guide explains why evaluation workloads fit batch so cleanly, what the discount actually is, where the operational gotchas hide, and how to fold batch into your eval harness so the savings show up on the next invoice rather than the next fiscal year.

Why evals are the textbook batch workload

Synchronous API calls exist to serve a user who is waiting. A person typing into a chat window, an agent mid task, a customer expecting a response in a second or two: those workloads need low latency and they pay for it. Evaluation runs are the opposite. Nobody is waiting on an eval. You kick off a few thousand graded completions, you go to lunch, and you read the scoreboard when it lands. The latency that synchronous pricing charges a premium for is worth nothing to you here, which means you are paying for speed you actively do not need.

Anthropic batch pricing reflects that. Work you submit as a batch runs asynchronously, returns within a generous window rather than in real time, and costs roughly half of the synchronous rate. For a workload where latency is irrelevant, a fifty percent cut for accepting a delay you would happily accept anyway is close to free money. Evaluation runs are the cleanest example of this because they are bulk, repeatable, and deadline tolerant by nature.

What the batch discount is worth on a real eval suite

Consider a team that runs a nightly regression suite of five thousand prompts, plus a grading pass where a stronger model scores each output. That is ten thousand completions a night, every night, before anyone touches a feature. Run synchronously, that suite carries the full token rate on both the generation and the grading. Moved to batch, the same ten thousand completions cost about half. Over a quarter of nightly runs, the difference is not a rounding error. It is a meaningful slice of the API line on your invoice, recovered for accepting results in the morning instead of the moment.

The saving compounds when you add the work that eval teams do beyond the nightly suite: model selection sweeps, prompt ablations, red team batteries, and the large one off graded runs that accompany every launch. None of these is latency sensitive. All of them are pure batch candidates. A team that routes every non interactive eval through batch typically finds that the eval line, often a surprisingly large share of total spend, drops by close to half on its own.

The model routing layer underneath the batch saving

Batch pricing is one lever. Model routing is the other, and on evaluation workloads the two stack. Many teams default to running evals on the strongest model because it feels safer, then pay top rates across the whole suite. In practice the generation step and the grading step often have different needs. A great deal of regression generation can run on Sonnet or even Haiku where the task is well defined, with the stronger Opus model reserved for the grading pass or the genuinely hard cases.

Routing the suite across Opus, Sonnet, and Haiku by difficulty, rather than running everything uniformly on Opus, typically cuts aggregate spend by forty to seventy percent before batch is even applied. Layer batch on top and the eval line becomes a fraction of what a uniform synchronous Opus suite would cost, with no loss in the signal you actually use to make decisions. The discipline is to ask, for every step in the eval, what is the cheapest model that still gives a trustworthy result, and to run that step in batch.

Operational details that trip teams up

The most common mistake is treating batch as a drop in replacement and being surprised by the timing. Batch returns within a window, not instantly, so an eval harness that blocks on a synchronous response needs a small redesign to submit a batch, poll for completion, and collect results. This is a modest engineering change, usually a day of work, and it pays for itself in the first month. Build the harness to submit overnight and read in the morning and the latency window stops being a constraint at all.

The second trap is mixing latency sensitive and latency tolerant work in the same path. Keep the interactive evals that gate a deploy, the ones a human is waiting on, on the synchronous path, and push everything bulk and scheduled to batch. A clean split by latency need is what lets you capture the batch discount without slowing down the one or two checks that genuinely need to be fast.

The third is forgetting that batch and caching combine. If your eval prompts share a large common prefix, a fixed system prompt, a rubric, a long set of instructions, prompt caching can cut the input cost of that shared context by up to ninety percent, and that saving applies inside batch as well. An eval suite with a heavy shared prefix is a candidate for both levers at once.

Building the batch eval harness in practice

The shift from synchronous to batch is small in code and large in savings, and it is worth describing concretely so the engineering case is obvious. A synchronous eval harness sends a request and blocks until the answer comes back, one after another, which is simple to write and expensive to run. A batch harness instead assembles the whole set of eval prompts into a single submission, hands it off, and stores a handle. It then polls for completion on a schedule, and when the batch finishes it collects every result at once and writes them to the scoreboard. The logic is no harder. It is simply reordered so the waiting happens once, in the background, rather than thousands of times in the foreground.

The practical pattern most teams settle on is to submit the nightly suite at the end of the working day and read the results the next morning. The batch window comfortably covers an overnight run, so the latency you are accepting costs you nothing in real terms: the engineers who care about the scoreboard were asleep while it ran. For larger one off runs, a model selection sweep or a launch validation battery, the same pattern applies on a longer clock. Submit, let it run, collect. Once the harness is built this way, every future eval inherits the discount with no further effort.

It helps to keep a single switch in the harness that decides, per run, whether work goes to the batch path or the synchronous path. The handful of evals that genuinely gate a deploy, where a human is waiting on a green light, flip to synchronous. Everything else defaults to batch. That one switch is what lets a team capture the discount across the bulk of its eval volume while never slowing down the one or two checks that have to be fast.

Keeping results clean when you move to batch

A reasonable worry is whether moving evals to batch changes the results. It does not. The same model produces the same quality of output whether the request is processed synchronously or in a batch window. Batch is a scheduling and billing decision, not a different model or a degraded one, so your scores, your regressions, and your comparisons remain directly comparable to your synchronous history. The only thing that changes is when the answer arrives and what it costs.

Where teams do get into trouble is consistency of configuration across runs, and that is true regardless of batch. Pin the model version, the prompt, the rubric, and the grading logic so that a change in score reflects a real change in behavior rather than a drift in setup. If anything, moving to a disciplined batch harness encourages this, because you are assembling the whole run as one well defined unit rather than firing off ad hoc synchronous calls. A clean batch run is often a more reproducible run.

One more clean result habit pays off here: separate the generation step from the grading step in your records, and track the cost of each. Because both steps can move to batch and both can be routed to an appropriate model, keeping them distinct lets you see exactly where the eval budget goes and tune each independently. It also makes it obvious when a grading pass is over specified on the top model and could drop to a cheaper one without losing trust in the scores.

Questions buyers ask about batch evals

Does batch change which model I can use? No. Batch is a processing mode that applies across the models, so you can route a batch eval to Opus, Sonnet, or Haiku exactly as you would synchronously, and the half rate applies on top of whatever model you choose. The right move is usually to route the bulk of the eval to a cheaper model and reserve the strongest model for the grading or the hard cases, then run the whole thing in batch.

What about the cases where I need a result fast? Keep them synchronous. The point of a clean split is that the one or two evals that gate a deploy, where a person is waiting, stay on the fast path, while the thousands of latency tolerant evals move to batch. You do not have to choose between speed and savings across the board. You choose per run, and the default for bulk work is batch.

How quickly does the saving show up? On the next invoice. Unlike structural changes such as commitment resizing, which land at renewal, moving evals to batch changes the rate you pay immediately on the work you move. That is what makes it one of the first levers to pull: it is low risk, low effort, and the payoff is fast. For a team running large nightly suites, the eval line can drop close to half within a single billing cycle, with no change to the results and no renegotiation required.

How this fits the bigger optimization picture

Moving evals to batch is one of the fastest wins in a Claude deployment because it is low risk, low effort, and touches a workload that is almost always larger than the team realizes. But it is one move in a wider program. The same buyer side discipline that says run evals in batch also says size your committed spend to real consumption, route every workload to the cheapest model that holds quality, cache aggressive shared context, and negotiate the overage rate and the unused commitment treatment before you sign. Each lever is worth something on its own. Together they are the difference between an invoice that climbs every quarter and one that stays flat while usage grows.

If you want the full sequence, our token optimization playbook lays out every lever in order, from the quick wins like batch and caching to the structural ones like commitment sizing and model routing, with the numbers behind each. It is the same method we use when we sit on the buyer side of a Claude negotiation and run the optimization underneath it.