Moving work from Opus to Sonnet or Haiku only saves money if quality holds. Here is the disciplined way to test cheaper Claude models against your real workload, prove the savings, and ship the change with confidence rather than hope.
The objection to model routing is always the same, and it is reasonable: we cannot risk quality. Engineering leaders have shipped products that work, and the prospect of swapping a proven model for a cheaper one feels like trading a known good outcome for an unknown saving. The fear is legitimate, but the conclusion most teams draw from it is wrong. The answer to the quality risk is not to keep paying for the most expensive model everywhere, it is to test properly, so the decision rests on evidence rather than fear. Tested well, moving large parts of a workload from Opus to Sonnet or Haiku produces no quality loss a user would ever notice, and the savings are substantial. Tested badly, or not at all, the change is a gamble nobody should take. This piece is about doing it well, so you can capture the saving without betting the product on it.
The first mistake teams make is testing cheaper models against a vague sense of good enough. Without a defined quality bar, every comparison collapses into impression and opinion, and the loudest worry wins. Before you test anything, write down what quality means for each task type in your application, in terms specific enough to score. For a classification task, that might be accuracy against a labeled set. For an extraction task, it might be field level precision and recall. For a generation task, it might be a rubric covering correctness, completeness, tone, and format adherence, scored by human raters or by a strong model acting as judge. The point is to make quality measurable, because a measurable bar lets you answer the only question that matters: does the cheaper model clear it for this task? An undefined bar guarantees you never will, and the test becomes a debate instead of a decision.
A test is only as good as the data behind it, and the most common failure is testing on examples that do not reflect production. Pull a real sample of traffic for each task type, large enough to be representative and including the hard cases, the edge cases, and the messy inputs your application actually receives, not just the clean examples that everything handles well. If your application does several distinct things, build a separate set for each, because a model that holds quality on classification may not hold it on nuanced generation, and a blended score hides exactly the differences you need to see. Where you have historical outputs that were judged good, use them as a reference. The goal is an evaluation set that, when a model passes it, gives you real confidence the model will pass in production, because the set looks like production.
With a bar and a set in place, the test itself is straightforward: run the same evaluation set through Opus, Sonnet, and Haiku, holding everything else constant, and score each against the quality bar for that task. Honesty matters here. Use the same prompt unless you are deliberately testing prompt changes, because a cheaper model that fails on the Opus prompt may pass on one tuned for it, and you want to know which question you are answering. Score blind where you can, so the rater does not know which model produced which output, because knowing biases the judgment toward the expensive model. Record not just pass or fail but the margin, because a model that barely clears the bar is riskier than one that clears it comfortably, and the margin tells you how much headroom you have when production throws something harder than your test set.
The output of a good test is rarely a single verdict. It is a map: this task type holds on Haiku, this one holds on Sonnet, this one genuinely needs Opus, and this one is borderline and deserves a closer look. That map is the whole point, because it lets you route each task to the cheapest model that clears its bar rather than making one global choice. Most teams discover that a large majority of their volume holds on Sonnet, a meaningful share holds on Haiku, and only a small core of genuinely hard tasks needs Opus. Routing against that map, rather than running everything on the top model, is what produces the forty to seventy percent reduction in aggregate spend that disciplined model selection typically delivers, with the quality on each task held by design rather than by overspending.
Some tasks will sit right at the bar, passing on the cheaper model most of the time but failing on the hard inputs. These deserve more than a coin flip. Options include tuning the prompt for the cheaper model, which often recovers the margin; splitting the task so the easy inputs route cheap and the hard ones escalate; or using a cheaper model first with a confidence check that falls back to a stronger model only when needed. A fallback structure can capture most of the saving while protecting quality on the cases that need it, because the expensive model runs only on the small fraction of traffic that actually requires it. The borderline cases are where careful engineering pays off, and where a blunt all or nothing choice leaves money or quality on the table.
A test proves the model holds on your evaluation set at a point in time. Production is not static: inputs drift, usage patterns change, and a model that held at launch can slip as the work it sees evolves. Ship the change with monitoring in place, sampling production outputs against the same quality bar you tested against, so you catch drift before users do. This is not a reason to delay the saving, it is the discipline that lets you take it safely. A team that routes aggressively and monitors continuously captures far more savings, more durably, than one that stays on Opus out of caution and never measures whether the caution was warranted.
Testing cheaper models is where token optimization stops being theory and becomes a number on the invoice, and that number is also the baseline you negotiate from. A workload that has been tested and routed to the cheapest sufficient model consumes far fewer expensive tokens, which means the commitment you make to Anthropic is sized against optimized demand rather than waste. This matters at the contract table, because committing to inflated usage locks the inefficiency into your agreement for the full term, and unused commitment is generally lost rather than refunded. The buyers who negotiate best arrive having already tested and routed, so they commit to real, optimized demand and negotiate the rate on that. The test that protects your quality also protects your contract, because it ensures every token you commit to is one you actually need.
A test can fail in ways that look like a verdict but are really artifacts of method, and these false conclusions are expensive because they either keep you overspending or push you into a regression. The most frequent is testing on the wrong data: a sample of easy, clean inputs makes every model look fine and hides the gap that only shows on hard cases, while a sample skewed toward edge cases makes a perfectly good cheaper model look worse than it is in production. Build the evaluation set to mirror the real distribution of your traffic, hard cases included in their real proportion, and the result will predict production rather than mislead you. The second mistake is judging by impression rather than the bar, where a reviewer glances at a few outputs, prefers the Opus phrasing, and concludes the cheaper model is worse, when measured against the task's actual quality bar the difference was cosmetic and irrelevant to the user. Score against the defined bar, not against a preference for the familiar model.
A third mistake is testing the cheaper model on a prompt tuned for the expensive one and treating the failure as the model's fault. Models respond differently to instructions, and a prompt optimized for Opus may underperform on Haiku not because Haiku cannot do the task but because the prompt was never written for it. When a cheaper model falls short, try a prompt tuned for it before concluding the task needs the stronger model, because a small prompt change often recovers the margin and converts a fail into a pass. A fourth mistake is testing once and treating the result as permanent. Production drifts, prompts change, and new model versions ship, so a conclusion from six months ago may no longer hold in either direction. Re run the evaluation periodically and whenever inputs or models change, so your routing reflects current reality rather than a stale snapshot.
The most robust pattern for capturing the saving without risking quality is a tiered fallback, and it deserves explanation because it resolves the tension that makes teams hesitant. Instead of choosing one model for a task, you run the cheaper model first and apply a check to its output, escalating to a stronger model only when the check signals the cheap answer is not good enough. The check can be a confidence signal, a validation against the task's requirements, or a lightweight quality classifier, depending on the task. The economics are favorable because the expensive model runs only on the fraction of traffic the cheap model could not handle, so you pay the premium precisely where it is needed and the cheap rate everywhere else. Done well, a fallback captures most of the saving a full downgrade would deliver while holding quality on the hard cases that a blunt downgrade would have failed, which is exactly the outcome the cautious engineering leader wanted and assumed was unavailable.
Get a quote for a bounded engagement. Fixed fee or gainshare, no risk to you.
Get a QuoteWeekly intelligence on Anthropic pricing moves and the buyer side counters that work.