AI Spend Audit: How to Reduce Costs With Open Models

In 2026, mentions of open-weight or open-source models on US earnings calls and at investor conferences ran about six times higher compared to the last year. The companies doing the talking are no longer only tech firms. PNC, CH Robinson and Siemens have all discussed open-weight models in recent weeks.

The clearest example is Tinder. Its annualized AI spend climbed from roughly $1 million in January to roughly $10 million in July, and its CTO, Vinay Kuruvila, told the FT the company has started routing some queries from non-technical staff to open-weight models to bring costs under control. AT&T says open models already handle about 40% of its AI workload and that it wants to reach 70%, on a system that processes around 45 billion tokens a day. The FT also reports that open-weight models went from 7% of the tokens flowing through Vercel’s AI Gateway in December to 56% in August.

None of these companies announced that they left their frontier vendor. They did something less dramatic and more useful: they split their traffic. Simple, high-volume requests go to cheaper open models, and the work that genuinely needs a frontier model stays there. When “open-weight” starts appearing in CFO vocabulary, it means AI has stopped being an innovation line item and has become a cost line that someone has to explain.

This article covers how to find the movable workloads in your own request logs, how to price the move with real numbers, how to prove quality before you switch, and how to roll out without a bad week. The model comparisons use current list prices and independent benchmark data, and the worked example uses Claude Opus 5.5 and Sonnet 5.5 as the baseline against DeepSeek V4.1 Flash and Qwen3.8 Flash.

Start with the Bill Instead of the Model

Most teams begin an AI cost review by asking which model is best. That is the wrong first question. The right first question is which of your workloads are paying frontier prices for tasks that do not need frontier capability, and you cannot answer it without request-level data.

Pull a month of logs from your gateway, proxy or provider dashboards and make sure each call carries the following fields:

• Application, team and feature (the business purpose, not just the API key)
• Model and version
• Input tokens, output tokens and cached tokens
• Latency and retry count
• Some signal of outcome, such as user acceptance, a downstream validation result or a thumbs rating
• Cost per call

If you do not have feature-level tags today, add them before anything else. A week of tagged logs tells you more than a quarter of invoices. Once you have them, sort features by monthly spend and look at the top ten. That list is your audit scope. Everything below the top ten is rounding error until the big ones are handled.

Sort Workloads by How Bounded They Are

The single best predictor of whether a workload can move is how tightly bounded the task is. A task with a fixed output shape, a clear definition of correct, and a limited amount of reasoning is a strong candidate. A task that requires long chains of dependent decisions, or where a subtle mistake is expensive, is not.

You do not have to guess at where the capability gap lives. Public benchmark data for Claude Opus 5.5 versus DeepSeek V4.1 Flash, drawn from the Vals AI benchmark suite as of late September, shows that the gap is far from uniform. On the aggregate Vals Index, Opus 5.5 scores 69.7 and V4.1 Flash scores 57.9, so Flash lands at roughly 83% of the flagship overall. The task-level picture matters more than that single number.

The pattern is consistent with the table above it. Bounded professional tasks (documentation, structured analysis, code generation that can be tested) sit within single digits of the flagship. Long-horizon agent work and competition-grade reasoning is where the open model falls away. This is why the audit is a workload exercise and not a vendor decision. The same model can be an easy yes for your ticket summarizer and a clear no for your autonomous incident-response agent.

Price the Move with Real Numbers

Here are the list prices that matter for this comparison, per million tokens. DeepInfra rates for the two open models are as tracked on OpenRouter and public price trackers in early October 2026, and they change often, so check current rates before you build a business case.

Per-token prices are not a business case on their own, so take a concrete workload. Imagine a feature that handles 1 million requests a month, each averaging 3,000 input tokens (a prompt, some retrieved context, an instruction) and 500 output tokens. That is 3 billion input tokens and 500 million output tokens monthly.

Against a Sonnet baseline the gap is smaller but still large: Qwen3.8 Flash comes in about 21x cheaper and V4.1 Flash about 17.5x cheaper at list price

Those multiples are the ceiling, not the forecast. Three things pull the real number down, and an honest business case accounts for each.

• Verbosity: Reasoning-heavy open models can generate far more output tokens per task than a flagship tuned for concision. Artificial Analysis found that the first DeepSeek V4 models produced several times the median token volume on its Intelligence Index. Measure tokens per completed task on your own workload, not just price per token. Independent cost-per-task figures are a better guide than list prices: at V4 Flash’s launch, Artificial Analysis estimated an average cost of about 3 cents per benchmark task, against $3.15 for Claude Fable 5. That is still roughly 100x, but it is a very different statement from comparing rate cards.
• Tokenization: Different model families split the same text into different numbers of tokens, and Anthropic has noted that its recent tokenizers produce meaningfully more tokens for the same text than older ones. Compare cost per task rather than cost per million tokens.
• Fallbacks: No cheap model clears every request. The realistic design is a tiered one, where the cheap model answers first and the hard cases escalate to the frontier model. The cost of that escalation belongs in the model. Here is the same workload with DeepSeek V4.1 Flash as the first tier and Opus 5.5 as the fallback.

At a 4% escalation rate, the tiered design runs at about 15x cheaper than all-Opus, which comes to roughly $245,000 saved per year on this single feature. If Flash also emits double the output tokens, that same 4% scenario lands near 13x. The savings are real and large, and they are very sensitive to how often the cheap model needs help. That sensitivity is precisely why the next two steps exist.

One more fairness note for finance readers. The baseline has been moving too. Opus 5.5 launched at $4 and $20 per million tokens, down from $5 and $25 for Opus 5, and Anthropic says its tests show about 40% lower cost per task on typical workloads because the model uses fewer tokens. Price your current model at today’s rates, with your real cache-hit rate, since cached input on Opus 5.5 is $0.20 per million. A stale baseline will overstate your savings and cost you credibility with the CFO.

Prove Quality on Your Own Traffic

Public benchmarks tell you where to look. Your own logs tell you whether the move is safe. Build a small evaluation for each candidate workload before any traffic shifts.

• Sample real requests. Pull 500 to 1,000 production requests per workload, stratified across the input types you see (short and long, easy and awkward, every language you support). Freeze this set. It becomes your regression suite for every future model change.
• Capture the baseline. Run the set through the incumbent model and store the outputs along with any known outcomes, such as what the user accepted or what the downstream system did with the result.
• Grade with the cheapest reliable method. For structured tasks, use programmatic checks: schema validity, exact-match on key fields, unit tests for generated code. For open-ended tasks, use an LLM judge, but calibrate it first by having humans grade a subset of 100 or so items and checking that the judge agrees with them. For high-stakes workloads, keep human review in the loop.
• Write the acceptance bar before you look at results. For example: at least 97% of baseline task success, no increase in severe errors, format validity above 99%, and tokens per task within an agreed multiple of the baseline. Teams that set the bar after seeing the numbers tend to move it.
• Look at the tail, not the average. An average quality score of 97% can hide a cluster of failures on one customer segment or one document type. Slice results by input category and read the worst fifty outputs by hand.

Then run the candidate in shadow mode: production traffic goes to both models, users only see the incumbent’s answer, and you compare. Shadow mode costs a little extra for a couple of weeks and removes most of the guesswork.

Roll Out with a Safety Net

Once a workload clears the bar, the rollout should be boring. The design that most enterprise teams converge on has five parts.

Cheap model first, frontier on escalation

Define explicit triggers: output fails schema validation, a confidence score falls below a threshold, the request matches a risk category, or the user asks again. Small decision models, which return a choice and a calibrated confidence score without generating text, are well suited to the routing step because they add very little latency and cost.

Canary the traffic

Start at 5% of a single workload, hold for several days, then move to 25%, then 100%. Keep the old path warm until the new one has survived a full business cycle, including month-end or any other seasonal peak.

Keep a kill switch

Every routed workload needs a one-line configuration change that sends 100% of traffic back to the incumbent. Test it before you need it.

Monitor what finance and engineering care about 

Track escalation rate, task success rate, severe-error rate and tokens per task alongside cost per accepted task. A rising escalation rate is the early warning that your savings are eroding.

Pin versions and re-run the suite

Open-weight providers release updates quickly, and some releases are previews whose behavior can shift between versions. Pin model versions in production and re-run your frozen evaluation set before adopting any new one.

Running the Cheap Tier on DeepInfra

DeepInfra serves open models through an OpenAI-compatible API, so adding a cheap tier to an existing pipeline means pointing a second client at a different base URL. Here is a minimal tiered pattern with schema validation and an escalation path.

import json
from openai import OpenAI

cheap = OpenAI(
    api_key="YOUR_DEEPINFRA_API_KEY",
    base_url="https://api.deepinfra.com/v1/openai",
)

CHEAP_MODEL = "deepseek-ai/DeepSeek-V4.1-Flash"  

def is_valid(text: str) -> bool:
    try:
        data = json.loads(text)
        return {"category", "priority"} <= data.keys()
    except (ValueError, AttributeError):
        return False

def classify_ticket(ticket: str) -> str:
    response = cheap.chat.completions.create(
        model=CHEAP_MODEL,
        messages=[
            {"role": "system", "content": "Return JSON with category and priority."},
            {"role": "user", "content": ticket},
        ],
        temperature=0,
    )
    answer = response.choices[0].message.content

    if is_valid(answer):
        return answer          # cheap tier handled it

    return call_frontier(ticket)  # your existing incumbent client
copy

The validation function is where your escalation rate is decided. Log every escalation with the reason, and review them weekly, because that log tells you exactly which requests the cheap tier cannot handle and whether the threshold needs adjusting.

The Bottom Line

The shift the Financial Times describes is not a story about companies abandoning frontier models. It is a story about companies discovering that a large share of their AI traffic never needed one. At list prices, DeepSeek V4.1 Flash and Qwen3.8 Flash cost between roughly 17x and 42x less than Claude Sonnet 5.5 and Opus 5.5 on a typical workload. After verbosity and fallback costs, a realistic planning number for a well-chosen workload is closer to 10x to 15x. That is still the difference between a six-figure line item and a five-figure one for a single feature.

The gap in capability is real and uneven. It is small on bounded professional tasks and very large on long-horizon agent work, so the audit has to be done workload by workload, with your own logs and your own acceptance bar. Teams that do that work end up with a routing layer, a frozen evaluation suite and a cost-per-accepted-task metric that finance trusts. Those three assets matter more than any one model choice, because they let you take advantage of the next price drop in an afternoon instead of a quarter.