Your Cheapest AI Model Might Be Your Most Expensive Decision

The procurement lead thought she had a clean win for the quarterly review. The company had moved its AI workloads to a model that ran forty per cent cheaper per million tokens. The comparison was sound, the pricing was real, and on paper the call was obvious.

Three months later the AI bill was higher than before the switch, and no one could tell the CFO why. The rate had gone down. The bill had gone up. Both were true. The reason sat where the vendor comparison never looked: the support workflow now needed more retries, longer prompts, and a round of manual review before each ticket could be closed. The saving showed up in procurement. The overrun showed up everywhere else.

That gap is where AI budgets leak, and it never appears in the rate you are quoted. It appears in the work it takes to turn a model’s output into something you can use.

The sticker price is not the cost

I read Jan Iłowski’s blog post from July 2026, and it names a problem I run into with clients constantly. His title is built to provoke: price per 1M tokens is meaningless (Iłowski, 5 July 2026). It takes apart the number most finance teams have quietly adopted as their unit of AI cost, and it does it in three moves.

Start with the obvious one: a token is not a standard unit. Every AI vendor splits text into tokens differently, so the same paragraph is worth more tokens on one model than on another. Iłowski points to the same text resolving to 160 tokens in one model and 200 in another, and to Anthropic changing its tokeniser so that identical input now splits into roughly thirty per cent more tokens (Iłowski, 5 July 2026). Compare two vendors on the per-token rate and you are comparing prices in two currencies without the exchange rate. The smaller number is often the larger bill.

Then there is the part you cannot see. Reasoning models spend most of their tokens on internal working you never read, billed at the same rate as the output you do. You pay for that thinking, it is often the larger share, and it never shows up as anything you can inspect.

The one that matters most is the third, and it needs a definition. A task is one piece of real work: a support ticket resolved, an invoice’s fields extracted, a function that compiles. What it costs is everything you pay to finish it correctly, including the tokens the model burns thinking and the cost of a second run when the first answer is wrong. That is a different number from the price of a token, and it is the only one worth trusting.

Measure it that way and cheap-per-token and cheap-per-task stop lining up. A low-priced model can take a support ticket, reason through a few thousand hidden tokens, misread the customer’s account, and answer the wrong question, so the ticket comes back and runs again. A dearer model reads the same ticket, settles it in far fewer tokens, and closes it first time. Cheaper on the rate card. More expensive on the closed ticket.

Iłowski put numbers to it. Ranked on cost per completed task instead of list price, the order flips. One of the strongest reasoning models came in around $0.99 a task despite charging more per token than its rivals. A cheaper, weaker model was the lowest of the lot at four or five cents. And one premium model ran to about $2.29 a task, dearer than models that score higher than it on the usual benchmarks (Iłowski, 5 July 2026). His numbers, his benchmark, one week in 2026, and the model names will age fast, so treat them as an illustration and not a price list. The cheapest rate card and the cheapest result are routinely different models.

The old cloud billing mistake is back

None of this is new. It is the oldest error in cloud finance, back for a second run with sharper marketing. A reserved instance with a low hourly rate looks cheap until it sits half idle, and then it isn’t. Cheap storage stops being cheap the second you pay to pull the data back out. We learned that lesson, slowly. Then AI showed up with a brand-new unit to be billed in, and we forgot it overnight and grabbed the sticker price again.

Price per token is the mechanism. Cost per outcome is what you are actually buying.

Sometimes the right call is to spend more per token

Now the problem changes shape. Buying the cheapest model per token looks like discipline on a procurement scorecard. It is the same reflex that had us reserving the lowest-hourly instance we could find and then leaving it half idle.

This is how I think about FinOps in general: less bill reduction, more capital allocation. So the question I ask on a model choice is not which one has the lowest rate. It is which one finishes the workload at the lowest acceptable mix of cost, error rate, and operational load. Sometimes that is the cheap model. Sometimes the honest answer is to pay more, because a stronger model clears the task in one pass, with no rework queue behind it, and works out cheaper per finished job than the bargain you ran three times.

The most expensive model per token is often the cheapest per decision, and the reverse is just as common.

You cannot govern a cost you cannot see

That measurement is harder than it sounds, because the biggest part of the cost is hidden. Reasoning tokens are the AI version of untagged spend: real, large, and unattributed. If most of your consumption is invisible, allocation breaks. You can still produce a clean chargeback report, and it will be clean in the wrong way, because the bulk of the bill is thinking you were never shown. The tokeniser gap makes it worse. Two models that count the same work differently cannot be compared on price per million tokens at all, because you would be holding two rulers and calling the difference a result.

A three-question test for AI cost

Before the next model switch gets waved through because the rate card looks better, ask three questions.

• Can you see it? Not the rate card, but the real cost of finishing one task on this workload, reasoning tokens included. If your reporting stops at list price, you are steering by the number Iłowski just called meaningless.
• Can you explain it? If one model costs three times another per finished task on a lower token price, can you say why in a sentence a CFO and a CTO both accept? Usually it is retries, reasoning depth, first-pass accuracy, or tokeniser inflation.
• Can you act on it? When the answer changes next quarter, and it will, can you move the workload without reopening a contract written in the wrong unit? Visibility you cannot act on is just trivia.

What a working version looks like

A workable first version does not need a platform. Start with one row per workload, not one per model. Each row carries the task, the model serving it, tokens in and out, an estimate of the reasoning tokens, the first-pass success rate, retries, human review effort, and the cost per successful result. The rate card can sit in the appendix. It should not drive the decision. For most teams the first cut of this is a spreadsheet, and that is fine. The discipline matters more than the tooling.

The FinOps Foundation now builds its AI guidance on exactly this. J.R. Storment writes that the point “is not to minimize token consumption. It is to connect token consumption to value,” and warns that treating tokens “as a cost discipline alone tends toward false economies” (FinOps Foundation, May 2026). Its 2026 survey put 98 per cent of practitioners on managing AI spend, up from 31 per cent two years earlier, with FinOps for AI their top forward-looking priority (FinOps Foundation, February 2026, 1,192 practitioners).

The point is not to minimize token consumption. It is to connect token consumption to value.

— J.R. Storment, FinOps Foundation

From there it is a sequence you can start this quarter.

1. Instrument every model call. Capture tokens in and out, the reasoning tokens, the model, and whether the call succeeded, each one tagged to a workload and an owner. Reasoning tokens are rarely on the invoice, so measure them at the call boundary and reconcile against the response metadata.
2. Decide what counts as success, then divide total cost by successful outcomes. This is the part teams skip, and it is the part that matters: a validation rule for an extraction, a small graded eval set for an answer, a human spot-check on a sample. Get it right and you have a cost per result you can defend to a CFO. Skip it and you are back to counting tokens.
3. Score each model against the quality the task actually needs, then route down to the cheapest one that clears the bar and escalate only where it fails. The Foundation points to routing work that cut cost by up to 98 per cent against a GPT-4 baseline while holding accuracy (FinOps Foundation, May 2026, citing FrugalGPT).
4. Pull the levers that cut tokens without touching quality: cache repeated context, batch anything that is not real-time, trim prompts and output formats. The same source reports context compression removing seventy to eighty per cent of the tokens sent to the model.

Then govern on the unit. Show each team the cost of its own outcomes, alert when cost per result moves rather than when raw spend does, and make a defensible cost per outcome the gate a new AI initiative has to clear.

I see the same failure over and over: a team routes everything to the cheapest model per token, then pays for it three times in retries, weak first-pass accuracy, and clean-up that lands in someone else’s cost centre and never shows up next to the AI spend.

A fintech I worked with, anonymised, had genuinely good procurement discipline. They ran a careful bake-off, locked the lowest per-token rate in the category, the exact figure the board had told them to drive down, and pointed a document-extraction workload at it. Six weeks later the cost per correctly extracted document was higher than the expensive model they had rejected, because the cheap one needed several passes and a queue of human corrections to reach the same accuracy. Good unit-price discipline, real money lost. What fixed it was not a new model but a new metric: the number in the contract and on the dashboard moved from price per token to price per validated document, and the model choice sorted itself out once the measure was honest.

The procurement lead from the top of this piece was not wrong about her forty per cent. She was measuring the wrong thing. Change the unit, and the rest follows.