Cost per token is an infrastructure metric. Cost per successful task is a business metric. Here’s the one equation that connects them — and why it flips the model you should deploy.
Every model you can call — GPT, Gemini, Claude, Qwen, DeepSeek — is sold on cost per token and paid for on cost per successful task. Those are not the same number, and the gap between them is where AI budgets quietly die. Modern systems increasingly route each request to a different model by complexity instead of using one model for everything — so which number you optimize is a live architectural decision.
The reason: a frontier model is priced against labor substitution, not compute. Its anchor isn’t a GPU-hour — it’s an engineer’s hourly rate. So once output can be wrong in a way a human must fix, the token bill becomes a rounding error next to the fix.
Cost per token is an infrastructure metric. Cost per successful task is a business metric.
The Real Cost of One Attempt
TCO = Ctoken + P(fail) × L
- Ctoken — Token cost for one attempt.
- P(fail) — Probability the output requires human intervention (won’t compile, breaks a schema, hallucinates a column, etc.).
- L — Fully loaded human repair cost (hourly rate × repair time).
A cheap model minimizes the first term and quietly inflates the second. The first appears on your API invoice. The second appears on your payroll.
The Equation That Decides Everything
Take any budget model (B) and any frontier model (F).
The frontier model is cheaper whenever:
CF + PF × L < CB + PB × L
Rearranging gives the entire decision framework:
ΔC < Δp × L
Where:
- ΔC = Additional token cost of the frontier model.
- Δp = Reduction in failure probability.
- L = Human repair cost.
Pay the premium only when the labor it saves is worth more than it costs.
That saved labor (Δp × L) is the reliability dividend.
It grows when:
- the cheaper model fails more often, or
- your engineers become more expensive.
Nothing vendor-specific appears in the equation. It survives every pricing change the AI labs make.
TCO crossover: the budget model overtakes the frontier model as tasks get harder
The budget model stays cheaper until task complexity drives its expected rework cost high enough that Δp × L exceeds the token premium. Left of the break-even, route to the cheaper model. Right of it, route to the frontier model. The crossover depends on ΔC / L, not on any specific vendor.
One Example (The Only Part That Expires)
Using Anthropic’s current pricing:
| Model | Input / MTok | Output / MTok |
|---|---|---|
| Claude Haiku 4.5 | $1 | $5 |
| Claude Fable 5 | $10 | $50 |
Accounting for Fable’s ~30% tokenizer expansion, one hard agentic request (40k input / 8k output) costs roughly:
- Budget model: $0.08
- Frontier model: $1.04
Therefore:
ΔC ≈ $0.96
Assume (illustratively):
- Budget model failure rate = 45%
- Frontier model failure rate = 8%
- US senior engineer = $150/hr
- Average repair = 30 minutes
Therefore:
L = $75
Δp = 0.37
| Token Cost | Expected Rework | Total Cost | |
|---|---|---|---|
| Budget Model | $0.08 | 0.45 × $75 | $33.83 |
| Frontier Model | $1.04 | 0.08 × $75 | $7.04 |
Despite costing 13× more per token, the frontier model is 4.8× cheaper per completed task.
If pricing changes tomorrow, only these numbers change.
The equation does not.
The Break-Even Depends on Payroll
Solve for the minimum reliability improvement required:
Δp* = ΔC / L
| Engineer | Repair Time | L | Required Δp |
|---|---|---|---|
| US Senior ($150/hr) | 60 min | $150 | 0.6 percentage points |
| US Senior ($150/hr) | 15 min | $37.50 | 2.6 percentage points |
| Offshore Senior (~$15/hr) | 30 min | $7.50 | 12.8 percentage points |
| Offshore Senior (~$15/hr) | 15 min | $3.75 | 25.6 percentage points |
A San Francisco engineering team can justify the frontier model with almost any measurable reliability improvement.
A lower-cost engineering team only benefits on genuinely difficult work.
Same models.
Same pricing.
Completely different routing decision.
Where the Framework Breaks
- On raw API cost, the budget model is still dramatically cheaper.
- On simple tasks, both models succeed almost every time, so human repair cost approaches zero.
- Most production systems compare against mid-tier models rather than the absolute cheapest one. The extreme example simply makes the economics easier to see.
- Your own evaluation pipeline should measure Δp. The equation is universal; the numbers belong to your workload.
Route on Complexity, Not on Price
The optimal architecture is straightforward.
Route boilerplate work—CRUD operations, SQL generation, extraction, classification—to the cheapest model, where both failure probability and repair cost remain low.
Route ambiguous integrations, system design, architecture, planning, and difficult reasoning to the frontier model, where preventing even one expensive engineering mistake easily pays for the additional token cost.
Using an expensive model for trivial work wastes API budget.
Using a cheap model for architectural work wastes engineering time.
Only one of those appears on your cloud invoice.
Cost per token is what you’re billed.
Cost per successful task is what you actually pay.
Prices use Anthropic’s published pricing for Claude Fable 5 and Claude Haiku 4.5. Failure rates and labor costs are illustrative assumptions. Replace them with your own measurements—the decision rule ΔC < Δp × L remains the same.
This article was originally published by samir on HackerNoon.
