Token prices are easy to compare, so they get compared. A model that costs a third as much per million tokens looks like a saving before anyone has run it on real work.
But nobody buys tokens for their own sake. A business buys completed work: an extracted invoice, a resolved ticket, a passing test. Tokens are an input, like electricity. The number that matters is what it costs to get a task done to the standard the workflow needs.
A worked example
The numbers below are chosen to make the arithmetic clear. They are not measurements.
Suppose a workflow can run on either of two configurations, and that every failure has to be caught and fixed downstream—by a retry, an escalation, or a person—at an average cost of $1.00.
| Configuration | Per attempt | Succeeds | Cost of failures | Per task |
|---|---|---|---|---|
| A | $0.02 | 70% | 30% × $1.00 | $0.32 |
| B | $0.05 | 95% | 5% × $1.00 | $0.10 |
B costs two and a half times as much per attempt, and less than a third as much per completed task. The exact numbers don’t matter; the structure does. When failure has a cost, the cheapest configuration per token is often not the cheapest per outcome.
The ratio that improves by failing
Cost per successful task has its own trap. If you only count the tasks that succeeded, a system can improve the ratio by quietly giving up on the hard ones. A router that abandons the most difficult tenth of its traffic reports a lower average cost—and, if the abandoned tasks drop out of the denominator, a better cost per success too.
So the ratio should never travel alone. Report it with the counts behind it: tasks completed, failed, stopped, deferred, and escalated. A task stopped to save money is not a success, and it shouldn’t be counted as one.
What goes in the numerator
Everything it took to get the work done:
- Inference, including retries, escalations, and failed attempts.
- Verification: tests, checks, model judges, and human review where they apply.
- Billed tool calls.
- The experiments that found the configuration.
- The optimizer’s own overhead.
If a cost isn’t available, say so, rather than leaving it out silently.
What goes in the denominator
Tasks that met the workflow’s definition of success—a definition the customer sets, including the mistakes that are never acceptable. It has to exist before any optimization starts. An optimizer is only as good as its objective, and an objective written in tokens rewards truncation, abandonment, and cheap mistakes.
What would change our mind
- Where failures are cheap to detect and free to retry, token price is a reasonable approximation. The gap shrinks as the cost of failure falls.
- Where success can’t be observed—no ground truth, or outcomes that arrive weeks later—cost per successful task can’t be computed directly. It has to be estimated from proxies, and those proxies have to be checked against real outcomes.
The figures in this post are illustrative. None are measured.