The invoice went up. The unit price went down.
Both statements are true, and the distance between them is where a large share of crypto-native AI teams are currently bleeding capital. This is not a modeling error. It is a structural feature of how inference is priced and how agents consume it.
Start with the deflation. Over the past two years, the price of a fixed capability tier of tokens fell roughly one to two orders of magnitude. Frontier-class inference that quoted at $30/$60 per million input/output tokens now clears at $2โ$3/$10โ$15. Lightweight tiers trade between $0.05 and $0.50 per million. Epoch AI's constant-capability pricing series puts the annual decline across a much wider band โ roughly 9x to 900x depending on task, median in the low tens. Pick any point estimate inside that range. The direction does not change.
Now multiply by volume. A 30-turn tool-calling agent re-sends its history on every round. Cumulative input tokens land at 30x to 100x a single-turn query. A 10x unit-price decline times a 30x volume increase is a 3x increase in the bill. That arithmetic does not appear in the vendor's launch post, and it is the most common single cause of budget overrun I encounter in agent deployments.
I ran this against my own stack last quarter. Ten thousand transactions a day, 99.9% execution success, MEV-resistant routing across DEX venues. The infrastructure held. The cost line did not behave the way the model-selection discourse promised, and the discrepancy sent me back to the invoice line items rather than the model card.
The first thing to establish is who actually pays. Three billing structures coexist and they carry completely different optimization incentives. Metered API billing puts the marginal cost on the caller โ that is the only regime where "save tokens" is an economically meaningful instruction. Subscription products absorb the per-call cost at the vendor, which means the end user's marginal cost is zero and the rational behavior is overuse, not restraint. Enterprise seat licensing amortizes cost across headcount and moves the incentive to a procurement department that optimizes on contract size, not token counts. Any cost advice that does not state which of these three it addresses is not advice; it is a category error.
For the metered-API reader โ the developer, the independent builder, the small desk โ the second thing to establish is the denominator. A claimed 60% saving against "all-frontier-everywhere" is a different claim than 60% against an already-routed baseline. The baseline defines the headline. When I audited ERC-20 contracts in 2017, the discipline that survived was not cleverness; it was a written checklist where every claim had a verifiable input. I apply the same rule here. If a cost-optimization guide does not publish its baseline, its percentage is marketing.
With that framing, the cost stack sorts cleanly by return on engineering hour. The order matters more than any individual technique.
Context and caching engineering sits at the top. Cached input typically bills at 10% to 50% of list โ Anthropic at roughly 0.1x, OpenAI in the 0.25x to 0.5x band. Batch APIs apply a flat 50% discount on top. Stack the two for offline work and you clear 90% reduction without touching model capability, which means the quality risk is close to zero. The engineering constraints are mundane and absolute: static prefixes โ system prompt, tool definitions, long reference documents โ must sit at the front of the prompt to be cacheable, and any timestamp, session ID, or randomized ordering injected into the prefix destroys the cache key. I have watched teams lose six figures of annualized savings to a timestamp in a system prompt.
Output constraints sit second, and they are chronically underweighted. Output tokens are priced at 3x to 8x input across current tiers โ $3/$15, $1.25/$10, $0.27/$1.10. Every sentence of unprompted preamble, every restatement of the input, every "certainly, here is" is billed at the expensive rate. Structured output schemas, hard max_tokens ceilings, and explicit prohibitions on recapping the prompt routinely deliver 20% to 60% reduction on interactive workloads. Most tutorials aim all their fire at the input side because input is the larger token count. The larger token count is not the larger invoice.
Task-level model selection sits third, and it is the tier everyone reaches for first because it requires no engineering. It is also the tier with the highest quality-regression risk, for a specific reason: degradation is silent. A downgraded model returns well-formed output, in the correct schema, with no error code, and the content is wrong. There is no exception thrown. The loss surfaces weeks later as a mispriced position or a failed settlement. That asymmetry is why I treat model substitution as a change requiring an evaluation set, not a configuration toggle.
The breakeven math is unkind. Take a call profile of 2,000 input and 500 output tokens. Frontier tier at $3/$15 per million costs about $0.0135 per call; light tier at $0.15/$0.60 costs about $0.0006. The spread is roughly $0.013 per call. Building a reliable routing layer โ evaluation set, regression harness, fallback logic โ is realistically 20 engineering hours. At $100 an hour that is $2,000. You need on the order of 150,000 calls to break even on the engineering, before you have counted a single quality incident. For most early-stage products, the router costs more than the tokens it saves.
Below that sit RAG substitution for long context and distillation into a fine-tuned small model. RAG is not universally cheaper โ it is cheaper on reuse. A single pass over 100,000 tokens of context is simpler and often cheaper than standing up retrieval. The crossover appears on the second, tenth, hundredth pass, where RAG re-sends 2,000 retrieved tokens per round instead of the full corpus. That is a one-to-two-order-of-magnitude difference in the regime that agents actually operate in. Distillation has the highest ceiling and the heaviest fixed cost, and it should be the last tool reached for, not the first.
Here is the part the guides skip. The largest cost levers are supply-side, and they are controlled by the API vendor's product design, not by your prompt craft. Prefix caching, continuous batching, KV-cache optimization under MQA/GQA/MLA, PagedAttention, speculative decoding, FP8 and INT4 quantization โ each of these cut the GPU-seconds per token by 1.5x to 4x. They are the real source of the price decline, and they are being shipped whether or not you write a clever prompt. Inference is memory-bandwidth bound in the decode phase, not FLOPS bound, which is why HBM capacity and KV-cache compression move cost more than raw compute throughput.
The corollary is uncomfortable for anyone writing tutorials. Vendors have a direct incentive to internalize every optimization that also improves their own GPU utilization, and prefix caching does exactly that โ it removes duplicate prefill work on their side too. What the vendor internalizes becomes a default setting, and the user-side technique premium evaporates. The half-life of a token-optimization playbook is six to twelve months. Standardization is the silent killer of alpha.
There is also a semantic trap in the framing itself. Through 2024 and 2025, "token maxxing" in the builder community described the opposite behavior โ deliberately stacking tokens at inference time, long chains of thought, test-time compute, spending more to buy capability. A guide that borrows the phrase to mean thrift is either running an intentional irony or has misread its own vocabulary. I have seen readers execute the reverse of a document's thesis because of a title like this, and in a bear market that misread compounds.
So what is the actual position? Ledgers do not lie, only the auditors do. The invoice says spend is rising while unit price collapses, and both lines are honest. That combination means the correct objective is not absolute reduction but budget reallocation โ under a fixed monthly ceiling, cheap caching and constrained output buy you more attempts, longer context, more agent steps, more shots at the rare event that pays. Cutting tokens to cut tokens is an accounting exercise. Converting saved tokens into additional capability is a trading decision.
We trade the protocol, not the promise. Watch two signals over the next two quarters: whether caching and batch defaults get silently enabled across major providers, which would retire most user-side advice overnight, and whether reasoning-model output multipliers โ currently 10x to 50x normal output length โ become the dominant line item in agent budgets. If they do, the entire conversation shifts from prompt hygiene to inference-time budget governance. The teams that instrument their cost per successful on-chain action before that shift arrives will be the ones still solvent after it.