A billing audit I ran last quarter started with a question any desk lead should be able to answer in an afternoon: how many tokens did we actually pay for?
It took eleven days. Visible output on a mid-complexity agent task: about 900 tokens. Billed units on the same task: just under 31,000. The other 30,000 were reasoning tokens โ work the model did internally, work the buyer never saw, work the invoice charged for anyway.
Nine hundred against thirty thousand. Not a rounding error, not a pricing quirk. A 34x spread between what the product looks like from outside and what it costs from inside, paid by every team shipping an agent loop into production โ whether or not they have measured it. I have run batched inference clusters and I have audited billing pipelines, and the gap between those two views of the same workload is the most under-discussed number in the AI trade right now.
I have spent sixteen years watching markets misprice things. This one has a new home: on-chain. Decentralized compute networks โ GPU marketplaces, verifiable inference protocols, tokenized model-serving layers โ have raised two years of capital on a three-part pitch. Cheaper than hyperscalers. Provably correct. Permissionless supply. Two of those are hard. One is close to impossible, and it happens to be the one the money depends on.
The moment metering becomes the product, opacity becomes the business model. On-chain, opacity gets a token, a staking contract, and a governance forum where nobody wants to ask the ugly question: what exactly are we being billed for?
The part of the trade everyone agrees on
The bull case for decentralized inference is correct at the demand level, and I want to be fair about that. GPUs are expensive. Hyperscaler margins are fat. Idle capacity sits in a hundred data centers across a dozen jurisdictions behind a dozen power contracts. Route demand to supply, settle on a public ledger, slash the cheaters, and you get a cheaper market with stronger guarantees than a handshake and an SLA nobody reads.
Demand for inference is not going away. Training is capital-intensive, lumpy, and episodic โ a lottery ticket drawn every eighteen months. Inference is metered, continuous, recurring. It is the annuity. And the money in an annuity is never in the headline. It is in the spread between the meter and the cost of serving.
That is where the pitch wobbles. This is a supply-of-verification business wearing the costume of a supply-of-compute business, and the two price nothing alike.
In a normal metered market, the unit is defined by physics. A kilowatt-hour is a kilowatt-hour. A gigabyte is a gigabyte. A gas unit is a gas unit, priced by an open fee market anyone can read in real time. Boring meters are the foundation of every liquid commodity market on earth.
Inference has no physical unit. The unit is defined by whoever writes the billing code โ and the billing code is written by the party that also collects the revenue.
In a normal metered market, the unit is defined by physics. In inference, the unit is defined by whoever writes the billing code.
Put that on a blockchain and it does not improve. It gets worse. A ledger forces you to name a unit and defend it forever, in public, against every counterparty with an incentive to argue. That is exactly the pressure you want on a settlement layer. It is also exactly the pressure most of these networks were not built to survive.
One visible token, thirty invisible ones
Inference cost is not one number. It is at least three: prefill, decode, and memory movement.
Prefill processes the prompt โ compute-dense, parallel, cheap per unit of output. Decode emits tokens one at a time โ memory-bandwidth-bound, serial, and the real cost driver. Then there is the KV cache, the attention state held in memory so the model does not re-read the whole context on every step.
Stack reasoning on top and the arithmetic stops being intuitive. A reasoning model burns tokens internally before emitting a single visible character. On my audit the ratio ran roughly 34 to one. Inside an agent loop it multiplies rather than adds โ every tool call, retry, and reflection step carries its own invisible trace, and each one bills.
Here is what matters for token design. The same visible output can cost wildly different amounts depending on batch size, quantization, cache hit rate, and speculative decoding. A 4-bit quantized serve on a warm cache and a full-precision cold serve can emit identical text at cost ratios that would embarrass a payday lender.
So a network settling per token has two options and both bleed. Bill the average and you overcharge cache hits until routers send traffic elsewhere. Bill actuals and you undercharge misses until operators exit. The clearing price sits in a band that moves constantly, and the only party who can see the whole band is the operator โ who is also the counterparty setting the meter.
This is proof-of-work with one crucial difference. In PoW the work is self-verifying. Hash the header, check the difficulty, done. In inference the work is silent. There is no nonce to check.
The verification tax
Three families of verification get pitched. They are not close to equivalent.
Trusted execution environments are the practical option. Run the model inside a confidential enclave, attest to the binary, sign the output. Fast and cheap. It also asks the buyer to trust a silicon vendor's attestation chain, the operator's build pipeline, and whoever holds the signing key. That is a real reduction in trust surface, and I have used it in production. It is not trust minimization. It is trust relocation. When I audited DeFi protocols after 2020 I learned to weight audit history over audit branding. Same rule here. A TEE attestation from a vendor nobody has stress-tested is a claim. It is not evidence.
Zero-knowledge proofs are the principled option. Prove correct execution without re-running it. The problem is arithmetic. For large models, proving cost is not a small multiple of inference cost โ it is an order of magnitude or more on top. You are buying correctness with a resource scarcer than the thing being verified. If the margin is thin, the proof eats it.
Optimistic systems are the economics-first option. Post an output, stake a bond, allow a challenge window. This works exactly as well as the challenger's ability to reproduce the computation โ same weights, same numerics, same metering visibility as the accused. It also only works when the value at risk exceeds the cost of adjudication. Nobody opens a dispute over a $0.0003 inference call.
Small inferences are economically unverifiable. Not hard. Unverifiable. The security budget is inversely proportional to transaction frequency, and inference is the highest-frequency workload in software.
Which brings the crux, and it is the crux I would put in front of any fund allocating here. You can slash an operator for a wrong answer. You cannot slash one for a padded bill, because padding is not a wrong answer โ it is a correct answer priced by a lie. Catching it requires the metering function itself to be verifiable. No network I have audited has that.
You can prove the answer. You cannot prove the price.
Verification is not a feature you switch on. It is a cost center you fund, and in a bear market everything that is not revenue gets cut first. So when I look at these networks I do not read benchmark tables. I look for a cost-to-serve disclosure, and I check whether the number reconciles against anything.
Memory is where the margin lives
Every operator I know who is actually profitable is profitable because of memory, not FLOPS. KV cache occupancy, high-bandwidth memory, prefix reuse, continuous batching โ those decide the marginal cost of a token far more than raw compute does.
And memory is the one input that refuses to decentralize. A warm cache is local. You cannot route a hot attention state across a network of heterogeneous operators without a transfer cost that eats the savings.
That produces a wedge the market keeps missing. Identical visible output, served in the same minute, can carry marginal costs that differ by an order of magnitude depending on where the cache sits and whether the batch is full. In a centralized cluster you bury that inside utilization. In a decentralized network you must expose it, because every operator is a separate P&L and every operator routes for their own margin.
Which is why memory-adjacent names in the DePIN basket have repriced harder than the compute names. The market worked out, correctly, that the scarce asset is not the FLOP. It is the place to put the state.
Watch what operators do rather than what protocols say. When a network starts charging separately for cache reads โ and one will โ that is the tell that the compute layer has quietly admitted where its costs actually live.
Anti-distillation is anti-audit in a different jacket
There is a second control point and it gets the least airtime.
Any serious model owner who lets a network serve their weights fears exactly one thing: paired input-output data is a distillation set. Hand a competitor enough traces and they train a cheap clone. The countermeasures are predictable โ encrypted weights inside an enclave, gated access, no raw log export, no visible reasoning trace.
Every one of those is defensible commercially. Every one also removes an audit surface. The trace you hide from a competitor is the same trace a safety researcher needs in order to detect deceptive behavior in an agent. We have lived this in DeFi. Privacy tooling and obfuscation tooling are structurally identical technology. The code does not know your intent, and neither does the settlement layer.
The blind spot: this sector is using trust-minimized rails to deliver a trust-maximized black box, then printing "verifiable" on the marketing page.
That holds while it works. It becomes a liability the first time a regulated buyer โ a bank, a fund, an insurer with procurement and audit obligations โ asks for the metering trail. They will not ask about throughput. They will ask whether the invoice reconciles against a deterministic record. In most of this sector, right now, it does not.
Who bleeds in this version of the tape
Bear markets do not kill narratives evenly. They sort them by cash flow. After sixteen years of this I ignore which project has the best documentation and ask which one stops paying its own bills first.
Operators on older silicon are bleeding already. Depreciation does not pause for a sentiment cycle and utilization does not recover on a roadmap. Networks subsidizing supply with emissions are not selling inference; they are selling tokens with extra steps. When emissions deflate โ and they always deflate โ the supply layer does not shrink gracefully. It exits, and the utilization chart that looked like adoption flattens into a line.
Buyers will not fix this either. What an enterprise buyer actually wants is a number they can put in a budget and defend in a board meeting. Volatility in unit cost is not a feature. It is a reason to sign reserved capacity with a hyperscaler and eat the premium.
And for anyone holding these tokens rather than trading them, the question is not the narrative. It is what the staking contract actually secures. If slashing depends on an unverifiable metering function, the bond is collateral posted against a claim that cannot be adjudicated โ a token with an insurance premium attached and no claims process. I have seen how that unwinds. The exit is not gradual.
And the quiet part about exit liquidity, because it decides who survives. Liquidity is the only truth in a thin book. Long-tail compute and inference tokens routinely reprice 40 to 60 percent on volume that would not fill a single mid-cap order for five minutes. That price is not a valuation. It is a quote on a market that is not open. If your thesis needs a token to hold a level for a quarter, check depth before you check the chart.
Alpha isn't announced. It's hunted in the noise.
The contrarian read
Here is what I think this sector has backwards.
The pitch is that verification is the moat. It is not. Verification is a commodity input that converges toward zero margin as it matures โ that is what happens to every proven, reproducible cost center in financial history. The moat is the meter: an objective, auditable, on-chain unit of inference work that both sides are forced to accept. Whoever ships that owns settlement for the next cycle the way a fee market owns blockspace.
Which means the trade is not the road. It is the tollbooth. Cost observability, slashing oracles, metering standards, dispute resolution โ that is where the pricing power sits, and almost none of it is priced into a compute token's fully diluted value.
The second read is less comfortable. Perfect verification is self-cancelling. Make compute genuinely provable, portable, and fungible and you have built a utility. Utilities do not carry 40x multiples. A token's only defensible claim on cash flow is scarcity of the meter, not scarcity of the machine. Most of these networks have the relationship inverted.
And volatility is the tax you pay for entry, not exit. Every builder committing to a metered inference stack is locking in unit-cost variance before they know their traffic curve. That is a hedging problem wearing a procurement costume, and almost nobody on the buy side treats it that way.
Meanwhile, panic is just a mispriced option on volatility. If the top inference networks cut emissions in the same quarter, that is not a signal to sell into. It is the moment cost of serving finally clears against real usage, and the survivors get cheap for the first time in three years. When that print shows up I want my list written and my sizing ready โ not my thesis rewritten at 3 a.m.
What I'm watching
The loudest signal would be a metering standard a third party can independently reproduce. Not a benchmark. Not a leaderboard. A unit of work that two operators, given the same request, must agree on. That document is worth more than the next five GPU deployments combined.
The quieter one is whether price-per-million-tokens starts arriving with a cost-to-serve figure attached. The gap between those two numbers is the entire business, and in a bear market the gap is what closes โ not the headline.
The consequential one is behavioral. Whether a regulated buyer with a real audit obligation signs a material contract with an inference network that refuses to expose its metering. If that happens, verification is a market. If it does not, this sector has a token problem, not a technology problem.
In 2020 I closed out of a money-market pool in under four minutes by reading the contract, not the community. I did not wait for a statement. Same reflex applies here. Metering is the contract.
If you cannot price a unit, you do not have a market. You have a subscription with extra steps.
So which one are you holding?