At 09:41 on a Tuesday morning, a document landed in my inbox with nine numbered section headers, four comparison tables, a six-row risk matrix, a five-star valuation grid, a glossary, and a legal disclaimer. It contained no facts.
I read it twice. The first pass was the way anyone reads a research product: skim the headers, find the conclusion, check the rating. The second pass was the way an auditor reads it: locate the evidence, trace each claim to its source, verify the arithmetic.
There was nothing to trace. The upstream extraction stage — the part of the pipeline responsible for pulling discrete, verifiable information points out of a source article — had returned an empty array. Not a failed request. Not a timeout. Not a 502. A successful call that produced a list of length zero. The retrieval layer handed the analytical layer nothing, and the analytical layer, being a well-behaved function of its inputs, produced a document in which every single assessment cell read the same four characters: N/A.
The pipeline did not hallucinate. That is the news.
Three thousand and sixty words of framework. Zero information points. One correct refusal. The ratio is the story.
Anyone who has spent a decade in this industry watching confident prose get manufactured out of press releases and Telegram screenshots should find that refusal remarkable. It is also, I will argue, insufficient. A document that says nothing in the visual grammar of a document that says something is not a neutral object. It is a liability with good typography.
The Pipeline That Decomposes Before It Thinks
The architecture that produced this artifact is not unusual. It is now the dominant pattern in automated research tooling, and it has been for about three years. You split the work into two stages because a single-stage prompt that reads an article and emits an analysis tends to conflate extraction with interpretation — it will smuggle an opinion into a fact, then cite the fact as justification for the opinion.
The two-stage design is a control. Stage one reads the source and emits a structured record: title, source, article type, domain tags, core thesis, a list of information points, identified protocols, time sensitivity, source quality. Stage two takes that record and runs it against a fixed analytical framework — technical architecture, token economics, market positioning, ecosystem dependencies, regulatory exposure, team and governance, risk matrix, narrative and expectation gaps, supply-chain transmission.
I have built three of these. The first, in 2021, for a DAO that wanted to summarize governance proposals for delegates who would not read forty pages of Solidity-adjacent specification. The third, in 2026, for a system that governed AI agents executing on-chain transactions, where the extraction record itself had to be citable in a dispute.
Every one of them had the same structural weakness, and it is not the model. It is the validation boundary.
When stage one returns a payload, the pipeline checks it against a schema. The schema validates shape: these fields exist, these types match, this string is under this length, this enum is one of these values. An empty array satisfies the shape. "information_points": [] is a perfectly legal JSON document. It is also a semantically null document, and the validation layer has no opinion about semantics because semantics is not what schemas do.
So the request proceeds. Stage two receives a record with a null information list, a null project name, a null publication date, and a null source classification. It runs the framework anyway, because the framework is a template and a template does not require a subject. Nine headers. Nine sets of empty rows. Nine conclusions that read, in slightly different phrasings, "insufficient information to assess."
The fix is one line and nobody ships it. "minItems": 1 on the information point array, plus a short-circuit that returns a 422 with the message "extraction produced no usable content" instead of a 200 with a report. One line. My own first pipeline ran for eleven weeks before I added it, and I only added it because a delegate asked me why the summary of a three-thousand-word proposal was nine hundred words long and did not name the proposal once.
The failure was not that the model invented facts. The failure was that nothing in the system was authorized to say the word "stop."
N/A Is Not a Risk Rating
Here is where the null report did something genuinely correct, and I want to be precise about why it matters, because the distinction it preserved is the same distinction that separates a real audit from a compliance theater exercise.
The document could have written, in the risk matrix, "no material risks identified." That sentence is a conclusion. It asserts that a search occurred, that the search had a defined scope, that the scope was examined, and that nothing was found within it. It carries the implicit claim that the examiner was competent and the evidence was available.
The report instead wrote "N/A - information insufficient," and in the integrated assessment it went further, stating explicitly that under conditions of zero input, "no risk" and "extreme risk" are equally unprovable, and that it was deliberately declining to return a low-risk verdict because an empty input is easily misread as an absence of problems.
That is an auditor's instinct. In traditional assurance work the distinction is foundational and it is written into the standards. "No evidence of material misstatement" is not the same statement as "no material misstatement exists." The first describes the limits of your procedure. The second describes the world. Only one of them is something you can sign.
When I ran risk guideline revisions for an infrastructure protocol through the 2022 collapse, the most useful document I produced that year was not a risk assessment. It was a two-page list of the things we could not assess because the data did not exist — validator slashing history across three counterparty chains, the true composition of a bridged asset's backing, the counterparty exposure of a market maker whose books we could see only through the venue. Naming the blind spots did more to stabilize delegate confidence than any dashboard we shipped. Delegates had been assuming the dashboard covered everything. It covered what it covered. Saying so out loud was the entire value.
An unassessable item and a cleared item look identical in a table cell rendered in eight-point gray type. The governance layer has to disambiguate them, because no reader will.
This is the part of the null report I would hold up in a design review. It refused to convert absence of evidence into evidence of absence. It refused, specifically and in writing, with a reason.
Confidence Scores Are Not Evidence
Now the part I would reject.
The report attached a confidence label — high — to each of its "cannot be determined" conclusions. "Cannot infer. [Confidence: high (this is a definitive conclusion)]."
I understand the reasoning. The author of that clause was trying to say: I am highly confident that I cannot know this. Logically defensible. Operationally catastrophic.
A confidence score is a scalar attached to a claim, and readers weight it by position, not by semantics. A table in which nine conclusions each carry a high-confidence tag reads, to a skimming delegate, as a table of nine well-supported findings. The fact that the findings are all statements about the examiner's own ignorance is a subtlety that survives approximately four seconds of attention.
I have watched this exact mechanism operate in DAO governance, and it is the reason I rewrote our proposal template in 2020. Before the rewrite, proposals arrived with a section headed "Risks" that was routinely filled with the phrase "low risk" and no supporting analysis. Delegates read the word and moved on. After the rewrite, that section required three things per risk: the mechanism by which the risk materializes, the on-chain or contractual evidence that the mechanism exists, and the mitigation already implemented with a pointer to where it can be verified. If an author could not supply all three, the field was required to read "NOT ASSESSED" in capitals, and the proposal could not advance to a vote until a delegate explicitly accepted the unassessed item.
Turnout rose 40 percent across the first three votes under that template. Not because delegates had more information. Because the information they had was honest about its own boundaries, and honesty is legible in a way that hedging is not.
A notation that cannot be distinguished from a zero will be read as a zero. Every time. By everyone.
So: if a report cannot be assessed, the correct rendering is not a gray dash in a populated table. It is a table that has been removed, replaced by a sentence in the body text that a human has to read. Formatting is a claim.
What a Verifiable Refusal Looks Like
In 2026 I designed the governance layer for a set of AI agents authorized to execute treasury operations inside a DAO. The agents held signing authority on a Gnosis Safe with a per-transaction cap and a per-epoch cap. Every action they took had to be reconstructable after the fact by a human overseer who had not been present.
The design constraint that shaped everything was this: an agent that does nothing must be as auditable as an agent that does something. The failure mode we feared was not a bad trade. It was a quiet quarter in which no trades occurred and nobody could establish why — whether the agent had evaluated and declined, or had simply failed to evaluate at all.
We solved it with a four-part record written for every decision epoch, including empty ones. First, an input hash: the SHA-256 of the exact state snapshot the agent reasoned over, so the input is frozen and cannot be retroactively reinterpreted. Second, an extraction record: the discrete facts the agent pulled from that state, each with a span pointer back to the byte range in the source. Third, an inference trace: the rule identifiers the agent applied, with the thresholds that fired or failed to fire. Fourth, an output claim: the action taken, with every claim in the accompanying rationale carrying a provenance pointer to one of the extracted facts. A claim with no pointer fails validation and the epoch is marked incomplete.
That last check is where the null report would have passed my system and most pipelines would have failed. A refusal is an output claim. It gets a rationale. The rationale is "extraction array length zero, threshold not met, escalation triggered." It gets a provenance pointer to the extraction record, which is itself hashed and timestamped. Six months later, a delegate asking "why did the agent do nothing in March" gets an answer that does not depend on anyone's memory.
Cost of the provenance layer: roughly 14 percent overhead on token budget and about 90 milliseconds per epoch in signing and logging. On a treasury managing eight figures, that is not a cost. It is the price of being able to defend the treasury in a forum where the alternative is a delegate with a screenshot and a grudge.
Code is the only law that holds. But code that cannot produce a record of its own restraint is code that will be trusted exactly once.
The null report contained the seed of this. Each N/A carried a line reading "basis: information point list is empty." That is a provenance pointer. It is one hop deep where it should be traceable, and it points to a null object rather than a source document, but the instinct was right. The system knew what it did not know and it wrote down why.
The Bear Market Makes This Worse
I want to place this artifact in its market context, because the economics of a bear market are what turn a schema-validation oversight into a systemic problem.
When budgets contract, research headcount goes first. The analyst who used to read the filing and mark the ambiguous paragraph with a question is the most expensive line item in the function and the easiest to automate. What replaces her is a pipeline. The pipeline is measured on throughput, latency, and cost per report. Nobody puts "refusal rate" on the dashboard, because a refusal looks like a failed request and a failed request looks like a bug.
So the optimization pressure points one direction. A stage-two model that fills in blanks with plausible prose returns a complete document, passes the shape check, and scores well on every metric the operator tracks. A stage-two model that returns a 422 scores zero on all of them. The operator, who is not an auditor, sees a pipeline with a 97 percent success rate and one with a 71 percent success rate, and makes the obvious choice.
I have seen this in DeFi specifically. Through the 2022 drawdown, the number of risk teams at mid-cap protocols fell by roughly half, and the number of automated monitoring subscriptions rose. The monitoring tools were good at what they measured. What they measured was what had been instrumented. On-chain liquidity, oracle deviation, collateral ratios — all instrumented. Governance capture, key-holder concentration, the social durability of a liquidity commitment — not instrumented, therefore not measured, therefore, in the weekly report, not present. Every week the report said the protocol was fine, because every week the protocol's instrumented surface was fine. The report was not lying. It was reporting on a subset and rendering the subset as the whole.
That is the same failure as the null report, at a larger scale and with worse consequences. The null report at least announced its emptiness. The weekly risk dashboard announced nothing and was believed.
Survival in a contraction is not a function of how much you monitor. It is a function of whether you know what your monitors cannot see, and whether anyone is accountable for the gap.
The Contrarian Reading: The Refusal Was Not Enough
The consensus reaction to an artifact like this, from anyone who has thought about the problem for five minutes, is relief. The model did not hallucinate. Good model. The system degraded gracefully. Ship it.
I think that reaction mislocates the risk so completely that it produces a false sense of security, and I want to be blunt about why.
Hallucination is the failure mode everyone fears because it is checkable. A fabricated sentence can be compared against a source. A fabricated number can be recomputed. A fabricated quote can be found or not found. The entire apparatus of fact-checking exists because hallucination is falsifiable, and falsifiable failures are the manageable kind. They cost you a retraction and some credibility.
The null report is not falsifiable in the same way. There is no false sentence in it. Nothing in it can be contradicted, because nothing in it asserts anything about the world. And yet it arrived with nine section headers, four tables, a risk matrix with a six-row structure, a five-star rating grid, a glossary of technical terms, and a legal disclaimer — the complete visual grammar of institutional authority.
That document will be skimmed by someone with forty tabs open. The skim will register: framework applied, no red flags, rating table present. It will be filed. It may be cited, months later, as "the analysis we ran," by a person who genuinely believes an analysis was run.
The dangerous object is not the confident lie. The confident lie has an adversary. The dangerous object is the confidently formatted emptiness, because its adversary has nothing to hold. You cannot refute a document that never made a claim. You can only notice that it never made one, and noticing requires exactly the kind of close reading that the document's own formatting is designed to make unnecessary.
There is a second problem, and it is a governance problem rather than an epistemic one. The null report closed with a remediation path: supply four items — the article body, a named protocol, a publication date, a source classification — and six of nine analytical dimensions become assessable. That is a sales funnel embedded in a failure notice. It reframes the system's malfunction as a customer input deficiency. The pipeline that returned an empty array is not the party being asked to fix anything.
I have watched this pattern in DAO governance for six years. A proposal fails to reach quorum and the post-mortem blames delegate apathy. A treasury committee approves a grant with no milestone criteria and the retrospective blames the grantee for underdelivering. The mechanism that set the conditions is never in scope, because the mechanism wrote the post-mortem.
Governance isn't a vote. It's a verification. And a system that grades its own malfunction as a user error has already failed the verification it is pretending to perform.
The correct remediation order runs the other way. Fix the validation boundary first — reject empty extraction at the door, return a 422, and log the rejection as a first-class event with an alert. Then, and only then, ask the user for input. The distinction matters because one of these produces a pipeline that cannot silently degrade, and the other produces a pipeline that has learned to describe its degradation as a service offering.
What I Would Build Instead
Three changes, in order of cost, all of which I have implemented and can speak to.
Reject at the boundary. Schema validation is not enough; you need a semantic gate. Minimum information point count, minimum source character length, at least one named entity for any analysis that evaluates a project or protocol. When the gate fails, the pipeline emits a structured rejection with a machine-readable reason code, and that rejection flows into the same monitoring as a completed report. If your success metric cannot distinguish "reported" from "refused to report," you do not have a metric. You have a mood.
Render absence as absence. Kill the N/A convention in tabular output. A cell that could not be assessed should be a cell that does not exist, replaced by prose in the body that the reader must traverse. Tables imply completeness; a table with nine empty rows implies a completed evaluation of nine dimensions. Prose implies nothing and therefore claims nothing. The cosmetic cost is high — reports look less impressive — and the informational gain is the entire point.
Log the refusal with the same rigor as the output. Every rejection gets a hash of the input, the reason code, the threshold that failed, the timestamp, and the version identifiers of the extraction and analysis stages. This is the cheapest of the three and the most valuable over time, because it converts a category of invisible events into a dataset. Six months of refusal logs will tell you whether your retrieval layer is broken 30 percent of the time on a particular source domain, which is a fact you cannot learn from output logs, since the failures produce no output.
I audited a governance tooling vendor in 2023 that had, by its own account, processed 41,000 proposals. When I asked how many had failed extraction, the answer was that they did not track that. They tracked proposals processed, votes summarized, and average latency. Their success rate was, definitionally, 100 percent. That is not a measurement. That is a tautology with a dashboard.
Skepticism is the first line of defense. It is also the only one that has to be present before the first line of code is written, because after that you are auditing rather than designing.
The Question You Should Be Asking Your Pipeline
Somewhere in your stack, right now, there is a function that receives an array of extracted facts and runs a template over it. That function has never been asked what it does when the array is empty. It has an answer. It has always had an answer. You just have not seen it, because in production the array has never been empty for long enough for anyone to look.
Ask it. Run it with nothing. Read the output the way an auditor reads a representation letter — looking for what is asserted, what is omitted, and which of the two the author believes they have done.
If it returns a report, you have a machine that manufactures institutional credibility from air, and it is running on your letterhead. If it returns a rejection, you have a machine. And you will know, which is the only advantage that has ever mattered in this industry, and the one that no amount of formatting can substitute for.
Verify everything, trust nothing. Especially the documents that agree with you, and most especially the ones that are beautifully organized.