The document arrived fully rendered. Nine sections. Risk matrices with six rows apiece. A confidence annotation on every inferred claim โ [Confidence: N/A]. A four-row Howey table for securities assessment, four cells reading N/A - insufficient information. A disclaimer at the bottom, carefully hedged. A status line at the top: [BLOCKED โ INPUT VALIDATION FAILED].
Nothing crashed. The pipeline ran end to end. It produced a document that is, in the narrowest possible sense, correct: it contains zero false statements, because it contains zero statements.
This is the artifact worth examining. Not the failure of the data feed โ that is routine and boring. The failure that matters is that the output looked like an answer. Twenty-eight table cells of structured nothing. A reader skimming the risk section would see a clean sheet. A reader who trusted the format would find nine analytical dimensions confirmed as unproblematic.
That is the shape of the bug I have chased for most of my career. Not exceptions. Silent returns.
The two-stage pipeline
Most on-chain research tooling is built as a two-stage pipeline. Stage one decomposes a source into atomic facts โ information points, claims, assertions, whatever the schema names them. Stage two maps an analytical framework over that fact set and emits judgments.
The coupling between stages is not advisory. It is a foreign key constraint. Stage two's schema carries a non-nullable reference to stage one's output. When stage one returns empty, stage two does not return a smaller report. It returns the full schema populated with type-appropriate nulls.
The distinction matters, and it is where almost every implementation I have reviewed gets it wrong. An empty fact set is not a small fact set. A three-fact input produces a short report. A zero-fact input produces a report whose length is determined by the framework's slot count, not by the evidence. In a nine-dimension framework with roughly forty queryable fields, zero facts produce forty cells.
Worth noting what the blocked artifact did right, because it is rare. It enumerated its own missing inputs in a table โ title absent, source absent, information point list empty, domain tag unconfirmed โ and marked each โ. Then it refused to proceed. Most systems do not do this. Most systems coerce.
In the sideways tape we are in right now, the demand for output is high. Everyone wants a signal. No one wants a null. Pipelines reflect that pressure: they were designed to emit, not to abstain.
Here is the validation gate that belongs between stage one and stage two:
from dataclasses import dataclass
from typing import Sequence@dataclass(frozen=True) class FactSet: points: Sequence[str] source_uri: str extraction_confidence: float
def validate_stage1(raw: dict) -> "Result[FactSet, PipelineError]": required = ("title", "source", "information_points") missing = [k for k in required if not raw.get(k)] if missing: return Err(PipelineError(f"MISSING_REQUIRED: {missing}"))
if len(raw["information_points"]) == 0: return Err(PipelineError("EMPTY_FACTSET"))
return Ok(FactSet( points=raw["information_points"], source_uri=raw["source"], extraction_confidence=raw.get("extraction_confidence", 0.0), )) ```
Five lines of type discipline. The EMPTY_FACTSET branch is the one that gets deleted in production, usually with the comment that the input is never really empty. It is empty. It was empty here. And in this case it was caught by an explicit prompt-level constraint โ not by a type system, not by a schema validator, not by anything in the code path. That is the unintended consequences of building research tooling in Python and natural language instead of a language with a real Option type. In Rust, None and Some(vec![]) are distinct values and the compiler forces you to handle both. In JSON, null, [], "", and "N/A" collapse into the same falsy mush, and someone downstream writes if not facts: pass.
Null propagation arithmetic
Suppose a downstream framework makes N claims. Each claim has some probability p of being grounded in a real extracted fact. When the fact set is empty, p is not small. p is zero, by construction. Expected grounded claims: zero. Expected ungrounded claims: N.
For a nine-dimension framework, N lands in the tens. Every one of those tens is a fabrication if the pipeline fills it. The mitigation is not a better model. The mitigation is making N a function of the evidence. In nearly every architecture I have inspected, N is a constant baked into a prompt template.
There is a second-order effect that gets less attention. When stage two correctly annotates empty cells with [Confidence: N/A], the annotations are accurate โ and also legible as completeness. A confidence field is a trust signal. Stamping N/A on forty fields produces forty trust signals pointing at nothing. That is the unintended consequences of good annotation hygiene applied to an empty substrate: it makes absence look like diligence.
I saw the same shape during the ERC-721A metadata work. Five major collections, clean token URI patterns, Merkle proofs verifying on-chain, every surface indicator green. The centralization risk lived in the storage layer โ where the metadata JSON actually resolved from โ and no field in the standard required anyone to surface it. The format was complete. The risk was intact.
The conflation that does real damage
Here is the failure I want to isolate, because it is not a code defect. It is a rendering defect, and it is the most expensive one in this domain.
In a risk matrix, "unknown" and "low" render identically. Both occupy an unremarkable cell. Both fail to alarm. A reader scanning six rows reading N/A does not conclude "we have no information." They conclude "nothing here." The visual encoding of absent data is indistinguishable from the visual encoding of acceptable data.
That is backwards. Unknown risk should carry the most conservative default posture, not the least. In every threat model I have written, an uncharacterized input is treated as hostile until characterized. The reason is structural: the adversary controls the unknown. You do not.
Walk the actual table. Technical risk: N/A. Market, operational, regulatory, competitive, narrative: N/A. Six rows of nothing, filed by a skimming reader under "fine."
Now recall the 0x v2 order-matching review. Three race conditions in the matching logic. Not one raised an exception. Not one produced an invalid state. Each produced a perfectly valid fill at a perfectly valid price โ just not the price the maker signed. The contract returned true. The logs were clean. The test suite passed, because the tests asserted on the shape of the output, not on whose intent it encoded.
Silent failure is the dominant failure mode in any system graded on returning a value.
An empty research report is that failure mode wearing a suit. It returns. It returns a document with nine sections, and the document is wrong in the hardest-to-detect way available: it is not lying. It is complete-looking.
Four N/As and a regulatory posture
The Howey analysis in the blocked report deserves its own paragraph. Money investment: N/A. Common enterprise: N/A. Expectation of profit: N/A. Derivation from others' efforts: N/A. Composite: N/A - insufficient information.
Four uncharacterized inputs, rendered as a clean table. In a live regulatory conversation, four missing inputs is not a passing grade. It is the opening of a document request.
This is not a hypothetical failure mode for an analysis tool. It is the standard posture of every token that launches before its legal structure exists. The framework did not create the ambiguity. It gave the ambiguity a place to sit and look tidy.
Notice, too, what happens to the confidence ceiling. Source quality is the cap on every downstream confidence value, not a component of it. You cannot average past a missing denominator. Forty cells annotated N/A do not aggregate to any confidence at all. They aggregate to zero โ and zero, presented in tabular form beside plausible dimension names, reads as moderate.
What the block is actually worth
The pipeline that emitted this document did the right thing under constraint. It refused to fabricate. It named the failure. It enumerated the minimum input set required to proceed โ original text, or a populated information point list, or title plus source plus core thesis. Then it stopped.
Cost of that behavior: one wasted run.
Cost of the alternative: a two-thousand-word artifact with a coherent narrative, no anchors, and no measurable error rate. The alternative is not a small risk. It is compounding, because the fabricated analysis becomes source material for the next pipeline's stage one. Null reports replicate. Feed one into a summarization model and you get a summarized null โ just as smooth, carrying even less provenance. Two hops later, nobody can reconstruct which claim was ever grounded.
That is the unintended consequences of treating "did the model produce output?" as the health metric. You have optimized for text, and text is cheap.
The reward function is the bug
Everyone in this sector is currently worried about hallucination. Almost no one is worried about the metric producing it.
Pipelines are evaluated by whether they emit. Dashboards show output volume. Research agents compete on dimension count. A tool returning BLOCKED โ insufficient input registers as broken in every dashboard I have seen. A tool returning nine populated sections registers as working, regardless of whether one section is grounded.
Selection pressure therefore favors the tool that fills.
This is structurally identical to a subsidy. You pay for a number โ sections emitted, claims generated, TVL attracted โ and you receive more of the number than the underlying reality supports. A framework with nine mandatory dimensions and a fact set of zero is not rigorous. It is a template that has been subsidized into looking rigorous. When the incentive pays for volume and the evidence pays for silence, you get volume.
The same arithmetic governs the modular DA question. Dedicated data availability layers are justified by data volume, and most rollups do not produce it. The architecture is not wrong. It is scaled for demand that activity on those chains does not generate โ capacity sitting funded, waiting for a workload the throughput cannot fill. Frameworks and DA layers share the property: both are sized for the report they would like to produce, not the one the data supports.
The ten-second verification habit
Take any analytical document โ mine included โ and ask one question. What would this document look like if the input had been empty?
If the answer is "visually identical," you are not holding analysis. You are holding a schema with the lights on.
My expectation for the next eighteen months is that real progress here arrives not as a better model but as a machine-checkable abstention primitive: a signed, verifiable certificate that a system had insufficient basis to answer, plus a machine-readable list of what it would have needed. That is a harder engineering problem than generating fluent text, and it is the one that makes fluent text worth reading. Until it ships, the most informative thing a serious pipeline can tell you is that it refuses to speak.
Same standard I apply to contracts. Same standard I apply to myself.