Plain Strata logo

Plain Strata

Archives
Listen
Log in
Subscribe
October 10, 2026

What a checker is allowed to say when it could not check

Plain Strata logo Plain Strata

Hello,

Picture the laboratory that reads the form afterwards. A stack of replication reports from strangers, each one a single tick in one of two boxes: reproduces, or does not reproduce. Nothing on the page tells it which ticks came from a chemist who ran the experiment and which came from a chemist whose bench could not do what step four asked.

That laboratory is a decentralized AI network. Machines in different buildings, owned by different people, do AI work for strangers, and a class of participant called a checker says whether the work was done properly. The answer is written on a ledger for good, and it moves money. Nobody has settled what the list of possible answers is.

I catch myself thinking this is a coding detail, and it is not. A checker allowed to say only yes or no will, every time it cannot check something, file a lie about somebody else, often about an honest worker. The engineering that produces that lie is about four lines long.

In the episode the hosts add a bouncer to the story: a man at a club door who knows a dozen kinds of fake ID and is handed a passport from a country he has never seen. Real or forged, he cannot say. So here is the question I keep: what is a machine allowed to write down when it could not check, and who reads it?

Listen:

Apple Podcasts: https://podcasts.apple.com/kg/podcast/plain-strata/id6783455764?i=1000794601547

YouTube: https://youtu.be/nItEH7oRDh4?si=F7CiguX5spUDXGyT


The full piece, no need to click through:

A chemist in a second laboratory has a sheet of paper in front of her and a bench behind her, and nothing else. The paper came from another city: twelve steps, each naming a substance, a mass, a temperature and a duration. Her job is to run it and say whether the published result holds up. Steps one, two and three go fine. Step four tells her to weigh out five hundredths of a gram of a catalyst. Her balance reads to one decimal place. The number she needs sits below the resolution of the instrument she owns.

So what does she write?

She has three sentences available and they are not interchangeable. She can write that step four reproduces. She can write that step four does not reproduce. Or she can write that she could not follow step four at all. The first two are facts about the paper. The third is a fact about her bench, and it is the one everybody forgets, because it looks like a failure and nobody likes writing failures down.

Now watch what happens if the form she has to fill in has only two boxes. She has to tick one. Ticking "does not reproduce" accuses a paper she never tested. Ticking "reproduces" certifies a paper she never tested. Both are lies pointing in opposite directions, and the laboratory reading her form afterwards cannot tell which lie it received.

That is the episode, and the spine is one sentence. A checker allowed to say only yes or no will, every time it cannot actually check something, file a lie about somebody else. So the vocabulary a verification system is allowed to answer in matters more than the cleverness of the instrument doing the checking, and the whole field has spent three years building instruments and almost no time on the vocabulary.

This is worth half an hour because of where it sits. There is a ledger, which is a shared spreadsheet that nobody is allowed to lie to: it holds no models, only records of who did work and who got paid. Around it sit machines in different buildings owned by different people doing AI work for strangers, and a class of participant whose job is to say whether that work was done properly. Every design for checking strangers' work ends in one of those participants emitting an answer, which is written down permanently and moves money. None of those designs says what the set of possible answers is.

Three words carry this.

Validate comes from the Latin validus, meaning strong, sound, able to stand. Not verus, true. A validator is not asking whether a claim is true about the world, it is asking whether the claim stands up when you lean on it, which is a smaller question, and the smallness is the entire design. Keep the body of the word: something that stands, or does not stand, or that you never managed to lean on.

Verdict is vere dictum, truly said. A verdict is a speech act and not a measurement. Somebody says something and the saying is what the system keeps, so everything downstream of a checker consumes a sentence, never an observation.

Abstain is abs tenere, to hold oneself away, and the holding is active. An abstention is a thing you do rather than a thing that happens to you, and it carries information, which is exactly why it needs a box of its own on the form.

Notice what none of those three words is about: correctness. The vocabulary of this layer is the vocabulary of testimony, and testimony has always had three shapes, not two.

Go back to the bench, because the first telling hid something. Suppose step four asks for a mass her balance can perfectly well weigh, and the balance is broken. It drifts. It reads an empty pan as three tenths of a gram, and she knows it. She still writes the same sentence: "I could not follow step four."

The sentence is identical and the fact underneath it is a completely different fact. In the first case it is a limit, a property of a bench never built to weigh that small, and the word is the Latin limes, the marked track running along a boundary between fields, the same word Rome used for its frontier: not a flaw, a line somebody drew on purpose and published on maps. In the second it is a defect, a property of a bench that should work and today does not.

So there are two questions to ask of every outcome, not one. What is this sentence about, the paper or the bench? And if it is about the bench, is it a limit or a defect? Not about blame, which is the standard wrong turn here: both facts are about her bench, only one is anybody's fault, and fault decides nothing. What decides is whether the sentence is about the thing being checked or the thing doing the checking.

That distinction has teeth. A limit can be printed on the report, and whoever reads it can route around it by sending the sample somewhere with a better balance. A defect cannot be printed in the same voice, and the reason has nothing to do with embarrassment: a laboratory allowed to file "my equipment was broken" as a result has found a way to decline any work it dislikes, collect its fee, and never be caught, because catching it means proving something about the inside of somebody else's machine.

Now put the chemist on a computer and watch each piece of the scene turn into a piece of a system.

The paper becomes a trace: a record of what a program claims it did, step by step, with inputs and outputs named. For a piece of software acting on its own behalf, a sequence of tool calls, their arguments, and what came back. For a model answering a question, an identifier for the model, a commitment to the exact numbers it was running, an input and an output.

The second bench becomes a verification engine: something that takes a trace and walks it again. Depending on which way you have chosen to check strangers' work, that is an interpreter holding a copy of the same numbers, or a challenger's own hardware, or a mathematical checking circuit, or the code that inspects a signed statement from a chip.

The balance's resolution becomes a guard. Every engine has a set of things it knows how to check and a set of things it does not, and each checkable thing is a guard: a question the engine can actually answer. Did the output fingerprint match. Was this call inside the declared budget. Is this timestamp inside the window. The set of guards an engine implements is its resolution: finite, always smaller than everything, and a fact about the engine rather than about the work.

Here is the sentence to carry. The three outcomes are the three ways a single guard can end. A guard can hold. A guard can fail. Or a guard can turn out to be unevaluable. The first two are facts about the trace. The third is a fact about the engine, and the reason it keeps getting lost is that it is usually filed in a list beside two things that are nothing like it.

The laboratory on the other end becomes whoever consumes the sentence. On a chain that is a registry: a place where a checker's answer is recorded permanently by something that re-runs nothing and takes no view. The chain here is the notary, not the judge. It certifies who said what, when, unaltered. So the question of what a checker may say is identical to the question of what the registry's form permits, and a form with two boxes does not make the third situation go away. It makes it get recorded as one of the other two.

That is the whole argument in one line. A missing outcome does not produce silence. It produces a misfiled outcome. The chemist still has to write something.

The collapse happens in about four lines of ordinary engineering, so it is worth going all the way into them once.

Take one recorded step and one guard. The engine's job is to run that guard against the values the trace recorded, and physically it does two separate things in sequence, which almost everybody writes as one.

First it matches the recorded values to what the guard asks for. The guard is a small function expecting named inputs, and the trace carries a bag of names and values written by whoever produced it, so the engine binds one to the other: every name the guard needs present, no extra name handed over. That matching looks only at names. It inspects no values and runs no part of the guard.

Then, and only then, it calls the guard, which does the actual work.

Now consider three completely different things that can go wrong, and notice that all three can arrive as the same kind of error inside the same wrapper. The trace could have named a value wrongly, because the party under scrutiny renamed a field, or refactored honestly, or forged the record on purpose. The guard the trace points at could be missing from this engine's set entirely. Or the engine's own table of guards could contain something that is not a function at all, because somebody built it wrong this morning.

Wrap all of that in one error catch and report one word, and three causes produce one signal. The first is a fact about the trace. The second is a limit of the engine, structural and publishable. The third is a defect inside the engine and must never reach the report. Flattened together, they become a single accusation, and the accusation points at the party under scrutiny in all three cases, because that is who the report is about.

The fix is the whole episode in miniature. Keep the matching step outside the wrapper so a failure to match is reported as its own thing, and keep the wrapper around the guard call as narrow as it can possibly be. Then the width of the error catch stops being a question of style and becomes what it always was: a decision about who gets blamed. That is why the third verdict matters. The engineering that produces it is this small, and the lie that replaces it is this cheap.

Valid means every guard the engine was asked to evaluate held, and the scope of that claim is exactly the guard set, never one millimetre wider. It means nothing I know how to look for was wrong, which is not nothing was wrong. Most designs drop the resolution behind the word, which is how a weak checker and a thorough one come to publish the same verdict. Invalid means a guard was evaluated and failed, and it should name which one, because a reader who cannot tell whether the dispute is about the output or about a timestamp cannot act on it.

Unevaluable means nothing was learned about the work at all, and it must never be folded into either neighbour. Fold it into invalid, and every weakness in the checking becomes an accusation against honest workers, who then start avoiding anything the checker does not understand, at which point the verification layer has quietly become a standards body nobody elected. Fold it into valid, and "I could not check" has been recorded as "it checked out", which is the cheapest attack on any verification system ever built: do something the checker does not understand.

So the correct treatment of the third answer is neither acceptance nor rejection, it is routing: send the work to an engine with a different guard set, raise the number of independent checkers, or mark the claim uncovered and let whoever is paying price that. The limit-versus-defect axis finishes the rule in one sentence: a limit is a verdict, a defect is a non-response. A broken engine's correct public behaviour is to say nothing and let the timeout handle its absence, which closes the refusal valve by construction rather than by policy.

This pattern has a name and it shows up everywhere once you have it: the third verdict. Any judgment made about a partly observable world needs an answer meaning "I did not reach this", and a system that refuses to provide one gets it anyway, in the form of a bug.

Databases learned it hardest. The value a database uses for a missing entry is not false, it is unknown, and generations of quietly wrong reports come from treating the two as the same thing. Courts have had it for centuries: guilty, not guilty, the Scottish verdict of not proven, and the hung jury as a separate object again. English law's "not guilty" already means the case did not stand rather than the defendant did not do it, and that almost everybody misreads it tells you how strong the pull toward two values is. Medicine reports positive, negative and indeterminate, and the protocol for the third is re-collect the sample, which is routing. A laboratory filing indeterminates as negatives would be committing fraud, and nobody would call that a rounding choice. Accountants keep a fourth opinion beyond the usual three, the disclaimer of opinion: we could not obtain enough evidence to form a view, which is among the most useful things an accountant can publish and says nothing whatsoever about the accounts.

In every one of those the third answer comes paired with a protocol for what happens next, which is the half verification designs keep leaving out.

There is a real tension at the bottom of this, and it has no clean answer. A checker that reports its third verdict honestly is publishing its own limits, continuously, in a form machines can read, and a complete record of what your engine cannot check is also a map of where to attack you. The honest checker's output stream is a vulnerability disclosure, filed hourly, by the victim.

The obvious escape is to hide the guard set, and it does not work: a verdict is interpretable only if you know the resolution behind it, so hiding the resolution destroys the meaning of the word valid and nobody can read any of your answers. Hold both halves. The thing that makes a verdict legible is the thing that makes its issuer attackable, and every verification design in this field is already sitting on one side of that trade without having said which.

One scene, with every piece of this in it. A piece of software runs a job inside a sandbox. Underneath it, on a separate processor on the same machine, sits a watchdog of the kind Nvidia announced on 28 September 2026, which keeps enforcing policy after the host itself has been taken over. It exists because of exactly this distinction. A supervisor running on the host cannot tell "I found no violation" from "I was not running", and on a compromised host those two look identical from outside, so the host's own report carries no information at all. Putting the watcher on another chip is an attempt to make the engine's state reportable by something other than the engine: the limit-versus-defect axis built out of silicon.

The job produces a trace and wants paying. Run it past each of the four ways this field checks work, and the three answers appear every time.

Re-running it: load the state before a step, do the step again, fingerprint the result, compare. Gensyn published a model on 15 September 2026 with a fingerprint for every one of 80,957 training steps so strangers could do exactly this, at about five times the normal cost in speed. Match is valid, mismatch is invalid, and a step that cannot be replayed because the arithmetic was never pinned down is unevaluable, a permanent limit, and the reason that tool cannot audit a model built any other way.

Challenging it: accept everything at once and keep a funded window open in which somebody can object. No objection is valid, bounded by whether anybody with money at stake was awake. A successful challenge is invalid. But a challenger whose own re-run disagrees in the last decimals for reasons of arithmetic rather than dishonesty holds an unevaluable with nowhere to put it, and a protocol with two outcomes forces that into an accusation against an honest worker.

Proving it: a mathematical checker takes a small file and confirms the computation was performed correctly. Valid and invalid are crisp here, and so is the third answer, which is published as a document: the list of operations the proving system does not support. A model using one of them is unevaluable by that instrument forever, and the honest version of the product ships the list.

Sealing it: hardware signs a statement about what it is running, checked against the manufacturer's chain. A good statement is valid, scoped to exactly what was measured. A mismatch is invalid. A statement covering the main processor while the work happened on an accelerator it says nothing about is unevaluable, structural, and worth publishing loudly. An architecture presented at Korea Blockchain Week on 30 September 2026 put this instrument in its verifiable-compute slot, and the side-channel work published earlier this year says the same seal leaks the order in which memory was touched, so there is a class of question about that trace it has no guard for at all.

Then the answer is filed. On a chain that is a registry in three parts: who this is, how it has behaved, and what independent checks were run on it. The third part is a socket rather than a method, request in and answer out, and the specification deliberately does not say what fills it. What this episode adds is that whatever fills it cannot be a yes-or-no, and that the registry's form, rather than the checker's code, is where the third verdict lives or dies.

Two questions, both answerable. If a checker publishes a fingerprint of its own guard set so its verdicts carry their resolution, who checks that the engine really implements that set? The same problem one level up, and the usual answer is a deposit, which hands the question back to the economics.

And if five independent checkers each return one of three answers, what does the group return? Majority voting over three values is not obvious, and whether two unevaluables plus one invalid plus two valids is an accusation or a routing decision is a design nobody has published. The effort has gone into making individual checkers cleverer. The arithmetic of combining what they say is where the next real result comes from.


The two voices are AI. The research and writing are mine.

Decentralized AI, layer by layer.

Dastan,

Listen on Spotify and Apple. @plainstrata. Decentralized AI, layer by layer.

You just read issue #27 of Plain Strata. You can also browse the full archives of this newsletter.

← Newer Two honest millers, two seals that do not match Older → Who actually pays for the cheapest AI in the world
Spotify
Powered by Buttondown, the easiest way to start and grow your newsletter.