I ran across Thore Graepel's article in the MIT Technology Review Don't be fooled, LLM's don't reason, which sounded deliberately provocative. It's worth reading, so hopefully this post's discussion won't discourage a read (it's fairly short as these things go).

Graepel has what I can only characterize as nostalgia for the particular "reasoning structure" in the AlphaGo system that defeated Go champion Lee Sedol. He describes the seemingly paradoxical "move 37," which ultimately proved triumphant. He goes on to describe two modes of human reasoning: one fast and effortless, the other slower and deliberate.

You could colloquially call these two modes intuition and deliberation.

AlphaGo had a hunch and its game-exploration algorithms analyzed that hunch as having good potential to win the game. In Graepel's argument, "reasoning" requires:

• persistent epistemic state;
• explicit manipulation of that state;
• exploration of alternatives;
• evidence-backed belief revision.

His assessment of AlphaGo's hunch ("this may be a good move") plus its powerful tree search ("how will this particular move affect the game?") is that this is closer to true reasoning than an LLM's next-token prediction.

I end up not being persuaded by Graepel's thesis. His condemnation is overly simplistic: "The intermediate reasoning is still produced by the same next-token prediction process..." This is akin to saying a computer is merely a transistor state-transition machine.

Is chain of thought (CoT) reasoning?

A helpful way to think about CoT is that it is a reasoning workspace. Mickens' Linguistic Illegibility thesis and paper (discussed in this post) helps us realize that computation is the key, not the surface description we see as the CoT. A useful position paper in this area is LLM Reasoning Is Latent, Not the Chain of Thought, where Wang argues that LLM reasoning should be studied as latent-state trajectory formation, which I simply call "computation."

The computation is not necessarily the record we visibly observe as CoT.

Despite my disagreement with Graepel's main thesis, he says something I find intriguing:

Reasoning can ... be understood as a sequence of moves that change the epistemic state to advance knowledge and reduce uncertainty: deducing consequences, breaking problems into parts, and—crucially—deciding what question to ask, calculation to perform, or experiment to run next.

This sounds a lot like SMCC!

SMCC does not put Graepel's System 2 (deliberative evaluation) inside the model. It builds System 2 around the model.

SMCC records:

• the problem definition;
• constraints;
• decisions - what architecture has been accepted;
• evidence that has accumulated;
• which tasks depend on which conclusions;
• what the model/agent proposes to change;
• acceptances and rejections;
• and what therefore becomes authoritative for the next reasoning step.

The model does not have to carry all of this through thousands of transient tokens, which should improve software-engineering reasoning, hopefully dramatically.

The all-up reasoning system improves because it directly addresses major failure modes in long-running agentic coding sessions: forgotten premises and constraints, goal drift, contradictory assumptions, accidental re-litigation of previous decisions, stale dependencies, and seemingly reasonable local fixes that violate global constraints.

I have had many conversations with ChatGPT for over two years on agentic safety and security. This has created a strong epistemic substrate. Previous conclusions are summarized, concepts acquire stable names, earlier evidence remains available, new evidence modifies existing conclusions, and, critically, each new conversation starts with a compressed version of much of that accumulated state.

I think it is actively harmful to require LLM reasoning to perfectly reflect or mimic human intelligence or reasoning. It doesn't need to.

But human + model + persistent context is capable of sustained reasoning.

It is possible that, to complete Graepel's notion of epistemic state, we may eventually want to add a hypothesis primitive to SMCC. This will be explicitly assessed as SMCC is pressure-tested against real-world software-engineering problems rather than added simply because it sounds architecturally attractive.

Graepel's AlphaGo structure can be summarized as:

hunch → search → evaluation

A different (and I would argue more powerful) structure is:

human intuition + model intuition → explicit exploration → persistent epistemic state → human/model evaluation → revised state

SMCC, used in a pair-programming or similar setup, is exactly this kind of process.

I call the broader concept paired intellect.

The argument that "models don't reason like humans" seems much less important than the realization that pairing two different kinds of intellect gives us problem-solving power unlike anything we have had available before. The more useful question is whether the system can perform most or all of the operations we associate with higher-order reasoning: generate hypotheses, find and connect evidence, explore alternatives, observe contradictions, make revisions, maintain goals, and solve novel problems.

And by "system," I mean the paired intellect of model and human.

The paired-intellect journey can be messy. Think of how often you have lost an important insight in a long LLM chat conversation or had difficulty retrieving it later. The model can perform lightning-fast advanced analysis while the human applies domain expertise and life experience to establish a conclusion. A process like SMCC can make that result durable in a place where it really matters: agent-assisted coding.

One thing I observe myself doing with coding agents is supplying intuition via interruption. Sometimes it is just something I put into the next-turn prompt. Sometimes I actively interrupt an agentic turn and say, "um, wait..."

This interruption isn't friction. It is a fundamental part of the reasoning architecture.

This may be one of the major reasons I use a pair-programming style of agentic coding. Being "in the loop" keeps me in the flow of every piece of incremental progress and gives me the opportunity (actually, requires me) to inject intuition into the paired reasoning process.

And then the model does this amazing thing that I still marvel at: it rapidly regenerates the local possibility space given the correction.

This loop: two unlike intelligences, persistent state, and correction when necessary, is much more powerful than attempting to force the model into a human-intellect constraint.