Keeping an AI system running is not the same as keeping it accountable

Jodi Schiller  |  Signal Over Noise

Temporal has announced a $550 million Series E at a $12.55 billion valuation. Its argument is straightforward: AI companies can produce impressive agent demonstrations quickly, but the durable advantage lies in turning those demonstrations into products customers can trust. As AI moves from an interface into the plumbing that handles payments, onboarding, fulfillment, and other consequential work, the systems underneath it must keep running even when processes take days, dependencies fail, or infrastructure changes.

Temporal calls its answer Durable Execution. Developers write ordinary code using the languages, models, and tools appropriate to the task; Temporal keeps that work progressing reliably across systems for as long as necessary. It is an important infrastructure problem, and the company’s growth shows how urgently the market recognizes it.

But durable execution is only half of durability. A system can run flawlessly and still make the wrong decision, conceal disagreement, exceed its authority, mishandle confidential information, or produce an outcome no human can meaningfully challenge. Reliability answers whether the machinery continues. Governance answers whether it should continue, under whose authority, with what evidence, and with what path to correction.

The demo was never the product

For several years, much of the AI market has confused model capability with product readiness. A fluent response feels like intelligence. A successful demonstration feels like a finished system. A prototype completes a task once, and the organization assumes the task has been automated.

The prompt is not the product. The product is the complete information architecture around the prompt: the sources the system may use, the permissions it receives, the decisions it can make, the states it must preserve, the humans who can intervene, and the evidence available when something goes wrong.

This becomes obvious when AI moves into consequential workflows. An agent that drafts a marketing caption can fail cheaply. An agent that touches a payment, changes a customer record, summarizes a legal file, triages a medical message, or initiates a fulfillment action can create damage at machine speed. The relevant question is no longer whether the model can perform the task. It is whether the surrounding system can contain uncertainty, detect disagreement, and recover without erasing accountability.

Execution can be durable while judgment remains brittle

Traditional software reliability focuses on execution: retries, timeouts, state persistence, observability, and recovery from infrastructure failure. AI introduces a second failure domain. The system may complete every technical step and still misunderstand the situation. It may be confidently wrong. It may select a plausible answer where the evidence supports several. It may optimize for completion when the correct action is escalation.

That is why human in the loop is not enough. The phrase often functions as a ceremonial safeguard: a person is somewhere near the process, so the process is presumed accountable. But a human cannot provide meaningful oversight if the interface hides uncertainty, compresses conflicting evidence into a single recommendation, or presents a finished answer without showing how it was produced.

A governed workflow must preserve the conditions required for judgment. That includes provenance, permissions, disagreement, reversibility, and a record of what the system did. The human role must be designed, not merely asserted.

What durable governance requires

Durable governance begins before implementation, with workflow archaeology: examining how work actually happens rather than automating the official description of it. Every consequential process contains informal decisions, tacit knowledge, exceptions, workarounds, and power relationships. If those are not surfaced, automation does not eliminate them. It buries them inside a system that operates faster and is harder to question.

From that excavation, organizations can establish a practical constitution for the system:

• Bounded authority. The agent has explicit permissions and equally explicit prohibitions. It cannot quietly expand its role because a next action appears useful.

• Machine Side Cross Review. A consequential output can be examined by a second model or method rather than trusting one generative path. Agreement is evidence, not proof; disagreement is a signal requiring attention.

• Preserved disagreement. When models, sources, or reviewers diverge, the system records the divergence instead of collapsing it into artificial consensus.

• Publication checks. Before an output leaves the protected workflow or triggers an external action, the system verifies confidentiality, factual support, authorization, and the appropriate level of human approval.

• A Sovereign Witness Log. The system retains a comprehensible account of inputs, decisions, interventions, and outcomes so that affected people can reconstruct and challenge what happened.

• Outcome telemetry. The organization measures not only whether tasks completed, but whether the results were correct, contestable, safe, and useful over time.

None of these controls replaces durable technical infrastructure. They make that infrastructure worthy of trust. If execution persists across failures, governance must persist across handoffs, model changes, staff turnover, and pressure to move faster.

Build the constitution before the city

The industry’s current sequence is often backward. Companies build an agent, connect it to real systems, celebrate the productivity gain, and add governance after the first serious failure. By then, permissions have spread, undocumented practices have hardened, and nobody wants to slow a workflow the business has begun to depend on.

Governance cannot be a policy document attached after deployment. It must be expressed in the workflow itself: in access boundaries, escalation paths, review interfaces, retained evidence, and the right to stop or reverse an action. The constitution must exist before the city becomes too large to govern.

Temporal’s announcement is a sign of the market’s maturation. The industry is moving beyond the spectacle of generation toward the difficult work of dependable systems. That is progress. But as AI reaches deeper into the operations that move money and shape people’s lives, dependability must mean more than uninterrupted execution.

The next generation of trusted AI products will need two kinds of durability. Their processes must survive technical failure, and their governance must survive organizational pressure. They must remember not only where the workflow stopped, but who authorized it, what evidence it used, where disagreement emerged, and whether a human still had a meaningful choice.

Durable execution keeps the machinery alive. Durable governance keeps the machinery answerable to people. We will need both.