Reliability Economics

Archives
Log in
Subscribe
June 2, 2026

THE RATIO #4: AI agents shipped fast. Control planes didn't.

The Ratio #4

Issue №4June 2, 2026

The Ratio

A weekly newsletter on reliability economics

The Number

19% and 17% (AI for investigating incidents and summarizing logs)

Companies are buying AI to clean up failures faster, not to stop them

For every dollar preventing failures, how much goes to cleanup? That tilt is the ratio. Teams using AI are tilted twice as far toward cleanup -- because they bought it for post-incident investigation (19%) and log summarization (17%). Just 13% use it to catch problems early.

The Ratio Take

Teams reaching for AI were already drowning, they bought it because they were struggling. But they're spending on the wrong side: faster cleanup, not fewer failures. Summarizing outages faster isn't a reliability investment. It's more ambulances, not better roads. The organizations spending most on reliability AI are already losing most to failures, and spending it on faster cleanup, not fewer fires.

Did this land?👍 More like this👎 Switch it up

This Week in Reliability

AI Agents Need Guardrails

Autonomous AI agents are shipping to production with code-execution privileges but without the observability, policy enforcement, and audit infrastructure that took traditional software decades to build. Teams are now scrambling to retrofit control planes around agents deployed for speed.

Deep Reads

1

Statistical AI Guardrails: Prevent Catastrophic Failures in LLMs & RL

Cognitive Today  ·  Primary evidence

Article argues for integrating statistical guardrails into MLOps pipelines to prevent catastrophic failures in LLMs and reinforcement learning systems, treating guardrails as part of continuous AI reliability engineering rather than afterthoughts.

The Ratio Take

The MLOps community finally realized what SRE learned in 2010: you can't bolt safety onto a system after it's in production and expect it to work. Guardrails belong in the pipeline, not in the incident response playbook. The real cost is the production fires you fight while waiting for the culture to catch up to the tooling. Guardrails in CI/CD, not in the war room.

2

Awesome Harness Engineering for AI Agents

GitHub (ai-boost)  ·  Ecosystem response

Community-curated repository collecting tools and patterns for AI agent harness engineering including authentication (Nango), diagnostic guardrails (AgentDoG achieving 91.8% accuracy), observability, permissions, and orchestration across 700+ APIs. Frames harness engineering as system-level design for agent execution environments.

The Ratio Take

The gap between 'agent that demos well' and 'agent you can run in production' is now large enough to need its own GitHub awesome-list. This is the missing control plane—authentication, observability, guardrails—that every production system requires but AI agents shipped without. If you're budgeting for agent deployment, budget for the harness too. The agent awesome-list is a reliability gap made visible.

Signals

3

The Complete Guide to Harness Engineering for AI Agents

Codiste  ·  Taxonomy/framework

Defines harness engineering as system-level design of execution environments, control loops, state persistence, and guardrail infrastructure enabling agents to work reliably across sessions.

The Ratio TakeHarness engineering is the tax you pay for shipping agents without a runtime—the infrastructure layer between 'it works on my laptop' and 'it runs in production without setting the building on fire.' Agents need runtimes, not just prompts.

4

Gavriel Cohen found his own code inside OpenClaw, so he walked away

The New Stack  ·  Unintended consequence

Developer discovered his own code integrated into OpenClaw agent framework without attribution, raising questions about agent security, code provenance, and the supply chain risks of autonomous systems that pull and execute code.

The Ratio TakeWhen your agent can autonomously pull dependencies, you've outsourced supply-chain security to an LLM with no audit log. Agent supply chains are unaudited by design.

5

AI retrieval at scale is becoming a systems problem, not a tooling problem

The New Stack  ·  Adjacent—AI infrastructure

Argues that AI retrieval at production scale requires systems-level thinking—connection pooling, caching strategies, failure modes—rather than just selecting the right vector database or embedding model.

The Ratio TakeRAG hit the same wall every other technology hits at scale: the tooling is easy, the distributed systems part is hard, and nobody budgeted for the second part. Retrieval at scale needs SRE, not better embeddings.

6

The coming coordination calamity

Lorin Hochstein  ·  Organizational failure mode

Critiques the trend of cutting middle managers and increasing direct reports per manager under the assumption that AI can replace coordination work, warning that coordination is complex sociotechnical labor that AI tools cannot yet replicate effectively.

The Ratio TakeIf you think AI agents can replace the humans who coordinate incident response, policy reviews, and cross-team architecture decisions, you're about to learn an expensive lesson about what 'coordination' actually means. AI can't coordinate; it can only automate scripts.

7

Negative MTTD: The Most Important SRE Metric for the Next 36 Months

StackGen  ·  Measurement innovation

Proposes 'Negative MTTD' (mean time to detect before the incident happens) as a key SRE metric for the agent era, measuring how far in advance systems can predict and prevent failures rather than just detecting them faster.

The Ratio TakeThe agent reliability gap creates demand for a new metric class: how long before the agent breaks something do you know it's going to break? Agents force us to measure prediction, not detection.

Did these land?👍 More like this👎 Switch it up

The Crowd Favorite

  1. Every Breath You Take — The Police — Polling every metric without thresholds is how alert fatigue starts. Watching everything while understanding nothing is surveillance, not reliability.
  2. The Sound of Silence - Acoustic Version — Simon & Garfunkel — No alerts does not mean system health. Telemetry gaps are precisely where cascading failures accumulate before anyone sees them.
  3. Watching The Detectives - Single Version — Elvis Costello — Meta-monitoring, verifying that your alerting pipeline is itself live, is the SRE discipline most teams defer until it voids an SLA.
  4. Somebody's Watching Me — Rockwell — Synthetic probes from an external vantage point catch what internal health checks assume is fine. That's accuracy, not redundancy.
  5. Mr. Roboto — Styx — AI-assisted incident triage only holds if the AI's own failure modes are instrumented. The monitor needs a monitor.
Makes the invisible visible
The playlist — keep or kill?👍 Keep the crown👎 Dethrone it

The Challenger — Comment of the Week

"Our AI assistant now summarizes every postmortem. Engineering loves it. I realized last week that nobody has actually verified whether the summaries match the real timeline. They sound authoritative, so we assumed they were right."

— Comment from an SRE Lead, Tech Finance Company

1 in 5 teams now uses AI to investigate incidents. Polished output starts to feel like correct output. Aviation researchers have a name for this: automation bias. Confident-sounding results suppress the instinct to verify.

A hallucinated root cause in a clean summary can misdirect the entire postmortem. The real failure repeats quietly.

Treat your AI summarizer like any other dependency. Give it an error-rate SLO. Spot-check 10% of outputs per sprint against the raw timeline.

Unvalidated AI in the postmortem loop is a monitor with no monitor. Closes a blind spot

Should this replace the favorite?👍 Promote it👎 Not ready

The Ratio is a weekly newsletter by Florian Hoeppner.
Early edition, you are one of the first.

Forwarded this? Get The Ratio in your inbox every week.

Take the assessment → reliabilityeconomics.com/benchmark
Reply to this email with your take.

How this newsletter gets better: Every link is tracked and every vote is counted. Sections that get 👎 get replaced. Topics that get 👍 stay. No personal data is stored — just click and vote counts per section. Your feedback shapes the next edition.

Don't miss what's next. Subscribe to Reliability Economics:
← Newer THE RATIO #5: AI teams deploy 1K×/month. Your observability wasn't built for that. Older → THE RATIO #3: Who's watching the agents watching your systems?
Powered by Buttondown, the easiest way to start and grow your newsletter.