Reliability Economics

Archives
Log in
Subscribe
September 15, 2026

AI agents fail 60% of tasks—who pays for the cleanup?

The Ratio #18

Issue 18September 15, 2026

The Ratio

A weekly newsletter on reliability economics

 

The Number

5.63 vs 0.77 (100k+ vs 50k–99k orgs)

Organizations with 100,000+ employees have a median reliability ratio of 5.63. Organizations with 50,000 to 99,999 employees sit at 0.77. That's a 7x difference between adjacent size bands. Sample sizes: N=24 and N=46.

Neutral

Scale adds complexity faster than it adds prevention capacity. Think of it like a shipyard building ever-larger tankers but keeping the same number of hull inspectors, the vessel gets bigger, the inspection coverage gets thinner, and nobody notices until something cracks. The biggest organizations in this benchmark are not just slightly behind the tier below them. They are meaningfully more under-invested. 7x more. That gap doesn't come from negligence. It comes from organizational mass outrunning the budgeting logic that worked two mergers ago.

The very largest organizations are 7 times more under-invested in reliability than the peer group one tier below.
Did this land?
More like thisSwitch it up
 

This Week in Reliability

AI Agents Hit the Wall

Multiple reports this week show AI coding and SRE agents failing 40-60% of the time in production contexts, forcing teams to decide whether agent reliability problems justify the reactive cleanup cost or whether the prevention promise was oversold.

 

Deep Reads

AI's best coding agent fails 60% of the time — and the data backs it up

The New Stack  ·  Primary evidence

New benchmark data shows leading AI coding agents fail to complete 60% of real-world software engineering tasks, with success rates varying dramatically based on task complexity and context availability.

Neutral

This isn't about agent immaturity—it's about unmeasured cleanup cost. Every failed agent attempt creates technical debt, rollback work, and human verification overhead that never appears in the TCO calculation. If your agent succeeds 4 times out of 10, you're not saving 40% of engineering time—you're spending 60% extra on cleanup.

Failed agent runs create costs nobody budgeted for.

ilert AI SRE is generally available

ilert  ·  Vendor response

ilert launched AI SRE agent that performs root cause search in minutes instead of an hour, delivering hypotheses with evidence and proposed actions. Now available on all paid plans.

Firefighting

Compressing incident search from an hour to minutes is pure MTTR reduction—but notice what's missing: any claim that this prevents the next incident. This is reactive tooling with an AI label, and the economics only work if your on-call cost exceeds the license plus false-positive cleanup. The real question: does faster RCA create any feedback loop that reduces incident frequency, or does it just make the war room more efficient?

Faster diagnosis doesn't prevent the next fire.

Confessions of an Unrepentant Slop Snob

Charity Majors  ·  Quality framework

Charity Majors presents a framework for evaluating when AI involvement in engineering workflows is additive versus when it violates quality standards and creates long-term maintenance burden.

Neutral

The slop threshold is where agent economics flip from net positive to net negative—and most teams don't measure it until six months later when nobody can maintain the generated code.

Quality erosion is a lagging reliability cost.

Digital Experience Monitoring with Grafana Cloud: Session Replay, synthetic checks, and faster investigations

Grafana Labs  ·  Missing context layer

Grafana Cloud adds session replay and synthetic monitoring to answer who was affected, what they saw, and whether an issue warrants escalation—data that metrics alone can't provide.

Prevention

This closes the context gap that makes agent-driven RCA fail—without user session replay, your AI agent is guessing about impact and often escalates or ignores the wrong things.

Context gaps make agents guess wrong.

The Pulse: tech companies move to open AI models

The Pragmatic Engineer  ·  Cost optimization signal

Pinterest reports that open-source models post-trained on internal data deliver superior performance for their use cases at less than 8% the cost of comparable closed proprietary models.

Neutral

When agent economics force you to pick between failure rate and cost-per-call, open models let you optimize both—but only if you can afford the post-training infrastructure overhead.

Cost pressure drives model sovereignty.

Thoughts on the July 8th Telstra outage report

Lorin Hochstein  ·  What agents can't see

Analysis of Telstra's July 8 NTP-related outage and the independent review report, examining what the incident reveals about systemic reliability issues in complex telecom infrastructure.

Prevention

NTP failures are exactly the kind of deep system dependency that agent-driven automation misses—no amount of log parsing intelligence prevents timing drift at the protocol layer.

Agents see symptoms, miss root causes.

SRE Weekly Issue #534

SRE Weekly  ·  Human baseline

Weekly SRE newsletter curating incident reports, tools, and reliability engineering practices including Planetscale's Neki horizontal sharding solution for Postgres at scale.

Neutral

The steady drumbeat of manual incident curation still outperforms agent-driven aggregators—human editorial judgment filters what actually matters for reliability economics, not just what's trending.

Curation beats automation for signal quality.
Did these land?
More like thisSwitch it up
 

The Crowd Favorite

  1. Don't Stop Me Now — Queen — No rate limiter on the retry path means your recovery makes the outage worse. Exponential backoff isn't optional.
  2. Numb — Linkin Park — Alert fatigue is a signal-to-noise problem. Every low-value threshold that fires means the real emergency lands without priority.
  3. The Final Countdown — Europe — A deployment freeze without rollback criteria is just a deferred decision. It arrives mid-incident.
  4. In the Air Tonight - 2015 Remaster — Phil Collins — Burn rate alerts only work when they fire rarely enough that people believe them.
  5. Time — Pink Floyd — Most teams obsess over how fast they fix things and ignore how long the fire burned before anyone noticed. 47 minutes, in one case I reviewed.

Neutral

MTTD beats MTTR — if you measure it
The playlist — keep or kill?
Keep the crownDethrone it
 

The Challenger — The Over-Engineering Award

This week's award goes to the anonymous 100k+ employee cohort benchmarked at a 5.63 ratio of prevention spending to firefighting spending.

Your runbooks have runbooks. Your SLOs have SLOs. Your chaos experiments have their own post-mortems.

The 50k to 99k cohort next door runs at 0.77 and still ships on Friday.

Honest read: 5.63 is either peak reliability maturity or a prevention budget that quietly ate the product roadmap. The benchmark data can't tell you which. Your VP of Engineering probably can.

Neutral

Over-investment signal
Should this replace the favorite?
Promote itNot ready

The Ratio is a weekly newsletter by Florian Hoeppner.
Early edition, you are one of the first.

Forwarded this? Get The Ratio in your inbox every week.

Take the assessment → reliabilityeconomics.com/benchmark
Reply to this email with your take.

How this newsletter gets better: Every link is tracked and every vote is counted. Sections that get a down vote get replaced. Topics that get an up vote stay. No personal data is stored — just click and vote counts per section. Your feedback shapes the next edition.

Don't miss what's next. Subscribe to Reliability Economics:
← Newer AI SRE stopped asking permission—and shipped anyway Older → AI SRE isn't replacing toil—it's exposing maturity gaps
Powered by Buttondown, the easiest way to start and grow your newsletter.