The RatioA weekly newsletter on reliability economics | | |
|
The Number
2.5 vs 1.94 — unused SLOs vs none
Organizations that have SLOs but don't act on them show a higher failure-to-prevention ratio than organizations with no SLOs at all — 2.5 versus 1.94.
58 organizations have SLOs that don't influence decisions. Their median failure-to-prevention ratio is 2.5. 27 organizations have no SLOs at all. Their median ratio is 1.94. The group with unused SLOs sits 29% higher than the group with nothing defined.
Neutral
Unused SLOs aren't neutral. They correlate with worse reliability economics than having nothing at all. Teams spend time defining targets, then let them collect dust while the gap between what failures cost and what prevention gets funded keeps growing. This is like bolting a smoke detector to the ceiling without wiring it to a battery. Everyone in the building assumes they're covered. They're not. The measurement was never the problem. Measurement without anyone responding to it changes exactly nothing.
| Defining SLOs you don't act on leaves you worse off than never defining them. |
|
|
This Week in Reliability
Detection Gap Economics
Organizations are waking up to the cost of incidents that started long before anyone noticed. The gap between 'customer impact begins' and 'response starts' represents pure waste — and this week's content shows teams finally measuring it, instrumenting around it, and building tools to close it.
|
Great Circle · Primary evidence Brent Chapman argues incidents begin when customers experience problems, often well before internal detection and response. He outlines four investments that help close the detection gap. Neutral This is the reliability economics argument in its purest form: every minute of undetected customer impact is money burned with zero operational learning. The gap isn't a monitoring problem — it's a budget leak you can't even measure until you instrument for it. | Detection lag is the cost center nobody budgets for. |
|
Gremlin · AI blindspot angle Gremlin identifies five capability gaps in AI SRE tools: they excel at triage and known-pattern response but struggle with novel failure modes, cross-system causality, and behavioral anomalies that don't map to existing telemetry. The post emphasizes understanding these blind spots before production deployment. Firefighting AI SRE closes the MTTR gap but widens the detection gap if your telemetry has holes. Autonomous response only works when the system can see the problem — and most detection gaps live in the spaces between services, the drift nobody instrumented, and the emergent behaviors that weren't in the training data. | Faster response to invisible problems is still blind. |
|
Anyshift · Tooling response Anyshift's integration with PagerDuty surfaces blast radius context when an alert fires — in this case, showing what else could fail before a local checkout service fix becomes a broader incident. Prevention Closing the detection gap means showing responders what's about to break, not just what already did. | Blast radius visibility turns alerts into predictions. |
|
CNCF Blog · Instrumentation pitfall Service mesh observability with Istio and Kiali provides immediate visibility into request rate, latency, and error metrics, but metric aggregation artifacts can create misleading signals when zero-traffic services suddenly report activity. Neutral Instrumentation that lies to you during the detection window turns a gap into a canyon. | Bad telemetry is worse than no telemetry. |
|
Humainary · Next-gen instrumentation Humainary proposes rapid innovation in novel instrumentation that operates in near-real-time within local process space, enabling coordination across diverse instrument libraries within the same space-time region. Prevention If detection gaps live in the spaces between existing telemetry boundaries, you need instrumentation that can span them without waiting for data pipeline lag. | Detection needs to happen where the failure is. |
|
StackGen · Agent observability gap StackGen details their approach to debugging multi-stage AI agent workflows, focusing on visibility into agent decision chains and intermediate states that are otherwise opaque. Firefighting Agent-driven operations create a new detection gap: the space between what the agent decided and why it failed. | Autonomous systems hide their own failure modes. |
|
Dynatrace · Organizational constraint Dynatrace outlines six SRE trends including unshackling teams from traditional cost-center structures, enabling more strategic work, and fostering a culture that accepts failure as a learning mechanism. Neutral If SRE teams spend all their time responding to the detection gap, they'll never get budget to close it. | Culture shift requires closing the firefighting loop first. |
|
|
The Crowd Favorite
- Highway to Hell — AC/DC — Deploying without a practiced rollback is a one-way gate. Velocity into production is not velocity out.
- Purple Haze — Jimi Hendrix — Alert storms without hysteresis bury the causal signal in duplicate noise. Page deduplication is not optional.
- Gimme Shelter — The Rolling Stones — Co-located replicas share the blast radius of the host failure. Redundancy that doesn't cross fault boundaries isn't redundancy.
- What's Going On — Marvin Gaye — A cardinality cap on your metrics pipeline is a blind spot you agreed to in advance. Name it explicitly or rediscover it mid-postmortem.
- Born in the U.S.A. — Bruce Springsteen — Regional failover documented but never exercised fails at incident time. A DR test is the only evidence a runbook is true.
Prevention
| Five sharp failure modes before production hits back. |
|
|
The Challenger — Contrarian Take
Popular belief: An SLO you don't actively monitor is worthless documentation.
Teams with unused SLOs benchmark at a 2.5 prevention ratio. Teams with none sit at 1.94. That 0.56-point gap suggests the definition process, forcing engineers to reason about failure modes and acceptable thresholds, reshapes behavior before any dashboard activates. In surgical medicine, the WHO Surgical Safety Checklist cut major complication rates ~36% even when teams found it routine. The mechanism isn't active consultation. It's cognitive framing. Here is the thing: if a committee wrote your SLOs without operating the service, the framing is corrupt and the gap collapses.
Neutral
| Even unused SLOs outperform having none. |
|
|
The Ratio is a weekly newsletter by Florian Hoeppner. Early edition, you are one of the first.
Forwarded this? Get The Ratio in your inbox every week.
|
Take the assessment → reliabilityeconomics.com/benchmark Reply to this email with your take.
How this newsletter gets better: Every link is tracked and every vote is counted. Sections that get a down vote get replaced. Topics that get an up vote stay. No personal data is stored — just click and vote counts per section. Your feedback shapes the next edition.
|
|
|