2026-11-02
November 2, 2026
Most week-over-week movement in a delivery metric is routine variation. A process behavior chart tells you which movements deserve a meeting.
Open any engineering metrics review and watch the pattern. Lead time went up 12% this week, so someone offers a reason. Deployment frequency dipped, so someone else offers another. A story gets attached to every wiggle, the stories get written into the notes, and the notes become next month's "learnings." Nobody asks the prior question: did the system change at all?
Most of the time it did not. Any process that produces a number each week produces a number that varies, for reasons no one could name in advance. If you explain that variation, you are writing fiction about noise. Worse, you will act on the fiction: a reorg of the review process, a new rule for pull requests, a pep talk. Those interventions land on a system that was going to produce a similar number anyway, and the next wiggle gets credited to them.
The discipline that fixes this is old and cheap. Walter Shewhart developed control charts at Bell Labs in the 1920s, and Donald Wheeler, in Understanding Variation (1993), turned them into the process behavior chart (PBC) for people who are not statisticians. The chart costs a spreadsheet and ten minutes. What it buys you is a defensible answer to one question: what evidence do we have that the system changed?
The DORA metrics (lead time, deployment frequency, change failure rate, recovery time, and since the 2024 edition rework rate) are reported almost everywhere as snapshots. This quarter versus last quarter. This week versus last week. A snapshot comparison carries no information about expected spread, so every difference looks like news. That is the root of the reacting-too-much problem, and it is a measurement-literacy problem, not a tooling problem.
A process behavior chart adds the one thing the snapshot lacks: the natural process limits, calculated from the metric's own history. You plot the metric over time, draw a center line at the average, and compute an upper and lower limit from the average moving range, which is the mean of the absolute differences between consecutive points. In Wheeler's XmR form, the limits sit at the average plus or minus 2.66 times that average moving range. These are not targets and they are not goals. They are the voice of the process, a statement of what the system produces when nothing exceptional happens.
Three detection rules do most of the work. Mark Graban, whose book Measures of Success (2018) is the most readable treatment for operations leaders, summarizes them this way: a point outside the limits is a signal, eight or more consecutive points on one side of the average is a signal, and three of four consecutive points closer to a limit than to the average is a signal. Everything else is routine variation. Graban's practical advice is to use fewer rules, not more, because every additional rule raises the false-alarm rate.
Here is what that looks like in practice. These numbers are illustrative, not from a real team. Imagine median pull request lead time, in hours, over twelve weeks: 31, 34, 29, 33, 36, 30, 32, 35, 28, 33, 31, 34. The average is about 32 hours and the average moving range is about 3.9, which puts the natural limits at roughly 22 and 43 hours. Week five rose from 33 to 36, a 9% jump that a snapshot review would happily explain. It is inside the limits. It is noise. Then week thirteen arrives at 52 hours. That point is far outside the upper limit, and now the meeting is justified, because something outside the normal process happened. The right question is not "why is lead time bad?" but "what was different about that week?" A release freeze, a vendor outage, a key reviewer out sick. Week fourteen returns to 32, which tells you the process itself did not move. You fixed an incident, not the system.
That distinction is the whole point. Wheeler's framework separates two different jobs. A special cause (a signal) calls for finding what happened and, if it was bad, preventing its recurrence. Common-cause variation, the spread inside the limits, calls for something harder: a change to the system itself, because no amount of chasing individual points will move a stable process. Treating routine variation as a special cause is a mistake Deming called tampering, and it tends to make the process worse. Treating a real signal as routine is the opposite failure.
InfoQ published a useful applied case in December 2025: Egor Savochkin's walk-through of PBCs over DORA metrics for a real team. He found three stable plateaus over about three years, with change lead time near 25 hours after a legacy migration, near 10 hours after deployment automation, and near 3 hours after pair programming, each step visible as a sustained shift in the chart. Equally instructive is what the chart told him to ignore. A February rise stayed within normal variation, while a late-March signal traced to deployment tooling problems. Note the author's own caveat, which I would repeat to every leader: a PBC shows when the system changed, not why. Causation still needs context, and rigorous proof needs an experiment.
Why does this matter more in 2026 than it did in 2020? Because AI tooling has made every metric noisier and every intervention more tempting. The 2025 DORA report found that AI amplifies existing conditions, and a Faros summary of the report notes the shift away from low/medium/high/elite tiers toward per-metric buckets and seven team archetypes. When you roll out a coding assistant and lead time moves, you will be asked whether the tool worked. A snapshot gives you a story. A PBC with a baseline gives you a shift you can date, a rule that detected it, and a clear sense of whether the same limits still hold. That is the minimum evidence standard for an AI-ROI claim, and most decks do not meet it.
Two cautions. First, a stable process can be stably unacceptable. If lead time sits inside its limits at 32 hours and customers need 8, the chart is telling you to redesign the system, not to wait for a signal. Second, the chart is a conversation tool, not a surveillance tool. Once a PBC becomes a team ranking, you have created a target, and Goodhart's Law takes over. Use it on systems, and show it to the team that owns the process first.
Try this week. Pick one delivery metric your review already reports (lead time for changes is the easiest) and pull the last 15 to 20 weekly values from your tooling. Compute the average and the average moving range in a spreadsheet, draw the limits at average plus or minus 2.66 times the moving range, and bring the chart to your next metrics review. Open with one question: "Which of last month's movements actually fall outside these lines?" Count how many of the explanations you would have given go away.
What it is. A time-series chart of a metric with a center line and natural process limits calculated from the metric's own variation, used to separate signals from routine noise.
When to use it. Whenever a metric is reviewed on a cadence and people keep explaining small changes: lead time, deployment frequency, incident counts, support backlog, review turnaround, sprint throughput. It also fits before-and-after claims, such as "the new tool cut cycle time."
How to run it:
When NOT to use it. On metrics with very few or irregular data points, and as a way to rank individuals; for low-count "chunky" data, plot days between events instead of counts per period.
Example: a team ships a CI speedup and watches lead time for four weeks. If every point stays inside the old limits and no run rule fires, the honest report is "no detectable change yet."
DORA metrics and process behavior charts, InfoQ — A practitioner applies PBCs to change lead time and deployment frequency over three years and shows which movements were noise and which were real shifts. Read it for the worked cases.
Key takeaways from the 2025 DORA report, Faros — A secondary summary of the report's AI findings, seven capabilities, and move to team archetypes. Verify any figure you plan to quote against the report itself.
Engineering leadership in 2026, LeadDev — Scott Carey's June 2026 talk draws on a survey of more than 600 engineering leaders about how their roles and priorities are shifting. The full findings sit behind a free account.
"Before you ask why the number moved, ask whether it moved."
— Editor's note, not a quotation: today's issue in one line
Don't miss what's next. Subscribe to Critical Path: