Almost every study of AI code quality stops at the benchmark, or at best the pull request. This one kept going. Researchers inside a company whose products serve billions of people tracked a year of code changes, with a record of which bytes a model actually wrote, then followed that code through review and into production. By the end of the year, AI tools were writing roughly 70% of submitted code.

The reliability finding is the opposite of the usual fear. AI-generated changes broke the build more often on the way in, but once merged they were rolled back slightly less often than human-written ones. The headline failure mode everyone worries about, AI code taking down production, didn't show up in a year of data.

"This indicates that while AI-generated code requires more automated pipeline cycles to stabilize, once it passes initial review and automated checks, it is less likely to require an immediate rollback than human-authored code."

What showed up instead was cost, in two places. AI-written functions consumed 5 to 8% more compute and memory, for a boringly specific reason: the models wrote explicit loops where a human would have called a library, and copied objects where a human would have moved them. Nothing is wrong with any single line. It's just a fraction more work, repeated everywhere, forever. The second cost landed on reviewers, who left nearly twice as many blocking comments on AI changes.

Both of those are money leaving the business, and neither appears on any dashboard an engineering leader currently watches. Incident count and revert rate, the metrics most teams use to judge whether an AI rollout is going well, are precisely the two this study found were fine. A team watching only those would call it a success while its infrastructure spend and review queue absorbed the difference.

The researchers also looked for signs that longer or more thorough review caught these inefficiencies. They found nothing.

"The difficulty of isolating these specific architectural trade-offs during standard review suggests that conventional proxy metrics for human effort do not adequately capture the cognitive friction of evaluating AI-generated code."

Asking reviewers to look harder is not a fix here. Nobody blocks a merge over a loop that should have been a library call. The plausible controls are upstream and automated: a static check that fails on these specific patterns, or feeding the models the internal libraries they are failing to reach for.

The authors are generous about why this happens.

"We argue that the observed weaknesses are largely artifacts of historical default system configurations and a lack of context-aware prompting in early implementations."

That's a reasonable guess, and it's also the one claim the study can't check: the model identities behind the data were masked before analysis. Their fix was tested on a handful of hand-picked functions in a lab, not on the production codebase. Treat "better prompting fixes it" as an untested hypothesis, not a finding, particularly when it arrives in a vendor deck.

One limit to hold onto: this is C++ at an unusually well-instrumented company, and the waste it identifies is about memory copies and container use, which is a C++ preoccupation. What travels is the shape of the result, not the percentage. The costs of AI-assisted development are landing somewhere other than where teams are looking, and finding them takes instrumentation most organisations haven't built.

Sources

1. Tran, M., Lewis, F., Yang, K. et al., "Characterizing the Quality Profile of AI-Generated C++ in Production" — https://arxiv.org/abs/2608.06640