It seems like every parent gripes about the education app their kid’s school uses. Our school district heard these complaints, said “hold my beer!” and built their own.

They applied absolute consistency, which means that the per-class communication granularity that makes sense for high schoolers is also applied to the kindergartners. I am now in separate chats for kindergarten math and reading and writing and social studies and science. (They don’t even do social studies and science in kindergarten!)

There is a lot to be said for consistency. It means having one thing to build and deploy. One thing to manage and maintain. A single data schema for their developers to worry about. And what I get out of it is a breakdown of a kindergarten I don't recognize.

Speaking of well-structured things that only sometimes loosely represent reality, let’s talk about software delivery metrics.

What does it mean to measure software delivery in 2026?

We as an industry had some agreed upon metrics and approaches. Now we’re in an era when the very nature of software development is changing, and there are open questions on what to measure and how to measure it.

People are hungry for guidance on how to adapt, but many of our typical resources for gathering data are losing resolution.

The landscape for publicly available telemetry data is changing rapidly. APIs are closing or imposing limits. Also, as more user questions are asked to LLMs directly, the signal gained from data sources like Google Trends or Stack Overflow weakens. The net effect is that telemetry is becoming privatized. RedMonk has historically relied on a lot of these data sources, and now we are reevaluating whether there is enough signal remaining in them to continue things like our language ranking analysis.

There are still great resources available from companies that choose to share lessons from their data, like the CircleCI state of software delivery report. But the nature of what data is available and from whom is materially different now than it was in the pre-AI era.

Surveys are also challenging. We are still building a shared understanding and vocabulary of what it means to work with LLMs, which means surveys and studies have new confounding variables.

On top of this, some of the voices that provided longitudinal guidance are growing quiet right as the industry’s need for help is highest. For example, DORA announced they are not running their annual survey and report for 2026. Even though I had questions about DORA’s direction with the report last year, it feels like a loss to have one fewer guiding voice in the industry for now.

Just like my school’s app contains a version of kindergarten that is not reflective of reality, many of the measurement approaches we rely on describe a state of software development that increasingly no longer exists.

We know we are all going to have to adapt to this new environment, and we are all collectively trying to figure out where this industry is headed and what the patterns are. But as of now, we don’t know what the answer to measurement is, let alone how to consistently apply it across a range of organizations. Studying the industry's transition to AI is proving to be as challenging as actually making the transition.

-Rachel

Links Roundup

• Annie Sexton’s piece on compression and prediction is one of those rare reads that falls straight into "I wish I’d written this" territory.

• Really interesting story here about Soundslice, a music education company. When a bug kept cropping up, it turned out it was because of ChatGPT hallucinating a feature. What do you do when ChatGPT is telling people you have a feature that you actually don't? Maybe implement the feature.

• A lovely piece about finding each other's humanity in "a dense mire of processed and reprocessed text and transcripts and emails where nobody is really talking to anybody at all."

• This cognitive load is something many of us feel daily. We're suspicious of the writing of others, and highly sensitive to even the appearance of AI writing in our own posts. 

• Our own Stephen O'Grady has written a fair bit about open weights models and why they aren't actually open source, and here is Steven J. Vaughan-Nichols writing on the same topic in the Register, summing up many of the pressing issues. "The AI industry likes to abuse the word ‘open.’ It appears in product releases, research papers, policy debates, and investor presentations. A company publishes model files to Hugging Face, developers run them on their own GPUs, and the release is quickly described as an ‘open source model.’ Not necessarily. It may only be open-weight. The difference is more than a technicality."

Recent RedMonk Research

• A long time ago Steve O’Grady wrote a book about how software developers were taking over the world. Now, it’s their agents turn. A few thoughts on the new New Kingmakers (and now those of you who’ve been asking for a follow up know why Steve kept putting it off). RM clients mentioned: Cloudflare, Docker, GitHub, Google (DORA) and Microsoft

• A lot has happened with open weights models this summer - so much so that Steve had to rewrite this piece, once from scratch, three times. But if only to get this published before something else happens to force him to tear it all up again, here are some angles to consider when understanding the role and significance of open weight models from an industry perspective. RM clients mentioned: Amazon, Google, IBM, Microsoft

• How does a database best known for running on developer laptops help a cloud win workloads? Rachel Stephens’ take on Amazon Web Services’ acquisition of DuckLabs: The Mighty Duck. RM clients mentioned: AWS and Google (BigQuery)

• Back when Kate Holterhoff was a web developer and QA engineer, she regularly tested the mobile experience of the sites her team built (typically using BrowserStack & Google Chrome's Device Mode), so when she learned about recent moves in the iOS simulator space I had some questions about the state of native mobile app development, and, specifically, why simulators are blowing up. RM clients mentioned: Google, AWS, and Expo

• What’s the Difference between Agent Experience (AX) & Generative Engine Optimization (GEO, aka AI SEO, AEO, LLMGEO)? TLDR: GEO gets you mentioned, AX determines what happens next, but there's so much more to say!! At RedMonk we've been speaking a lot about this subject, so here’s Kate’s breakdown targeted to practitioners in the software space. She focuses on what each term means, a bit about their history, and why both matter today. RM clients mentioned: Google

• Historically, registries answered a simple question: “Can I download this package?” Today’s tooling increasingly asks a different question: “Should I and how?” The Artifact is Free, Assurance is the Product. RM clients mentioned: The Eclipse Foundation, Microsoft/GitHub, Docker, Google, Red Hat, AWS, and Chainguard

• James Governor wrote a post about the eval and observability convergence, in the context of Dynatrace acquiring Arize AI. Open Telemetry is going to be a common glue for observability and AI observability. RM clients mentioned: Dynatrace and LaunchDarkly

• Just riffing off an interesting post by Per Buer of Varnish Software: The agents are coming for the web and the web isn’t ready. RM clients mentioned: Cloudflare, GitHub, and Fastly

• Stephen O'Grady inspired James to bash this out with special appearances by Render and Fly.io - competitors, both seemingly well positioned for the agent era. RM clients mentioned: Cloudflare, Fly, GitHub, MongoDB, and Render

Recent Videos and Appearances

• In this MonkCast, Rachel Stephens talks with Sehjung Hah, a product marketing engineer at Broadcom, about how VMware Cloud Foundation 9.1 is shifting private cloud operations from click-driven dashboards toward programmable, API-first workflows — From Dashboards to APIs: Day Two Operations in VCF 9.1 with Sehjung Hah

• What does it mean to trust software? For this RedMonk Conversation, Kate Holterhoff sits down with John Ellis, President of Codethink and the contributor to the Eclipse Trustable Software Framework, to pull that question apart. Evidence Over Certificates: John Ellis on the Eclipse Trustable Software Framework

• In this RedMonk conversation, Stephen O’Grady sits down with Boris Bialek, VP of Industries at MongoDB, and Raman Jatkar, Head of Product Management for Purple Fabric at Intellect Design, to examine where agentic AI and data platforms converge: The Data Behind the Agents with MongoDB’s Boris Bialek & Intellect Design’s Raman Jatkar

• James Governor speaks with Jason Willeford, Global Lead for Secure Global Support at Red Hat, to discuss digital sovereignty and why it’s become such a big deal for European enterprises: The Digital Sovereignty Questions Every Enterprise Should Be Asking with Jason Willeford

• Most conversations about AI agents stay on what agents can do. Less attention goes to what they leave behind: logs, traces, and tool-call chains at a volume nobody’s pipeline was built for. Nikhil Mungel, Head of AI R&D at Cribl, talks with Kate Holterhoff about why that is becoming observability’s cost and noise problem: Why AI Agents Are Blowing Up Your Telemetry Bill with Nikhil Mungel

• Why does ChatGPT recommend one DevTool over another, and what can founders do about it? In this RedMonk Conversation, Kate Holterhoff sits down with Adam DuVander, Principal Consultant at EveryDeveloper and author of Developer Marketing Does Not Exist, to explore how AI assistants are transforming software discovery. Why ChatGPT Recommends Your Competitor: LLMs & the Future of DevTool Marketing with Adam DuVander

• In this conversation recorded at Fastly Xcelerate, James Governor talks with Jefferson Frazer, Director of AI at Shutterstock, about how a two-decade-old media company reinvented itself for the age of foundation models. From Stock Photos to Trillions of Artifacts: Shutterstock’s Data Journey with Jefferson Frazer

• Generated code is arriving faster than maintainers can read it. RedMonk’s Steve O’Grady discusses this new reality with Jason Brooks and Brian Proffitt, both managers in Red Hat’s Open Source and AI Program Office. “Somebody is always paying the token bill”: AI and Open Source with Brian Proffitt & Jason Brooks

• RedMonk’s Rachel Stephens sits down at VMware Explore to discuss the TrueSource by Broadcom announcement — RedMonk Quick Take: VMware Explore 2026

• Kate Holterhoff talks with Victoria Melnikova, head of new business at Evil Martians, about what product market fit looks like for developer tools in 2026. Good Wigs & Golden Paths: Dev Tools in the AI Era with Victoria Melnikova
  

Sponsor The 2026 Monktoberfest

The Monktoberfest, now at a new venue for 2026, is a unique opportunity to reach an elite audience of technical people that transcend traditional developer categories. Just as you can’t find beers like this anywhere else in the world, you won’t find a crowd like this anywhere else in the world. From CEOs to hardcore developers, sysadmins to marketers to DBAs, the person sitting next to you is probably responsible for some of the software you use every day.

Sponsorship opportunities are available. Please contact Morgan Harris with any questions you might have about the programs.

RedMonk Recommends

Kai has worn a lot of hats in his career, often multiple simultaneously. As an engineer, product manager, developer advocate, and customer manager, he honed his versatility at Civo, helping navigate its rapid shift from 5 to 100 people.

He led the launch of KubeQuest (a gamified onboarding path aiming to make complex cloud tech accessible for beginners), built Civo's beta referral program, and set up the core integrations and reporting the company ran on. Equal parts customer advocate and technical operator, Kai excels at catching non-negotiable user needs before they become build-blocking issues and isn't afraid of stepping into the breach of engineering to debug distributed systems and ship fixes himself.

Having worked in craft beer before transitioning into tech, Kai built his customer-focused foundation brewing and selling beer in London, running BrewDog’s European e-commerce operations, writing their customer service playbook, and managing the digital marketing of an online beer distributor. Seeking bigger scale, he transitioned into tech via the Makers bootcamp in 2019. What he may lack in the way of a traditional CS degree, he makes up for in thinking on his feet, stepping cleanly into senior engineering territory by shipping core features and debugging complex, distributed systems.

What he's after: solutions engineering, developer advocacy, platform engineering, or any hybrid role solving customer problems in code and explaining them back clearly. Based in Chicago, Kai is open to in-person, hybrid, or remote, and is authorized to work for any US employer.

Connect on LinkedIn or catch him at this year's Monktoberfest to chat tech, hiring, or cask conditioned ales.

Meet the Monks

Events we'll be attending:

• All Things Open: October 18-20 in Raleigh, NC

• Cloudflare Connect: October 19-22 in San Francisco, CA

• Oracle AI World: October 25-28 in Las Vegas, NV

• IBM TechXchange 2026: October 26-29 in Atlanta, GA

• Github Universe: October 28-29 in San Francisco, CA

• Kubecon NA: November 9-12 in Salt Lake City, UT

• AWS re:Invent: November 30 - December 4 in Las Vegas, NV

Events we'll be hosting:

• The Monktoberfest 2026: October 1-2 in Portland, ME