In this issue...
A solar eclipse is coming up on August 12th. It should be visible from a lot of places in Europe, the best being Spain, but an Italian developer has created a great map which you'll find in the Topical section. Speaking about Europe, don't miss the European Data Journalism Network analysis of megafires on the continent. Among a few tools linked in this issue, my favourite must be img2threejs, allowing users to uploade a single image and create a model entirely using AI-generated (but allegedly token-efficient) threejs code, not mesh files or code. There's also the intriguing story of Anthropic's latest cybersecurity incident.
·
Last week I had a few deeply intellectual, satisfying conversations about using AI in production to replace human processes. As you know, I've worked with AI for quite some time while, still, describing myself as an AI-skeptic (although, more correctly, I'm a hype-skeptic). The main topic that caught my attention was how to set up proper quality controls and guardrails in AI. I couldn't help but repeat my usual stance: humans get it wrong, too, in a measurable way. Which means, I think, that we should be reframing the AI deployment question as "*what is your error baseline, and is AI improving it on it?"
Thinking of this brought back memories of a story from my time working in Italian healthcare over 20 years ago.
Our product was a set of software packages to automate blood and urine testing in hospital labs, so that the processing and reporting would be faster to reach doctors and patients and, we strongly suggested, more accurate than manual processes. In a world that was, back then, rather close to microscope-and-paper, we still had customers challenging the accuracy of the tests. Some of them were resistant to automate, despite strong assurances, quality checks, and clear guardrails and monitor. Things changed massively when the news told in this academic paper became known: "In February 2007, three organs from an human immunodeficiency virus (HIV)-positive donor were transplanted at two hospitals in the Tuscany Regional Health Care Service, owing to a chain of errors during the donation process. ... During the donation process, the result of the lab test performed for evaluation of organ suitability was mistakenly transcribed from positive to negative. This wrong negative result was then included in the donation record without any cross-check."
While this situation was, likely, a rare occurrence, it did make me thing about quality checks and guardrails in human processes – or lack thereof: single points of failures, no controls, no measurements, no alarms. It's remarkable that we worry about all these points when we're talking about automated processes, but we rarely do (or did) when their equivalent was human-led.
Obviously, I do appreciate that automation makes the consequences of mistake have worse, much worse impacts, especially in processes involving large scales. But the reality is that humans can get things disastrously wrong, too. The first step to automating a human process properly is to ask: what is the rate of human error in the pre-automation version of that process? That is your comparison to develop and deploy an AI.
But the most important consequence of this is that if you can't answer the question, you shouldn't be automating that process – with AI or other technology – until you can.
·
Quantum #669 had an open rate of 33% and a click rate of 12%.
The most clicked link was the quirky We > Ultrarich billionaire budget simulator.
·
This Freedom of Information request to Transport for London is truly another level.
·
The Quantum of Sollazzo grove now has 50 trees. It helps managing this newsletter's carbon footprint. Check it out at Trees for Life.
'till next week,
Giuseppe @puntofisso.bsky.social
