"AI safety" is AI hype
"AI safety" is just more AI hype—and our policymakers are dangerously distracted by it
By Emily
In early March, SV Angel put out a statement in the form of an open letter with the headline "Build AI for a Better Future". The initial signatories included OpenAI, Microsoft, Google, Mistral AI, Hugging Face, and other companies and organizations as well as individuals with a variety of affiliations. It was self-parodying, but still I couldn't resist doing a parody of the letter anyway, entitled "Build 'AI' for a More Exploitative Future". I posted it across social media sites, referring to the original letter as "the latest open letter from the 'AI Safety' crowd".
The most remarkable response was the numerous people who replied (publicly and privately) to let me know that I was wrong to associate the SV Angel letter with "AI Safety". Nevermind that the rhetoric of the doomers and the boosters is nearly indistinguishable. Nevermind that there is overlap between the signatories of the SV Angel letter and last year's "AI Pause" letter (for a take down of that one, see here). No, (a subset of?) people who identify as working on "AI Safety" would like the world to know that they are very, very different from the people who are concerned with "building AI for a better future".
I shouldn't have been surprised—the AI doomers and AI boosters are locked in their own conversation with each other, one that keeps the focus on the technology (and not the people building, selling and using it, much less the people being exploited in its development and harmed by its use), and in that conversation, the doomer to booster cline defines the whole space. (Unfortuantely, a lot of the press reporting on all of this falls for that narrow view, too.)
But that view is too narrow, especially as a lens through which to consider the regulation of the development and use of so-called "AI" systems. As a case in point, here's a quick review of Billy Perrigo's recent piece in TIME "Exclusive: U.S. Must Move ‘Decisively’ to Avert ‘Extinction-Level’ Threat From AI, Government-Commissioned Report Says". I'm pretty confident that even the folks taking me to task for calling the "better future" letter "AI Safety" would claim this piece as "AI Safety". But it's dripping with AI hype just like the "better future" letter, all while pulling the focus away from the actual harms being done in the name of "AI", as usual.
Ready? Here we go...
Right from the lede, Perrigo uncritically platforms x-riskers, associating the x-risk idea not with the actual outfit the published the report, nor any context as to who they are, but with the fact that the report is commissioned by the US government:
Paragraph two then pretends to take just a little distance stating that "AGI is a hypothetical technology" but then cites "the leading AI labs" (aka the people selling the hype) as expecting "AGI to arrive within the next five years or less".
By paragraph three, we still don't know who these report authors are, nor why their opinion should be of interest (except that they won the government contract), but we are informed that they spoke with "more than 200 government employees, experts, and workers at frontier AI companies—like OpenAI, Google DeepMind, Anthropic and Meta".
The use of "frontier" in that description is particularly appalling, evoking both Star Trek and the idea that these companies are building the future as imagined in science fiction, one the one hand, and on the other settler colonialism where the lands and life worlds of Indigenous peoples of North America were seen as empty space to be conquered and tamed.
Paragraph four has some of the recommendations from the report:
Some notes: The harm is framed in terms of the "computing power" of the models, a trope familiar from the "AI Pause letter" and grounded in the idea that larger and larger models are the path to "AGI"; that somehow these piles of linear algebra, if they just reach critical mass, will combust into consciousness and agency. And then, as always, the bar is conveniently just beyond what's already been built. The rest of the paragraph locates the danger of these models in their "weights", that is the result of training them, rather than the large-scale theft of creative work, the exploitation of workers to sanitize system output, the ongoing pollution of the information ecosystem, the environmental impacts of compute-hungry models, the reproduction of systems of oppression in system output, the use of algorithmic decision making to shirk collective responsibilities of care, and so on. No, what the reports authors suggest should be punishable by jail time is the release of model weights. The rest of it? Business as usual. Oh, and where else should we put our focus? On "alignment", i.e. the fool's errand of trying to get the output of text synthesis machines to "align" with "human values", again keeping the focus entirely on the technology and not how it's being used, and on whom.
The only dissenting voice included in this piece is Greg Allen, but his dissent (as quoted) is focused only on being skeptical that the US government will take the actions suggested in the report. There is no skepticism of the doomerist viewpoint, nor pointer towards the urgent, actual harms being done in the name of "AI" (despite Perrigo's own previous reporting on some of those harms).
The authors of the report (Jeremie and Edouard Harris) are finally named in the 9th paragraph, and introduced as brothers who have doggedly been seeking some branch of the US government that might be receptive to their message.
Then we learn that there are two kinds of risks identified in their report:
In the first of these, we're asked to imagine that "AI" systems are somehow much, much more useful than old-fashioned information access (web search, or even going to a library), for bad actors looking to get info on various types of weapons. This already requires an imaginative leap about the potential future "capabilities" of large language models. The second is an even bigger leap, resting on the belief that they are about to combust into consciousness.
The article goes on to describe the report as warning that industry AI labs are incentivized to prioritize speed (of scaling up their systems) over "safety". I have no doubt that tech companies are incentivized to engage in harmful practices in the pursuit of scale and profit, but the harms aren't the imaginary "safety" risks that the Harris brothers and Perrigo would have us all focus on.
All in all, Perrigo's TIME piece is completely lacking in any critical review of the report it is focused on. Instead, he writes "The report's recommendations, many of them previously unthinkable, follow a dizzying series of major developments in AI that have caused many observers to recalibrate their stance on the technology." Note how "previously unthinkable" validates the recommendations as reasonable/or "thinkable" now, while the same sentence expresses only awe rather than any critical distance for the hype show that the tech cos have been putting on since the release of ChatGPT (if not longer).
... and in the very next sentence, he appeals to some imagined common experience of being led to consider existential risk. (The rest of that paragraph talks about some polling data that suggests that the doomers have succeeded in scaring the public at large, but doesn't actually substantiate the claim that it's the "dizzying series of major developments in AI" that is scaring people.) Perhaps that's how Perrigo understands his own experience, but if we're going to have journalism that actually holds power to account here, we're going to need journalists to resist the urge to be impressed.
Our book, The AI Con, is out on May 13, 2025, but you can pre-order it now where ever fine books are sold! 