AI Doomerist Fandoms and Why We Call it Fan Fiction
By Emily
We've taken a part a few AI doomer artifacts on the podcast, including most recently the "AI 2027" web page in Episode 56 but also a Google DeepMind paper-shaped object about "dangerous capabilities" in Episode 36 (with guest host Ali Alkhatib). In both of these episodes and elsewhere, we've talked about these artifacts as bad fan fiction. I wanted to dig into that metaphor a bit further.
I want to be very clear that, even as we are intentionally quite derisive about these hype artifacts, the point of the analogy isn't that fan fiction is inherently poorly done, flimsy, or unserious. Fan fiction can be excellent—well-crafted and joyful. For those who aren't familiar, fan fiction, or derivative works, are works of fiction that build on the characters and worlds developed by other people. So the problem isn't that someone is writing fan fiction per se, but rather that they are writing fan fiction but passing it off as a) original and b) non-fiction.
I am not a fiction author, but my impression is that fan fiction can be an easier entrée into fiction writing because the fan fiction author can leverage another author's worldbuilding and character development. Other members of the same fandom already know and understand the world the story takes place in and already have a sense of attachment to the characters. This means that the fan fiction author can focus in on just one component of crafting a story, without having to attend to the full range. It also means that fan fiction tends to have a limited audience (other members of the same fandom) but, luckily, one that is already established.
My experience of reading Google's "dangerous capabilities" paper-shaped object and especially AI 2027 was that they, too, were hitting familiar beats. There's a world (of which I am no fan) that Nick Bostrom and other TESCREAL-ists have laid out. The AI 2027 authors, part of the "Rationalist" tendency (the "R" in TESCREAL) and so deep in their own fandom that they probably don't see it as such, produced an artifact that relied for its coherence on presumed familiarity with that imagined world and its logic.
If these documents were presented as what they are, derivative works based on TESCREAList worldbuilding, they would be somewhat less problematic. But Google's paper-shape object went up on arXiv (where computer scientists working in ML/AI believe they can find scientific writing) and the AI 2027 web page, with its high polish, was a deliberate attempt to influence policymakers. (Policymakers who are already distracted enough by both the boosters' baubles and the doomers' disasters.)
In using the metaphor of fan fiction to ridicule these artifacts, while trying not to besmirch fan fiction itself as an enterprise, I've been calling them "bad fan fiction". But I think that doesn't quite thread the needle in the right direction. I'm not, after all, a member of the relevant fandom and thus can't actually judge their quality as derivative works. I think a better way to put this is that they are dishonest or disguised fan fiction, masquerading as science and/or serious policy advice.
Our book, The AI Con, is now available wherever fine books are sold!
