Synthetic media is giving me the uncannies
Logo: Eniko
Note: No large language model was used in the production of this text. No kitten was strangled either, and technically the set of events which failed to happen is infinite, but this particular non-event matters.
This week has been an interesting one, for someone doing a PhD involving 'generative AI', i.e. proprietary, online systems backed by large language models. My topic is on students use in assessment, or specifically the attitudes of faculty towards that, but the encroachment of the technology on all aspects of society is hard to avoid.
ISBNs (or lack thereof) in Canada
I found out from one of our hippest librarians, via a doctoral pal, that Canada was going to stop issuing ISBN numbers for books made up of synthetic text. We applauded the decision, although, like all legal decisions, it will be tested at the point of enforcement. The adverb in 'primarily generated by AI' is load bearing here. Detecting synthetic text is tricky: detection systems are themselves opaque machine learning systems, and, in the case of student work in assessment, both false positives and false negatives will tend to penalise certain demographics which are already the victims of educational inequities. As per a recent item in Mystery AI Hype Theatre 3000, "pangram is not an oracle".
Google's SynthID technology, which, they had deployed in production for two years with little fanfare, is the one Anthropic made a big fuss of announcing they were going to use for watermarking. Before it, making synthetic text (pictures are another matter) recognisable as such was presumed, if not impossible, certainly moot. These watermarks defend against false positives, but they relie continuous sequences of tokens, as generated, which makes them easy to defeat with a modicum of edition. Still, the mere possibility of watermarking, detection and norms of disclosure is getting the slop-mongers very upset, as illustrated by the reaction of the Substack community upon an announcement to this effect. Substack's take:
Norms around AI use and its disclosure are unsettled. But we have made the assessment that while the use of AI isn’t necessarily a problem, a lack of transparency around it definitely is. (Substack, 2026)
Bear in mind, this is Substack claiming lack of transparency is problematic. The same Substack which perfectly knowingly makes money from hate speech, and enables its speakers to monetise it. So if they draw a line...
Still, tomes of slop not getting an a ISBN number (in Canada) seems to constitute progress. At least symbolically, it acknowledges that synthetic text and human utterances are categorically different, and should be treated differently. If anything because people use effort as a heuristic for quality (Kruger et al., 2004). Which goes towards explaining the 'social penalty' AI users suffer (Reif, Larrick and Soll, 2025).
A case in point in the synthetic tome genre
The same week, the same doctoral pal showed me what, for this very reason, I would hesitate to call a book. A 'paperback bound printout', let's say, on father's rights. It has, infuriatingly, received an ISBN. The person-credited-as-author on this text self-publishes, the pal tells me, up to twenty such book-shaped-objects a month. In Silicon Valley, they call this kind of grift 'high-agency' (Kriss, 2026). This one is meant to give clarity to fathers as to their rights under the law, but contain no references: acts are mentioned, because statistically the words will come after one another, but not paragraphs, let alone secondary sources.
In a twist of irony, the blurb at the back of the... thing, explained that the man credited with its authorship went through a divorce and was offered a lot of advice, but little in the way of facts. Yet, without some kind of truth grounding, surely text generated stochastically is only factually correct by accident. It didn't take much flicking through to spot a sentence of LLM output presuming a human editor would check it for factuality:

But with an average of 36 hours from idea to final proof, this is to be expected. This must be what they call 'Divorced Dad Energy'. At least he is not weaponising the low barrier of entry to the production of a legitimate looking text, but he is certainly making the most of how low the barrier is, using the short production cycle typical of social media influencers to reward-hack the distribution algorithms of print materials, which print-on-demand services for self-published authors had already incentivised. The mass production of (lately, self-published) genre fiction as always been a thing, and these involved in it have long been early adopters of many a shortcut to production cycles.
Sloppocalypse now
Coincidentally, this happened the week of an event for PhD students with presentations (from both students and staff) much of which contained ostensibly synthetic images, without, of course, any disclosure. That evening a few of us were at the pub bemoaning the normalisation of undeclared use of synthetic media in all fields of life, not least of which higher education, which is highly ironic given students have been dragged in front of academic disciplinary panels for arguably lesser moral transgressions. The OIA website has made for gripping reading of late.
Us AI-critical PhDs will be putting out an open letter or position paper, and, in the interim, a blog on the Library's site is not a bad place to stake our claim. Because, before we had grand theft autocomplete (since then, grand theft autocracy, eh?) abstracting and absconding the product of human intellectual labour into weight matrices, the places where knowledge was stored, indexed, and made accessible was libraries. Libraries value authorship and its attribution. Libraries were, before the internet, the place were references could be checked. Libraries act as diffusion nexus and quality control for human knowledge; that is knowledge for, but also by humans. Libraries are not perfect, and this quality control has been, at times, gatekeeping. That said, they are barely problematic compared to the "generative AI" industry and its practices. And given that John Dewey was involved, this should speak to how bad the AI industry is.
Behind the Bastards | iHeart
There’s a reason the History Channel has produced hundreds of documentaries about Hitler but only a few about Dwight D. Eisenhower. Bad guys (and gals) are...
Our position is not that people should not produce synthetic media. This is entirely between them and their conscience. It is simply that:
- Undisclosed use of synthetic media implicitly equates it with genuine, human-made text and images.
- We should fight against the normalisation of such undisclosed use.
- In the absence of disclosure, we should if possible ask if the media is synthetic, and if so why this was not disclosed.
- We should, in particular, explicitly disclose the absence of AI use. To show it still occurs, the collective presumptions notwithstanding.
Turing's imitation game (1950) was a thought experiment, not a test, to be operationalised into a success criteria. In winning this game, LLMs have managed to plausibly pass for humans. We have to wonder whether it means we are the ones who _lost_the game, rendered unable to recognise other humans now that outwards signs of humanity are increasingly imitable by machines. Artificial Imitation, more like.
References
Kriss, S. (2026) ‘Child’s Play: Can the Democrats harness the new prairie populism?’, Harper’s Magazine. Available at: https://harpers.org/archive/2026/03/childs-play-sam-kriss-ai-startup-roy-lee/ (Accessed: 13 October 2026).
Kruger, J. et al. (2004) ‘The effort heuristic’, Journal of Experimental Social Psychology, 40(1), pp. 91–98. Available at: https://doi.org/10.1016/S0022-1031(03)00065-9.
Reif, J.A., Larrick, R.P. and Soll, J.B. (2025) ‘Evidence of a social evaluation penalty for using AI’, Proceedings of the National Academy of Sciences, 122(19), p. e2426766122. Available at: https://doi.org/10.1073/pnas.2426766122.
Turing, A. (1950) ‘Computing machinery and intelligence.’, Mind, 59(236), pp. 433–60.