Does GPT-3 Dream of Electric Artists?
Earlier this year, I received a copy of Stephan Keppel's Soft Copy Hard Copy in the mail. At around the same time, Nelis Franken, a student of mine, told me about GPT-3. In a nutshell, GPT-3 is a product of the artificial-intelligence (AI) software industry. The idea is relatively simple. There are algorithms that you can train by feeding them information and that then do certain things, based on what they've been trained with. This idea is widely used. It also has been responsible for a vast number of problems, such as when, for example, facial-recognition algorithms misidentity people of colour because the training sets were limited to mostly white people.
The idea behind GPT-3 is to produce text that reads as if it had in fact been written by humans. But it's all based on already existing material. Conceptually, this seemed like a perfect tool to use to write about the book, given it centers on copying and/or re-using materials. I had also reviewed an earlier book by the same artist and didn't want to repeat myself.
There has been a lot of talk how AI is going to replace a lot of human activities. I'm firmly in the camp of skeptics. There probably are a lot of very simple applications that AI could take over. But more complex tasks will probably be out of reach of computers. I doubt that anything that involves deeper creativity will become victim of some algorithm.
My understanding of GPT-3 is probably too limited. But I was skeptical that it would be able to produce something that would be on par with what I do. Here's what I mean by that: I don't mean to say that I'm some sort of genius. I'm clearly not. But my writing is more than merely a collection of words the formation of which is based on some of the other writers I've read. Even if I couldn't express what it is, I am convinced that there is something that I bring to the words, and nobody else has access to that.
Now this might sound pretty obvious and unremarkable actually. But this particular part, the stuff that I bring to the text, either has to be completely random for it to become part of an algorithm (machines can create random stuff pretty easily) or it simply should not matter (if it doesn't matter, you don't need to include it in your algorithm).
It's one thing, though, to think about this. It's quite another to see what's possible. Getting access to GPT-3 took a while. With the help of Nelis, I finally was able to get there (if you're reading this, Nelis: thank you so much!). Thus I set out to write an article with the help of the machine. Here it is (read it first before reading the rest of this email).
I accessed GPT-3 through a browser window that looks very similar to the editing screens I use for CPhMag.com or this Mailing List. The only difference is that there's a button underneath that says "Submit". Once you click on that button, you'll see new text appearing. But you have to give it a prompt. That prompt could be anything, a word, a list of words, a sentence, more than a sentence... There also are a few sliders to the right, the most obvious one setting the length of the response. I set that to a few hundred (this is in units of tokens, which according to the interface is roughly four characters).
I started things by writing
The surfaces of any modern city are a hodge podge of mass-produced materials, set against each other without regard for what sits next to what else.
...and then I hit "Submit". It was interesting to see the algorithm in action as more and more words were being added to my initial prompt. And probably not surprisingly, the algorithm cleared its first bar: the text was convincing in the sense that it read as if it had been written by a human being. Albeit not this particular one.
There were a number of problems to overcome during my work on the piece. First, fairly often, GPT-3 strayed far from the topic at hand, or it ran down a few very very particular avenues. At some stage, every prompt resulted in the production of a (fake) artist bio. I rejected all those results. In some ways, I can't blame the algorithm for what it did, given the way I ended up working (more details below). But I think I also spotted some of the source material that it had been fed in form. The intersection of art, photography, and architecture was just something it hadn't been trained with.
The second problem was a writer's one. The algorithm would not only change style but also voice. The text read as if it had been written by a human being, but the combination of my writing and the machine's read much too differently. What I ended up doing was the following: if the text made conceptual sense (regardless of whether I liked it or not; please note one more comment below), I edited it for style and voice to try to fuse everything together.
Given I was interested in what the machine would produce, I didn't reject any results that made sense in the article, as long as they were not detrimental to the overall purpose. In other words, while I wasn't afraid of making a fool of myself (if you've seen the end of the article, you know what I mean), I certainly did not want the machine to make fun of either the artist or the publisher, or write anything that would put them in a bad light.
It's one thing for me to be critical -- I will answer for that. But there's no accountability for the machine, even if at the end, the whole text is mine (I am the author of the whole piece).
There was one case where the machine suddenly inserted a fake quote. I thought about what to do with that. I liked the outcome. At the same time, I wanted to be transparent about what had actually happened. I ended up adding "One might imagine him expressing his ideas as follows:" in front of the fake quote and added a disclaimer afterwards. The disclaimer interrupts the flow of the piece a little bit. But it was important for me to be transparent about what had just happened.
On numerous occasions, the machine ended up running into a circle where it would repeat the same output where it would repeat the same output where it would repeat the same output... There's a slider to the right of the writing window that allows one to control this. When it first happened, I was confused. And I was reminded of how Instagram's machine translation does the same with Japanese text all the time. In particular, there tend to be blocks of "in the middle of" repeated multiple times in many of its translations.
On one occasion did I allow the machine to change voice, namely at the end. At some stage, it spit out this weirdly personal first-person material that I personally would never write this way. But it seemed like such a strange and oddly compelling text that I decided to keep it more or less as is (I made some very minor edits). This solved my final problem, namely how to end the piece. I had no idea how to prompt the machine to write something that could serve as an ending. By chance, I got it.
Given the machine's output, the whole article basically is a montage of blocks of either one or two paragraphs produced separately. Going beyond two paragraphs would always have the machine run too far from the task at hand. So I would give it a prompt (typically a single sentence), look at the response, edit it (if it was usable), and then use the longer text as a prompt. I would repeat the whole procedure until the machine ended up too far from the planned article.
The montaged blocks are (hopefully) being held together by my own text prompts. In some ways, you could argue that my prompts are prompted by the machine itself. That's true to some extent. But there also is the overarching goal of the article, and I set that goal. Not the machine.
I'd like to think that I learned a few things from this exercise. Whether or not I used the full potential of GPT-3 is debatable. I'm typically reluctant to try things to have them confirm my hunches or expectations. That said, I'm now not any more convinced than before that AI can replace serious criticism. From what I heard, it's already able to produce a lot of very basic writing. Apparently, you can have a machine produce PR copy. But then if you've ever read such texts, you know how low a bar that is.
In the end, the reliance on training sets and prompts might be AI writing machine's biggest weakness. At best, they will likely be able to contribute more of the same to an already existing pool of writing. That's good (hey, I like to read easy stuff every once in a while). But it's unlikely to pose a threat to all writers who dig a little deeper and who use their own creativity (and possibly eccentricity) to write.
I'm going to leave it at that. As always thank you for reading!
-- Jörg