Artefact 252
What’s my ideal LLM?
Part three. Congratulations for getting this far. (If you’ve been sent this, part 2 is here.) Your reward, if we can call it that, is this, the creative, upbeat, futures-facing instalment.
Imagine what we (could still yet) win!
Dude, stop killing the vibe
Earlier this summer, Mike Betts replied to something I had posted on Linkedin.
“I always have the impression that you are super progressive in your thinking, tools and methodologies - I’m surprised you’re not more accepting. But everyone has a complete right to view and treat LLMs as they see fit.
I feel like you’re exactly the kind of person and professional who should be building the AI future that works for us all - I’m convinced you would create something amazing!”
Mike; flattery will get you everywhere.
More seriously, I’ll start with that first paragraph.
I guess I have only been properly thinking about AI since about 2016. We were working with the Innovation team at Konica Minolta on various projects exploring prototypes and concepts of how automation and AI would fit into everyday business processes.
This is maybe my favourite film artefact from that time, not just because of the prototype and setup, but also because of the collective thinking processes and conversations in the lead up to it:
Then Andy Budd invited me to the second instalment of The Juvet Agenda, a conference he and the Clearleft gang organised in Norway.
For the three years of The Juvet Agenda (2017-19), a group of around twenty strategists, academics, designers, futurists and more gathered for three days of working out what this wave coming at us was, and what might (if anything) be done. Perhaps there should be a ten year anniversary gathering in 2027, to check-in on some of the issues raised, and see how that’s all going…
Back to Mike’s first point, then.
I think I’m perhaps ‘less accepting’ than most because I have been thinking for years from the broader perspective of who is trying to do what, here?
I think I am (healthily) skeptical about the actors and motivations behind the push to LLMs. It is really useful to pay less attention to what the mouths are saying, and more at what the hands are doing. It is a really easy space to make outrageous claims in.
Which makes Mike’s second point - “building the AI future that works for us all” - a great challenge to think about.
If not this, then what?
Following Follow the Rabbit
I mentioned Igor Schwarzmann in Part 2.
He and Johannes Kleske recently invited me to join them on their podcast Follow The Rabbit to talk about the Cognitive Debt idea. You can watch that episode here:
For the discussion, I had set up a Miro board to unpack the ‘Moments of Enlightenment’ model (see Part 2), and some of the talking points I wanted to cover.
(You can poke around on that board here, if you like.)
And again, the same kind of question came up at the end of the discussion.
If it’s not this generation of LLM-style tools, what might I like to see instead?
So, thanks to Mike, Igor and Johannes, I have been thinking since about the sort of requirements I would like to see in an LLM…
The LLM I’d like to use
It all starts with the data.
I’d like to use an LLM with much, much better data from the outset. What do I mean by better? Well, better for the people who created it in the first place; they have consented in some way to its inclusion in the training data set. Better in quality, too. It’s not a grab of ‘anything that happens to have been written in English’ (yeah, nice one, Meta).
Better sourcing then, and careful selection. Wikipedia is likely in every training data set anyway, but (so the claim goes) you need much, much more data than that to train better models.
Side note: there is clearly an argument to make that if a technology needs to burn through every scrap of data ever produced in order to work, then maybe it is not a very good technology?
Anyway, let’s imagine for now that we look for better data sources. Earlier this week, Dan Hon wrote about a possible “Model for the People”. In Dan’s example, he refers to the national copyright libraries that are specifically tasked with collecting together written works. e.g. here’s the The British Library‘s collection policy:
“If you publish anything in the UK and Ireland, you need to give one copy to the British Library. This system, called legal deposit, has existed in English law since 1662. It was updated in 2013 to include electronic publications.”
That might be a useful route too, to add more data in. Although, of course, we’re now only really gathering data in the English language because of how the training works. If I think about what older connections between words and concepts exist in the British Library’s collection, do we want to include everything?
By and large though, I’d like to be able to trust an LLM provider’s clear policies for data use; “we only use data sources which comply with our standards, which are XXX”.
Then we come to training.
This was something I wrote down during a conversation with Håvard Legreid we were having after my talk in Bergen:
Every time I used an LLM, I’d like not be thinking that I’m ‘burning the planet to make sand think harder’. Especially when I could do the thinking myself, and I’m fuelled mostly by sandwiches. It massively nags at me. Imagine what it would be like to be free from that nag with every keystroke?
I’d like a LLM that has constant commitment to smarter, more precise training methods. Transparency in where and how resources are used. Imagining you could choose a supplier who only use closed-loop liquid cooling.
It would be nice if, should I want to, I could also read detailed notes from the supplier on how they working in the various stages of training to shape the LLM - the equivalent of well-documented, open access code.
Now we get to the interface.
I believe that the chatbot interface on top of LLMs is a terrible idea. As Paul Ford points out here, “the last 75 years of software development are not suddenly irrelevant because of LLMs, and if you want to be a true platform for the future, then playtime is over—you have to actually make the thing work.”
The chatbot interface does not work.
It simplifies an unimaginably complex statistical model into the equivalent of Red Dwarf’s Talkie Toaster…
Side note: yes, niche, I know, but the 15yo has recently watched the first seven series of Red Dwarf (1988-1997) for the first time, and the toaster is “AI” references throughout are very in keeping with how we think about AI now, which perhaps points to the fact that we’re simply rehashing the same ideas in this area with infinitely more computer power.
And yes, I get that this is an innovative new field. We have never had to design interfaces that deal with anything like this before.
LLMs are a new form of hyper-quantitative data - rather than collecting data on a specific issue, industry etc, it asks you to collect all the data. And then, when the user puts in a specific research request, it works backwards from the whole world view it has created.
And yet, this hyper-quant is delivered through a qualitative interface. A system that pretends to talk to you, and as such hits all the meatware vulnerabilities that you have because hey, you’re human and joyfully stacked to the brim with pareidolia. If we see faces in light switches, I’m not surprised we think the LLM is ‘alive’
Nothing about an LLM interface shows you what is actually going on. And that’s to the detriment of every user specifically and the technology generally.
Sarah Gold made some great points here on why transparency and trust need to go hand in hand in the current AI space:
“…most people don’t see how these systems work. And that’s a problem. When users don’t trust AI, adoption stalls. People hesitate to use AI-powered services, avoid engaging with them, or abandon them altogether.”
Last month’s report from Internet Matters highlighted the dangers to young people of pushing LLM chatbots out unregulated and unexplained to anyone who wants to use them.
Earlier this year, I asked some undergraduate students during a guest lecture what was happening when they were using ChatGPT as their main browser, and there seemed to be no appreciation of why this text box should be different that other text boxes where you could also search for information.
I’d like to see an LLM where the process is prominent in the interface.
How it works is actually what it looks like.
For instance, (and to keep with the older sci-fi references), the first series of Westworld (2016!) has a delightful UI as a plot device.
Maeve (the android host) is shown by Felix (the tech admin) how the very words she speaks are statistically assembled on the fly, according to her character as designed.
Felix tells her “You can improvise a little, but most of what you say was designed upstairs”
(watch here if you want)
When I rewatched the first season earlier this year, it really jumped out at me again: Maeve is an LLM.
(There’s a great medium article by Daniel Manu which dives deeper into this too).
How might we have LLM interfaces which demonstrate the connective assemblage of language that’s going on behind the scenes? How might that start to introduce every user to the idea of what’s going on?
Once users can see an LLM working in its simplest form, how might we increase the operability of these systems?
The sheer number of different prompt approaches you can take to using an LLM by ‘typing text in the box’ is staggering. In various discussions and forums, I’ve had conversations with people where processes and approaches are shared (or not!) like recipes… or spells.
Can you remember the special words to make the magic work?
Any workflow set-up is largely left to the user to experiment with, learn, note down and remember to implement for themselves.
Or, I would guess in most cases, not noted down - what % of LLM users actually have repeatable methods they employ, rather than just freely typing into the box?
I’d like to see an LLM where the creation of the ‘media’ pulls more from other established interfaces as we deploy our ‘effects’
Yes, I’m talking plugins.
You might never have used GarageBand or Adobe Premiere. Basically, as the timeline unfolds along the media you’re creating, you can effects to alter the output, tweaking and tuning as you go.
I guess it goes back to Igor’s description of ‘Synthetic Media’ in the last newsletter - if we are making media, why can’t we make it using established tooling setups from other media forms?
Users might then have a more granular, visual understanding of the cause and effect of using an LLM. How does what I tweak here alter the output?
It could also mean that each project would be set-up and retain its own plugin settings - you would have a better idea of how you got there (and might do again).
There’s something interesting here that perhaps ties into this exploration of the future of civic tech interfaces by Dan Munz and Mark Headd:
“Instead of designing one interface that works acceptably for everyone, and must be managed and scaled that way, AI will make it possible to generate variations tailored to specific circumstances.”
Rather than one text box for everything, a whole a variety of different workspaces you can flip between, customise, save, share etc.
But John, this sounds… hard to learn?
Yes, but it should be hard to learn. The current drive to onboard as many users as possible to LLMs (in order to somehow, eventually pay back all that the investment money) demands that the starter interface is simple.
I’d like to use an LLM that’s honest for all parties about the complexity involved. I feel in a sector short on standards and long on lies, we need to have a proper conversation about what these technologies really are, and how we want them to be used in our communities, organisations and society at large.
(You should read Sharif’s piece “We need a serious public conversation about AI and the future of the UK” - HT Matt).
Imagined future states
It’s all very well creating a little speculative version of my perfect LLM. It is a really useful way to surface your own standards and ethics when it comes to this area. And maybe something you could do yourself, or with your team too?
Do I think anyone will make an LLM like this?
In all honesty, no - there are too many incentives involved to not design an LLM this way.
But there are plenty of other signals and trends we’ve been keeping track of that suggest the general AI narrative of Silicon Valley doesn’t unfold in the way they might hope.
For instance, there are enough legal cases in courts around the world now that we might see something really interesting happen with the province of data ownership. The current laws (and implementations of them) are clearly unfit for purpose, so new specific regulations are likely to emerge in the next few years. Perhaps an actual mass use-case for blockchain will be recording your written text as thoughts before it goes out to the LLM marketplace, and pays you back pennies in PRS-style annual revenues?
Alternatively, given the funding structures of the major LLM players currently, it might be highly unlikely that in five years time they all survive in their current forms. Perhaps then we’ll see a repeat of the end of previous technology booms, where properties are broken down and sold for parts. What are the chances of you bumping into GPT-3.5 when you buy a new smart toaster in 2030? How might older models find practical, efficient places of use in the future, when they are more ‘evenly distributed’.
Or maybe the next AI Winter will be even long and deeper than the last, but the LLM boom only collapses after the world has built out a massive renewable energy infrastructure in the short term to deal with the power requirements. In one way, it might be the greatest gift that LLMs could give us - the ability to quickly and cleanly reduce our reliance on fossil fuels.
Is this… the design futures part?
Ah, yes, I was going to talk about the two Design Futures courses from this summer;
With Rob Phillips and Gem Barton, I ran the first Futures Through Design at the Royal College of Art in London. It was the most successful launch of a new open course in their history, apparently - not only did it sell out, but scored 4.67 out of 5 in the feedback.
And with Toban Shadlyn, I ran Innovation and Future Thinking at IED in Barcelona, which was my tenth anniversary of teaching on the course and partnered with Bluewave Alliance to consider possible futures of coral in the Mediterranean.
Here are some photo diaries from the RCA course and the IED course though, should you want a sneak peek.
But given we’re over 2500 words now, why don’t I do that in a week or two.
Until then.
John V Willshire
14th August 2025