The Symbol Grounding Problem
Everything Is True
Ada Hoffmann's author newsletter
Before we get into today's post, some news:
RESURRECTIONS is up for pre-order on Kindle! (If you're not in the US, it may take a few days before it propagates to your country's version of Amazon; it took a day before it showed up for me in Canada, for instance.) If you've been looking forward to this book, please feel free to pre-order as soon as possible; the timing really does help. If you prefer a physical copy, I’ll let you know when those become available as well.
I don't remember if I announced this earlier, but I've been appointed to the SFWA Emerging Technologies Committee, where we'll do some projects and organizing related to the effect of new technologies - including, but not limited to, generative AI - on speculative fiction authors. It will be a lot of behind the scenes stuff that I mostly won't be able to publicly discuss, but I'm super excited.
And now, some cognitive science!
Every year - and now more than ever - I tell my first-year students about the symbol grounding problem.
It goes like this:
In the orthodox view of cognitive science, information processing consists of representations and procedures. Something - in your mind, in another animal's mind, or on a computer - creates representations that refer to the world, and uses them to carry out procedures - whether it's as simple as adding two numbers, or as complex as writing a novel. A representation can be fuzzy and diffuse (as in the brain, where our knowledge is stored as patterns of connection) but without some form of representation, there is no thought.*
All the representations in a human mind share some important properties.
First: all representations are symbolic.
To be symbolic means that a representation is not the same as the thing it represents. (We call the thing itself a referent.) The map is not the territory. As I like to tell my students, words like “water” are representations; but I can't drink a glass of the word "water." Whether a representation is a word, a picture, or something more abstract, it refers to something other than itself.
Second: all representations are grounded.
If the map is not the territory, then grounding is what connects them. I can't drink a glass of the word "water," but I can look at the glass, hear the word "water," and understand that one of them refers to the other - that the word "water" is not just a meaningless sound. Without grounding, all symbols would be meaningless; like Chinese characters to a person who doesn't read or speak any Chinese. To a limited extent, we could still follow rules and work with them; but the work would have no meaning to us, no known correspondence to anything we experience or understand.
So the symbol grounding problem is: how do representations become grounded? How do we learn to associate something abstract, like the word "water," with our actual sensory experience of the world?
Philosophers of mind can get very deep into this. What even is meaning in the first place, asks a philosopher. What even is a symbol? What are concepts? What are categories? How does that work?
We don't need to get into those weeds today. What I do want to emphasize is:
In order to give meaning to a symbol, we need to connect that symbol to our physical, sensory, embodied experience.
In order to connect a symbol to sensory experience, we need to have sensory experiences.
Every single human who has ever lived has access to embodied, sensory experience. (Yes, even humans with sensory disabilities; my students always ask about that.) It’s on that foundation that we build all our understanding of the world and of ourselves.
But a computer does not have sensory experience.**
In particular, large language models & text-to-image generators do not. (An image generator's processing of images is not sensory experience; the generator gets a lot of information saying, e.g., what pixel of the image is what color, but it has no experience of seeing things in the physical world: no sense at all that a visual stimulus corresponds to something that can be touched, interacted with, encountered, experienced.)
When you understand that generative AI's symbols are not grounded, it explains some of the most frustrating aspects of their behavior. It goes a long way to explaining why they can't stop themselves from confidently producing misinformation, or falling for weird prompt injection tricks, why they fail at commonsense reasoning despite explaining their answers articulately, or why they have so much trouble distinguishing between ethical and unethical speech. From ChatGPT's perpective, none of its words actually anything. They're just symbols it's pushing around.
It also explains why claims about generative AI becoming sentient are so laughable. One does not simply become sentient without having senses. When we anthropomorphize AI, it's because we look at their words and instinctively assume that they correspond to a subjective reality in which things are sensed and understood, the way they would if a human wrote them. But they simply do not.
If some spontaneous force of subjective consciousness ever did arise from a sufficiently large mass of ungrounded symbols, then its workings would be so utterly alien to us, and so divorced from the world we understand, that we still might not be justified in calling it thinking.
---
*This is merely the orthodox view! There's another school of thought saying some of our "cognitive" abilities are reflexive responses to sensory stimuli, and therefore don't require representations at all. I get into this a bit with my students in the later weeks of the course, but I'm not going to get into it here.
**A possible exception is robots, which have sensors to take in the world around them and effectors (limbs, etc) which can affect that world in some way. The question of whether a robot's representations could be grounded is important, and far from settled; there's some interesting work currently with engineers seeing if they can apply the same learning techniques that power large language models to a robot's learning of sensorimotor skills. ChatGPT, however, is not a robot.