June 12, 2016, 9:29 p.m.

Prototyping the Extended Mind

DisAssemble

There have been conversations I've had where, after a case of forgetfulness or curiosity, I've paused the conversation to look up pertinent information on my phone. “Paused the conversation” is perhaps a bit of a euphemism, “blithely ignored the other person” may be more apt. But I’m hardly unique - I’m sure you’ve done the same.

I’ve heard arguments that claim our memories will wither if we rely on smartphones to look everything up rather than attempting to remember it. I’ve listened to claim after claim that the art of conversation is sullied when people ignore others to look at their phone mid-conversation.

But allow me to take a rather provocative stance:

There is no functional difference between recalling information via your physical brain or via your phone. Our memory is as external as internal.

In academic literature, this is known as the extended mind hypothesis. The extended mind (EM) hypothesis is perhaps best exemplified by an an anecdote, drawn from the originators’ journal article:

Inga hears from a friend that there is an exhibition at the Museum of Modern Art, and decides to go see it. She thinks for a moment and recalls that the museum is on 53rd Street, so she walks to 53rd Street and goes into the museum. It seems clear that Inga believes that the museum is on 53rd Street, and that she believed this even before she consulted her memory. It was not previously an occurrent belief, but then neither are most of our beliefs. Rather, the belief was sitting somewhere in memory, waiting to be accessed.

Now consider Otto. Otto suffers from Alzheimer’s disease, and like many Alzheimer’s patients, he relies on information in the environment to help structure his life. In particular, Otto carries a notebook around with him everywhere he goes. When he learns new information, he writes it down in his notebook. When he needs some old information, he looks it up. For Otto, his notebook plays the role usually played by a biological memory. Today, Otto hears about the exhibition at the Museum of Modern Art, and decides to go see it. He consults the notebook, which says that the museum is on 53rd Street, so he walks to 53rd Street and goes into the museum.

Otto believes he has access to his memory, much as Inga does. Though it may take a second or two for Inga to remember it, it may only take a short time more for Otto to open his book to the page where he wrote down the address. Isn’t this just a difference of quantity of time rather than anything more fundamental?

There are certainly differences in the experiences of retrieving this information, and anyone who has studied memory will tell you memory is not analogous to a simple filing system. Nevertheless memory is a system of recall and retrieval, as is the extended mind.

With smartphones, the plausibility of the EM hypothesis is even greater. Our friend Otto is actually at a disadvantage than us smartphone-equipped and able-memoried folk. Not only does Otto not know where the MOMA is, he doesn’t know exactly where in his notebook that information is. And arguably, his access is slower than Google (since he may need to search through his notebook) and a smartphone contains knowledge that you haven’t necessarily recorded previously — it is a repository with nearly limitless encyclopedic qualities.

The thinner the division between access and realisation (i.e. full awareness) of the information the more convincing the extended mind hypothesis becomes.

Memory is not the only cognitive capacity extended through our tools. Examine web browsing — the activity which you are, or were just engaged in. How did you get here? Twitter? Medium? Reflect on the path that you’ve taken to achieve this route. There’s a particular quality to it in that you forged it; it reflected your thoughts in that your thoughts impelled the retrieval of the information on the screen. A feedback loop forms where your thoughts are splayed out on screen in a tangible way. Each new query forms a new thought which forms a new query, and so on.

In this way, it’s reflective of not only what you are thinking about but your thinking in and of itself. Your curiosity, your need or desire to remember something, are represented by the inputting of queries, or the clicking of links. Your browsing behaviour is a map, a record of your thought in much the same way as writing down something in a notebook is. Yet on the web, we can take it further — because you aren’t just recording your input on an empty page, you are interacting with, and reacting to, information. The content of what you are looking at and your reaction to it intertwine and become inexorably manifested to form an external mind.

But the manifestation of this thought is difficult to play with, to be “within”; it is insubstantial. The challenge is giving a corporeal form to our extended browsing mind, such that we can reflect on it and work with it as we do our own thoughts.

Yes. we have a we have our web histories, but they are simply lists of pages, not representations of our branching, query-laden thought process.

In viewing a map of our thoughts, we can recall what we were thinking about, how various thoughts (manifested as pages, clicks, and queries) are interrelated, and reflect on the nature of our curiosities and thinking patterns. Importantly, this also lets algorithms visibly work with us, within our extended mind.

Let’s take an example. In the gif below, a user googles a word she vaguely knows, ‘acedia’:

As she searches, her browsing — her extended mind — is mapped in an area above the browser.

She enters the Wikipedia article, then goes back to the Google search.

Slowly her cognition becomes visible.

Acedia, by the way is “ is a state of listlessness or torpor, of not caring or not being concerned with one’s position or condition in the world.”

Finally, we see her her clicking a link to a related topic, “ennui”.

How does “ennui” and “acedia” relate? They involve meaninglessness — a lack of purpose. Accordingly, we can see that the system recommends an article , “Leo Tolstoy on Finding Meaning in a Meaningless World”. Algorithms work to find relations between words that she searches, find common themes and ideas. From there is she is able to find similarly themed articles, based on her history (great care of course, needs to be taken with such a thing to avoid filter-like bubbles).

And perhaps she was feeling empty, forlorn, and was Googling these words. But on viewing her extended mind in this way, she could take a bird’s eye view, and perhaps realise something about herself she hadn’t otherwise known.

But it’s not just the work of semantic algorithms that could exist within an extended mind. Importantly, this “thinking” is all mapped for her to recall at a alter date. She could tag the grouping of browsing, or have it tagged automatically. Much like one might remember the name of a friend of a friend by recalling the closer friend, new understandings can be sought by recalling how one browsed.

Pages become nodes of activity that have been determined by the cognition of the user. In this way the cognition of the user and the system work hand in hand to aid the user.

Interfaces are our bodily proxies to an intangible world — the world of information.

However it’s difficult to be within our space within the realm because we don’t have the phenomenological awareness that we have in real life. What does it “feel” to think on the web?

Making this feeling more visible, more tangible, is a first step to narrowing the gap of experience. Yet this gap will never be crossed if we don’t make that first leap — the leap of understanding that our minds extend beyond the matter within our skull.

You just read issue #7 of DisAssemble. You can also browse the full archives of this newsletter.

Powered by Buttondown, the easiest way to start and grow your newsletter.