Human Synthesizer: Reinforcing data with our bodies
It's been a while! Jordan and I have been busy with life and fun workshops that we have been facilitating. If I had to play favorites, I'd choose the Human Synthesizer experience that I mentioned in the last newsletter. Jordan ultimately facilitated three, each one with different themes that the participants pulled out.
What we did
Most of the data that we encounter is far removed from our personal experiences. Our eyes might gloss over, or we might have difficulties connecting a chart's peaking line to actual events. Getting an audience to spend five minutes with a chart is enough of a challenge, let alone cultivating empathy to put faces to the numbers. Jordan and I are always trying to think of novel ways we can interact with data and make it more personal and intuitive - using our own data or employing all the senses to widen our minds as to what the data can mean, represent, and give us a fresh perspective. This is called by many names: Data visceralization, embodied data, somaesthetics, data representations, and ranges by having 'new age-y' connotations to a broader design meaning. It's not new that the body and mind work together to respond to and create an experience - theories rooted in bodily processes argue that language, reasoning, etc. are ultimately rooted in bodily processes. If so, how can we use our bodies to help us learn and assimilate information?
The Human Synthesizer (aka Data Choir) exercise was one such exploration of how to use our bodies to 'experience' non-personal data.
In three 45-minute sessions, we took AQI data from Los Angeles, CA, and Thimphu, Bhutan by bucketing the levels into the diverging color palette that we are all used to seeing by now when we check the daily AQI. We only had to worry about the first four colors; fortunately purple, "very unhealthy" and maroon, "hazardous" didn't appear in the datasets. Jordan and I picked out a sound for each and as each color or "level of concern" ticked by, we assigned groups to voice the sound that accompanied the data. In the one group, we collectively voiced all of the sounds, instead of being only assigned to one color/sound pair. After going through the first dataset, we listened to the recording and asked participants what they would change (if anything!), and we'd spontaneously incorporate some of the suggestions for the 2nd AQI dataset.
Ok, sounds cool. What's the point?
So many!! - Exploring performative aspects when working with data by using your whole body - Extending engagement with a data set - Creating a shared, collective data experience
As current researchers, we had specific questions, too:
- How does exploring data through performance and body change the way we relate to the data?
- How does the collective activity affect how people think about the data?
- Does coordinated action impact how people think about data?
- What will the participants be focused on? On what sounds they are making (micro) or (macro) the whole composition?
Sorry to disappoint, but naturally we just scratched the surface and don't have answers to all of these questions. Jordan is writing a whole chapter on the subject for her dissertation. Still, for now, I'll contain some learnings in this newsletter, focusing on the act of performing data, then listening to the entire performance. You can listen to one of our melodic performances below ↓
<audio
src='http://docs.google.com/uc?export=open&id=1rI8iSj0R41K9EfS6evFHmYKuiUbQ-bcd'
controls>
<track kind="captions" />
</audio>
Enough from me! Let's hear what the participants said so that you can draw your own conclusions.
What participants said
One theme really stuck out to me: the active doing vs the passive listening. Broadly speaking, what did participants get out of vocalizing the AQI scale? What about when they listened to the recording of themselves and other participants? Not to pick favorites, but a couple of participants immediately got to the heart of the exercise, and why I am so passionate about the idea of "data experiences":
Kit L:
I feel like with any type of data, if you're not directly experiencing, it, it's easy to ignore it, especially with like climate change data, or like that, ever see collapse and like getting to actually like, engage multiple parts, like the physical aspect of making sound and like also hearing it.
Yet, there was a streak of "green" in one of the datasets, where participants assigned to the healthy category had to repeat the same sound over and over across the dataset's days. One participant noted that his perception just became "a lot" instead of a distinctive number. It made me think of how difficult it is for us to imagine big numbers (if you say 1 billion light years away vs 100 billion, both are so big and abstract that they end up being "equal" to my ==imagination== although there's a 99 billion difference between them).
Yeah, and that's something where I actually, as I was doing those I found it hard to perform the right number. When I saw that long walk I was just like, Oh, keep going until it's over. So that was like lost on me as a performer as well.
Despite the difficulty in performing the "right" data, voicing the data still fostered an understanding of the dispersion of the AQI data.
I mean, even just in doing that I have much more of a sense of duration than I ever do from just looking at the data
Christian Basl:
You kind of acquire for the data [...] I had the feeling that I could interact more with the data, and I got to learn a little bit more of how the air quality is in some places...So the activity for me is more interesting than to actually listen to it.
Brian Kinnee:
I definitely agree that listening to it musically, it seems more like you either percussion or somebody using a soundboard, and like, you know, pushing the buttons on the onboard, or like a long time kind of thing. And so yeah, I agree that it's interesting. The hierarchy is not super clearly like it's distinguishable, and I found myself even as we were performing in it, like looking to see like Wait! What does the code mean like, What is this actually correlating to? And I think probably after we did it, I started to so see the songs with the different like scales ratings, but I It took me some time before we got there ownership, generative, experience The experience seemed different when we scratched the bucket assignment and we all voiced each color/sound pair.
Participants found that they were more focused on the details during the performance, while they had more of a sense of the big picture when they listened back to the recording.
Silvia Hüttner:
Yeah, I also say it was really fun. Um, especially then listening back [...] I was so focused on the data and to get it, you know, right and so on, that you maybe forget the other data points that you are not involved in vocalizing. But then, once you listen back, then you can enjoy the whole piece.
Lara:
It was great. It really made me feel/realize the relational/durational element of the data - I don't think I'm explaining it that well, but before I would have looked at the clean air data and thought 'today is bad' or 'tomorrow will be okay', this helped me take a longer view of the data. The physicality of producing the sound also emphasized the data (through the relationship between air and breath/body).
Our hope for the workshop was that participants would be more engaged with the data, and have a more embodied experience of data that they don't have first-hand experiences of.
The experience of being part of a choir making data sounds, instinctively gave me a deeper understanding of the data set: depending on how much I had to "sing" within my category showed me how strongly that category was represented in the data set - and at the same time, I listened carefully to the other categories, so I ended up with a good sense of what the air quality was like in a city.
Through sensory data representations, my goal is to facilitate empathy for data that doesn't directly relate to the audience. A sad truth is,
If you're not directly experiencing, it, it's easy to ignore it, especially with like climate change data
Although I think exercises like this could be useful tools, I still have some reservations.
Considerations
A few participants highlighted that it was a novel experience and that they hadn't done anything similar before. While this praise made my ego happy, the next moment I worried about the implications of these comments:
If the important part is a novelty, how do we preserve that in future experience designs?
It's an issue that no one has the answer to, and that artists constantly fight with (how to create new and ‘fresh’ work). If we started playing out datasets we worked with, would we eventually become in-sensitized to the exercise? Perhaps to a certain extent, but I do believe that seeing the numbers/colors and voicing them fosters a stronger connection to the data, if only because it forces your to spend more time with it.
Whatever you decide to call it - data visceralizations, representation soma-something - is not only a useful tool in creating empathy and a more memorable “data experience” but also in representing the uncertainty in data. When we were sonifying the AQI data, our voices weren't quite in unison, our pitch altering. These imperfections, and all the imprecisions that come with data visceralizations, convey data more honestly than clean lines and bar charts. They inherently and naturally highlight data's uncertainty, which every data visualization-er knows is not an easy feat!
If a personal anecdote stuck out as you read this, or if you have your own experiences embodying data, send me an email, or give me a shout on twitter.
Further Resources
A newsletter about sensory sketching, and representing data with all our senses.