Devlog #31
Hi there, and welcome to the Loud Numbers devlog - your weekly update on everything sonification. You’re getting this email because you signed up at loudnumbers.net, and you can unsubscribe at any time with the link in the footer.
We’re welcoming a lot of new members to our secret sonification club this week, which is wonderful! Please, make yourselves at home. The first rule of sonification club is that you must tell as many people as possible about sonification club, so this week we did our bit by talking about Loud Numbers at Outlier Conference.
It turns out that we’re not allowed to share the link to the recording of the talk outside conference attendees yet (if you’re an attendee and you missed it, message Duncan in the conference Slack and he’ll send you a link). The link will be available publicly in due course, and we’ll share it when it is.
In the meantime, what we can do is share answers to some of the questions that came up in the Q&A after the talk, lightly edited. So that’s what today’s newsletter is all about. If you’ve got questions that aren’t answered below, then hit reply and we’ll get right back to you. Enjoy!
- Duncan & Miriam
Outlier Q&A
Q: What are you using to map sound parameters? MIDI?
A: We’re using Sonic Pi for mapping sounds, and then Apple’s Logic Pro X for creating final tracks. We normally record the Sonic Pi audio output as separate tracks, and then mix them and add more sound layers in Logic.
Q: Have you found connection on an emotional level with the data?
A: Emotion is a really important part of sonification. We all have an emotional relationship with music. We’ve definitely found that it’s important to treat that relationship with care, and there’s an ethical imperative not to abuse it.
Q: Have you tried using silence to make a stronger point sometimes?
A: Actually yes! We’re working on a track that sonifies the decline of insect populations. The more insects the more sound. As the track progresses, it gets quieter and quieter.
Q: Can you sonify qualitative data (like text), or does it have to be numbers?
A: Great question. Is there a boundary between music driven by data and music that’s “inspired” by texts or other media? There’s a whole, fascinating, grey area between those two things. For parameter mapping the data does need to be numeric, but you can of course convert text to numbers and work with it that way. Or you can treat text as a kind of “loose” data - like we’re doing in our Boom & Bust track, where samples are positioned along a musical timeline.
Q: What’s the psychology behind why sonification works so well?
A: We’re not experts on psychology, but a lot of academic work has been done on sonification! Sara Lenzi is a really interesting person to follow on the academic side of things.
Q: Do you think it’s important to provide a key or score for sonifications, or is the context just for the composer, leaving the listener to enjoy the music?
A: Legends and visual backup can really help translate sounds but they can also so easily take precedence. We live in a visual culture + visualisation is more familiar to most people than sonification. We’re interested in experimenting with what happens when you don’t rely on visual backup. The intro to our tracks contain a kind of sonic legend, which is important to include if you want to effectively communicate data. But if you want to do something more artistic, then perhaps a legend (audio or otherwise) is less necessary.
Q: Can sonification help for accessibility?
A: Absolutely. Accessibility isn’t a specific focus of our project, but lots of people are doing great work on this. The best results come when sonification is combined with visualization (otherwise hearing-impaiired people can also struggle).
Q: What are the equivalents of “standard” charts like bar, pie and line in sonification?
A: Sonification is still an immature field, and there aren’t really “standards” yet. Data to pitch is a very common sonification, but like many pie charts it’s not always executed very well. Data to loudness is usually a better choice. But like visualization it depends so much on your data, the story and how your audience will experience it!
Q: How would you combine sonification with a visualization library like, for example, D3?
A: We’re aware of Tone.js, which seems like a good solution. We’ve not used it though! There are also a bunch of great sonification packages for Python and R. If you want some links, hit reply and we’ll hook you up.
Q: How much music theory goes into your sonifications?
A: Quite a lot! Miriam has a PhD in music. But that’s a personal choice, because we’re interested in how music and data structures interact or don’t. Duncan has no formal training, on the other hand, so you definitely don’t need a music theory background. Make punk sonifications!
Q: Have you tried to play your data sonifications to blind people? If yes, how were they experiencing the ‘readability’ of the ‘sonified’ data?
A: Not yet but we’d love to! Please get in touch if you can help us make that happen.
Q: Just like we have interactive visualizations, have you tried interactive sonifications?
A: Not yet. But Karim Douïeb recently published this fantastic project.
Q: It’s common for visual dataviz to have design guidelines and industry standards for aesthetics and readability (e.g. colour palette, typefaces and components…). Can you see that happening for sonified dataviz as well?
A: Definitely, but sonification is much less common so it’ll probably take longer for those guidelines and standards to emerge. We’re interested in publishing a “Loud Numbers sonification style guide” of some sort at some point. But we need to get the podcast finished first!