What Does Data Sound Like?
Acoustic Infographics: Making sense of complex information with sound
We are used to seeing data visualizations, but can might audio help us make sense of huge data sets? Are there ways our ears can help us understand complexity better than our eyes? Scientists at NASA suggest that by converting data into sound they are able to scan vast data sets in a matter of minutes. They call it a “bird’s ear view” of space.
Designers, data journalists and sound engineers are doing more and more with acoustic infographics and sonic explainers. Sound and data are not so far apart and there have been some really interesting attempts by newsrooms and artists to turn data into songs and sounds.
The Mueller Report Redactions Remix: PRI
When the Mueller report was released everyone wanted to know what was in it, but given the amount of redactions many also wondered what wasn’t in it. Ryan Cavis, a technical lead at PRX, and the team at PRI’s The World mapped every word and page and redaction in the report and turned it into a data set, and from there a MIDI file. They write, “Each eighth-note is one page of the report. The pitch varies based on the percentage of the page redacted. But volume also represents the amount of text on a page.” You can see the results, synced up to pages turning, below.
U.S. Home Prices, Sung As Opera : NPR
NPR’s Planet Money team turned to opera to illustrate the Case-Shiller home price index. “You can see the boom and bust all in one simple graph,” write Jacob Goldstein and David Kestenbaum. “But when we go on the radio to talk about home prices, a graph isn’t much good to us — nobody can see it. So we converted the Case-Shiller graph into musical notes.”
An Olympic Musical: New York Times
Kevin Quealy (a graphics editor at the New York Times) has a great Tumblr of charts and infographics, which is where I found a few audio infographics related to the Olympics. The NYT graphics team and Amanda Cox worked map the finishes of close races with sound. This was based on a project Cox did for the 2010 winter Olympics.
Above is Amanda Cox’s original audio infographic and below is one of the new videos she made with the New York Times for the London games. There are many more versions and videos over at the Kevin Quealy’s Tumblr.
Gun Deaths as Piano Notes: PRI
Sophie Chou, a data journalist for PRI, created a haunting audio track out of data tracking the number of deaths from mass shootings over the course of one year. In this interview with Storybench about how she created the sonification she talks about the emotional impact data as audio can have over data as charts and graphs. (Composer Joshua Clausen then took the audio and turned it into a remarkable and moving coral piece.)
Using Brain Wave Data to Convert a Seizure to Song: Brian Foo
There has been a lot of discussion about virtual reality and empathy, but artist Brian Foo believes that turning data into songs can also spark empathy. He writes about his project, Rhapsody in Grey, “This song generates a musical sequence using EEG brain wave data of an anonymous epilepsy patient. It examines the periods before, during, and after a seizure. The goal is to give the listener an empathetic and intuitive understanding of the brain’s neural activity during a seizure.”
Hearing Colors: Radiolab
It’s no surprise that RadioLab has toyed with the form. In the segment below they use a choir to represent the light spectrum, specifically that of a rainbow. The voices in the choir represent the various colors, and the hosts explore how different animals see a rainbow based on what colors they can see. However, being that this is radio, we get to hear how the rainbow would sound, rather than see it.
Listening to Radioactive Decay: The Radioactive Orchestra
Jihii Jolly pointed me to this examples. She writes in her post about the infographic, that visualizing how energy moves is difficult. Enter the Radioactive Orchestra. “The Radioactive Orchestra was developed into an online application for anybody, where users can pick up to five isotope frequencies, place them on virtual mixing board and edit their pitch and tempo. A visual music video of sorts is also automatically played to the left of the mixing board. These beats can then be saved and shared.”
Singing The Euro Zone Crisis: Marketplace
The complicated financial system and recent debt crisis in the European Union is tough enough to explain in print — how do you do it on radio? Marketplace from American Public Media decided to use a Barbershop Quartet.
Algorithms as Audiblization
The person who created the video below calls it an “audibilization.” Here is how he describes it: “This audibilization is just one of many ways to generate sound from running sorting algorithms. Here on every comparison of two numbers (elements) I play (mixing) sin waves with frequencies modulated by values of these numbers. There are quite a few parameters that may drastically change resulting sound.”
Audio in Arugla Algorithms: FiveThirtyEight
In 2016 I asked Jody Avirgan, FiveThirtyEight’s podcast host, “What does data sound like?” Avirgan pointed to one of his stories: “Can You Taste The Algorithm In This Arugula?” in which he explored “how farms big and small are using data to control sunlight, nutrients, air circulation and more.” In the video he looks out over a field of greens and asks “Where is the data?” Getting an interviewee to point out how data manifests itself in the phisical world — the sound of shades closing automatically to control temperatures, or sprinklers turning on in response to sensors — is one way give voice to the data.
