Wired for sound, and more

Wired for sound, and more

The LIVELab at McMaster University will give researchers new insights into what happens in our minds and bodies when we make and listen to music
October 15, 2014

“Music is a very social activity,” says Laurel Trainor, Director of the McMaster Institute for Music and the Mind.

It’s a brave assertion in an age when digital audio players in our cars and pockets often make one aspect of musical experience — listening — a solitary activity. But those digital files are still created by people working together, and judging by the packed confines of funky basement clubs and the crowded lobbies of glistening concert halls, we still crave the experience of live performance.

At the core of the LIVELab is a 100-seat
performance space bristling with sensors,
motion-capture cameras, speakers, micro-
phones and video screens.

What’s the appeal? What alchemy binds performers and audience members together to create the intoxicating elixir of live performance? Trainor hopes that some fresh answers to these questions and others will emerge from a unique new laboratory at McMaster University, funded in part by the Canada Foundation for Innovation.

The Large Interactive Virtual Environment Lab (LIVELab for short) is built around an intimate, hundred-seat performance space that’s a technological marvel. A network of microphones and speakers can recreate the acoustics of almost any performance environment by manipulating what the audience hears as reverberations. Close your eyes and the vocal group on stage may sound at one moment like they’re singing in a cathedral, and at the next like they’re sitting in your living room. A bank of video screens covering one wall adds a visual dimension.

The LIVELab building was designed
by Hamilton architects McCallum
Sather. The glass lobby and other
sections around the core do double
duty as places to prepare and
organize large numbers of audience
members and musicians.

But the real power of LIVELab lies in its capacity to monitor the brain activity, physiology and movement of up to four performers and 32 audience members simultaneously during a live performance. Sensor-studded skullcaps capture patterns of attention, thinking and emotion in the brain. Other devices track changes in muscle tension, respiration, heart rate and sweating that indicate emotional response. And 24 motion-capture cameras on the walls record the movements of audience members, musicians or both.

Using these tools, the LIVELab will allow researchers to explore almost every aspect of performance: the complex and dynamic relationships between multiple performers, between performers and audience, and even between the audience members themselves. What they learn will ultimately help musicians, dancers and other performers understand and connect even more powerfully with their audiences.

“A lot of these individual technology components can be found in other places,” says  Trainor, who holds degrees in both music performance and experimental psychology. “But the way that we've put them together is unique. It allows us to study musical performance, dance and other kinds of social interaction in ways that aren’t possible anywhere else.”

In one of the first performances at the LIVELab, the
Afiara String Quartet used real time audience feed back
to shape their performance.
Credit: Paulina Rz

The variety of information produced by the LIVELab’s technologies will enable researchers to test many different hypotheses. And Trainor expects that the sheer volume of data will give rise to unexpected new avenues of enquiry as sophisticated machine learning algorithms search for patterns and connections researchers might have missed.

Even before it was completed, LIVELab’s potential for new insight into performance captured the interest of the Afiara String Quartet. Afiara is a young, Toronto-based ensemble that performs in clubs and alternative venues as a way of building a new generation of listeners for classical and contemporary chamber music. “Our main thing is to engage with audiences,” says cellist Adrian Fung, “and that means we need to know what they’re feeling.”

The attraction of the LIVELab is obvious. After an initial meeting with Trainor, Afiara became one of the first groups to perform in the facility. Audience members were able to respond to the performance via wireless tablets. The results were displayed in real time on the video wall, allowing the players to view if their listeners perceived the music to have energy or tension or if it seemed calm or neutral.

The LIVELab’s Disklavier grand piano
can record every nuance of a keyboard
performance, then play it back on the
instrument or replicate it on another
disklavier connected by the Internet.

“For the last third,” Fung recalls, “I could see that they were really with us. I often feel electricity from an audience, but this time I really felt it in my scalp.” Now the quartet is working closely with Trainor on using physiological measures from audience members in real time during performances to help them fine-tune their attempts to connect with, educate and respond to a younger audience.

The LIVELab is also generating interest beyond the performance community. Sound recording firms and media producers are eyeing the space as a venue for testing new technologies. And hearing aid designers plan to use the LIVELab to simulate challenging real-world settings — from busy streets to noisy restaurants — as part of developing and testing new products.

“It seems like everybody who comes into the lab thinks of some new way to use it,” says Trainor. She’s not surprised; although LIVELab is rooted in performance, she believes it could be used to simulate interaction in classrooms, boardrooms, presentations and meetings, and that the results will generate new insight into human dynamics in education, business and community life. “If you look at the challenges we're facing in the world, almost all of them involve human interaction.”

And human interaction — in music, performance and beyond — is what the LIVELab is wired to explore.

Read more about the opening of McMaster University’s LIVELab.