Sign In

Remember Me

Invited Essay — Future of Work: Musical Performance in Virtual Reality

Future of Work: Musical Performance in Virtual Worlds

Music Island: Five Years of “Virtual Concerts in the Park”

Editor’s Introduction

Today’s guest post continues our theme on the Future of Work in a somewhat different direction: the performing arts and music.

The rise of digital music and music sharing has challenged the existing economics of the music business. And at the same time these very technologies offer artists new ways to connect with fans, customers, and other artists. A shared virtual reality environment presents an entirely new creative means for performing music and sharing music and other art as well as potentially providing a platform for selling or exchanging it.

Linda Rogers has been organizing musical performance in Second Life for half a decade. Linda shares some of the history, issues, and challenges she encountered along the way to five years of virtual reality performances in which she quite arguably and somewhat accidentally invents an entirely new art form — the shared virtual reality musical performance.

I hope you all enjoy reading this article as much as I did.

Introduction:

Time was that when orchestras and ensembles wanted to reach out to find new audiences, they took their instruments out to the park and shopping malls where teens and young families tended to gather. In today’s world, increasingly, people are meeting online.  Together with some intrepid, tech-savvy musicians and ensembles, I’ve been out there in the virtual community, presenting virtual concerts in the new virtual park of Second Life since 2007.  This article attempts to answer the following questions: Who has been playing classical or “art” music professionally in Second Life? What were their expectations, their experiences and what are the lessons we have learned?  Who are our audience members?  What do they find rewarding about concerts in virtual reality?  And lastly, what lessons do virtual concerts have to offer to default world concert presenters?

In a time-stressed world, where recession is forcing more workers to work longer hours [1], the classical symphony orchestra and indeed all of the live performing arts increasingly compete with internet, television, movies and video games for the shrinking leisure time available.  Many of us are spending longer hours at work around the planet.  Over 60 % of Americans report listening to and loving classical music,  but only 5 % report regularly attending classical concerts by their local symphony orchestra. [2]  The most frequent location for listening to classical music is now the family car while traveling between work and home. Clearly enjoying music through virtual extensions of our selves online is a logical next step in the evolution of performance.  It is here.  It is happening.  And thousands are finding listening and performing opportunities in virtual reality environments.

What is it?

What do we mean by a “live concert” in virtual reality:  Live concerts come in three basic types, 1) music-streaming with avatar animation; 2) video-streaming and 3) multi-media.  During a music-streaming concert a musician or ensemble performs live in their home or studio and broadcasts using some rather simple to use software (Winamp, Simplecast, SAM or Nicecast for Mac) and basic, low-cost equipment (microphones, external sound mixer, computer)  sending the pre-mixed sound to a Shoutcast server.  This music signal is imported into a particular address in Second Life at the MP3 quality level and shared in live time with individuals from around the globe who choose to log on and share a musical experience.  The additional challenge for the performers is being present in the virtual world in their own avatar shapes, playing animations and interacting with the audience.  Some employ assistants to help with either avatars or broadcasting. The April 20, 2011 concert by Milan’s Ars Cantica Choir and Alessandro Marangoni, excerpted on YouTube provides an example of a music-streaming concert with avatars on stage[3].  Some ensembles prefer to stream live video into Second Life, usually via Quicktime.  (Flash video capability is possible within Second Life but still problematic at this writing.)  Thomas Coard of the Fribourg Conservatory in Switzerland has produced a number of video concerts as part of his early music program as demonstrated in his 2010 broadcast with the Faire Winds ensemble [4].  Sometimes video streamed into SL is filmed in the context of a concert before a live audience.  Often, organizers will enable interaction between the live and virtual audiences for a “mixed reality” event.  This second type of event perhaps holds the most promise for large-scale adoption by music presenters, as it can be built on and augment existing music series events.  The third, and most rarely produced variety of musical event involves using the virtual platform itself in the creation of new art forms.  This can involve using Second Life sounds  (built in midi sounds) to create music, as performed by the Avatar Orchestra Metaverse [5] or combining streamed music with particle effects, scripted objects, images, and animations only possible in a virtual environment.  In Montgomery and Helsinki’s 2009 “Storm in a Teacup”[6], virtual audience members were asked to board a virtual tea bowl.  Once seated on the bowl, Montgomery’s original electro-acoustic score was cued to begin as the tea bowl vehicle carried the audience through the 3D display of Helsinki’s photos that had served as inspiration for the composed music.  Storm sounds, breaking windows and boiling kettle sounds punctuate the score as sound samples collected by and used by the composer. At a dramatic moment in the work, audience members are dumped like tea-leaves scattered by a fortune-teller, falling willy-nilly 200 virtual meters . . . to the last movement of the work.  On the ground, the music completes with a final calm movement amidst photos of a Japanese tea ceremony.   Other examples of not-possible-in-real-life events include the collaboration of a composer and a particle artist for music and visual effects with scripting of particle behaviour in live chat and live time, and a collaboration between a composer and a 3D sculptor to create a visual and musical environment simultaneously.

Figure 1. Audience whirls amid music and light during a 2009 virtual reality event with composer, Paul Kwo
 

The origin of Music Island concerts:

When I first ventured into Second Life in 2006, I brought my personal curiosity about virtual worlds, and some years of experience as an administrator and producer in classical music, new music and theatre in Canada and the USA.  The existence of live music in virtual reality was a surprise to me and I quickly became a volunteer with a classical music group that was assisting a handful of musical pioneers who were giving concerts in settings that were not always welcoming or well-engineered for the presentation of serious art music.  The venues we first encountered were usually of two types, clubs or historical simulations of castles or concert halls.  Virtual clubs are the most prolific of music venues in Second Life to this day.  They are often created to attract traffic to commercial simulations for the purpose of selling products, renting virtual property or both. Clubs are usually designed for a mixture of live music and DJ’s spinning recorded music.  Music at clubs is the backdrop for socializing and dancing.  Music with difficult rhythms and requiring undivided listener attention was unwelcome and unsuitable in SL’s commercial clubs. In addition sim owners operating clubs were not always educated or accommodating to the needs of live musical events in Second Life, nor were they always present to deal with difficulties.

Figure 2. 2007 A griefer (troublemaker) attacks an early music concert in a club setting with noise and visual spam
 

Historical sims offered a different challenge.  Many were built for physical accuracy and beauty, not realizing that the visual detail in the number of “prims” (basic building blocks within Second Life) and the high resolution graphics used would lower the capacity within their sim.  Servers would become overloaded trying to draw and re-draw such complex scenes plus avatars attending the events.  We tried to offer concerts in some simulations that crashed (abruptly shut down, logging off all avatars) with only 10 to 20 avatars in attendance.  Some of these simulations were built for history education purposes, but far more were built by individuals wishing to roleplay as royalty in their own fantasy kingdoms. Once again, music was only a prop or the backdrop for socializing.

In addition to the lag factor in texture dense simulations, some musicians felt uncomfortable with the message sent by the presentation of classical music in castles and historical simulations.  They did not want to cast this music as something dead and mounted in a museum, nor did they want to perpetuate the snobbery of court dress.  Some of our founding members’ goal in bringing their music into virtual reality was to expand the reach of their music to new audiences, not to duplicate an elitist atmosphere.

 

Figure 3. A typical Second Life historical or vanity opera house

With all these thoughts and opinions percolating in our minds, we had a day-long consultation with interested artists, builders, sound technicians and experienced virtual event organizers.

Venue Design:  We decided on a look and feel of a seaside summer music festival, the kind of place where people could be expected to attend concerts in shorts or sun-dresses, and where a simplicity of design would be welcome and natural.  Our builder, Jon Seattle (SL), kept the textures low resolution, limited the number of textures and depended on simple basic shapes for the stage and seating.  Incorporating one of the better suggestions from our consultation, the venue, Music Island at Sea Turtle[7]  was created on a sim line, so that the stage area and audience are present virtually in two separate simulations in the same region.  This arrangement insulates performers from the lag (slowdown) caused by audience density, allowing them as much stability as possible in virtual reality while they are performing.

  Figure 4. Music Island at Sea Turtle Island’s simple summer music festival amphitheatre design

 

Who is performing in virtual reality and what are their goals?

Musicians in virtual reality are professionals at every stage of development, amateurs, music students and music educators.  The goals they bring with them are varied and some are more easily achieved in the virtual environment than others.

Because virtual reality is so new, some leading artists and organizations have managed to create a great deal of buzz with concerts in Second Life. When the Royal Liverpool Philharmonic streamed a video of a concert into virtual reality, it created headlines in the New York Times[8]  and the Liverpool Telegraph, [9] and many other newspapers due to the novelty of the activity. When major artists such as pianist Lang Lang have appeared in Second Life, they have been rewarded with both coverage in the mainstream press and also through social media spin with amateur YouTube videos and articles in the blogosphere.

For emerging artists, Second Life usually is only one of a number of online strategies they are exploring for promotion and dissemination. The virtual community is an international cyber tribe and savvy artists and arts organizations are finding that virtual reality is a great way to reach beyond regional and national borders. Second Life also has its own media, and the popular news, radio and television stations covering events in virtual reality (e.g. Treet TV) have proven to be a great way to communicate with this select group of high tech audience members.  For emerging artists, the Second Life requirement to have a pseudonym hobbles name recognition. As virtual reality evolves into a serious platform for art, corporations, and learning, this role-play with fictional names seem more and more out-dated. The 2010 Linden Lab move to allow “display names” within the Second Life community may, in part, be in response to those who wish to link their real names and careers to virtual world projects. On Music Island we have long been getting around the name recognition issue with posters, T-shirts and even virtual CD stands with links to performers’ real world websites.  Many performers such as the North Carolina, USA violin duo, Duo Appassionato prefer to superimpose real photos with avatar photos in their virtual concert advertising.

  Figure 5. Concert poster for a virtual performance by  the Duo Appassionato

 

Many are curious about the economics of SL music.  In 2010, I was invited to a meeting, along with other music presenters, with Pete Linden and some Linden Lab marketing consultants.  The marketers hoped to garner success stories about musicians performing within Second Life. It was clear after some period of discussion, that the marketing team’s idea of success was based on dollars earned by musicians in virtual reality.  While there are many reasons to perform online, the virtual economy does not easily scale to provide a livelihood for musicians.  200 units of the inworld currency equals less than one US dollar, and the value of many virtual goods and services scale at 1/200th of real world value.  Musicians who would earn anything from $250 to $5000 (US) in orchestras, small venues or concert hall solo concerto performances, are playing in virtual reality for the equivalent of only $20 to $50 dollars.  It is clear that they find rewards beyond immediate financial rewards.  However, for some musicians, the ability to monetize time that would otherwise be downtime or unpaid rehearsal time, is a small perk that other sorts of online dissemination (YouTube, Livestream) does not offer.  Indirect financial rewards are realized through online CD and Itunes sales, and promotion of ticketed real world concert events.

Much more important to professional musicians’ earning potential is the international networking that is possible within the virtual community.  Musicians have been assisted in organizing tours, have had free accommodation made available to them while touring and some have been able to break into new markets through virtual world connections.  Italian pianist Alessandro Marangoni made his North American debut in 2010 through a virtual world connection.  In the same year,  Torben Asp of Denmark was able to play for the first time in the US.  Duo Appassionato and the Schumann Duo, two virtual world duos, teamed up for a series of concerts on the US East Coast.  Virtual performances are, because of the current limitations of the medium, very small in numbers. A typical virtual audience size is only 30 to 60 listeners, but it only takes one good connection to make a booking or tour possible for an artist—and that concert or tour could be very lucrative.  At Music Island, starting in 2011, we have been extending the reach of virtual concerts through a Livestream web television station, http://www.livestream.com/musicisland) allowing individuals not present in virtual reality to watch concerts on the web or their mobile devices.

The opportunity to try out and perfect new repertoire is one way that many musicians use virtual performances.  Performer Mario Metsovarra Torres, (Arimo Teixeira in Second Life) has spoken about how his life as Arimo has helped him to find his own style, combining influences from latin guitar, rock, jazz and his Aztec heritage in the video “Life is Art”[10].   During 2009-10, music educator Thomas Coard of the Fribourg Conservatory in Switzerland decided to launch a trial educational project consisting of video concerts with his recorder students at the conservatory.  During summer 2009 we worked out the bugs in coordinating video streamed concerts from his home studio for broadcast at Music Island.  Throughout the Fall and Winter season, students performed at concerts prior to their performance exams at the conservatory.  The ability to perform their material before a virtual audience proved a good test of where more work might be needed to make performance secure under pressure.  A unprecedented 100% of Thom’s students received passing grades on their performance exams.

Some musicians use virtual performances to keep skills sharp that they have little opportunity to exercise in their professional careers.  Orchestral musicians lack opportunities to play challenging concertos but can do so easily in virtual reality.  Many musicians play more than one instrument but their career demands them to play one more often than the other.  In Second Life a violist can exercise their violin skills, an early music specialist composes electronic music and a classical pianist can sing arias.  These are actual examples of the way in which some Music Island performers are using virtual concerts.

The aspect unique to classical music in Second Life is the quality of the audience experience. A podcast or webcast can deliver the same sound quality and serve promotional purposes, but all of these are solitary experiences. By contrast, concerts in Second Life are joyfully social; audience members are celebratory in their anticipation and appreciation of the music in a way rarely matched in real life concerts.  Small wonder then that some musicians and music educators with more long-range goals, find virtual reality a great asset to audience development and to provide music appreciation added value to their concerts. Unique to the medium, listeners silently text appreciative comments, hurrahs, and questions that they hope someone more informed will be able to answer.  Some types of activities just work better in virtual reality.  In 2010, the Schumann Duo was scheduled to perform a concert but due to an illness decided that they were best to offer a “open rehearsal” instead.  I was concerned because usually “open rehearsals” can be difficult to follow in the best of cases.  I was not sure how it would go over in the virtual medium.  However, because both performers were wearing head sets, the audience could hear every word, and the performers could see the audiences comments and questions on screen and respond almost immediately.  Certainly it was the most successful and lively open rehearsal I have ever witnessed.

Conversations quickly reveal that many of those attending classical concerts in Second Life have little or no experience of live classical music.  In a survey of audiences at two concerts in 2008, over 80% of those in attendance had never been at a live classical music event.  While music series in the mainstream arts are having trouble attracting new audiences to conventional concert stages, it is demonstrated that the Internet virtual audience is open to the experience of art music. It seems worth it to step into the virtual world to reach out to this new audience.  However at the time of writing, skepticism still seems high.  The cartoon like appearance of avatars, the inaccuracy of playing animations and the reputation of virtual worlds as places for adult entertainment are all reasons that musicians give for being dis-interested in virtual performances.  Those who understand the appeal are sometimes daunted by the technical requirements that include upper-end computers, a good graphics card and a high-speed internet connection.  Over the years some projects that have been discussed have later been cancelled due to lack of adequate computer resources, lack of equipment (such as a mixer), lack of technical expertise, and lack of support from colleagues or organizations for such an experimental effort.

As an arts manager, my initial interest in virtual music was learning but after becoming an evangelist for the medium, I hope to be able to create a sustainable virtual arts project.  At this point, the economic model for that is elusive.  In the industrialized world, a typical classical music organization derives 50% or less of its income from earned revenue (ticket sales, merchandizing), the remaining 50% is derived from a combination of government grants, foundation support, corporate sponsorship and donations from individuals and businesses.  In the virtual world, the norm for live music is to provide free events with an opportunity for individuals to donate through “tip jars”.  As long as there is a wealth of free concerts in virtual reality, it would be difficult for any series to introduce ticketed events, apart from one-time appearances of a major artist. The numbers able to attend virtual concerts are limited by the technology.  While in theory 100 avatars can fit in an empty simulation, the reality is that once the sim is filled with buildings, some simple scripted objects like notegivers, breaking waves in the ocean and sea-gulls overhead, the capacity of a sim is reduced.  During the four years Music Island has been presenting concerts in virtual reality, I have seen avatars become increasingly complex and their ARC (avatar rendering cost) increase dramatically.  Our record for numbers of avatars attending a concert at Music Island was 83 in 2008.  I doubt with 2011 avatars, that would be possible.  With good 3D building practices, today it is possible for a sim to remain stable with 50 avatars in attendance.  A venue built on the junction of 4 simulations would be able to have a maximum of about 200 avatars at an event.  This is the current situation for virtual audience.  While this might seem small to rock music enthusiasts used to concerts in stadiums, many chamber concerts and new music concerts are performed in small halls with intimate audiences. However, when music organizations think of electronic dissemination, targeting large numbers is often in the forefront of their thinking.  Virtual reality is unable to provide those numbers.  Instead, virtual performances, are better thought of as being similar to the small intimate audience development events that orchestras and ensembles often provide as adjunct activities.

It is the nature of arts funding provided by governments and foundations to be national or regional in scope.  The goal of funding bodies is to support the arts and culture of a geographic area for the good of national arts, citizens and local tourism. But music in virtual reality is very international.  At Music Island, I have created a flag for every new country a concert has been streamed from by a musician.  Currently there are 18 flags flying.  When we ask our audiences to shout out where they are logging into the concert, typically we find people attending from over 15 nations.  Time zone, not national borders dictates the origin of virtual concerts.  Such an international music experience, while one can argue serves the goals of art, artists and even world peace, is regrettable, ineligible for traditional arts grants.  A nationally-based demonstration project with an international component, might be palatable to traditional funders but only with the buy-in of a national, regional or municipal orchestra, ensemble or music series.

With no earned revenue, small private donations from “tips”, Music Island has continued to exist as a voluntary organization.  Currently, we operate with inkind donations of space, performances and administration as well as brokered sponsorships of stream and some incidentals.  Donations are sufficient to cover stream costs and the minimal contribution to tier costs requested by our hosting region. This fragile balance makes us fearful for the future of our series and we know that succession planning demands a more sustainable model.

What lessons can bricks and mortar concert halls learn from virtual concerts?

Virtual concert audiences gave as the top reason for preferring to hear concerts in virtual spaces, the one hour format and the ease with which they could attend concerts.  The Culture Track 2011 Study (AMS Research, NY) found that leading deterrents to attending cultural events were cost, inconvenient times and difficulty in getting to the event.[11]  The 2.5 hour concert format, usual in concert halls for the last century, may no longer fit the way that we live and disruptive technologies will make concert halls obsolete if presenters fail to embrace rather than fight technological innovation.  In order to adapt to the new reality, orchestras and ensembles may have to think about the way that they schedule and disseminate their programming, something that will take buy-in from all stakeholders.  As the Knight Foundation reports, “leisure trends are moving people further and further away from fixed, static experiences, and anyone who thinks that classical music concerts are somehow immune to the shifting sands of cultural tastes and consumer behavior is sadly mistaken.” [12]  The issues in implementing shorter concerts and in disseminating concerts virtually involve negotiating new clauses in entrenched union contracts that have been based on both traditional concert length and traditional forms of recording and broadcasting.

Virtual concert attendees value the ability to join the concert, or leave it, at their convenience. This is a much-valued attribute of park concerts enjoyed by generations of families, able to pick up and go as soon as the fussiest child could no longer be jollied into a better mood. These concerts have traditionally been free.  Indoor bricks and mortar concert halls are designed in a way to maximize seating to monetize the space as much as possible.  The result is an inability for people to take a seat without disturbing others.  Because of this, traditionally late-comers are not allowed to take their seats until a break In the program enables them to take their seat without disturbing others.   Artistically, the concern is that the experience for the audience is compromised when works are not heard in their entirety.  Musicians train for decades to present music they highly value to appreciative audiences. While they are tolerant and understanding of the needs of family audiences to come and go to tend to the shorter attention spans of children, they can be demoralized by the inattentiveness of adult audiences. The question then arises, “why is this same restlessness not cited as demoralizing to SL musicians. The answer may be that –like the parent departing with a restless child— sudden departures are often not volitional as virtual audience members may be forced to leave early due to a computer connection failure.  Offering concerts simultaneously in real and virtual spaces, as we did in February 2008 with the Leeds Sinphonia concert at Music Island in the virtual work and in Leeds England in the concert hall, allows virtual audiences to come and go at will while the musicians and audience in the concert hall remain undisturbed.

Another plus cited by the virtual audience is their freedom to text during concerts, making the concert more of a group experience.  In addition to the social aspect of texting, program notes, links to additional information, and images can be conveyed to the audience, making the experience a more guided and interactive experience.  Some orchestras in the USA have experimented with dedicated PDA devices that audience members can use to access livetime listening aids, texting them information about the works they are hearing “Consumer reactions to the device have been encouraging.  Evaluation results suggest that the Concert Companion, experience will appeal to those who prefer a more active learning experience in the concert hall and to those who are comfortable multitasking. The technology is expensive now . . . it is only a matter of time before real-time interpretive commentary is available to concertgoers on a regular basis, whether beamed wirelessly to a hand-held device or projected onto a screen above the stage, like supertitles. What orchestras have learned with certainty is that some people really enjoy embedded interpretation in their concerts, and others really don’t.” (Knight Foundation (2004) Smart Concerts: Orchestras in the age of edutainment, pg. 16) .  For those who enjoy embedded interpretation, concerts in virtual reality offer an affordable alternative to equipping an entire audience with handhelds, an alternative that hopefully will continue to be explored by more professional musical ensembles and orchestras.

No discussion of virtual music practice is complete without the mention of the word “fun”.  Musicians, audiences, and presenters all cite the way that virtual concerts have re-charged their batteries, opened up creative channels, and warmed their hearts.  This more than anything else is the reason we keep on doing what we are doing—even in the face of financial and technical challenges.

 


[1] Snowdon, Graham (2011, June 20). Employees prepared to work longer for less, Guardian .  Retrieved from http://www.guardian.co.uk/money/2011/jun/20/employees-work-longer-for-less

[2] Knight Foundation (2003).  Magic of Music, brief no. 1, pg. 3

[3] Miranda, Kate (Producer) (2011) Via Crucis [YouTube video] http://youtu.be/cduQQGhVooY

[4] Miranda, Kate (2010) Faire Winds Ensemble. [YouTube video] http://youtu.be/m5dL5Qq_wcY

[5] http://avatarorchestra.blogspot.com/

[6] Caulfield, Christine. (producer and composer) (2010) Storm in a Teacup  [YouTube video] http://youtu.be/WJfnWhJPMw8

[7] Linden Lab. [Second Life Slurl] http://slurl.com/secondlife/Sea%20Turtle%20Island/57/23/22

[8]  Midgette, Anne (2007, September 18). Watching a Cyber Audience Watch a Real Orchestra Perform in a Virtual World, New York Times.  Retrieved from  http://www.nytimes.com/2007/09/18/arts/music/18seco.html?ei=5124&en=06cf09c87b780968&ex=1348632000&adxnnl=1&partner=permalink&exprod=permalink&adxnnlx=1309285601-XZqu8acnT9Or7j22vvup9A

[9] Norris, Jeffrey (2007, September 17) For a new concert experience, get a second life. The Telegraph.  Liverpool, UK.  Retrieved from http://www.telegraph.co.uk/news/uknews/1563192/For-a-new-concert-experience-get-a-second-life.html

[10] Wise, Deborah (producer) (2011) Art is Life [YouTube Video] http://youtu.be/4lh7GDGFROw, part of a doctoral dissertation of the same title for the University of Colorado

[11] AMS Research (2011) Culture Track 2011, pg. 42

[12] Knight Foundation. (2004) Smart Concerts: Orchestras in the age of edutainment, issues brief #5, page 5.

7 Comments

  1. Kate Miranda is the perfect person to write this article! I hope she will make this a regular event so the world can track, learn and experience the growing world of virtual music performances through her intelligent, astute, and eloquent observations! Well done Kate!

  2. Thanks all for the positive comments and I want to thank Peter for inviting me to share my thoughts here. I really do hope that our experiment at Music Island might be a small butterfly flapping its wings that will change the future of classical and “art” music.

  3. Again I am expanding my understanding of the virtual worlds thanks to you.

    You are so full of good ideas. Keep it up, Kate!

    Stephan66 Ewing

  4. Excellent article: good historical perspective on classical music in Second Life and clear picture of benefits of the immersive, 3D virtual platform for live music.

  5. It’s great to read such a positive and well-documented article on virtual presence. I can use this in my own research, and I’ll pass it on to others

    Thanks, Kate!