|
Facilitating Musical Creativity: In Collectivity and Mobility
Click
here to download pdf version. (99kb)
by Atau Tanaka
Sony CSL Paris
6 rue Amyot
F-75005
Paris FRANCE
tanaka [at] csl [dot] sony [dot] fr
http://www.csl.sony.fr/atau
Keywords
Social music, locative media, peer-to-peer, ad-hoc networks, sensor interfaces, interaction, online communities
Abstract
We present two projects that facilitate collective music creativity over wireless networks. One system is a participative social music system using sensors and portable computing devices. The second is a musical work of locative media art that uses off the shelf mobile phones and GPS devices to create abstract audiovisual narratives from user movements through town. We discuss how network and community infrastructures affect the creative musical process, and the implications for artists creating new content for these formats. The projects described are real-world examples of collaborative systems as musical works.
Introduction
We are living in an increasingly mobile culture. But while society has the means to be mobile, what are the real issues in creating a culture of mobility? Commercial offerings simply attempt to transpose existing media, such as television, to portable devices. Meanwhile unexpected usage emerges in a grassroots fashion, for example the explosion of SMS. How can we leverage these forces and channel these energies to create deeper cultural experiences for a mobile society?
Personal music players revolutionized the notion of pervasive music over 25 years ago. The Walkman allowed the music lover to bring his/her music with him wherever he went, and allowed him to create a private aural sphere, defining his own musical environment while on the move, all while not disturbing those around him. Today’s personal music systems have merged with recent developments in networked information access. The container of the music has gone from physical media, such as the cassette or CD, to data files. The storage capacity of modern personal music players is vast enough for a user to carry his whole music collection with him at all times. Meanwhile in stationary fixed-line networks, developments in decentralized, peer-to-peer architectures have been seized upon by music fans to exchange, share, and copy music. This has led to heightened grass-roots inter-user interaction exemplified by the explosion of user generated content on sites like YouTube and Flickr. While this user level exchange can be regarded as a social phenomenon, it often takes place under anonymity, among users who do not know each other. Social software has applied complex network theory [1] and concepts from Social Computing to connect acquaintances in six degrees of separation networks. The work presented here merges the mobility of portable devices with the connectivity of social software to create pervasive, social music systems.
Networks and mobile devices have become new infrastructures on which to deploy music. However, typical commercial offerings are limited in their vision and do not exploit these media to their full musical potential. Downloadable music services commoditize music, treating it as a dead medium. Rather than using peer networks simply for file sharing, we propose a vision that respects both music and networks as dynamic, living forms. This vision is founded on a history of artistic practice that lends insight to musical creativity on these media. This leads to the vector of artistic creation pointing out potential new end-user content formats.
The work presented is situated at the intersection of electronic music, interaction, and social computing. The projects facilitate building creative communities as a fundamental part of the musical process. The act of musical creation thus incorporates notions of reciprocity, engagement, roles, belonging and awareness, to create live Social Music Systems.
There is substantial precedence of composers working with networks [2, 3, 4, 5] Early works used radio and telephone networks to allow audience participation to become a musical material itself [3]. Digital networks have been used for remote performance, as well as for local communication among musicians onstage [2]. The Internet has been used to allow audience participation [6, 7, 8]. The history of network music has been published in the literature [9]. A practice of network music compositions has led up to the current work presented here [10].
If networks have significant latency for real time applications, this means that they exhibit specific temporal characteristics. Seen in this light, it is the same musical concern as when composers consider the acoustical characteristic of the concert space in which their work will be performed. Composers of sacred music in the medieval age took advantage of the long reverberation times of cathedral architectures to write long, slowly moving lines, at times “hiding” secular melodies within the cantus firmus. Be-bop musicians exploited the intimacy and short reverberation time of jazz clubs to play blazingly fast solos. It is taken as a given by trained musicians that playing a be-bop solo in a cathedral would make little acoustical and thus little musical sense. Network transmission latency viewed in this way becomes the acoustic of the network, to be recognized and exploited as one does when composing for specific resonant physical spaces [11].
Interaction in music has been a concern since early digital systems of the 1970’s [12]. It is also claimed that interactivity is a fundamental quality inherent in music, independent of technology, digital or not [13]. A summary of the field of gestural music is found in [14]. We apply the notion of idiomatic writing, taken from traditional musical instrument composition, to identify salient musical characteristics of interactive technology [13]. Rather than viewing technical limitations as such, this view considers them as qualities giving these media their distinct musical voice to be exploited creatively.
Social Music Systems
Most prior work in the area of interactive music and network music is primarily aimed at trained musicians. The current work seeks to put those technologies to the service of democratizing musical creativity. The artistic mission, then, takes on a social aspect.
The projects described here facilitate building creative communities as a fundamental part of the musical process. The process of musical creation incorporates notions of reciprocity, engagement, roles, belonging and awareness, to create live Social Music Software.
The goal was to create a socially oriented music system not just for passive music consumption, but for participative creative listening [15]. The objective was not to coax listeners to become content creators, but to make the act of listening a process that was in a feedback loop with the evolution of the music being listened to, the notion of active listening.
The music was to be a dynamic media flow that represented the collective action of the community of listeners. In this regard, it needs to represent the social dynamic of the group. At the same time, each user needs to have a sense of his own part in the whole. By balancing these two requirements, we hoped to address the issue of reconciling personal listening with social connectivity. We introduce the notion of reflexive translucence to discuss these needs.
The state of activity of each individual user needed to be detected in a non-invasive fashion. We wished to pick up subconscious response to music listening to see how the music might be affecting her instantaneous behavior, stress level, excitation. The state of the group of listeners as a whole needed to be captured at any given moment. This meant sensing gross behavior of the community’s constituent members in a single flow of information. This includes presence and geographical/topological localization. The choice of technique would depend on the scale of the topology, be it in a contained interior space or a wide urban environment.
A music engine would be used to generate a music based on the contextual information. The musical result would then be rendered as a flow of digital audio data, to be delivered to each connected listener by streaming over mobile wireless networks. Since all processes, from incoming contextual information, to outgoing audio delivery, take place over the network, there is inherent latency and transmission delay. This would have an impact on the feel of the system for each user
Malleable Music
Malleable Music places similar concepts of participative music mixing in a mobile environment. It uses wireless ad-hoc networks to create participative, shared musical experiences amongst listeners.
Mobile devices endowed with sensors are in communication with each other and with a music engine. Communities of friends enter a virtual space, where each person is identified by a distinct part in the music. Subconscious gestures made in the act of listening — be it gripping the device, or tapping along to the beat — are picked up by sensors. The server side music engine aggregates this user-level context with system-level context based on localization, transforming the ensemble into a musical context. This musical context modifies a piece of music on the fly, resulting in a “social re-mix”, a single audio stream of music that contains the social information, and that is listened to by all participants as a shared musical experience.
Visually an avatar represents each user in the virtual space. By associating a distinct musical part with each user, we create a musical analogue, the notion of a sonic avatar. In the way that the movements and deformations of the visual avatar give an indication of the state and actions of each user to the others, the relative amplitude of any given part in the music represent the presence of the user represented by that sound. Treatments on the sound such as filtering and echo effects communicate the behavior of that user on their device. The total music then represents two contexts: personal and social. The mix of parts is an indicator of human presence, or social context. The effects and variation of any given part directly reflect individual user gesture captured by sensors on the device, and represent the personal context of each participant.
Malleable Music was used in two different types of musical situations. First, to remix a well-known pop song. Each of three participants took a part from the song — one becoming the voice part, another taking the percussion, and the third represented by the brass section. The original recording was analyzed and dissected to isolate segments where these individual parts were prominent. The music engine filtered each part based on incoming sensor data from each device, and mixed them based on the relative proximity of the users, maintaining the rhythm and tempo of the music. What was originally a three minute pop song became an extended game-like experience that that created a sense of common engagement on the part of the users. The original song was recognizable, with the act of listening enhanced by a sense of proactive involvement in the evolution of this “malleable” version.
|
 |
| |
The second situation where the system was used was in a theatre workshop (Figure 1, see above). The participants were a group of actors and actresses, playing out a script in a neutral “black box” space. The users had free circulation in the space. The elaboration of this blank space by a theatre set provided an “environment”. Movement of each participant was monitored continuously by a single overhead video camera. The devices tracked each individual user’s gestures (the personal context), and the camera system tracked the location of the three devices (the community context). These two layers of contextual information fed the synthesis and generation of a sound environment that was played back to each device as well as a surround-sound speaker system in the space. The actors/actresses to acted out a script, articulating and moving in an expressive way. The system created the sound environment for the piece based on these movements and gestures. As the participants created the soundtrack of the theatre piece through the act of playing out the work, there was an intimate link between the theatrical expression and the accompanying sound environment.
Net_Dérive
Malleable Music demonstrated the possibility of responsive music systems interacting with small groups of associated users. The detection of human presence took place either in an indoor setting in the case of the theatre workshop, or through a location-mapping simulator. The focus of Malleable Music was to put in place a generative music engine that could create musical streams that responded to both personal and community contexts. This was applied to a known piece of music as well as to an abstract soundtrack for a theatre piece. The next step was to try these techniques out in a real mobile environment, on the streets, out in the wild.
By transposing the musical act from stage to street, we displace the locus of creation and creativity, not just physically, but socially. Mobile communication devices are meant to connect groups of people. Musical concerts, similarly, are situations that bring people together for a common purpose. The difference with the previous situations is that mobile users dispersed in the streets do no share a physical space and lack the possibility of eye contact in the way that musicians together onstage or an friends attending a concert together.
Acts of music making and music listening take place in direct human contact. Could these be extended to network mediated contact? Can we elicit commonalities to make a community-based musical process, creating a shared experience among users?
With Net_Dérive we sought to answer these questions in the context of a musical work of locative media. We sought to extend the practice of location-based artworks by using the music engine from Malleable Music to sonify the movements of participants tracked by GPS devices.
The mobile system consisted of a piece of wearable computing – a spandex scarf in which were embedded two mobile phones and a Bluetooth GPS module (Figure 2, left). The technical architecture of the system has been previously described [16] as has the conceptual basis recreating the games of the Situationists [17]. Here we describe what the experience was like for the participants.
While the users wandered the streets on their own, out of visual contact of the others, the fact that they started their journeys at the same time, each donning the white device-laden scarf and headphones, gave them a sense of belonging to a common mission. Once off on the walk, the wearable, with the graphics displayed on the phone screen, and the soundscape streamed and heard on the headphones created for each user a kind of bubble around them. A radar-like display onscreen gave the participant an idea of their location relative to the others. This had a direct correspondence to a series of rhythmic blips generated by the music engine based on the GPS data. This was mixed with processed street sounds as picked up by each mobile and upstreamed to the engine to be treated, mixed, and streamed back to all the participants. This total musical output that was a blend of geographical representations and found-sound transformations gave the participant a sense of connection to the other users, and an abstracted yet heightened awareness of the surrounding urban environment. This is quite distinct from the feeling of isolation that most users of personal music players report.
Conclusions
Artists have always invented new forms in response to new media. Communities of users have a tendency to self-organize and create new forms of communications. Here we try to catalyze creativity along these lines. We feel that notions of social computing coupled with artistic creation can combine to point out ways in which technology evolution can be assimilated directly in cultural production, ultimately leading to possible new forms of musical content. We discuss how network and community infrastructures affect the musical creative process, and the implications for artists creating new content for these formats.
The projects described here look at the creative potential of sensor systems, network infrastructures, and collaborative systems. They create new contexts for artistic practice that redefine the respective roles of artist and spectator. They respond to the claims of the democratizing force of technology by using such architectures to conceive works that open up the creative process in such a way that they include the end-user in a process that is not completed until the moment the work is beheld.
Despite these changes or shifts in the role of the author, composer, or artist, this does by no means obviate their existence nor put in question the notion of an artistic work. The goal is not to create totally user driven systems, nor random processes. Instead, the act of authoring for such systems requires the artist to conceive of open forms that nonetheless articulate his original creative vision, that become an act of creative expression all while letting go of absolute control, and forsaking making a frozen art object. Instead, the creative vision must remain intact through shifting contexts and situations while the work is rendered at run time as a function of environmental variables. The work of music is not finished until perceived, but is nonetheless an artwork, and is a work of the artist in question, albeit with the participation and contribution of the viewers and listeners.
This for us is the true notion of interaction – not a simple technological consideration, but a social one where deterministic and hierarchical processes are opened up to shared processes. By conceiving of artistic structures in this way, we open up the possibility for conceiving of new forms and formats. These forms are characterized by a modularity, where structure and asset are separated, where open forms allow for an implicit form. It becomes an act not just of empowering the audience, but of sensitizing the artist to a new set of responsibilities, one where the locus of the creative act is displaced. Creating work for these new contexts requires a coming to terms with the notions of sociability and community and their representations in abstract creative works.
References
1. A.-L. Barabasi and R. Albert, “Emergence of scaling in random networks”, Science, Vol. 286(5439) pp. 509-512, (October 1999).
2. S. Gresham-Lancaster, “The Aesthetics and history of the Hub: The Effects of Changes Technology on Network Computer Music”, Leonardo Music Journal Vol. 8: p. 39-44. (1998).
3. Neuhaus, Neuhaus, http://www.ubu.com/sound/neuhaus_radio.html
4. G. Föllmer, Soft Music, http://crossfade.walkerart.org
5. Föllmer, G. and E. Ungeheuer, Netzmusik-Stand der elektroakustischen Musik oder Musik von anderen Planeten? Ein Printchat, in Elektroakustische Musik - Handbuch der Musik im 20, E. Ungeheuer, Editor. 2002, Laaber-Verlag: Jahrhundert Band 5 Laaber. p. 303-316.
6. Yamagishi, S. and K. Setoh. Variations for WWW — Network Music by MAX and the WWW in International Computer Music Conference. 1998.
7. Stelkens, J. peerSynth: a P2P Multi-User Software Synthesizer with new techniques for integrating latency in real time collaboration. in International Computer Music Conference. 2003.
8. Jordà, S., Faust Music On Line (FMOL): An Approach to Real-time Collective Composition on the Internet. Leonardo Music Journal, 1999. 9(5-12).
9. Barbosa, A., Displaced Soundscapes: A Survey of Network Systems for Music and Sonic Art Creation. Leonardo Music Journal, 2003. 13: p. 53-60.
10. Tanaka, A., von Telepräsenz su Co-erfahrung: ein jahrzehnt netzwerkmusik (From Telepresence to Co-experience: A Decade of Network Music. Neue Zeitschrift für Musik, 2004. 5: p. 27-28.
11. Tanaka, A., Composing as a function of Infrastructure, in Surface Tension: Problematics of Site, K. Ehrlich and B. LaBelle, Editors. 2003, Errant Bodies Press: Los Angeles.
12. Spiegel, L., Graphical GROOVE: memorial for the VAMPIRE, a visual music system. Organised Sound, Cambridge University Press, 1998, 3(3): p. 187-191.
13. Tanaka, A., Interaction, Experience and the Future of Music, in Reinventing Music: Social and cultural impacts of new music technology, O'hara and Brown (eds), (2005), Kluwer Academic Press: London.
14. Wanderley, M. and M. Battier, eds. Trends in Gestural Control of Music. 2000, IRCAM: Paris, France.
15. Tanaka, A., Tokui, N., Momeni, A. “Facilitating Collective Musical Creativity.” Proceedings ACM Multimedia, Singapore. 2005. pp. 191-198.
16. Tanaka, A., Gemeinboeck, P. “A Framework for Spatial Interaction in Locative Media.” in Proceedings of New Interfaces for Musical Expression (NIME06), Paris, 2006. p. 26-30.
17. Knabb, K. Situationist International Anthology. Bureau of Public Secrete: Berkeley, CA, U.S.A. (2002.)
Author Biography
Atau Tanaka was born in Tokyo, and was raised in the U.S. He bridges the fields of media art, experimental music, and research. In Paris has worked at IRCAM, was Artistic Ambassador for Apple France, and conducts research at Sony Computer Science Laboratory (CSL) Paris. Atau performs with sensor based musical instruments and composes for mobile network systems. His works include solo and ensemble concert works and exhibition installations. His work has been presented at Ars Electronica, SFMOMA, Eyebeam, V2, ICC, and ZKM. He has received support from the Fondation Daniel Langlois and has been mentor at NESTA. For the 2007 season, he is Artistic Co-Director of STEIM in Amsterdam. He has joined Newcastle University in the U.K. as Chair of Digital Media.
Citation reference for this Leonardo Electronic Almanac Essay
MLA Style
Tanaka, Atau. “Facilitating Musical Creativity: In Collectivity and Mobility.” “LEA-ACM Multimedia” Special Issue, Leonardo Electronic Almanac Vol. 15, No. 5 - 6 (2007). 10 May 2007 <http://leoalmanac.org/journal/vol_15/lea_v15_n05-06/atanaka.asp>.
APA Style
Tanaka, Atau. (Apr. 2007) “Facilitating Musical Creativity: In Collectivity and Mobility,” “LEA-ACM Multimedia” Special Issue, Leonardo Electronic Almanac Vol. 15, No. 5 - 6 (2007). Retrieved 10 May 2007 from <http://leoalmanac.org/journal/vol_15/lea_v15_n05-06/atanaka.asp>
|
SUBSCRIBE
to LEA | Site Guide | Contact
