Breaking the Wall
Playful interfaces for audience participation and artistic expression in musical live performances
ART-BASED RESEARCH (FWF PEEK)
With Breaking The Wall, we explore how to use technology to involve the audience in music performances. We design and develop different interactive systems together with artists. These systems will be deployed in a public performance in spring 2017 in the Kuppelsaal of the Vienna University of Technology.
The main research question we want to answer is:
“Which new ways of artistic expression emerge in a popular form of music performance when using playful interfaces for audience participation to facilitate interactivity among everybody involved?”
The results of the project will be situated at the interdisciplinary intersection of art, music and technology. We will present structured and evaluated insights into the unique relation between performers and audience, leading to tested and documented new artistic ways of musical expression that future performances can build on. We will further deliver a tool-set with new interfaces and collaborative digital instruments. The results of the project will be highly relevant to musical practice, and contribute to theory from the areas of media arts and musicology.
This section reports about ongoing work and prototypes created together with musicians.
DJ-mediated Distributed Smartphone performance
Together with the composer, DJ and musician Electric Indigo (Susanne Kirchmayr) we develop a performance with the goal to break up the traditional relation between artist and audience by making the audience an extension of the artistic intent and expression of the performance. By using an app developed during the course of the project (Poème Numérique by Christoph Bartmann) the smartphones of the attendees are transformed into mobile sound sources. On the one hand these can be manipulated in real-time by the composer and musician, on the other they are freed of a static location-dependance by the behaviour and movement of the audience. This way the audience (as a swarm) is able to independently affect the distribution of sound thereby acting as a dynamic mediator between performer and performance space. This bridge is realised technologically through the use of ultra sound (high-frequency sound IDs), which connects the audience's smartphones to the performer without the need to join a WiFi or using a mobile phone network, thus lowering the technological threshold for participation.
multi-sensory Audience experience
The collaboration with the performance artists Fuckhead (Didi Bruckmayr, Chris Bruckmayr et al.) focuses on the technological and dramaturgical connection of body, sound, light and room. Through this multi-sensory experience, the audience should be able to reflect on and question digital surveillance and technological authority as it may be part of technology-mediated audience participation. This kind of embodied and technological intervention creates an experimental situation where accepted customs, habits, and eerines convene interchangingly. Technically, the audience will be tracked in the room and becomes a characteristic parameter of the whole performance architecture.
3D Tracking of Multiple Balloons Using Computer Stereo Vision
The developed balloon tracking system focuses on interactive audience participation in live concerts using multiple balloons. The balloon tracking system is able to track 3D positions of several balloons during the concert in real time. Technically, the system uses two static cameras placed in front of the audience to record videos of the balloons from two different angles. In every frame of these video streams, all balloons are detected in 2D. After detection in 2D, the system calculates their 3D position. Once the system has calculated the positions, it sends the position of each balloon using OSC. The system is designed to be robust enough to work in different kinds of conditions that may occur during the live concert, such as fogginess and dynamic lighting. The system is made in C++ using the computer vision library OpenCV. The presented system was implemented by Karlo Dumbović as part of an internship in collaboration with the Computer Vision Lab.
Game-based audience participation in live performances
Two diploma theses (Naida Comaga & Lukas Gartlehner, in progress) explore the use of game-based approaches to live music audience participation. The theses analyse the use of game mechanics in performance arts, putting a focus on collaborating with stakeholders from the areas of audio engineering, the arts and game design. Goal of the theses is to build a prototype for game-based interaction and evaluate it from artist and audience perspectives in a live concert setting.
Project-related publications, talks and theses are listed below.
Hödl, O., Kayali, F., Fitzpatrick, G. & Holland, S., 2016. TMAP Design Cards for Technology- Mediated Audience Participation in Live Music. In Workshop position paper for CHI 2016. pp. 1–4. (Download PDF)
Kayali, F., Bartmann, C., Hödl, O., Mateus-Berr, R., Pichlmair, M., 2016. Poème Numérique: Technology-Mediated Audience Participation (TMAP) using Smartphones and High-Frequency Sound IDs. In Proceedings of the INTETAIN 2016 8th International Conference on Intelligent Technologies for Interactive Entertainment, June 28-30 2016, Utrecht, NL.
Hödl, O., 2016. The Design of Technology-Mediated Audience Participation in Live Music. PhD Thesis, Vienna University of Technology. (Download PDF)
Bartmann, C., 2016. Exploring audience participation in live music with a mobile application. Master Thesis, Vienna University of Technology. (Download PDF)
Comaga, N., 2016. Game-based audience participation in live performances (tentative title). Master Thesis, Vienna University of Technology.
Gartlehner, L., 2016. Game-based audience participation in live performances (tentative title). Master Thesis, Vienna University of Technology.
Pfeiffer, D., 2016. Simulation in Audience Participation for Live Music. Bachelor Thesis, Vienna University of Technology.
Eigner, T., 2015. Investigating a color music interaction system in a collaborative music performance. Master Thesis, Vienna University of Technology.
Dumbovic, K., 2016. 3D Tracking of Multiple Balloons Using Computer Stereo Vision. Vienna University of Technology, Human Computer Interaction Group (E187/2) and Computer Vision Lab (E183/2)
Hödl, O. Music Computing from Opera to Avant-garde. Forum Medientechnik, All Around Audio Symposium. FH St. Pölten 23.11.2016
Hödl, O. Interaktive Publikumsbeteiligung in Live-Musik. Symposium PopNet Austria. Wien 10.12.2015 - 12.12.2015.
Kayali, F. Towards Bridging The Gap In A Musical Live Performance. Vienna Talk 2015 on Music Acoustics - "Bridging the Gaps". Wien, 16.09.2015 - 19.09.2015; in: Hödl, O. Kayali, F. Fitzpatrick, G. Holland, S. Proceedings of the Third Vienna Talk on Music Acoustics, (2015), S. 221.
EVENT COLLABORATION AND CO-ORGANISATION
2. Waves Music Hackday. WUK, Wien 01.10.2016
1. Waves Music Hackday. Alte Post, Wien 03.10.2015
People and Institutions
Breaking The Wall is funded by FWF PEEK. It is a collaboration between the Vienna University of Technology, the University of Apllied Arts Vienna and the University of Music and Performing Arts. The team is comprised of artists and researchers that cover diverse areas such as media arts, computer science, Human-Computer-Interaction, game design, musicology, ethnomusicology, technology and interface design.
Dr. Fares Kayali is a researcher, educator and game designer living and working in Vienna, Austria. Fares has headed the design and development of several digital games, which were finalists at IndieCade and the Independent Games Festival and he has presented his work at renowned international conferences including Games for Change, GDC, GLS, CEGC, Games for Health, and DiGRA. He works as a senior postdoctoral researcher at the Vienna University of Technology and the University of Applied Arts Vienna, and he is co-founder of the Positive Impact Games Lab (http://piglab.org). Fares lectures in game design at several Austrian universities. His research interests include game design, serious games, game-based learning, music and interactivity, media and game art, games for health and HCI in health care. He leads this project and also is principal investigator of the game-based learning project “Sparkling Games”.
Oliver Hödl holds a PhD awarded by the Vienna University of Technology. Currently, he works as a multi-disciplinary researcher and artist with an interest in Human-Computer Interaction, interactive and collaborative music making and multimedia art. He is affiliated as research fellow at the research group Cooperative Systems at the University of Vienna and the Institute for Design and Assessment of Technology at the Vienna University of Technology. In research he focuses on studying user experiences, art-based research approaches, and using HCI-related qualitative and quantitative research methods. His newly developed music instruments and interactive concerts have led to performances throughout Europe, USA and Australia. Furthermore, he has worked on projects around game design, urban mobility, safety and disaster solutions, and healthcare IT.
Prof Ruth Mateus-Berr is Associate Professor at the University of Applied Arts Vienna, Department for Design, Architecture and Environment for Art & Design Education, artist, scientist, and multi sensual design researcher. She graduated in 1991 from the University of Applied Arts in Vienna and received her doctorate in 2002 for her research on the design of the Carnival at the times of the National Socialism (Univ Prof Manfred Wagner, Univ Prof Karl Vocelka), published 2007.
Thomas Wagensommerer lives and works as a media artist in Vienna, Austria. He studied Digital Media Technology at the University of Applied Sciences St. Poelten, Philosophy at the University Vienna (without graduation) abd Transdiciplinary Art (TransArts) at the University of Applied Arts Vienna. He is a lecturer for Experimental Media at the University of Applied Sciences St. Poelten and a member of the research / artistic staff at the University of Applied Arts Vienna. He is an artistic assistant of the Breaking The Wall project.
Uli Kühn lives and works in Vienna as a sculptor, media artist and musician. He also is a lecturer and research assistant at the University of Applied Arts Vienna. His work is situated on the border between music, media art, performance and experimental digital film production and has been shown at venues including diagonale graz, ars electronica, flmfest dresden, soundframe, lames, o1 kunstradio, triennale linz lentos, moozak.org, flmfest dreseden and urban art forms.
Johannes Kretz, born 1968 in Vienna, studies (composition, pedagogy) at the music academy Vienna with F.BURT and M.JARRELL and mathematics at the University Vienna • 1992/93: studies at IRCAM, Paris with Marco Stroppa and Brian Ferneyhough • co-founder of NewTonEnsemble Vienna, of the international composers group PRISMA, of ikultur.com and of aNOther festival Vienna • teacher for music theory and composition at the conservatory of Vienna • since 1997: teacher for for computer music at the university for music and performing arts Vienna, since 2001 also music theory, since 2004 also composition, since 2009: habilitation in composition, associate professor • Since 2008: Head of ZiMT ("center for innovative music technology") of the university for music and performing arts Vienna, since 2013 dean/head of department of the Institute for composition and electro-acoustics • Founding member of NewTonEnsmble Vienna, of the European Bridges Ensemble, the international composers group PRISMA, of the performance duo TOUCHING, and of ikultur.com. Co-curator of aNOther festival Vienna together with Wei-Ya Lin und Mahdieh Bayat.
Susanne Kirchmayr aka Electric Indigo, born 1965 in Vienna, Austria, works as musician, composer and DJ. She started her career 1989 in Vienna, lived in Berlin from 1993 to 1996, founded the database and network for women in electronic music and digital arts "female:pressure" in 1998 and has received several awards in Austria and Germany. She is a music composer and sound designer for the project.
Dr. Didi Bruckmayr, born 5.4.1966, lives near Linz/Austria. Performance artist, singer, actor, producer, award winning multimedia artist, scientist and dancing fool. touring the clubs and international festivals since 1985. Frequently working for contemporary opera and theatre.
Chris Bruckmayr, Mag., Sound Artist. Produces and releases dark techno music under the name raum.null on vinyl only label Belgrade dubs / Belgrade. Performances with Fuckhead & raum.null at Ars Electronica festival 2014-2016 and at the Heart of Noise festival 2016. Show producer at Ars Electronica SPAXELS GmbH.
Dipl.-Ing. Christoph Bartmann is an Austrian information scientist. He studied business informatics at the Vienna University of Technology and works as a project manager and IT-consultant. His interests include mobile applications, music, audience participation as well as data and business process management.
Bsc. Naida Comaga is a master student at TU Wien in Vienna, Austria, majoring in Media Informatics. She is currently working on her Thesis, "Audience interaction through the game based interactivity/participation" under the supervision of prof. Peter Purgathofer and Dr. Fares Kayali. Her research is based on the ubiquitous technology interfaces, human computer interaction, game design and gamification of the different interaction interfaces in a two particular areas: music and performance.
Lukas Gartlehner is currently studying media informatics at the Vienna University of Technology. In his academic projects he focuses on the intersection of music, games and interaction design. He works as a software developer and writes his master thesis on audience participation as part of the "Breaking the Wall" project.
Geraldine Fitzpatrick, Peter Purgathofer, Thomas Gorbach (Vienna University of Technology)
Hande Sağlam (University of Music and Performing Arts Vienna)
Simon Holland (Music Computing Lab, Open University)
fares <at> igw <dot> tuwien <dot> ac <dot> at