Immersive virtual musical instrument
 | This article needs additional citations for verification. (November 2018) |
An immersive virtual musical instrument, or immersive virtual environment for music and sound, represents sound processes and their parameters as 3D entities of a virtual reality so that they can be perceived not only through auditory feedback but also visually in 3D and possibly through tactile as well as haptic feedback, using 3D interface metaphors consisting of interaction techniques such as navigation, selection and manipulation (NSM).[1] It builds on the trend in electronic musical instruments to develop new ways to control sound and perform music such as explored in conferences like NIME.
Development
[edit]Florent Berthaut created a variety of 3D reactive widgets involving novel representations of musical events and sound, that required a special 3D input device to interact with them using adapted 3D interaction techniques.[2]
Jared Bott created an environment that used 3D spatial control techniques as used in known musical instruments, with symbolic 2D visual representation of musical events.[3]
Richard Polfreman made a 3D virtual environment for musical composition with visual representations of musical and sound data similar to 2D composition environments but placed in a 3D space.[4]
Leonel Valbom created a 3D immersive virtual environment with visual 3D representations of musical events and audio spatialization with which could be interacted using NSM interaction techniques.[5]
Teemu Mäki-Patola explored interaction metaphors based on existing musical instruments as seen in his Virtual Xylophone, Virtual Membrane, and Virtual Air Guitar implementations.[6]
Sutoolz from su-Studio Barcelona used real time 3D video games technology to allow a live performer to construct and play a fully audio visual immersive environment.[7]
Axel Mulder explored the sculpting interaction metaphor by creating a 3D virtual environment that allowed interaction with abstract deformable shapes, such as a sheet and a sphere, which parameters were mapped to sound effects in innovative ways. The work focused on proving the technical feasibility of 3D virtual musical instruments. Gestural control was based on 3D object manipulation such as a subset of prehension.[8]
Early work was done by Jaron Lanier with his Chromatophoria band and separately by Niko Bolas who developed the Soundsculpt Toolkit, a software interface that allows the world of music to communicate with the graphical elements of virtual reality.[citation needed]
References
[edit]- ^ Bowman, Doug; Kruijff, Ernst; LaViola, Joseph; Poupyrev, Ivan; Stuerzlinger, Wolfgang (2010). "3D User Interfaces: Design, Implementation, Usability". In Mynatt, Elizabeth D; Hudson, Scott E; Fitzpatrick, Geraldine; Hudson, S; Edwards, K; Rodden, T (eds.). Proceedings of the 28th international conference on Human factors in computing systems (CHI'10). New York: Association for Computing Machinery. ISBN 978-1-60558-929-9.
- ^ Berthaut, Florent; Desainte-Catherine, Myriam; Hachet, Martin (2010). "DRILE: an immersive environment for hierarchical live-looping". Proceedings of the 2010 International Conference on New Interfaces for Musical Expression (NIME10), Sydney, NSW, Australia. pp. 192–197.
- ^ Bott, Jared N.; Crowley, James G.; LaViola, Joseph J. (2009). "One Man Band: A 3D Gestural Interface for Collaborative Music Creation". Proceedings of the 2009 IEEE Virtual Reality Conference (VR '09). IEEE. pp. 273–274. ISBN 978-1-4244-3943-0.
- ^ Polfreman, Richard (2009). "Frameworks 3D: Composition in the third dimension". Proceedings of the International Conference on New Interfaces for Musical Expression (NIME) 2009. pp. 226–229.
- ^ Valbom, Leonel; Marcos, Adérito (2007). "An Immersive Musical Instrument Prototype". IEEE Computer Graphics and Applications. 27 (4): 14–19. doi:10.1109/MCG.2007.76. ISSN 0272-1716. PMID 17713229.
- ^ Mäki-Patola, Temu; Laitinen, Juha; Kanerva, Aki; Takala, Tapio (2005). "Experiments with Virtual Reality Instruments". Proceedings of the 2005 International Conference on New Interfaces for Musical Expression. pp. 11–16.
- ^ Wynnychuk, J; Porcher, R; Brajovic, L; Brajovic, M; Platas, N (2002). "sutoolz 1.0 alpha: 3D software music interface". Proceedings of the 2002 International Conference on New Interfaces for Musical Expression. pp. 1–2.
- ^ Mulder, Axel G. E. (1998). Design of virtual three-dimensional instruments for sound control (PhD thesis). Simon Fraser University.