Toward a Framework for Interactive Systems to Conduct Digital Audio and Video Streams
Eric Lee, Thorsten Karrer, Jan Borchers · Computer Music Journal · 2006
Following advances in commodity hardware and computing power, interactive conducting systems have grown in complexity and capability. Modern conducting systems incorporate research from a variety of disciplines, from motion tracking, to gesture recognition and interpretation, to digital signal processing. Frameworks have emerged in recent years to enable rapid development of such complex systems, including Max/MSP for manipulating and processing synthesized and sampled audio (Puckette 2002), and EyesWeb for gesture tracking (Camurri et al. 2003). Today's computers are, moreover, able to handle large chains of complex filters and other operations on digitally sampled audio and video streams in real time. In contrast, modern computer music systems often do not take full advantage of these capabilities by continuing to use synthesized music, usually MIDI-based; even less incorporate video. The advantage of using synthesized music over digitally sampled audio streams is a higher level of semantic access to the data, such as beats, notes and voicings. However, digital audio and video recordings can offer a higher level of fidelity and realism: today's synthesizing technology is still unable to reproduce, for example, the unique character of the Vienna Philharmonic playing in their Golden Hall of Vienna's Musikverein. Part of the problem can be attributed to the difficulty of working with time-based effects in current multimedia frameworks such as Apple's QuickTime