CAVI: A Coadaptive Audiovisual Instrument–Composition
Çağrı Erdem, Benedikte Wallace, Alexander Refsum Jensenius · 2022
This paper describes the development of CAVI, a coadaptive audiovisual instrument for collaborative human-machine improvisation. We created this agent-based live processing system to explore how a machine can interact musically based on a human performer’s bodily actions. CAVI utilized a generative deep learning model that monitored muscle and motion data streamed from a Myo armband worn on the performer’s forearm. The generated control signals automated layered time-based effects modules and animated a virtual body representing the artificial agent. In the final performance, two expert musicians (a guitarist and a drummer) performed with CAVI. We discuss the outcome of our artistic exploration, present the scientific methods it was based on, and reflect on developing an interactive system that is as much an audiovisual composition as an interactive musical instrument.