Virtual Ensemble Assembly: Musicality in Separation
Kaitlin Pet, Ilana Shapiro, Christopher S. Raphael · Zenodo (CERN European Organization for Nuclear Research) · 2022
Remotely assembled music today depends on a predetermined “reference track”. While reference tracks ensure all parts sound ‘in sync’ when combined, they restrict a musician’s ability to pick their own tempo or perform other timing-related musical gestures. Working with expert ensembles, we present experiments that address this problem. We first created a remote assembly pipeline allowing artists to have temporal freedom during the recording process. We then develop three algorithms that preserved players’ original timing intentions while ensuring relative synchrony between parts: 1) direct modeling of chamber music expertise, 2) optimization of desirable performance qualities, and 3) performance simulation based on competing goals. Though the resulting music we assembled did not capture the full expressive range of an in-person performance, our work both increases the quality and ease of making remote recording and introduces exciting new applications for aiding synchronous rehearsal and performance.