MIAMI: A Mixed Reality Interface for AI-based Music Improvisation
Ryogo Ishino, Nao Tokui · 2022
In this research, we created a mixed reality interface and system for improvisation with deep generative music models, named MIAMI, which stands for "a Mixed reality Interface for Ai-based Music Improvisation". This was achieved by linking a 3D interface using Microsoft Hololens, a Python program that loads and executes generative models, Ableton Live which is a Digital Audio Workstation (DAW), and a Max for Live(M4L) device running on it. By incorporating the interface itself into the live space, we are able to achieve physical movements and visual augmentation that were not possible in conventional improvisational performances using music generation interfaces, demonstrating the possibility of a new musical experience.