Learning the nuance of musical instrument acoustics
Chris Donahue · The Journal of the Acoustical Society of America · 2019
While acoustic modeling successfully captures the broad strokes of how musical instruments produce sound, it falls short of capturing the more nuanced attributes required to synthesize entirely convincing reproductions. For example, there are innumerable (effectively random) factors that affect the waveform produced by a single violinist performing the same musical gesture multiple times. Using machine learning, we can implicitly learn a distribution of these factors by modeling a collection of instrument waveforms, which could potentially lead to more convincing synthesis. In this talk, I will discuss our work on using generative adversarial networks, an unsupervised machine learning technique, to synthesize instrument waveforms. I will also speculate about how similar strategies might be paired with existing acoustic models to produce digital instruments which are indistinguishable from their real counterparts.