Enhancements for Networked Music Performances
Marco Fink · 2018
The availability and capability of today’s internet allow several novel challenging interactive multimedia applications like Networked Music Performances (NMP). A Networked Music Performance is an online artistic collaboration with musicians located at different geographic locations connected using the internet. While offering manifold artistic possibilities, many technical challenges like the resulting latency and the possibility of packet loss have to be considered. This work depicts three enhancements for NMP applications which improve error robustness, the algorithmic delay, and the spatial listening experience, respectively. To counteract the possibility of packet loss or network jitter-caused tardy arrival of packets, this work derives two methods to conceal errors during audio replay at the receiver side. The first, auto-regressive model-based variant facilitates concealing the audible impact of missing packets with high quality but is computationally expensive. Several ways of computing the auto-regressive model are presented and compared. The second method, based on wave-form substitution, constitutes an efficient, cheap alternative. The proposed methods are evaluated subjectively with a listening test and objectively with measurements of perceptual quality. The application of audio codecs in NMP sessions is inevitable in most scenarios due to the restricted data rate and in particular the upload rate of private internet accesses. Besides reducing the data rate the codec must feature a small algorithmic latency to restrict the overall latency to a certain extent. A novel audio coding approach which features smaller delays than widely used low-delay codecs and a clearly reduced data rate in contrast to delay-less codecs is presented. It is constructed using the Adaptive Differential Pulse Code Modulation (ADPCM) codec approach in subbands in combination with a Vector Quantizer (VQ) resulting in the Vector-Quantized Adaptive Differential Pulse Code Modulation (VQ-ADPCM) codec. The proposed codec is capable of encoding broadband audio with a data rate of 64 kbit/s and algorithmic delay of about 1 ms. The perceptual quality is compared to well-known codecs using perceptually motivated measurements. The last contribution is intended to improve the acoustic spatial scenery within a NMP. For this purpose, a pseudo stereo conversion method providing a broad stereo panorama for single channel sound sources is derived. The method enhances the spaciousness of the stereo mix at the receiver without adding timbral coloration or reverberation and therefore offers an improved listening experience for NMP participants. The proposed method is based on the design of a complementary filter pair, which can be applied in time- and frequency-domain. Additionally, the integration within a virtual surround mixer based on Head-Related Impulse Responses (HRIRs) is demonstrated. Virtual surround mixing allows the arbitrary positioning of several sound sources in a virtual room. The extension with the proposed pseudo-stereo approach even facilitates to define sound sources of a certain size instead of single point sources. The three proposed enhancements are purely based on digital signal processing and therefore can be implemented in the software layer of any NMP system without demanding any changes to the actual musical performance, the utilized hardware, or the available network structure.