Distributed and Rate-Adaptive Feature Compression
Aditya Deshmukh, Venugopal V. Veeravalli, Gunjan Verma · 2024
We study the problem of distributed and rate-adaptive feature compression in a sensor network, wherein a set of distributed sensors observe disjoint multi-modal features, compress them, and send them to a fusion center containing a pretrained learning model for inference for a downstream task. To gain insight, we first analyze the case where the pretrained model is a linear regressor. We obtain the form of optimal quantizers assuming knowledge of underlying regressor data distribution. Under a practically reasonable approximation, we then propose a distributed compression scheme which works by quantizing a one-dimensional projection of the sensor data. We also propose a simple adaptive scheme for handling changes in communication constraints. For the case when the pretrained model is a general learning model, we propose a VQ-VAE based compression scheme, which is motivated by the fact that VQ-VAE based compression works by quantizing low-dimensional latent representations, which matches the strategy obtained for pretrained linear regressors. We further show that the adaptive strategy proposed for case of linear regression can also be applied effectively to the VQ-VAE based compression scheme. We demonstrated the effectiveness of the VQ-VAE based distributed and adaptive compression scheme on MNIST Audio+Video and CIFAR10 datasets.