Nonlinear Information-Theoretic Compressive Measurement Design
Liming Wang, Abolfazl Razi, Miguel R. D. Rodrigues, Robert Calderbank, Lawrence Carin · 2014
We investigate design of general nonlinear func-tions for mapping high-dimensional data into a lower-dimensional (compressive) space. The nonlinear measurements are assumed contami-nated by additive Gaussian noise. Depending on the application, we are either interested in recov-ering the high-dimensional data from the non-linear compressive measurements, or perform-ing classification directly based on these mea-surements. The latter case corresponds to clas-sification based on nonlinearly constituted and noisy features. The nonlinear measurement func-tions are designed based on constrained mutual-information optimization. New analytic results are developed for the gradient of mutual infor-mation in this setting, for arbitrary input-signal statistics. We make connections to kernel-based methods, such as the support vector machine. Encouraging results are presented on multiple datasets, for both signal recovery and classifica-tion. The nonlinear approach is shown to be par-ticularly valuable in high-noise scenarios. 1.