Continuous basis pursuit and its applications
Eero P. Simoncelli, Daniel Tranchina, Chaitanya Ekanadham · 2012
Transformation-invariance is a major source of nonlinear structure in many real signal ensembles. To model this structure, we develop a methodology for decomposing a signal into a sparse linear combination of continuously transformed features. The central idea is to approximate the manifold(s) of transformed features(s) by linearly combining interpolation functions using constrained coefficients that can be recovered via convex programming. The advantage of this approach over traditional sparse coding methods is threefold: (1) it is built upon a more accurate probabilistic source model for transformation-invariant ensembles, (2) it uses a more efficient dictionary, and (3) both structural and transformational information can be extracted separately from the representation via well-defined mappings, providing transformation-invariant and -equivariant information, respectively. The method can be used with any linear interpolator, and includes basis pursuit denoising as a special case corresponding to nearest-neighbor interpolation. We propose a novel polar interpolation method with which our method significantly outperforms basis pursuit on a sparse deconvolution task. In addition, our method outperforms the state-of-the-art in identifying neural action potentials from voltage recordings on multiple simulated and real data sets. The advantage of our method is primarily due to its superior handling of near-synchronous action potentials, which overlap in the trace and are not recoverable by standard spike sorting methods. Finally, we develop a hierarchical formulation in which successive layers encode more complex features and their associated transformation parameters. A two-layer time- and frequency-shiftable representation is learned from speech data. The second layer encoding compactly represents sounds in terms of acoustic features such as harmonic stacks, sweeps, and ramps in time-frequency space. Despite its compactness, synthesis reveals that it is a faithful representation of the original sound and yields significant improvement over wavelet thresholding techniques on an acoustic denoising task. These two applications demonstrate the advantage of representations which separate content and transformation, and our proposed methodology provides an effective tool for computing such a representation.