Recovering Hidden Components in Multimodal Data with Composite Diffusion Operators
Tal Shnitzer, Mirela Ben‐Chen, Leonidas Guibas, Ronen Talmon, Hau‐Tieng Wu · SIAM Journal on Mathematics of Data Science · 2019
Finding appropriate low-dimensional representations of high-dimensional multimodal data can be challenging, since each modality embodies unique deformations and interferences. In this paper, we address the problem using manifold learning, where the data from each modality is assumed to lie on some manifold. In this context, the goal is to characterize the relations between the different modalities by studying their underlying manifolds. We propose two new diffusion operators that allow us to isolate, enhance, and attenuate the hidden components of multimodal data in a data-driven manner. Based on these new operators, efficient low-dimensional representations can be constructed for such data, which characterize the common structures and the differences between the manifolds underlying the different modalities. The capabilities of the proposed operators are demonstrated on 3D shapes and on a fetal heart rate monitoring application.