Adaptive multi-modal positive semi-definite and indefinite kernel fusion for binary classification
Maximilian Münch, Christoph Raab, Simon Heilig, Manuel Röder, Frank-Michael Schleif · 2022
Data and information are nowadays frequently available in multiple modalities like different sensor signals, textual descriptions, graph structures, and other formats.The maximum information from these heterogeneous representations can be obtained by fusing the various modalities by specific embeddings or proximity measures.Current approaches are widely limited in the fusion model and the applied measures, especially when the given data is non-vectorial.We propose a model to learn the spectral properties of the different inner product representations in a joined optimization problem.The approach is evaluated on various multimodal data and compared to modern multiple-kernel learning and baseline techniques.* MM and MR are supported by the Bavarian HighTech agenda and the Würzburg Center for Artificial Intelligence and Robotics (CAIRO).Additionally, we thank Dr. Benjamin Paaßen for the invaluable discussions about this research topic during a fantastic boating trip.