Multi-Modal Generative Models for Learning Epistemic Active Sensing
Timo Korthals, Daniel Rudolph, Jürgen Leitner, Marc Hesse, Ulrich Rückert · 2019
We present a novel approach of multi-modal deep generative models and apply this to coordinated heterogeneous multi-agent active sensing. A major approach to achieve this objective is to train a multi-modal variational Auto Encoder (M2VAE) that integrates the information of different sensor modalities into a joint latent representation. Furthermore, we derive an objective from the M2VAE that enables the maximization of the evidence lower bound via selection of sensor modalities. Using this approach as a direct reward signal to a multi-modal and multi-agent deep reinforcement learning setup leads intuitively to an epistemic active sensing behavior that coordinately resolves the ambiguity of observations.