From Percepts to Semantics: A Multi-modal Saliency Map to Support Social Robots’ Attention
Lorenzo Ferrini, Antonio Andriella, Raquel Ros, Séverin Lemaignan · ACM Transactions on Human-Robot Interaction · 2025
In social robots, visual attention expresses awareness of the scenario components and dynamics. As in humans, their attention should be driven by a combination of different attention mechanisms. In this article, we introduce multi-modal saliency maps, i.e., spatial representations of saliency that dynamically integrate multiple attention sources depending on the context. We provide the mathematical formulation of the model and an open source software implementation. Finally, we present an initial exploration of its potential in social interaction scenarios with humans and evaluate its implementation.