Joint Data-Driven Analysis of Visual-Odometric Anomaly Signals in Generative Ai-Based Agents
Giulia Slavic, M. Bracco, Lucio Marcenaro, David Martín, Carlo S. Regazzoni, Pamela Zontone · 2024
This paper presents a data-driven model that, by exploring the correlation between the data coming from two sensors (GPS and camera), allows us to explain the odometry anomalies by analyzing video data. Our approach uses a Markov Jump Particle Filter (MJPF) to process the odometry data of a vehicle, extracting its features and identifying instant anomalies. Simultaneously, the object causing the anomaly is detected and tracked in the video data. After that, the correlation between the odometric trajectories and the object trajectories is determined in both normal and abnormal cases. Another correlation coefficient is then employed to calculate the distance between the computed correlations. The proposed method is evaluated using multi-modal data collected from a vehicle operating in a closed environment, where pedestrians represent anomalies. We show that our system is able to distinguish which video anomaly better explains the odometry anomaly.