An Intelligent Situation Interpretation System Based on Multimodal Information Fusion
Xinwei Wu, Yongjie Hou, Licheng Sun, Hongbin Ma · 2025
This paper proposes a multimodal intelligent situation interpretation system for wargame scenarios. This system does not directly extract situation information from images using networks; instead, it employs methods with stronger reasoning capabilities, relational understanding, and multimodal fusion to conduct situation analysis.It integrates Multi-Object Tracking (MOT), Situation Analysis (SA), and SmartVoice (SV) modules. The MOT module to achieve accurate operator tracking, providing trajectory data for situation analysis. The SA module calculates key metrics such as operator speed, maneuver direction, and group dynamics while fusing trajectory and textual data to generate battlefield keywords. The SV module uses a large language model and text-to-speech synthesis to transform these insights into real-time wargame interpretations. Experimental results demonstrate high tracking accuracy, rapid model convergence, and effective interpretation generation, showcasing the system's potential for real-time tactical insights and decision support.