A Multimodal Data Fusion and Semantic Communication Approach for Telematics
Shaojiang Liu, Jiajun Zou, Zhiping Wan · 2024
With the development of Intelligent Transportation Systems (ITS), Telematics plays a key role in improving road safety, optimizing traffic management, and promoting autonomous driving technologies. However, Telematics faces multiple challenges such as multimodal data fusion, semantic communication and collaborative decision making. To address these issues, this paper proposes a multimodal Transformer network based on a cross-modal self-attention mechanism for efficiently fusing heterogeneous data from multiple sources of cameras, LIDAR, and millimeter-wave radar. By introducing the self-attention mechanism, the proposed network is able to deeply explore the higher-order correlations between different modal data, and significantly improve the accuracy and robustness of environment sensing. In terms of semantic communication, this paper combines information bottleneck theory and variational autoencoder (VAE) to design an efficient semantic information encoding and compression method, which reduces the communication load and optimizes the transmission efficiency.