Low-latency MLLM Inference with Spatiotemporal Heterogeneous Distributed Multimodal Data
Xiangrui Xu, Sicong Liu, Zhiwen Yu, Lehao Wang, Bin Guo · 2024
Distributed sensing systems have been widely applied in various Internet of Things (IoT) scenarios, and the emergence of the Multimodal Large Language Model (MLLM) has opened up new possibilities for these systems. However, the spatiotemporal heterogeneity and asynchronous arrival of distributed mobile data make achieving low-latency, high-accuracy MLLM inference extremely challenging. In this paper, we propose a framework of MLLM inference with spatiotemporal heterogeneous distributed data to achieve low-latency, high-accuracy MLLM inference in distributed sensing systems.