A Survey on Vision-Language Models for Multimodal Federated Learning Tasks
Sheng Yang, Zhennan Chen, Yuanguo Lin, Guorong Cai, Xi Chen, Han Yu, Pengcheng Wu, Qiang Yang · 2025
Multimodal Federated Learning (MMFL) enables privacy-preserving training on decentralized visual-text data, facilitating collaborative learning across clients with heterogeneous modalities. Effectively modeling multimodal heterogeneity in such settings is crucial. Vision-Language Models (VLMs), renowned for their powerful visual-text alignment capabilities, emerge as promising solutions for enhancing MMFL. However, deploying large-scale VLMs in decentralized MMFL environments presents significant integration challenges, ranging from misaligned modality semantics to client-specific constraints, extending beyond traditional federated optimization issues. This survey presents the first comprehensive review of VLM-based MMFL approaches. Specifically, we: (i) outline the foundational concepts of existing VLMs and MMFL frameworks; (ii) categorize current research into four representative tasks: cross-modal content generation, visual question answering (VQA), multimodal retrieval, and multimodal sentiment analysis (MSA); (iii) explore key scientific challenges in MMFL research, including high communication overhead, cross-client semantic drift, multimodal fairness and bias, and privacy-fusion trade-offs; and (iv) conclude by outlining open research directions, emphasizing the development of lightweight, communication-efficient architectures, privacy-aware multimodal alignment, fairness-aware training objectives, and robust methods for handling non-IID multimodal data distributions.