Examining the Impact and Limitations of Distributed Large Language Models and Multimodal Systems
Klaus Elli · HAL (Le Centre pour la Communication Scientifique Directe) · 2025
Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) have emerged as transformative advancements in artificial intelligence, enabling unprecedented capabilities in natural language processing, multimodal understanding, and generative AI. These models, powered by massive datasets and distributed computing frameworks, have significantly impacted diverse fields, including healthcare, education, robotics, and creative industries. However, despite their success, LLMs and MLLMs face substantial challenges related to computational efficiency, bias, interpretability, hallucinations, and security vulnerabilities. This survey provides a comprehensive overview of recent progress in distributed LLMs and multimodal models, highlighting key architectural innovations, training paradigms, and real-world applications. We explore critical challenges, including the environmental impact of large-scale model training, ethical concerns surrounding bias and misinformation, and the limitations of current reasoning and knowledge integration capabilities. Furthermore, we discuss emerging trends and future research directions aimed at enhancing model efficiency, reliability, and alignment with human values.