A Modular Perspective on the Evolution of Deep Learning: Paradigm Shifts and Contributions to AI
Yicheng Wei, Yifu Wang, Junzo Watada · Applied Sciences · 2025
The rapid development of deep learning (DL) has demonstrated its modular contributions to artificial intelligence (AI) techniques, such as large language models (LLMs). DL variants have proliferated across domains such as feature extraction, normalization, lightweight architecture design, and module integration, yielding substantial advancements in these subfields. However, the absence of a unified review framework to contextualize DL’s modular evolutions within AI development complicates efforts to pinpoint future research directions. Existing review papers often focus on narrow technical aspects or lack systemic analysis of modular relationships, leaving gaps in our understanding how these innovations collectively drive AI progress. This work bridges this gap by providing a roadmap for researchers to navigate DL’s modular innovations, with a focus on balancing scalability and sustainability amid evolving AI paradigms. To address this, we systematically analyze extensive literature from databases including Web of Science, Scopus, arXiv, ACM Digital Library, IEEE Xplore, SpringerLink, Elsevier, etc., with the aim of (1) summarizing and updating recent developments in DL algorithms, with performance benchmarks on standard dataset; (2) identifying innovation trends in DL from a modular viewpoint; and (3) evaluating how these modular innovations contribute to broader advances in artificial intelligence, with particular attention to scalability and sustainability amid shifting AI paradigms.