The Evolution of Convolutional Neural Network Architectures: A Review
Deyong Mei, Junyi Gong, Mingding Qu, Shiquan Bian · IEEE Access · 2025
Convolutional Neural Networks (CNNs) are one of the most important and successful algorithmic architectures in deep learning, especially effectively in processing data with a grid-like topological structure. Over the past decade, CNNs have demonstrated exceptional performance across various domains and continue to evolve actively as a major focus of research and development. The existing reviews on CNNs primarily provided either a broad overview or applications and implementations across specific areas, the contents of which in terms of the concepts, algorithms, and principles that underlying the fundamental architectural components often lack sufficient depth and breadth, providing only elementary introductions to a few conventional algorithms and models. In this review, we presented a comprehensive and systematic analysis of the algorithms and theoretical foundations that form the basis of the core building blocks for CNN architectures, covering convolutional algorithms, activation functions, spatial downsampling strategies, regularization techniques, normalization techniques, fully-connected layers, optimization algorithms, and classic CNN-based models. Our analysis specially emphasizes on the evolutionary trajectory of CNNs from the mathematical principles underlying the models and algorithms, which serves as a valuable complement to previous research efforts. By delving into the underlying logic of CNN architectures, we expect to provide a comprehensive framework rather than an overview for researchers who are interested in architectural innovation of CNNs. Furthermore, we investigated the current challenges and limitations of CNNs and proposed promising directions for future research.