A novel sub-network level ensemble deep neural network with a regularized loss function to improve prediction performance
Jalil Toosifar, Yahya Forghani, Seyyed Abed Hosseini, Nasser Shoeibi · Journal Of Big Data · 2025
Deep neural networks have been widely applied in various fields—such as image recognition, natural language processing, and robotics—achieving remarkable success. Ensemble learning is a well-established technique to improve the accuracy and generalization of neural networks by aggregating multiple models. Unlike conventional ensemble methods that combine the final outputs of independent models, this paper introduces a novel sub-network-level ensemble approach which integrates neurons, layers, and blocks within a single network. The proposed method leverages a checkpoint-based aggregation mechanism to combine multiple snapshots of a model during training, thereby ensuring greater diversity and reducing overfitting. Additionally, a regularized loss function is introduced to enhance diversity at different levels of the network, leading to improved generalization. Experimental results on standard benchmark datasets—including CIFAR-10, CIFAR-100, MNIST, and Fashion-MNIST—demonstrate that the proposed method outperforms traditional ensemble techniques, achieving accuracy improvements ranging from 0.08 to 7.05%. Furthermore, a computational complexity analysis confirms that the proposed approach maintains a balanced trade-off between accuracy and computational efficiency compared to existing methods.