Optimizing Neural Network Training: A Markov Chain Approach for Resource Conservation
Ke Wang, Xianting Huang, Cong Tan, Siu‐Ming Yiu, Zicong Chen, Xiaolin Lei · IEEE Transactions on Artificial Intelligence · 2024
The additional resource consumption generated during the repeated training processes of neural networks, including time and computational power costs, is a problem of significant concern. We, therefore, propose a method that utilizes Markov chain to predict the training outcomes of neural networks with the same structure. This method’s training is based on prior experience to optimize the parameter adjustment process, thereby reducing the number of times training must be started from scratch and lowering time costs. By predicting training outcomes and reducing forward and backward propagation computations, among other factors, computational resource consumption significantly decreases. Simultaneously, since Markov chain represents a clear mathematical model, the properties of probability transition offer greater interpretability compared to traditional methods. In an era where explainable artificial intelligence is equally crucial, a more transparent training method could have greater application potential in many important scenarios. The dual benefits they provide exemplify the advantage of our approach. Regarding the critical part, we have theoretically and experimentally demonstrated that, under certain conditions, the neural network training process possesses Markov property and becomes a Markov process after clustering.