Performance comparison of full-batch BP and mini-batch BP algorithm on Spark framework
Jiewen Zheng, Qingli Ma, Wuyang Zhou · 2016
Full-Batch update and mini-batch update are two most widely used algorithms in back-propagation(BP) neural network, to deal with the huge training time and computation cost in the learning process. Parallel computing can improve the computation efficiency and have implemented these two algorithms on Mapreduce framework. In this paper, we implement these two algorithms on Spark framework and evaluate the performance by extensive experimental results. We verify that, Spark framework outperforms Mapreduce in implementing full-batch update algorithm due to its innovative design philosophy and lazy evaluation mechanisms. In addition, mini-batch update algorithm will cost less training time than full-batch update and we also notice that the moderate batch size k will perform better, although accurate analysis is not completed. Our work obtain novel knowledge in recognizing the performance in Spark framework for these two algorithms.