A Distributed Class-Incremental Learning Method Based on Neural Network Parameter Fusion
Hongtao Guan, Yijie Wang, Xingkong Ma, Yongmou Li · 2019
Class-incremental learning has received considerable attention due to the better adaptability to constantly changing characteristic of online learning. The neural network is suitable for class-incremental learning because of batch training style and fine-tuning. However, as the rapid growth of data centers and data category, data isolated island problem presents a new challenge to the current machine learning model: how to provide an accurate training method in a distributed and incremental manner. Existing solutions cannot adapt to incremental classes without aggregating data. In this paper, we propose a distributed class-incremental learning framework, called DCIL, which iteratively updates new class parameters and fixes existing class parameters to achieve high training accuracy. Especially, we propose MF, a parameter fusion method to integrate the parameters of multi-nodes into a new global parameter. In order to match the requirements of identical columns corresponding to the known classes, we propose a single node class-incremental learning, called SCIL. To evaluate the performance of DCIL, we conduct experiments on 5 datasets under various parameter settings on both 2 and 4 nodes distributed scenarios. Extensive experiments confirm that DCIL outperform the general baselines and achieve classification accuracy closes to the method of data aggregating.