Partnership-based Collaborative Learning Approach from Decentralized Data
Ye Chen, Jingguo Dai, Xinduo Su, Yaorong Fan, Wendong Bian, Peng Ye, Guang Chen, Yuezhen Huang · 2021
When training models to implement machine learning tasks in the Internet of Things (IoT) environment, large-scale collection of the data stored on decentralized devices is inappropriate in view of the individual privacy, and federated learning as an alternative to centralized machine learning method has been proposed. However, in such a method, individual updates of the participating devices before the global aggregation only adopt biased native information, which may be sensitive to the specific classes that have larger data size. In this paper, considering that the integration of more balanced estimates could be superior to the aggregation of skewed calculations, we propose a novel machine learning approach that enables decentralized IoT devices to cooperatively train a global model based on the direct and indirect interactions among devices while keeping the raw data locally. We term our approach Partnership-based Collaborative Learning Approach (CLAP). The proposed approach consists of two stages, including the top-down stage and the bottom-up stage. In the former stage, to improve the model accuracy of the device itself, each device is allowed to make active choice to find partners, so that it can directly interact with partners to reduce the influence of biased data distributions on individual estimates. In the latter stage, from a global perspective, considering that single devices may only offer partial information about the given task, CLAP tends to aggregate the results of the diverse nodes to provide more extensive features, leaving a more accurate collective estimate. In order to study the effect of direct interactions among partners in a local context, outcomes of the nodes in the simulation are determined by three components: their own estimates, the estimates of partners and the effect degree of the estimates of their respective partners. Experimental results demonstrate that CLAP based on direct and indirect interactions performs better than other distributed learning methods, and two characteristics including diversity and conditional dependence make contribute to the superior performance of CLAP.