A communication-efficient model of sparse neural network for distributed intelligence
Yiqiang Sheng, Jinlin Wang, Zhenyu Zhao · 2016
In this paper, we propose a communication-efficient model of sparse bidirectional neural network to intelligently process distributed data. The basic idea of the proposal is a modified bidirectional communication between the core and the edge of Internet by model parameters. The formulation and the procedures of the proposal are investigated. In theory, we prove that the proposed neural network is sparse, while a typical neural network is dense. In practice, a tree topology of computer cluster with a core machine and M edge machines is designed to implement the proposal, where M is the number of distributed datasets. The MNIST image database is split into M parts on the edge machines to simulate the distributed datasets from Internet of Things. Simulation shows the communication cost is greatly improved with the same level of accuracy in comparison to the state-of-the-art model. More importantly, it is naturally secure and private to communicate between the core machine and the edge machines through the model parameters, instead of the original data.