Energy-aware Preprocessing for Distributed Training in D2D Edge Network with Non-iid Data
Jiaxin Wu, Jigang Wu, Long Chen, Yifei Sun · 2021
Inherent non-iid characteristic of heterogeneous devices' local dataset slows down the model training process and decreases the training accuracy. To tackle this problem, we propose a dataset reconstruction scheme to transform training device's non-iid dataset into iid dataset via data exchange among trusted devices. Two data exchange modes are proposed to serve the scheme. Considering the transmission overhead, we formulate an optimization problem named NORS to minimize the total energy consumption of devices. We then design an approximation algorithm named X-GTR to obtain a near-optimal auxiliary devices set for dataset reconstruction, while meeting the variance constraint of NORS. For comparison, we propose a baseline algorithm that randomly selects auxiliary devices for dataset reconstruction. Numerical results show that, with the total number of devices increasing, our proposed algorithm outperforms baseline algorithm on reducing energy consumption.