Coke: Communication-Censored Kernel Learning Via Random Features
Ping Xu, Zhi Tian, Zhe Zhang, Yue Wang · 2019
Distributed kernel-based methods are attractive in nonlinear learning tasks where either a dataset is too large to be processed on a single machine or the data are only locally available to geographically-located sites. For the first case, we propose to split the large dataset into multiple mini-batches and distribute them to distinct sites for parallel learning through the alternating direction method of multipliers (ADMM). For the second case, we develop a decentralized ADMM so that each site can solve the learning task collaboratively through one-hop communications. To circumvent the curse of dimensionality in kernel-based methods, we leverage the random feature approximation to map the large-volume data into a smaller feature space. This also results in a common set of decision parameters that can be exchanged among sites. Motivated by the need to conserve energy and reduce communication overheads, we apply a censoring strategy to evaluate the updated parameter at each site and decide if this update is worth transmitting. The proposed COmmunication-censored KErnel learning (COKE) algorithms are corroborated to be communication-efficient and learning-effective by simulations on both synthetic and real datasets.