Sparse and Debiased Adaptive Huber Regression in Distributed Data: Aggregated and Communication-Efficient Approaches
Wei Ma, Junzhuo Gao, Lei Wang, Heng Lian · Statistica Sinica · 2023
Distributed estimation and statistical inference for linear models have drawn much attention recently, but few studies focus on robust learning in the presence of heavy-tailed/asymmetric errors and high-dimensional covariates.Based on adaptive Huber regression to achieve the bias-robustness tradeoff, two classes of sparse and debiased lasso estimators are proposed using aggregated and communicationefficient approaches.To be specific, an aggregated ℓ1-penalized and a multi-round ℓ1-penalized communication-efficient adaptive Huber estimators are respectively proposed in the first stage to handle the distributed data with high-dimensional covariates and heavy-tailed/asymmetric errors.To correct the biases caused by the lasso penalty, a unified debiasing framework based on the decorrelated score equations is considered in the second stage.In the third stage, hard-thresholding is used to produce the sparse and debiased lasso estimators.The convergence rates and asymptotic properties of the proposed two estimators are established.The finite-sample performance is studied through simulations and a real data application to Commu-