Robust Sparse Online Learning through Adversarial Sparsity Constraints
Zhong Chen · 2024
In this paper, we propose a novel robust sparse online learning framework named Adversarial Sparse Online Learning (ASOL) for high dimensional data streams, which is implemented by the adversarial sparse learning over the ℓ1-norm and ℓ12-norm constraints. Our main idea is to exploit adversarial learning between ℓ1-norm based constraint on the current weight vector and ℓ12-norm based constraint on the incremental weight matrix, achieving robust sparse solutions of the online weight vectors through such adversarial sparsity constraints. Compared with projection-based or ℓp-norm based sparsity-introducing mechanisms, our proposed ASOL algorithm is to find an equilibrium and achieve stable sparse solutions by simultaneously considering the sparsity structure of the current weight vector through the ℓ1-norm constraint as well as the correlations and sparsity structure of previous weight vectors through the ℓ12-norm constraint. Extensive empirical studies have demonstrated the effectiveness and efficiency of the proposed method.