Sharpness and Gradient Aware Minimization for Memory-based Continual Learning
Lam Tran Tung, Viet Nguyen Van, Phi-Hung Hoang, Khoat Quang Than · 2023
Memory-based Continual Learning methods (CL) preserve performance on old data by storing a small buffer of seen samples to re-learn with current data. Despite their impressive results, these methods may still obtain sub-optimal solutions as a result of overfitting training data, especially on the limited buffer. This can be attributed to their employment of empirical risk minimization over training data. To overcome this problem, we leverage Sharpness Aware Minimization (SAM), a recently proposed training technique, to improve models’ generalization, and thus CL performance. In particular, SAM seeks for flat minima whose neighbors’ loss values are also low by simultaneously guiding a model towards SAM gradient direction corresponding to low-loss regions and flat regions. However, we conjecture that directly applying SAM to replay-based CL methods whose loss function contains multiple objectives may cause gradient conflict among them. We then propose to manipulate each objective’s SAM gradient such that their potential conflict is minimized by adopting one gradient aggregation strategy from Multi-task Learning. Finally, through extensive experiments, we empirically verify our hypothesis and show consistent improvements in our method over strong memory-replay baselines.