Efficient and Stable Test Time Adaptation for Dynamic Reality Scene
Juxin Liao · 2024
Test-time adaptation (TTA) is designed to adapt machine learning models to test sets with dataset shifts during the testing phase, without revisiting the training data. However, deploying TTA methods in real-world applications often leads to significant performance degradation. Specifically, TTA may fail to improve or even harm model performance when the test data exhibits the following characteristics: 1) mixed distribution shift, 2)online imbalanced label distribution shift, which are common scenarios in practice, 3) small batch sizes. While existing TTA methods can handle these situations to some extent, they struggle to balance both performance and efficiency, two critical aspects in real-world applications. In this paper, we apply the proposed Efficient and Stable Test-Time Adaptation (ES-TTA) method to address these real-world scenarios, demonstrating superior performance and efficiency compared to current state-of-the-art TTA methods. ES-TTA efficiently and stably enhances TTA in three ways: 1) filtering out high-entropy corrupted samples, 2) encouraging model weights to converge towards flat minima on the loss surface, 3) utilizing a sharpness-sensitive data selection strategy to improve TTA efficiency. Extensive experiments on the large ImageNet-c dataset show that our proposed ES-TTA can be effectively and efficiently applied to complex real-world scenarios.