Re3adapter:Efficient Parameter Fing-Tuning with Triple Reparameterization for Adapter without Inference Latency
Lihong Qiao, Rui Wang, Yucheng Shu, Ximing Xu, Baobin Li, Weisheng Li, Xinbo Gao · 2024
With the rise of large-scale model applications, leveraging these models as the base network for efficient transfer learning has garnered increasing attention. Currently, parameter-efficient transfer learning methods have made significant improvements in reducing the number of trainable parameters but introduce latency during inference. In this study, we propose an enhanced adaptation of the adapter using a reparameterization technique, revamping the activating layers into linear layers. This modification retains the high-dimensional fine-tuning capability of the adapter for visual tasks while avoiding additional inference latency. We name this plug-and-play module the Re3adapter, which optimizes the model with only 0.26% of the parameters and introduces no inference latency. Experimental results demonstrate its clear advantages in traditional classification and medical tasks.