LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Weiming Shen, Limao Xiong, Yuhao Zhou, Xiaohua Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Miao Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang · 2024
Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.