Text Classification Based on Multi-Head Attention Mechanism
Donghui Wang, Fan Zhou, Zhenyu Yin · 2024
Aiming at the limitations of traditional classification methods in single text information extraction, this paper proposes a multi-head attention classification model that introduces a bias term. The model employs a multi-head attention mechanism in place of a single attention mechanism to obtain effective information from multiple perspectives. By introducing bias vectors in each attention head, the model is able to focus on different aspects of the input in multiple dimensions, thus capturing key information more accurately. In this paper, the researchers carried out experiments using the IMDB dataset. The findings demonstrate that the suggested model displays notable enhancements in text categorization tasks, thus confirming the efficacy of the approach.