Feature Fusion Transformer Network for Natural Language Inference
Lei Sun, Hengxin Yan · 2022 IEEE International Conference on Mechatronics and Automation (ICMA) · 2022
Natural Language Inference (NLI) is a branch of Natural Language Processing (NLP) whose main task is to determine the relationship between a sentence pair (two sentences). Early attempts were made to use Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) to solve such problems, butit was difficult to achieve ideal results. Until the last few years, satisfactory results have been achieved using various methods based on attention mechanisms. Attention mechanism has unique advantages in a wide range of receptive fields, but it is difficult to extract local structure features effectively, while neural networks represented by CNN are good at local feature extraction. Therefore, we first verified the above point by experiments, then proposed the inference network models Feature Fusion Transformer Network (FFTN). The model is improved based on the Transformer model structure, retaining the non-local feature extraction advantages of the self-attention mechanism, combining the convolution method to enhance the feature attention of the local domain, and efficiently combining local and non-local features. Finally, excellent accuracy performance is achieved on the SNLI and MultiNLI datasets.