daminglu123 at SemEval-2022 Task 2: Using BERT and LSTM to Do Text Classification
Daming Lu · Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) · 2022
Multiword expressions (MWEs) or idiomaticity are a common phenomenon in natural languages.Current pre-trained language models cannot effectively capture the meaning of these MWEs.The reason is that two single words, after combined together, could have an abruptly different meaning than the compositionality of the meanings of each word, whereas pretrained language models reply on words' compositionality.We propose an improved method of adding an LSTM layer to the mBERT model to get better results on a text classification task (Subtask A).Our result is slightly better than the baseline.We also tried adding TextCNN to mBERT and adding both LSTM and TextCNN to mBERT.We participate in SubTask A and find that adding only LSTM gives the best performance.