DEEPYANG at SemEval-2020 Task 4: Using the Hidden Layer State of BERT Model for Differentiating Common Sense
Yang Bai, Xiaobing Zhou · 2020
Introducing common sense to natural language understanding systems has received increasing research attention.To facilitate the researches on common sense reasoning, the SemEval-2020 Task 4 Commonsense Validation and Explanation(ComVE) is proposed.We participate in sub-task A and try various methods including traditional machine learning methods, deep learning methods, and also recent pre-trained language models.Finally, we concatenate the original output of BERT and the output vector of BERT hidden layer state to obtain more abundant semantic information features, and obtain competitive results.Our model achieves an accuracy of 0.8510 in the final test data and ranks 25th among all the teams. Related WorkRecent years, the NLP community have witnessed a variety of works that enhances machines ability to perform deep language understanding which goes between the lines, rather relying on reasoning