Does BERT Understand Idioms? A Probing-Based Empirical Study of BERT Encodings of Idioms
Minghuan Tan, Jing Jiang · 2021
Understanding idioms is important in NLP.In this paper, we study to what extent a pretrained BERT model is able to encode the meaning of a potentially idiomatic expression (PIE) in a certain context.We make use of a few existing datasets and perform two probing tasks: PIE usage classification and idiom paraphrase identification.Our experiment results suggest that BERT indeed is able to separate the literal and idiomatic usages of a PIE with high accuracy.It is also able to encode the idiomatic meaning of a PIE to some extent.