Evaluation of Conversational Skills for Commonsense
Dilyara Baymurzina, Yuri Kuratov, Denis Kuznetsov, Daniel Kornev, Mikhail Burtsev · Computational Linguistics and Intellectual Technologies · 2021
Lack of commonsense is one of the most challenging problems in the field of conversational AI.Despite the recent significant progress in NLP driven by pre-trained language models, commonsense reasoning is still out of reach.We propose an approach to evaluate conversational commonsense usage.We use the approach to evaluate conversational skills of the socialbot during interaction with users.Analysis of data with joint manual and automatic annotations allowed us to identify automatic metrics tied to commonsense.We also develop two commonsense conversational skills that combine commonsense knowledge graphs completion model COMeT [6] and templatebased approach.