Entity Contrastive Learning in a Large-Scale Virtual Assistant System
Jonathan Rubin, Jason Crowley, George Leung, Morteza Ziyadi, Maria Minakova · 2023
Conversational agents are typically made up of domain (DC) and intent classifiers (IC) that identify the general subject an utterance belongs to and the specific action a user wishes to achieve.In addition, named entity recognition (NER) performs per token labeling to identify specific entities of interest in a spoken utterance.We investigate improving joint IC and NER models using entity contrastive learning that attempts to cluster similar entities together in a learned representation space.We compare a full virtual assistant system trained using entity contrastive learning to a baseline system that does not use contrastive learning.We present both offline results, using retrospective test sets, as well as online results from an A/B test that compared the two systems.In both the offline and online settings, entity contrastive training improved overall performance against baseline systems.Furthermore, we provide a detailed analysis of learned entity embeddings, including both qualitative analysis via dimensionalityreduced visualizations and quantitative analysis by computing alignment and uniformity metrics.We show that entity contrastive learning improves alignment metrics and produces wellformed embedding clusters in representation space.