Assessing self-awareness and transparency when classifying a speaker's level of certainty
Heather Pon-Barry, Stuart M. Shieber · 2010
This paper is about using prosody to automatically detect one aspect of a speaker's internal state: their level of certainty.While past work on classifying level of certainty used the perceived level of certainty as the value to predict, we find that this quantity often differs from a speaker's actual level of certainty as gauged by self-reports.In this work we build models to predict a speaker's self-reported level of certainty using prosodic features.Our data is a corpus of single-sentence utterances that are annotated with (1) whether the statement is correct or incorrect, (2) the perceived level of certainty, and (3) the self-reported level of certainty.Knowing the self-reported level of certainty, in conjunction with the perceived level of certainty, allows us to assess what we will refer to as the speaker's transparency.Knowing the self-reported level of certainty, in conjunction with the correctness of the answer, allows us to assess what we will refer to as self-awareness.Our models, trained on prosodic features, correctly classify the self-reported level of certainty 75% of the time.Intelligent systems can use this information to make inferences about the user's internal state, for example whether the user of a system has a misconception, makes a lucky guess, or needs encouragement.