Automatic annotation of COMMUNICATOR dialogue data for learning dialogue strategies and user simulations
Kallirroi Georgila, Oliver Lemon, James Henderson · 2005
We present and evaluate an au-tomatic annotation system which builds “Information State Update” (ISU) representations of dialogue context for the COMMUNICATOR (2000 and 2001) corpora of human-machine dialogues (approx 2300 di-alogues). The purposes of this annotation are to generate train-ing data for reinforcement learning (RL) of dialogue policies, to gen-erate data for building user simula-tions, and to evaluate different dia-logue strategies against a baseline. The automatic annotation system uses the DIPPER dialogue manager. This produces annotations of user inputs and dialogue context repre-sentations. We present a detailed example, and then evaluate our an-notations, with respect to the task completion metrics of the original corpus. The resulting data has been used to train user simulations and to learn successful dialogue strategies. 1