k-nearest neighbor Monte-Carlo control algorithm for POMDP-based dialogue systems
Franck Lefèvre, Milomir M. Gašić, Filip Jurčíček, Simon Keizer, François Mairesse, B. Thomson, Kai Yu, S.J. Young · 2009
In real-world applications, modelling dialogue as a POMDP requires the use of a summary space for the dialogue state representation to ensure tractability. Sub-optimal estimation of the value function governing the selection of system responses can then be obtained using a grid-based approach on the belief space. In this work, the Monte-Carlo control technique is extended so as to reduce training over-fitting and to improve robustness to semantic noise in the user input. This technique uses a database of belief vector prototypes to choose the optimal system action. A locally weighted k-nearest neighbor scheme is introduced to smooth the decision process by interpolating the value function, resulting in higher user simulation performance.