Targeted feature dropout for robust slot filling in natural language understanding
Puyang Xu, Ruhi Sarikaya · 2014
In slot filling with conditional random field (CRF), the strong current word and dictionary features tend to swamp the effect of contextual features, a phenomenon also known as fea-ture undertraining. This is a dangerous tradeoff especially when training data is small and dictionaries are limited in their cov-erage of the entities observed during testing. In this paper, we propose a simple and effective solution that extends the feature dropout algorithm, directly aiming at boosting the contribution from entity context. We show with extensive experiments that the proposed technique can significantly improve the robustness against unseen entities, without degrading performance on enti-ties that are either seen or exist in the dictionary. Index Terms: slot filling, feature dropout, spoken language un-derstanding