Comment-guided Learning: Bridging the Knowledge Gap between Expert Assessor and Feature Engineer
Xiang Li, Wen-Pin Lin, Heng Ji · 2011
Abstract—As more and more natural language processing systems utilize human assessment on system responses, it becomes beneficial to discover some hidden privileged knowledge (such as comments) from assessors. We present a simple, low-cost but effective comment-guided learning approach to exploit such knowledge declaratively in an automatic assessor. Our approach only requires a small set of training data, together with comments which are naturally available from human assessment. To demonstrate the power and generality of this approach, we apply the method in two very different applications: name translation and residence slot filling. Our approach achieved significant absolute improvement (15 % for name translation and 8 % for slot filling) over state-of-theart systems. It also outperformed previous methods such as recognizing textual entailment (RTE) based fact validation. Furthermore, it can be used as feedback to significantly speed up human assessment. Keywords-comment-guided learning; assessment; feature engineering I.