Inter-sentence features and thresholded minimum error rate training: NAIST at CLEF 2013 QA4MRE
Philip Arthur, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura · Cross-Language Evaluation Forum · 2013
This paper describes the Nara Institute of Science and Technology’s system for the main task of CLEF 2013 QA4MRE. The core of the system is a log linear scoring model that couples both intra and intersentence features. Each of the features receives an input of a candidate answer, question, and document, and uses these to assign a score according to some criterion. We use minimum error rate training (MERT) to train the weights of the model and also propose a novel method for MERT with the addition of a threshold that defines the certainty with which we must answer questions. The system received a score of 28% c@1 on main questions and 33% c@1 when considering auxiliary questions on the CLEF 2013 evaluation.