Toshiba ASKMi at NTCIR-4 QAC2.
Tetsuya Sakai, Yoshimi Saito, Yumi Ichimura, Makoto Koyama, Tomoharu Kokubu · NTCIR · 2004
Toshiba participated in NTCIR-4 QAC2 Subtask 1: this is our first QAC participation. Using our newly developed Japanese QA system called ASKMi, we submitted two runs, one using Okapi/BM25 for document retrieval (TSB-A) and the other using Boolean AND constraintsbefore applyingOkapi/BM25(TSB-B). We achieved the 5th best performance among the 17 participants (8th and 9th among the 25 submitted runs). This paper briefly describes ASKMi, and analyses the formal run and oracle run results using Reciprocal Rank, as well as a new performance metric called Qmeasure which can handle multiple-answer questions and answer correctness levels.