Overview of the NTCIR-7 ACLIA IR4QA Task
Tetsuya Sakai, Noriko Kando, Chuan‐Jie Lin, Teruko Mitamura, Hideki Shima, Donghong Ji, Kuang‐Hua Chen, Eric Nyberg · NTCIR · 2008
This paper presents an overview of the IR4QA (Information Retrieval for Question Answering) Task of the NTCIR-7 ACLIA (Advanced Cross-lingual Information Access) Task Cluster. IR4QA evaluates traditional ranked retrieval of documents using wellstudied metrics such as Average Precision, but the retrieval task is embedded in the context of cross-lingual question answering. That is, document retrieval is treated as a component of the entire question answering system. This paper concentrates on how relevance assessments for the Simpified Chinese, Traditional Chinese and Japanese IR4QA test collections were obtained, and the outcome of the formal IR4QA evaluation using the three collections. For the relationship between IR4QA and the entire ACLIA task cluster, we refer the reader to the overview paper of ACLIA [17]. For details of the individual IR4QA systems, we refer the reader to the participants’ reports.