Overview of the NTCIR-7 ACLIA Tasks: Advanced Cross-Lingual Information Access
Teruko Mitamura, Eric Nyberg, Hideki Shima, Tsuneaki Kato, Tatsunori Mori, Chin-Yew Lin, Ruihua Song, Chuan‐Jie Lin, Tetsuya Sakai, Donghong Ji, Noriko Kando · 2008
This paper presents an overview of the ACLIA (Advanced Cross-Lingual Information Access) task cluster. The task overview includes: a definition of and motivation for the evaluation; a description of the complex question types evaluated; the document sources and exchange formats selected and/or defined; the official metrics used in evaluating participant runs; the tools and process used to develop the official evaluation topics; summary data regarding the runs submitted; and the results of evaluating the submitted runs with the official metrics. input to the IR4QA task, which produced retrieval results that were subsequently fed back into the end-toend QA systems. The modular design and XML interchange format supported by the ACLIA architecture make it possible to perform such embedded evaluations in a straightforward manner. More details regarding the XML interchange schemes and so on can be found on the ACLIA wiki [6]. 1.