IBM's PIQUANT in TREC2003
John M. Prager, Jennifer Chu‐Carroll, Krzysztof Czuba, Christopher A. Welty, Abraham Poovakunnel Ittycheriah, Ruchi Mahindru · 2003
For the most part, the system we used for TREC2003 was a smooth evolution of the one we ran in TREC2002. We continued to use our multi-source and multi-agent architecture. For Factoid questions we used all of our previous answering agents with an additional pattern-based agent, an enhanced answer resolution algorithm, and increased coverage of the Cyc sanity checker. We will devote a portion of this paper to performing a post-mortem of our experiences with Cyc this year. For List questions, which we did not attempt previously, we ran our Factoid system with different parameters. For Definition questions we took an entirely new approach, which we call QA-by-Dossier, and which will be the other focus of this paper. While we think that our system performed reasonably well in this subtask, the NIST evaluation results do not reflect this, raising some questions about the Definition subtask specification and evaluation.