Performance Evaluation Techniques for an Automatic Question Answering System
Tilani Gunawardena, Nishara Pathirana, Medhavi Lokuhetti, Roshan Ragel, Sampath Deegalla · International Journal of Machine Learning and Computing · 2015
Automatic question answering (QA) is an interesting and challenging problem.Generally such problems are handled under two categories: open domain problems and close domain problems.Here the challenge is to understand the natural language question so that the solution could be matched to the respective answer in the database.In this paper we used a template matching technique to perform this matching.The first part of the paper discusses about an automatic question answering system that we have developed using template matching techniques.The approach adopted is an automated FAQ (Frequently Asked Question) answering system that provides pre-stored answers to user questions asked in ordinary English and SMS language.The system also has techniques to overcome spelling and grammar mistakes introduced in questions by its users and therefore user-friendly compared to restricted syntax based approaches.The second part of the paper studies three techniques for performance evaluation in the above system which are based on template matching approach: 1) Random classification of templates, 2) Similarity based classification of templates, 3) Weighting template words.