Handling Compound Hindi OOV words in web queries
Amit Asthana, Sanjay Kumar Dwivedi, Anand Prakash Dwivedi · Research Square · 2022
Abstract Handling of Out of Vocabulary (OOV) words is still a problem in NLP. If there is a word for which morphological analyser is not able to find a morpheme, that word is known as OOV word. These words if not identified, may restrict to understand the proper meaning of the sentence. It may also have severe impact on the IR system involving the queries. Detection and identification of OOV words in information retrieval is a challenging task. This problem may become more challenging in case of cross lingual information retrieval (CLIR) due to issues in query translation. The objective of this paper is to understand the impact of web queries involving these words on the retrieval effectiveness of web searches. Subsequently, we have also proposed an algorithm to successfully detect and handle the impact of Hindi web queries involving compound OOV. Our results have shown increased precision of 8.53% for one-word web queries involving only OOV word and 15.68% with queries having more than one word having at least one OOV word.