A comparative study of lexical similarity approaches for Malay short text
Madihah Mohammad, Madihah Mohammad, Arifah Che Alhadi · AIP conference proceedings · 2023
The short text retrieval is gaining tremendous attention due to rapid exposure and widespread usage of social networks. This type of text is widely used in various applications such as spam filtering, tweet classification, question-answer systems, web search and so forth. Thus, the effective analysis of these texts is of vital need. But, short text retrieval often becomes complex because they are usually contains repeated words, less features and overloaded with noise. Different lexical similarity approaches have been developed to tackle the problem of lexical analysis. However, most of them are in English, and the solutions for problems involving Malay are of limited availability. This paper examined the reliability of lexical similarity approaches focusing on term-based approaches for Malay short text analysis. The Cosine, Overlap coefficient and Jaccard similarity methods were used to quantify the similarities of some news title written in Malay news. Results show that the selected approaches term-based has potential to be used to analyze Malay short texts.