EmbeddingROUGE: Malay News Headline Similarity Evaluation
Yeong Tsann Phua, Yew Kwang Hooi, Mohd Fadzil Hassan, Matthew Teow Yok Wooi · 2022
The evaluation metric is an essential part to evaluate text generation tasks such as news headline generation. The ROUGE evaluation metric is still the standard evaluation metric for most text summarization tasks. Nevertheless, this metric has its weakness due to its nature of measuring exact lexical overlapping between the candidate text and reference text. In this paper, we would like to attempt to propose a word embedding-based ROUGE named EmbeddingROUGE. The proposed design of this embedding-based evaluation metric aim to overcome the weakness of the ROUGE metric. Generally, the experiment shows that the EmbeddingROUGE demonstrates superior evaluation score over the original ROUGE.