WRF: Weighted Rouge-F1 Metric for Entity Recognition
Lukas Weber, Krishnan Jothi Ramalingam, Matthias Beyer, Axel P. Zimmermann · 2023
The continuous progress in Named Entity Recognition allows the identification of complex entities in multiple domains.The traditionally used metrics like precision, recall, and F1-score can only reflect the classification quality of the underlying NER model to a limited extent.Existing metrics do not distinguish between a non-recognition of an entity and a misclassification of an entity.Additionally, the dealing with redundant entities remains unaddressed.We propose WRF, a Weighted Rouge F1 metric for Entity Recognition, to solve the mentioned gaps in currently available metrics.We successfully employ the WRF metric for automotive entity recognition, followed by a comprehensive qualitative and quantitative analysis of the obtained results.