An empirical study of Definition Modeling with LLMs for the main languages of Latin America
Erica Kido Shimomoto, Edison Marrese-Taylor, Erique Reid · 2024
In this paper, we propose the first multilingual study on definition modeling focusing on languages of Latin America. We collect monolingual dictionary data for Spanish and Portuguese and perform an in-depth empirical study to test the performance of large language models in the definition modeling of monosemic words. A comprehensive human evaluation of the LLM-generated definitions validates our choice of COMET as an automatic evaluation metric, highlighting the zero and few-shot capabilities of these models in this new task and also showing their shortcomings and the limitations of COMET for automatic evaluation.