Evaluating bilingual lexicon induction using comparable corpora
Martin Laville · HAL (Le Centre pour la Communication Scientifique Directe) · 2023
Bilingual lexicon extraction (BLI) has as its objective the creation, in an automatic manner from bilingual corpora, of lexicons between two languages. It is most often used in the general domain, where the extracted lexicons can be used in machine translation or information retrieval. BLI systems work on large amounts of data and the results seem to be highly satisfactory. However, the evaluation data contains many errors, which could lead to a re-evaluation of the systems. A second and more marginal context of use of BLI systems is in specialized domains, where the objective is to obtain translations that are not available in classical dictionaries. Specialized corpora (about only one subject) are poorly supplied with data and it is complicated for BLI systems to obtain as good results as in the general domain. It is therefore necessary to adapt the approaches to take into account this particularity. In this thesis, we improve the results obtained in specialized domains by proposing the use of data selection techniques. Then, we focus on the evaluation process in general domain and more particularly on some biases present in evaluation data such as the overpresence of very frequent or graphically identical word pairs and we propose a more accurate and unified evaluation process that takes into account these weaknesses.