Looking for the best Evaluation Method for interlingua-based Spoken Language Translation in the Medical Domain
Marianne Starlander, Paula Estrella · Archive ouverte UNIGE (University of Geneva) · 2011
This paper focuses on the quality of rule-based machine translations collected using our open-source limited-domain medical spoken language translator (SLT) tested at the Dallas Children’s hospital. Our aim is to find the best-suited metrics for our Interlingua rule based machine translation (RBMT) system. We applied both human metrics and a set of well-known automatic metrics (BLEU, WER and TER) to a corpus of translations produced by our system during a controlled experiment. We also compared the scores obtained for both type of evaluation with those obtained on translations produced by the well-known statistical machine translation (SMT) system GoogleTranslate in order to have a point of comparison. Our aim is to find the best-suited metric for our type of Interlingua RBMT SLT system.