Finding the System that Suits you Best: Towards the Normalization of MT Evaluation
Paula Estrella, Andréi Popescu-Belis, Nancy L. Underwood · Archive ouverte UNIGE (University of Geneva) · 2005
The Framework for the Evaluation of Machine Translation, FEMTI, brings together the many disparate metrics and methods which have been devised for MT and helps evaluators to design an evaluation plan based on the context of use intended for the system. FEMTI allows therefore the generation of more standardized and reusable evaluation plans. By evaluators we mean not only developers and programmers, but also end users, managers, and anyone else with a stake in the acquisition or deployment of a system. Thus, the use of FEMTI is not limited to experts in the field of MT. In this paper we describe FEMTI and the latest enhancements we are making to it, in particular the interfaces which not only allow evaluators to create their own tailor-made evaluation plans, but also to contribute their experience and expertise in constantly improving the resource for the community at large.