Summary Evaluation with and without References
Juan‐Manuel Torres‐Moreno, Horacio Saggion, Iria da Cunha, Éric SanJuan, Patricia Velázquez-Morales · Polibits · 2010
We study a new content-based method for the evaluation of text summarization systems without human models which is used to produce system rankings.The research is carried out using a new content-based evaluation framework called FRESA to compute a variety of divergences among probability distributions.We apply our comparison framework to various well-established content-based evaluation measures in text summarization such as COVERAGE, RESPONSIVENESS, PYRAMIDS and ROUGE studying their associations in various text summarization tasks including generic multi-document summarization in English and French, focus-based multi-document summarization in English and generic single-document summarization in French and Spanish.