A Critical Reassessment of Evaluation Baselines for Speech Summarization
Gerald M. Penn, Xiaodan Zhu · 2008
We assess the current state of the art in speech summarization, by comparing a typical summarizer on two different domains: lecture data and the SWITCHBOARD corpus. Our results cast significant doubt on the merits of this area’s accepted evaluation standards in terms of: baselines chosen, the correspondence of results to our intuition of what “summaries” should be, and the value of adding speechrelated features to summarizers that already use transcripts from automatic speech recognition (ASR) systems. 1 Problem definition and related literature Speech is arguably the most basic, most natural form