Using the Amazon Mechanical Turk to Transcribe and Annotate Meeting Speech for Extractive Summarization
Matthew Marge, Satanjeev Banerjee, Alexander I. Rudnicky · 2010
Due to its complexity, meeting speech provides a challenge for both transcription and annotation. While Amazon’s Mechanical Turk (MTurk) has been shown to produce good results for some types of speech, its suitability for transcription and annotation of spontaneous speech has not been established. We find that MTurk can be used to produce highquality transcription and describe two techniques for doing so (voting and corrective). We also show that using a similar approach, high quality annotations useful for summarization systems can also be produced. In both cases, accuracy is comparable to that obtained using trained personnel.