Improving spoken language translation by automatic disfluency removal: evidence from conversational speech transcripts
Sharath Rao, Ian Richard Lane, Tanja Schultz · 2007
Machine translation of spoken language has made significant progress in recent years, however, translation quality is still limited due to specific idiosyncrasies of spoken language; including the lack of well-formed sentences and the presence of disfluencies. In this paper, we investigate the effect of disfluencies on Statistical Machine Translation (SMT) and introduce an Automatic Disfluency Removal scheme as a pre-processing step prior to translation. On Broadcast Conversation (BC) transcripts the proposed approach demonstrates that up to 8% relative improvement in BLEU can be obtained via Automatic Disfluency Removal. Furthermore, we show that the detrimental effect of disfluencies on SMT differs across disfluency types.