Improving Disfluency Detection by Self-Training a Self-Attentive Model
Paria Jamshid Lou, Mark S. Johnson · 2020
Self-attentive neural syntactic parsers using contextualized word embeddings (e.g.ELMo or BERT) currently produce state-of-the-art results in joint parsing and disfluency detection in speech transcripts.Since the contextualized word embeddings are pre-trained on a large amount of unlabeled data, using additional unlabeled data to train a neural model might seem redundant.However, we show that self-training -a semi-supervised technique for incorporating unlabeled data -sets a new state-of-the-art for the self-attentive parser on disfluency detection, demonstrating that self-training provides benefits orthogonal to the pre-trained contextualized word representations.We also show that ensembling selftrained parsers provides further gains for disfluency detection.