Utterance disfluency handling in Indonesian-English machine translation
Khaidzir Muhammad Shahih, Ayu Purwarianti · 2016
We propose a hybrid technique on handling the utterance disfluency for Indonesian-English machine translation. The handling is done as the preprocessing in the machine translation system. In the preprocessing, we classify utterance disfluency using a combination of statistical based technique and rule based technique. We used 4 types of disfluency such as filler (for filled pause and discourse marker), rough copy, noncopy, and stutter. For the statistical based, we employ lexical features and CRF algorithm. In the experiment on the disfluency classification, we compared the statistical based only method and the hybrid method. The experimental result has shown that using a hybrid method achieved higher performance than only the statistical based method. The disfluency classification result is employed in the machine translation and enhance the BLEU score of Indonesian-English machine translation.