Inference Strategies for Machine Translation with Conditional Masking
Julia Kreutzer, George Foster, Colin Cherry · 2020
Conditional masked language model (CMLM) training has proven successful for nonautoregressive and semi-autoregressive sequence generation tasks, such as machine translation.Given a trained CMLM, however, it is not clear what the best inference strategy is.We formulate masked inference as a factorization of conditional probabilities of partial sequences, show that this does not harm performance, and investigate a number of simple heuristics motivated by this perspective.We identify a thresholding strategy that has advantages over the standard "mask-predict" algorithm, and provide analyses of its behavior on machine translation tasks.