Parallel Acceleration of IBM Alignment Model based on Lock-Free Hash Table
Siyuan Jing, Gaorong Yan, Xingyuan Chen, Peng Jin, Zhaoyi Guo · 2016
Word alignment is still the most time-consuming task in statistical machine translation. This paper deeply analyzes the performance bottleneck of MGIZA++, which is the most commonly used word alignment tool, and proposes a lock-free programming method to speed up the modern word alignment parallel algorithm. Firstly, a word alignment oriented linear lock-free hash table is introduced. Secondly, three sub-tasks of IBM alignment model, i.e. model initialization, expectation calculation and normalization, are parallel designed and implemented. Experimental results show that the method proposed in this paper can significantly reduce the execution time compared with MGIZA++.