A Parallel Training Research of Chinese Part-of-Speech Tagging CRF Model Based on MapReduce
Tao Liu, Lei Lin, Luo Chen, Wei Xiong · 2013
Conditional random field(CRF) model bears a major drawback of low training efficiency for large-scale data processing.A parallel method of conditional random field model training based on MapReduce is proposed to solve the problem.The method designs parallel algorithm for feature selection and parameters estimation of CRF model to achieve a parallel iterative scaling algorithm.Experiments show that the method improves the efficiency and reduces time cost significantly while guaranteeing the training result correctness.