Unsupervised Chinese Word Segmentation with BERT Oriented Probing and Transformation
Wei Li, Yuhan Song, Qi Silvia Su, Yanqiu Shao · Findings of the Association for Computational Linguistics: ACL 2022 · 2022
Word Segmentation is a fundamental step for understanding many languages.Previous neural approaches for unsupervised Chinese Word Segmentation (CWS) only exploit shallow semantic information, which can miss important context.Large scale Pre-trained language models (PLM) have achieved great success in many areas.In this paper, we propose to take advantage of the deep semantic information embedded in PLM (e.g., BERT) with a self-training manner, which iteratively probes and transforms the semantic information in PLM into explicit word segmentation ability.Extensive experiment results show that our proposed approach achieves a state-of-the-art F1 score on two CWS benchmark datasets.The proposed method can also help understand low resource languages and protect language diversity.1