Developing NLP models for Taiwanese Hokkien with challenges, script unification, and language modeling
Jeng‐Shin Sheu, Shun-Yi Xu, Aftab Ahmad · Journal of the Chinese Institute of Engineers · 2025
In recent years, artificial intelligence (AI) has advanced significantly in natural language processing (NLP), marked by powerful pre-trained language models from major companies. While these models excel across domains, they predominantly cater to widely spoken languages, while neglecting important ones like Taiwanese Hokkien. This paper identifies challenges hindering Taiwanese Hokkien’s integration into NLP and proposes solutions. We use deep neural networks to train a translation model to convert diverse Taiwanese Hokkien texts into a unified script. Using this unified text, we train a pre-trained language model and a generative model for Taiwanese Hokkien. These models play a pivotal role in the realm of NLP for Taiwanese Hokkien, enabling us to explore various aspects of integrating Taiwanese Hokkien into the AI landscape. The downstream tasks, built upon the pre-trained language model, showcase the successful acquisition of a universal language representation for Taiwanese Hokkien. Moreover, the generative model impressively generates fluent Taiwanese Hokkien text.