A Topic-Constrained Sampling Method for Text Generation
Wenyi Ran, Jiaqiang Wan, Zhiqiang Wang, Haobo Zhu, Xinyu Qiu · 2024
Pre-trained large language models (PLLMs) can be fine-tuned to incorporate relevant knowledge and linguistic conventions of a local corpus, though this process typically involves high training costs and computing resources. This paper employs a local corpus to construct a Latent Dirichlet Allocation (LDA) topic model, and then generates topic-constrained text by the LDA. On the one hand, this method is easy to implement; on the other hand, it enhances the model's generalization capabilities on local corpus. Experimental results reveal that the proposed method achieves better performance than the baseline model in both diversity and generalization.