Human Evaluation for Translation Quality of ChatGPT: A Preliminary Study
Yanqing Zhao, Min Zhang, Xiaoyu Chen, Yadong Deng, Aiju Geng, Limin Liu, Xiaoqin Liu, Wei Li, Yanfei Jiang, Hao Yang, Yu Han, Shimin Tao, Ning Xie, Xiaochun Li, Miaomiao Ma, Zhaodi Zhang · 2023
ChatGPT has shown promising results for Machine Translation (MT).However, whether it is comparable to standard translation models and performs well in some specific domain remains as an open question.In this paper, we conduct human evaluations on its translation performance in three domains using the Direct Assessment (DA) method.The evaluation result shows that ChatGPT as a whole achieves comparable performance with standard translation models, especially in the general domain.However, ChatGPT's performance is inferior in terms of translating domain-specific terminologies, and it appears to be informal when it comes to the Information and Communications Technology (ICT) and biomedical domains, where the text style is distinctively different from that in the general domain.