Are Large Language Models Actually Good at Text Style Transfer?
Sourabrata Mukherjee, Atul Kr. Ojha, Ondřej Dušek · 2024
We analyze the performance of large language models (LLMs) on Text Style Transfer (TST), specifically focusing on sentiment transfer and text detoxification across three languages: English, Hindi, and Bengali.Text Style Transfer involves modifying the linguistic style of a text while preserving its core content.We evaluate the capabilities of pre-trained LLMs using zero-shot and few-shot prompting as well as parameter-efficient finetuning on publicly available datasets.Our evaluation using automatic metrics, GPT-4 and human evaluations reveals that while some prompted LLMs perform well in English, their performance in on other languages (Hindi, Bengali) remains average.However, finetuning significantly improves results compared to zero-shot and fewshot prompting, making them comparable to previous state-of-the-art.This underscores the necessity of dedicated datasets and specialized models for effective TST.