Non-Parallel Text Style Transfer using Self-Attentional Discriminator as Supervisor
Kuan Feng, Yanmin Zhu, Jiadi Yu · 2021 IEEE International Conference on Big Data (Big Data) · 2021
Non-parallel text style transfer aims to rephrase a sentence with another style while reserving its content relying on non-parallel data. Most existing methods can be divided into two groups: 1) separating content and style of the input text and 2) directly modeling the style transfer process. To the best of our knowledge, all these existing works lack fine-grained supervisory signals during training, which leads to difficulty in achieving a good balance between content preservation and style satisfaction. However, the study on supervisors which could provide fine-grained supervisory signals for training transfer models has received relatively less attention. Thus, we propose a self-attentional discriminator and a training strategy for training an attentional transfer model by leveraging the fine-grained supervisory signals from the proposed discriminator. Specifically, our discriminator provides token-wise style/content weights by performing self-attention between the sentence vector and the token embeddings, which forms a shortcut in back-propagation leading to more accurate gradients. The style/content weights also pose a better content alignment constraint and improve the interpretability of the training procedure by identifying stylized tokens. The training of our transfer model is end-to-end via Gumbel-Softmax with the pre-trained discriminator. Experiments on two datasets with automatic and human evaluations as well as theoretical and empirical analysis demonstrate the effectiveness of our method.1