Incorporating Stylistic Lexical Preferences in Generative Language Models
Hrituraj Singh, Gaurav Verma, Balaji Vasan Srinivasan · 2020
While recent advances in language modeling has resulted in powerful generation models, their generation style remains implicitly dependent on the training data and can not emulate a specific target style.Leveraging the generative capabilities of a transformer-based language models, we present an approach to induce certain target-author attributes by incorporating continuous multi-dimensional lexical preferences of an author into generative language models.We introduce rewarding strategies in a reinforcement learning framework that encourages the use of words across multiple categorical dimensions, to varying extents.Our experiments demonstrate that the proposed approach can generate text that distinctively aligns with a given target author's lexical style.We conduct quantitative and qualitative comparisons with competitive and relevant baselines to illustrate the benefits of the proposed approach.