An Empirical Analysis of the Writing Styles of Persona-Assigned LLMs
Manuj Malik, Jing Jiang, Kian Ming A. Chai · 2024
There are recent efforts to "personalize" large language models (LLMs) by assigning them specific personas.This paper explores the writing styles of such persona-assigned LLMs across different socio-demographic groups based on age, profession, location, and political affiliations, using three widely-used LLMs.Leveraging an existing style embedding model that produces detailed style attributes and latent Dirichlet allocation (LDA) for broad style analysis, we measure style differences using Kullback-Leibler divergence to compare LLM-generated and human-written texts.We find significant style differences among personas.This analysis emphasizes the need to consider socio-demographic factors in language modeling to accurately capture diverse writing styles used for communications.The findings also reveal the strengths and limitations of personalized LLMs, their potential uses, and the importance of addressing biases in their design.The code and data are available at: https://github.com/ ra-MANUJ-an/writing-style-persona Example from the training corpus That was the funniest thing so far this season.Sam SCREECHING and stabbin' wights all around in battle fury while more fall on him like throw pillows.Associated 20 Style Descriptors, ordered by score 'The author uses uncommon phrases.','The author uses descriptive words.','The author uses colorful language.','The author uses an energetic style.','The author uses a clever play on words.','The author is vivacious.','The author is using words to create a vivid and engaging atmosphere.','The author is using vivid descriptions.','The author is using punctuation to create a sense of tension and suspense.','The author is using male pronouns.','The author is intense in their writing.','The author is dramatic.','The author is captivating.','The author has a distinct and memorable style.','The author is creating a sense of anticipation and excitement.','The author is using a playful style.','The author is describing a current event.','The author uses victorious language.','The author is using a lighthearted tone.','The author uses singular subjects.'