Towards a Knowledge Management Framework for LLM-Generated Personas in Collaborative Systems
Lucas Nóbrega, Luiz Felipe Martinez, Yuri Fernandes de Andrade Lima, Carlos Eduardo Barbosa, Matheus Argôlo, Herbert Salazar, Geraldo Bonorino Xexeo, Jano Moreira de Souza · 2025
With the advancement of Artificial Intelligence (AI) technologies, the world has been increasingly reshaped across multiple domains - healthcare, finance, education, and creative fields. This expansion transforms individual tasks and facilitates complex collaborations where humans and machines work to- gether in unprecedented ways. Through machine learning and, more specifically, large language models (LLMs), AI now plays a crucial role in enhancing productivity. Recent technological advancements have significantly boosted the text generation capabilities of LLMs, enabling them to simulate personas that can represent groups or specific well-known individuals. By simulating expert perspectives, LLMs provide insights to guide human decision-making across various collaborative frameworks in Computer-Supported Cooperative Work (CSCW). Personas, as simulated identities, have proven helpful in group brainstorming sessions, consensus-building exercises, and virtual advisory boards, which help represent diverse viewpoints and facilitate group decisions. Among these CSCW methods, the Delphi method is a well-established approach for reaching consensus among experts through iterative feedback and structured discussions. This work introduces a framework to manage and evaluate LLM-generated personas simulating expert input in Delphi studies. Our contributions include a framework designed for evaluating LLM personas in Delphi studies using a text similarity validation technique comparing real and simulated expert opinions and an evaluation of a Delphi about Brazilian higher education, assessing the LLMs' reliability in real-world scenarios. Findings show LLMs can effectively replicate expert perspectives, enhancing AI integration in specialized decision-making.