LEXI: Large Language Models Experimentation Interface
Guy Laban, Tomer Laban, Hatice Güneş · 2024
The recent developments in Large Language Models (LLMs) mark a significant moment in the research and development of social interactions with artificial agents. These agents are widely deployed in a variety of settings, with potential impact on users. However, the study of social interactions with agents powered by LLMs is still emerging, limited by access to the technology and to data, the absence of standardised interfaces, and challenges to establishing controlled experimental setups using the currently available platforms. To address these gaps, we developed LEXI, LLMs Experimentation Interface, an open-source tool for deploying artificial agents powered by LLMs in social interaction behavioural experiments. Using a graphical interface, LEXI allows researchers to build agents and deploy them in experimental setups along with forms for collecting self-reported data while collecting interaction logs. The outcomes of usability testing indicate LEXI’s broad utility, high usability, and minimal mental workload requirement, with benefits observed across disciplines. A proof-of-concept study exploring the tool’s efficacy in evaluating social human–agent interactions was conducted, resulting in high-quality data. A comparison of empathetic versus neutral agents indicated that people perceive empathetic agents as more social, and write longer and more positive messages towards them.