Challenges in Using Conversational AI for Data Science
Bhavya Chopra, Ananya Singha, Anna Fariha, Sumit Gulwani, Chris Parnin, Ashish Tiwari, Austin Zachary Henley · 2025
Large Language Models (LLMs) are transforming data science, offering assistance in coding, preprocessing, analysis, and decisionmaking. However, data scientists face significant challenges when interacting with LLM-powered agents and implementing their suggestions effectively. To explore these challenges, we conducted a mixed-methods study comprising contextual observations, semi-structured interviews (n=14), and a survey (n=114). Our findings reveal key obstacles, including difficulties in retrieving contextual data, crafting prompts for complex tasks, adapting generated code to local environments, and refining prompts iteratively. Based on these insights, we propose actionable design recommendations, such as data brushing for improved context selection and inquisitive feedback loops to enhance communication with conversational AI assistants in data science workflows.