Toward Conversational Decision Support Systems: Integrating LLMs in the Operations Research Methodology
Mariusz Kaleta · Annals of Computer Science and Information Systems · 2025
This paper introduces the concept of Conversational Decision Support Systems (C-DSS)-a novel, agent-based framework that leverages Large Language Models (LLMs) to enhance the Operations Research (OR) methodology.We focus on the modeling and coding stages of decision support systems, where language-based interaction is crucial.The paper evaluates the effectiveness of LLMs in generating mathematical models and AMPL code for a curated set of 20 LP/MILP artifacts.Four architectural setups are analyzed: a monolithic LLM agent (M/C), its enhancement with a code verifier (M/C+V), agentbased decomposition with RAG-enhanced coding (M+C R +V), and full specialization with RAG-enhanced modeling and coding (M R +C R +V).Experimental results on two benchmark problems reveal that the targeted retrieval-augmented generation technique (RAG) significantly improves performance for complex modeling patterns such as piecewise functions, indicator constraints, and nested logic.We also propose a broader vision of C-DSS as a multi-agent ecosystem-including agents for visualization, explanation, verification, and orchestration-suggesting a path toward more explainable, adaptable, and intelligent decision support systems.