Enhancing CNC Machine Operator Accessibility through a Multimodal Chatbot
Harsh Sanchaniya, Dhruv Sinha, Ashish Joshi, Sneha Kothimbire, Bharati P. Vasgi, Punam Chavan · 2025
Modern CNC machining presents significant operational complexities and data interaction challenges, often creating accessibility barriers for a diverse operator workforce. This paper details the design, development, and accessibility-focused evaluation of a Flutter-based mobile conversational assistant tailored for CNC machine operators. Developed with industry collaboration, the system aims to bridge the accessibility gap by translating complex, real-time telemetry data (spindle speed, feed rate, alarms) into easily understandable, actionable insights. The architecture leverages IoT data streams, structured storage, efficient querying, and automated data processing. Crucially, it employs a multimodal interface (text and voice), multilingual support, and a conversational interaction model powered by a Large Language Model (LLM) with Retrieval-Augmented Generation (RAG). Specific features like hands-free continuous conversation mode and visual adjustments directly target physical, cognitive, and linguistic accessibility needs. By providing intuitive, context-aware guidance through natural language, the assistant empowers operators with varying technical literacy and language backgrounds, reduces cognitive load, facilitates hands-free information access, and aims to foster a more inclusive and efficient shop floor environment. Initial findings suggest significant potential in reducing task completion times and improving usability compared to traditional interfaces.