AI Agents in Physical Computing : Through Natural Language Interaction
Anas Aljoudi · KTH Publication Database DiVA (KTH Royal Institute of Technology) · 2026
This thesis explores how AI agents and natural language interaction can support beginner interaction with physical computing systems. Using a research through design approach, the project combined literature research, interviews, prototyping, user testing, thematic analysis, and iterative refinement to investigate how users interact with AI assisted Arduino workflows. The research focused on how users shape physical system behavior through natural language, how they interpret AI generated outputs, and how they negotiate control when interacting with physical hardware. The project resulted in an interactive website platform prototype that integrates natural language prompting, AI assisted code generation, wiring visualization, and hardware feedback within a single workflow. The findings show that AI agents reduce barriers related to programming, but users still experience challenges related to wiring, hardware interpretation, and physical understanding. The thesis contributes design knowledge for future AI assisted physical computing systems by highlighting the importance of transparency, visual guidance, and physical understanding during interaction.