P‐3.9: Enhancing Embodied Intelligence in Human‐Computer Interaction: MISE – A Multimodal Interaction Framework for Smart Environments
Yi Yin, Hengyu Cao, Jun Lin · SID Symposium Digest of Technical Papers · 2025
In this paper, we introduce MISE (Multimodal Interaction for Smart Environments), a novel framework that will use input from several sensors to increase the embodied intelligence of human‐computer interaction (HCI). Overcoming the limitations of current systems, which may make use of only single modalities or do not fully understand the physical environment, MISE aims. We demonstrate comprehensive environmental perception and precise control by fusing vision, voice, point cloud, temperature, and tactile data via MISE. On experimental results, we show that MISE improves the object retrieval accuracy by 3.5% and that it achieves a 2.7% better performance on complex tasks compared to existing models. The framework shows good robustness and flexibility for being used in smart homes, industrial automation, and assistive healthcare platforms. This research shows how MISE can be used to dramatically enhance the way humans interact with intelligent agents in a much more natural, efficient way.