Intelligent interaction using gesture recognition and building information modeling
Zixuan Li, Yukang Wang, Xiaoping Zhou, Zhenghao Ouyang · Intelligent Buildings International · 2025
Human–Computer Interaction of Building Information Modeling (HBIM) has been widely applied in intelligent construction and other fields due to its strong usability. Existing HBIM operate Building Information Modeling (BIM) models through input devices like mice and keyboards, relying on devices and exhibiting low efficiency. Gesture interaction offers a new direction for non-contact HBIM due to its device-free and efficient. This study proposes an intelligent interaction scheme for BIM models based on gesture recognition. To facilitate natural and intuitive human-machine communication between gestures and BIM models, we custom-designed six target gestures for HBIM. Secondly, a convolutional neural network-based gesture recognition algorithm was developed to identify gesture features. Finally, various operations were implemented through the BIM model control module. The performance of our approach was evaluated in a real indoor environment, achieving a real-time accuracy rate of 98% for BIM model operations. Experimental results demonstrate that the approach high-precision recognition enhances interaction efficiency, enabling model interaction in diverse environments. This study complements HBIM from the perspective of non-contact interaction and inspires more intelligent BIM model interaction methods.