AI-Enable Augment Reality Application to Medical Education
Linqin Cai, Yu Gan, Lanrui Liu · 2025
Different from the traditional augment reality (AR) system for medical operation training, AR applications for medical image diagnosis aims to identify the abnormalities or lesions in immersive virtual environment, while provides auxiliary decision support for treatments. This paper presents an AI-enable AR pipeline for medical education to help students diagnose medical image based on deep learning and AR technology. Specifically, we firstly present an intelligent AR framework, allowing students to explore medical AR environment through headband glasses and mobile devices. Secondly, we propose a DCAM-Unet network based on 3D Unet deep network and dilated convolutional attention mechanism for the detection and segmentation of lesions or abnormality features in MRI images, aiming to improve the real-time performance and segmentation accuracy. Thirdly, we applied 3D surface rendering and optimizing methods to reconstruct the segmented mask data of MRI images. Finally, we performed experiments to verify our algorithms, and presented a case of AR application for medical education.