Gesture Recognition Based on the Fusion of Millimeter-Wave Radar and Camera
Chuang Ren, Liang Zhang · 2023
To overcome the limitations encountered in traditional gesture recognition methods, such as sensitivity to illumination, this study proposes an integrated approach that combines millimeter wave radar and camera information. By leveraging millimeter wave radar technology, valuable motion information including distance, speed, motion reversal, and hand movement angles can be captured. Concurrently, camera images are employed to extract vision features of gestures. The fusion of these two modalities enables accurate capturing of spatial position, shape, and motion information during the recognition process. A series of experiments was conducted to assess the performance of the proposed method, comparing it with conventional camera-based gesture recognition methods. The experimental results substantiate the significant improvement in recognition accuracy and robustness achieved by the gesture recognition method that integrates millimeter wave radar and camera information.