Uncalibrated Multi-view 3D Human Pose Estimation with Geometry Driven Attention
Victor Galizzi, Bertrand Luvison · 2024
To make up for the inherent challenging nature of 3D pose estimation, most multi-view frameworks rely on camera calibration, often leading to impractical or constrained architectures. Accurate human pose estimation is key to en-hancing human-computer interaction, gaming, health, sport and surveillance systems. By capturing precise and reliable body positions, our approach enables efficient and innovative downstream tasks. We leverage monocular 3D pose estimations and a novel geometry driven attention mechanism inside of a transformer lightweight architecture to produce high precision, occlusion aware refined 3D poses, with varying number of uncalibrated cameras. Our method shows competitive results on the in-lab dataset Human3.6M and in the in-the-wild environment of SkiPose PTZ-Camera, both in camera frames or in a disentangled person centric referential allowing practical downstream uses. Our approach matches state-of-the-art performance on Human3.6M, while being at least 3 times lighter. On the SkiPose base acquired under particularly difficult conditions, our results exceed those of the state of the art by being at least 3 times faster.