Multi-View 3D Human Pose and Shape Estimation with Epipolar Geometry and Mix-Graphormer
Hai‐Kun Wang, Mohong Huang, Yang Zhang, Ke Song · 2023
This paper focuses on the estimation of 3D human pose and shape from multi-view images. Despite significant progress in monocular 3D human pose and shape estimation, monocular methods still suffer from limitations such as occlusion, pose ambiguity, and depth blur. Multi-view images, which contain depth information, can be used to address these issues. We utilize Epipolar Geometry to establish point correspondences across different views, thus enabling multi-view feature fusion. We propose a Mix-Graphormer encoder that uses graph convolutional networks to establish fine-grained local interactions and obtain the correlation between key points and mesh vertices. To better utilize the features of the multi-view fusion heatmap, we propose a mix multi-head self-attention mechanism (Mix-MSA), which extracts features from the target human body mesh template and the fused multi-view heatmaps while integrating their interaction information. Experiments are conducted on the Human3.6M dataset and the MPI-INF-3DHP dataset. The results demonstrate that our model is competitive with state-of-the-art 3D shape and pose models.