Multi-scale Feature Injection for Occluded 3D Human Pose and Shape Estimation
Yunhui Shi, Yangyang Ge, Jin Wang · 2023
3D reconstruction has been applied to many research fields such as robots and computer vision with the fast development of technology. Despite significant progress, current 3D human pose and shape estimation methods still remain challenge to recovery 3D human mesh under occlusions. Previous works use a Iterative Error Feedback (IEF) loop to construct the regressor and often have disregarded information at occluded regions that make them difficult to handle occlusions. However, we argue that occluded regions have strong correlations with human body so that they can offer effective information for 3D human pose and shape estimation. To address this, we propose a multi-scale feature injection network MFINet, that utilizes the information at occluded regions as a secondary clews to enrich the image features in a coarse-to-fine manner. In MFInet, given the image feature at current scale, a Transformer-based module, called feature inject transformer module (FIM) is used to inject human feature into occluded region by considering their correlation. To this end, experiments show that our method is effective in both object and subject results on several benchmarks including Human3.6M, 3DPW, LSP and COCO.