CoDSDF: Constrained Decoding of SDF Quantized Vectors for Single-View 3D Reconstruction Using Conditional Generation
Deli Zhu, Yijie He, Yunong Yang, Haibin Fu · 2024
The Virtual Reality faces a shortage of personalized 3D resources, posing a challenge in easily extracting real-world objects and converting them into editable 3D shape meshes. Single-view reconstruction, which transforms easily accessible information such as a single image into specified 3D shapes, can be regarded as a unique conditional generation task. This paper proposes the CoDSDF network, which can generate high-quality detail-rich 3D mesh for multiple categories. We quantize 3D SDF shapes into a combination of low-dimensional discrete feature vectors and then model the distribution of these combinations, which are stored in the codebook, to associate them with image information as target conditional inputs for the generator.The CoDSDF incorporates SDF quantized vector decoding to retain more details in the generated. Additionally, a conditional encoding network built with Split-Attention combines more global and local features to predict the necessary conditional distribution inputs based on the image, guiding the generator to achieve precise reconstruction. CoDSDF captures more 3D structural details, such as holes, from a single image, demonstrating excellent single-view reconstruction results across 13 categories, including chairs, airplanes, and sofas. The generated data can be converted into point cloud meshes and integrated into digital media workflows.