3D Reconstruction: A Feature Fusion and Transformer-Based Feature Extraction Network for MVSNet
Junaid Jamshid, Zhang Zhen, Xuzhi Wang, Wanggen Wan, Gao Changyu, Sun Xuetao · IEEE Access · 2025
Three-dimensional (3D) reconstruction creates digital 3D models from multiple Two-dimensional images and is widely applied in robotics, cultural heritage preservation, industrial inspection, and immersive media. Deep learning has significantly improved the accuracy and robustness of 3D reconstruction methods by learning rich feature representations. However, challenges remain, including capturing features at multiple scales, handling low-textured or repetitive regions, and balancing accuracy with memory and speed, especially for high-resolution or large scenes. To address these issues, this paper introduces a novel Transformer-based Feature Fusion Pyramid Network (TF-FPN) for efficient and accurate 3D reconstruction. The proposed architecture integrates two main components: (1) a frequency-aware multi-scale fusion module that adaptively combines structural and textural information across different scales, and (2) a coarse-stage feature matching transformer that establishes reliable cross-view correspondences using lightweight self-and cross-attention mechanisms at the lowest resolution. Furthermore, a memory-efficient cascaded multi-view stereo network framework, termed TF-MVSNet, is introduced based on the TF-FPN module. This framework employs variance-based cost aggregation and channel reduction to minimize memory overhead without sacrificing accuracy. Extensive evaluation demonstrates that TF-MVSNet frame work achieving superior accuracy in DTU dataset (0.313mmerror) and Tanks and Temples (F − score: 60.13%). We further show that replacing the baseline feature extractor with TF-FPN in MVSNeRF improves the rendering quality metrics (PSNR: 25.90 27.50 dB) while lowering memory usage. The lightweight design and efficiency make TF-FPN suitable for real-time or resource-constrained 3D reconstruction tasks. Code is available at https://github.com/jjunaid88/TF-MVSNet.git.