Feature Correlation Transformer for Estimating Ambiguous Optical Flow
Guibiao Fang, Junhong Chen, Dayong Liang, Frank Van Reeth, Luc Claesen, Zhenguo Yang, Wenyin Liu · Research Square · 2022
Abstract Cost volume is widely used to establish correspondences in optical flow estimation. However, when dealing with low-texture and occluded areas, it is difficult to estimate the cost volume correctly. Therefore, we propose a replacement: feature correlation transformer (FCTR), a transformer with self-and cross-attention alternations for obtaining global receptive fields and positional embedding for establishing correspondences. With global context and positional information, FCTR can produce more accurate correspondences for ambiguous areas. Using positional embedding allows the removal of the context network; the positional information can be aggregated within ambiguous motion boundaries, and the number of model parameters can be reduced. To speed up network convergence and strengthen robustness, we introduce a smooth L1 loss with exponential weights in the pre-training step. At the time of submission, our method achieves competitive performance with all published optical flow methods on both the KITTI-2015 and MPI-Sintel benchmarks. Moreover, it outperforms all optical flow and scene flow methods in KITTI-2015 foreground-region prediction.