TPC-Tracker: A Tracker-Predictor Correlation Framework for Latency Compensation in Aerial Tracking
Xuqi Yang, Yulong Xu, Renwu Sun, Tong Wang, Ning Zhang · Remote Sensing · 2026
Online visual object tracking is a critical component of remote sensing-based aerial vehicle physical tracking, enabling applications such as environmental monitoring, target surveillance, and disaster response. In real-world remote sensing scenarios, the inherent processing delay of tracking algorithms results in the tracker’s output lagging behind the actual state of the observed scene. This latency not only degrades the accuracy of visual tracking in dynamic remote sensing environments but also impairs the reliability of UAV physical tracking control systems. Although predictive trackers have shown promise in mitigating latency impacts by forecasting target future states, existing methods face two key challenges in remote sensing applications: weak correlation between trackers and predictors, where predictions rely solely on motion information without leveraging rich remote sensing visual features; and inadequate modeling of continuous historical memory from discrete remote sensing data, limiting adaptability to complex spatiotemporal changes. To address these issues, we propose TPC-Tracker, a Tracker-Predictor Correlation Framework tailored for latency compensation in remote sensing-based aerial tracking. A Visual Motion Decoder (VMD) is designed to fuse high-dimensional visual features from remote sensing imagery with motion information, strengthening the tracker-predictor connection. Additionally, the Visual Memory Module (VMM) and Motion Memory Module (M3) model discrete historical remote sensing data into continuous spatiotemporal memory, enhancing predictive robustness. Compared with state-of-the-art predictive trackers, TPC-Tracker reduces the Mean Squared Error (MSE) by up to 38.95% in remote sensing-oriented physical tracking simulations. Deployed on VTOL drones, it achieves stable tracking of remote sensing targets at 80 m altitude and 20 m/s speed. Extensive experiments on public UAV remote sensing datasets and real-world remote sensing tasks validate the framework’s superiority in handling latency-induced challenges in aerial remote sensing scenarios.