Combining Old School Autoencoder with Cotracker for Improved Skin Feature Tracking
Wei-Pei Shi, Torbjörn E. M. Nordling · 2024
Background: Skin feature tracking enables quan-tification of human motion in an explainable way, making it suitable for clinical assessments. Accuracy is crucial, but no study has investigated state-of-the-art deep neural network-based point tracking models such as Cotracker. Cotracker jointly tracks points and has been shown to have better 3-pixel accuracy than five other state-of-the-art deep learning methods on the two most commonly used datasets for evaluation of single target point tracking. In 2021, Chang and Nordling introduced the Deep Feature Encoder (DFE) and demonstrated skin feature tracking so accurate that the errors cannot be excluded to stem from the manual labeling of the videos based on a$\chi^{2}$-test. Problem: How accurately can different methods track skin features and how to avoid the intrinsic weaknesses of the methods? Methods: We use videos of the Unified Parkinson's Disease Rating Scale postural tremor test recorded at two hospitals for benchmarking. DFE utilizes the encoder part of an autoencoder consisting of a five-layer convolutional neural network trained to reproduce skin crops without supervision. The residual squared error of the latent features of the encoder is then compared with crops to obtain a predicted position. We also propose Cotracker-DFE, using Cotracker to obtain an approximate position and subsequently cropping a small area that is fed to DFE to obtain a position predicted with a lower mean pixel error. Results: The mean Euclidean distance errors of Cotracker, DFE, and Cotracker-DFE are 1.2,0.8, and 0.8 pixels, respectively. DFE requires time-consuming computations, making it 35 times slower than Cotracker. Conclusion: The old school DFE provided more accurate skin feature tracking, while combining DFE with Cotracker provides the best overall performance, circumventing the lack of labeled data and computational resources required to fine-tune Cotracker.