Human-Centric Video Anomaly Detection Through Spatio-Temporal Pose Tokenization and Transformer
Ghazal Alinezhad Noghre, Armin Danesh Pazho, Hamed Tabkhi · IEEE Access · 2026
Human-centric Video Anomaly Detection (VAD) is important for safety-critical applications such as smart surveillance, traffic monitoring, and healthcare, yet it remains challenging because anomalies are rare, diverse, open-set, and environment-dependent. Existing pixel-based methods may suffer from background sensitivity, appearance bias, and privacy concerns, while pose-based methods often rely on isolated graph, reconstruction, prediction, normalizing-flow, or Transformer-based strategies. This reveals a key gap: the lack of a task-specific pose representation that jointly captures posture structure, relative motion, and temporal pose evolution for anomaly scoring. In this paper, we introduce SPARTA (Spatio-temporal Pose And Relative pose Transformer for Anomaly detection), a self-supervised human-centric VAD framework whose novelty lies not in the isolated use of pose data or Transformer architectures, but in the task-specific integration of pose representation, tokenization, and anomaly scoring. SPARTA introduces Spatio-Temporal Pose and Relative Pose (ST-PRP) tokenization, which jointly encodes absolute body configuration and relative motion displacement to expose both spatial and temporal pose dependencies to Transformer self-attention. In addition, SPARTA proposes a Unified Encoder Twin Decoders (UETD) Transformer core that combines current-sequence reconstruction and future-sequence prediction through two complementary non-autoregressive decoder branches. By fusing reconstruction-based and prediction-based anomaly scores, SPARTA detects abnormal posture and motion patterns that may be missed by single-branch models. Evaluations on SHT, HR-SHT, CHAD, and NWPUC show that SPARTA-H achieves the best average AUC-ROC of 75.87%, improving over the previous state of the art by 1.75 percentage points, and the best average EER of 0.29. SPARTA-H obtains AUC-ROC scores of 85.75%, 87.23%, 67.04%, and 63.48% on SHT, HR-SHT, CHAD, and NWPUC, respectively, while using only 0.5 million parameters with 5.96 ms average end-to-end latency. For more details and access to the implementation, please visit: https://github.com/TeCSAR-UNCC/SPARTA/tree/main