Toward Interpretable Multimodal Violence Detection With Knowledge Distillation and Modality-Aligned Preprocessing
Wendong Jiang, Chih-Yung Chang, Ming‐Yang Su, Yue‐Shi Lee, Diptendu Sinha Roy · IEEE Transactions on Systems Man and Cybernetics Systems · 2025
Social violence presents a compelling challenge to public safety, yet existing multimodal detection systems exhibit excessive reliance on RGB image semantics and opaque decision-making processes. Despite leveraging visual and auditory data, current models demonstrate RGB bias in feature prioritization, as evidenced by explainability analyzes, thereby limiting their generalization for behavioral understanding. Additionally, modality inconsistency and inefficient fusion mechanisms impair model transparency and training stability. To bridge these gaps, this study proposes modality-aligned preprocessing (VAJ) that structurally unifies visual-auditory features through conflict resolution and input optimization, explicitly suppressing color dominance while enhancing interpretable feature representations. Complementing this, we design DTVDS, an interpretable detection framework integrating knowledge distillation to transfer distilled behavioral insights from a cumbersome teacher network to an efficient student model. This dual strategy not only addresses computational overhead but also clarifies decision logic through simplified inference pathways. Evaluations on XD-Violence and UCF-Crime benchmarks demonstrate superior performance, with AP (89.64%) and AUC (88.35%) outperforming existing methods. Qualitative evaluations further validate interpretability, revealing modality-coherent attention maps and human-aligned rationale visualization. The proposed method advances violence detection by addressing persistent shortcomings in multimodal alignment and model explainability.