Transformer-Based Anomaly Detection in Satellite IoT: A Hybrid Approach with Rotary Embedding and Kernelized Attention
Youssef Maaod, Ahmed Yasser Lotfy Belaih, Nabila Ahmed Ali, Jomana Farag Soliman Mohamed, Farah Mohamed Mahmoud Ahmed, Aly Maher Abdelfattah, Mohamad Fouad, Mohamed Abd Elaziz · 2025
Satellite-based Internet of Things (IoT) systems provide crucial connectivity for remote applications; however, they face challenges such as limited bandwidth and noisy data. To address these issues, we propose a novel hybrid Transformer architecture for anomaly detection in satellite IoT imagery. Our model integrates Rotary Positional Encoding to capture spatial relationships and Kernelized Attention for efficient non-linear feature extraction. We compare proposed hybrid approach with Rotary-only and Kernelized-only Transformers, demonstrating that the hybrid model achieves superior performance with an accuracy of 95.62% and a ROC-AUC of 99.44%. The results highlight the potential of our method to enhance the reliability of satellite-based IoT systems, particularly in challenging environments.