TinyML for Computation-aware Transformer-based Anomaly Detection in Internal Combustion Systems
Iman Sharifirad, Jalil Boudjadar, Peter Gorm Larsen · 2025
This paper proposes a highly accurate, computation-efficient anomaly detection for an industrial internal combustion engine using a Transformer model. The proposed anomaly detection model enables identifying anomaly occurrences and quan-tifying how far the abnormality deviates from a normal state. This leads to informed decisions for efficient mitigation in the control loop of the combustion engine. To achieve computation-awareness while maintaining considerable accuracy, we have applied Pruning, a Tiny ML technique, to reduce the computation cost of the proposed Transformer model. The capacity analysis of the proposed model shows that, while trained on anomalies from a few combustion cylinders, our model can recognize and quantify anomalies from the entire set of cylinders. Furthermore, the analysis results demonstrated that Pruning reduced the computation cost by 8.4% and memory usage by up to 39% while the model's prediction performance barely decreased. Our transformer-based model to identify and quantify anomalies is deployable on the resource-constrained platform of the use case.