Knowledge Distillation for Anomaly Detection
Adrian Alan Pol, Ekaterina Govorkova, Sonja Grönroos, N. Chernyavskaya, Philip Coleman Harris, M. Pierini, I. Ojalvo, P. Elmer · 2023
Unsupervised deep learning techniques are widely used to identify anomalous behaviour.The performance of such methods is a product of the amount of training data and the model size.However, the size is often a limiting factor for the deployment on resource-constrained devices.We present a novel procedure based on knowledge distillation for compressing an unsupervised anomaly detection model into a supervised deployable one and we suggest a set of techniques to improve the detection sensitivity.Compressed models perform comparably to their larger counterparts while significantly reducing the size and memory footprint.