Dual Distillation: A Lightweight and Effective Intrusion Detection Model for IoT Security
Chonghao Pei, Lotfi Mhamdi · 2025
With the exponential growth of IoT devices, ensuring efficient and accurate network anomaly detection in resource-constrained environments has become a pressing challenge. A lightweight dual distillation approach is proposed, combining data distillation and model distillation. Data distillation is achieved through KMeans clustering and refinement, while model distillation leverages a random forest (RF) teacher model to guide a simplified multi-layer perceptron (MLP) student model. The approach is validated on CICIDS2017 and TON-IoT datasets, where dataset sizes are reduced to 20% and 40%, respectively. Experimental results demonstrate that the proposed method enhances MLP accuracy on CICIDS2017 from 93.17% to 99.66% and on TON-IoT from 89.00% to 93.14%. Moreover, the method significantly reduces model parameters compared to the RF teacher model while maintaining competitive performance, making it highly suitable for resource-constrained IoT environments.