Malware Detection in PDF Files using Machine Learning
Bonan Cuan, Aliénor Damien, Claire Delaplace, Mathieu Valois · Proceedings of the 15th International Joint Conference on e-Business and Telecommunications · 2018
In this report we present how we used machine learning techniques to detect malicious behaviours in PDF files.At this aim, we first set up a SVM (Support Machine Vector) classifier that was able to detect 99.7% of malware. However, this classifier was easy to lure with malicious PDF, we forged to make them look like clean ones. We first proposed a very naive attack, that was easily stopped by the establishment of a threshold. We also implemented a gradientdescent attack to evade this SVM. This attack was almost 100% successful. In order to fix this problem, we provided counter-measures to the latter attack. A more elaborated features selection, and the use of a threshold, allowed us to stop up to 99.99% of these attacks.Finally, using adversarial learning techniques, we were able to prevent gradient descent attacks by iteratively feeding the SVM with malicious forged PDF. We found that after 3 iterations, every gradient-descent forged PDF were detected, completely preventing the attack.