An Adaptive Approach to Recommending Obfuscation Rules for Java Bytecode Obfuscators
Yanru Peng, Yuting Chen, Beijun Shen · 2019
Bytecode obfuscation is an essential technique for protecting intellectual property and defending against Man-AtThe-End (MATE) attacks to Java/Android applications. Several bytecode obfuscators have been developed for modifying or refactoring Java bytecode (.class) so that it becomes hard to understand but remains fully functional. These obfuscators usually integrate a variety of obfuscation rules, allowing obfuscation algorithms to be combined and enforced on the applications. Meanwhile, it still remains a difficulty: Given a bytecode file f, which obfuscation rule(s) need to be applied such that f can get obfuscated sufficiently? This paper presents ORChooser (Obfuscation Rule Chooser), an adaptive approach to recommending a small number obfuscation rules for Java bytecode obfuscators. The key idea of ORChooser is, given a bytecode obfuscator, to (1)randomly select/unselect obfuscation rules for the obfuscator, and (2)calculate the obfuscation distance between the bytecode before and after obfuscation. Furthermore, ORChooser takes an iterative process to adaptively obfuscate the bytecode file f such that the obfuscated code is far away from f. We have implemented ORChooser and evaluated it on a state-of-the-art bytecode obfuscators: Android R8. The evaluation results clearly show the strength of ORChooser. In particular, within 5 iterations, ORChooser chose about 25% of obfuscation rules for R8, reducing more than 29% of the bytecode size. The similarity between the bytecode files before and after obfuscation is less than 27%, indicating that the ORChooser-supported obfuscators have obfuscated bytecode sufficiently and reduce its comprehensibility significantly.