Detecting Network-Based Obfuscated Code Injection Attacks Using Sandboxing
Stig Andersson, Andrew John Clark, George Mohay · QUT ePrints (Queensland University of Technology) · 2005
Intrusion detection systems (IDSs) are widely recognised as the last line of defence often used to enable incident response when intrusion prevention mechanisms are ineffective, or have been compromised. A signature based network IDS (NIDS) which operates by comparing network traffic to a database of suspicious activity patterns (known as signatures) is a popular solution due to its ease of deployment and relatively low false positive (incorrect alert) rate. Lately, attack developers have focused on developing stealthy attacks designed to evade NIDS. One technique used to accomplish this is to obfuscate the shellcode (the executable component of an attack) so that it does not resemble the signatures the IDS uses to identify the attacks but is still logically equivalent to the clear-text attacks when executed. We present an approach to detect obfuscated code injection attacks, an approach which compensates for efforts to evade IDSs. This is achieved by executing those network traffic segments that are judged potentially to contain executable code and monitoring the execution to detect operating system calls which are a necessary component of any such code. This detection method is based not on how the injected code is represented but rather on the actions it performs. Correct configuration of the IDS at deployment time is crucial for correct operation when this approach is taken, in particular, the examined executable code must be executed in an environment identical to the execution environment of the host the IDS is monitoring with regards to both operating system and architecture. We have implemented a prototype detector that is capable of detecting obfuscated shellcodes in a Linux environment, and demonstrate how it can be used to detect new or previously unseen code injection attacks and obfuscated attacks as well as well known attacks.