Atomizer: Fast, Scalable and Lightweight Heap Analyzer for Virtual Machines in a Cloud Environment
Salman Javaid, Aleksandar Zoranic, Irfan Ahmed, Golden G. Richard · 2012
In recent years process heap-based attacks have increased significantly. These attacks exploit the system under attack via the heap, typically by using a heap spraying attack. A large number of malicious files and URLs offering danger-ous contents are potentially encountered every day, both by client-side and server-side applications. Static and dynamic methods have been proposed to detect heap-based attacks in the literature, using various methodologies like NOZZLE. The main drawback with existing techniques is that they either consume too many resources or are complicated to implement. In this paper we propose Atomizer, which of-floads process heap analysis for guest VMs to the privileged domain using Virtual Machine Introspection (VMI). Atom-izer APIs can be used to implement various heap analyzing algorithms on processes running inside a VM. A simple heap-spray detection algorithm using Atomizer was implemented to determine the effectiveness of Atomizer. Use of Atomizer cannot be detected by in-guest VM malware, has minimal impact on the cloud server, is very effective in detecting heap spraying malwares, and is simple to deploy. Our architec-ture is particularly applicable to cloud environments where virtualization is used to host guest VMs.