Semantic view re-creation for the secure monitoring of virtual machines
Wenke Lee, Martim Carbone · 2012
The insecurity of modern-day software systems has created the need for security monitoring applications, such as anti-virus tools. These applications conduct passive monitoring of the system's state and active monitoring of the system's events. Two serious deficiencies are commonly found in such applications. First, their lack of isolation from the system being monitored allows malicious software to tamper with or disable them. Second, the lack of secure and reliable monitoring primitives in the operating system compromises their visibility, making them easy to be evaded. A technique known as Virtual Machine Introspection attempts to solve these problems by leveraging the strong isolation and mediation properties of full-system virtualization. It isolates the monitoring application in a separate, security virtual machine, from where it can securely monitor a guest virtual machine by leveraging the hypervisor's view of resources. This separation creates, however, a problem known as semantic gap, which can be defined by the loss of a high-level view of the guest's state and events from the part of the monitoring application. It occurs as a result of the low-level separation enforced by the hypervisor between the guest and the security virtual machine. This thesis proposes and investigates novel techniques to overcome the semantic gap, advancing the state-of-the-art on the syntactic and semantic guest view re-creation for security applications that conduct passive and active monitoring of virtual machines. In the space of passive monitoring, we propose a new technique for reconstructing a syntactic view of the guest OS kernel's heap state. By applying a combination of static code and dynamic memory analysis techniques, we are able to reconstruct a map of the guest OS's dynamic kernel objects. Our key contribution over previous work is the accuracy and completeness of our analysis, which translates into stronger monitoring capabilities for security applications. Although sufficient for certain types of integrity checking applications, a syntactic view of the guest state is not enough for others that require access to information at a higher level. With this in mind, we propose a technique that combines the security of out-of-VM monitoring with the semantic awareness of in-VM monitoring. By allowing out-of-VM applications to invoke and securely execute API functions inside the monitored guest's kernel, we eliminate the need for the application to know details of the guest's internal data structures. Our key contribution over previous work is the ability to overcome the semantic gap between the monitoring application and the guest OS in a robust and secure manner, by relying on the guest's own code. A security monitoring solution cannot be complete without an active monitoring component that intercepts and evaluates guest events as they happen. In this space, we propose a new virtualization-based event monitoring technique based on the interception of kernel data modifications as opposed to code execution trapping. Our key contribution over previous work is the ability to monitor high-level operating system events without the need for in-guest components and without the same circumvention problems of code execution hooks, and the ability to automatically re-create the syntactic context of guest kernel memory accesses.