Automated methods for creating diversity in computer systems
Elena Gabriela Barrantes Sliesarieva, Stephanie Forrest · 2005
The pervasive homogeneity of computer systems attached to the Internet represents a serious security threat. Once an effective attack is crafted against one machine, it can quickly and easily be used against thousands of identical systems. A possible response to this situation can be found using biological diversity as inspiration. In nature, diversity provides a defense against unpredictable threats by maximizing the probability that some individuals will survive. Diversity in computer systems could confer security benefits by protecting against attacks that rely on homogeneity. Reducing the uniformity in existing systems is, however, a non-trivial task, as standardization must be maintained at many interface points. This dissertation assesses the costs and benefits of automated diversity by implementing one interface and one implementation randomization. The interface diversification scheme is a machine language randomization, named Randomized Instruction Set Emulation (RISE), intended as a protection against the threat of code-injection attacks, which insert malicious machine-language code into programs. RISE protects against these attacks by creating a unique machine code per process by mapping all executable bytes of the process to a random mask. When injected code attempts to execute, its code is also mapped to the mask, but given that it was not correctly encoded, the emulator ‘decodes’ it to random bytes, causing the attack to fail. Although the attack will not execute as intended, there is a small probability that random bytes will be interpreted as instructions. Therefore, an analysis of the risks associated with the execution of random instructions is presented. Implementation diversity is used as defense against Denial of Service (DoS) attacks which lock protocols into low-performance emergency states allowing the program to operate under stress without crashing. An attacker with knowledge about the stress assessment algorithm can trick the program into going into one of the emergency states states. The diversity approach presented randomizes parameters used in the TCP congestion control algorithm. The randomization is tested against a DoS attack that throttles the throughput of a system by forcing TCP to go into a congestion avoidance mode. The diversification achieves the objective of keeping a portion of the hosts in the attacked network operating at larger bandwidths than if they were all using the same standard parameter values.