A Comprehensive Approach to Mitigate Return-Oriented Programming Attacks: Combining Operating System Protection Mechanisms and Hardware-Assisted Techniques
Xingnan Zhang, Jingjia Huang, Yue Feng · 2023
This paper proposes a comprehensive approach to mitigate ROP (Return-Oriented Programming) attacks by combining internal operating system protection mechanisms and hardware-assisted techniques. Through extensive literature review, we identify the effectiveness of ASLR (Address Space Layout Randomization) and LBR (Last Branch Record) in preventing ROP attacks. We present a process involving buffer overflow detection, hardware-assisted ROP attack detection, and the use of Turing detection technology to monitor control flow behavior. We envision a specialized tool that views and analyzes the last branch record, compares control flow with a baseline, and outputs differences in natural language. This tool offers a graphical interface, facilitating the prevention and detection of ROP attacks. The proposed approach and tool provide practical solutions for enhancing software security.