Kernel-Assisted Debugging of Linux Applications
Tobias Holl, Philipp Klocke, Fabian Franzen, Julian Kirsch · 2018
On Linux, most---if not all---debuggers use ptrace debugging API to control their target processes. However, ptrace proves unsatisfactory for many malware analysis and reverse engineering tasks: So-called split-personality malware often adapts its behavior in the presence of a debugger, yet ptrace makes no attempt to hide from a target process. Furthermore, ptrace enforces a strict one-to-many relation meaning that while each tracer can trace many tracees, each tracee can only be controlled by at most one tracer. Simultaneously, the complex API and signal-based communications provide opportunities for erroneous usage.