On the Security of Machine Learning Beyond the Feature Space
Erwin Quiring · Digitale Bibliothek Braunschweig (Verbundzentrale Göttingen (VZG)) · 2021
Machine learning is increasingly used in security-critical applications, such as malware detection, face recognition, and autonomous driving. However, learning methods are vulnerable to different types of attacks that thwart their secure application. So far, most research has focused on attacks in the feature space of machine learning, that is, the vector space underlying the learning process. Although this has led to a thorough understanding of the possible attack surface, considering the feature space alone ignores the environment machine learning is applied in. Inputs are usually given as real-world objects from a problem space, such as malicious code or PDF files. Hence, an adversary has to consider both the problem and the feature space. This is not trivial, as both spaces have no one-to-one relation in most application areas and feature-space attacks are thus not directly applicable in practice. As a result, a more thorough examination is required to understand the real-world impact of attacks against machine learning. In this thesis, we explore the relation between the problem and the feature space regarding the attack surface of learning-based systems. First, we analyze attacks in the problem space that create real objects and that mislead learning methods in the feature space. A framework is developed to examine the challenges, constraints, and search strategies. To gain practical insights, we examine a problem-space attack against source code attribution. The created adversarial examples mislead the attribution in the majority of cases. Second, we analyze the mapping from problem to feature space. Using the example of image scaling, we study attacks that exploit the mapping and that are agnostic to the learning model or training data. After identifying the root cause of these attacks, defenses for prevention are examined against adversaries of different strengths. Furthermore, the feature space also has an inherent connection to the media space of digital watermarking. This space is a vector space in which watermarks are embedded and detected. As adversaries target this process, attacks and defenses have been extensively studied here as well. Linking both spaces allows us to transfer attacks, defenses, and knowledge between machine learning and watermarking. Taken together, this thesis provides a novel view on the security of machine learning beyond the feature space by including the problem space and the media space into the security analysis.