A robustness testing approach for RESTful Web Services
João Agnelo · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · 2020
Robustness is the degree to which a certain system or component can operate correctly in the presence of invalid inputs or stressful environmental conditions.With the increasing complexity and wide-spread use of software systems, obtaining assurances regarding their robustness has become of vital importance.This is especially true in mission-or businesscritical systems where a failure may have severe consequences on the business or in human lives.Web services, in particular, are often exposed to abnormal and malicious input that could lead to serious issues, such as loss of data and information disclosure.Most modern web companies, including Google, Instagram, Spotify and Slack, are supported by REST services (also known as REST APIs), a type of software system that follows the relatively loose REST architectural style.In this type of scenarios, heterogeneity is prevalent and software is sometimes exposed to unexpected conditions that may activate residual bugs, leading service operations to fail, which may result in financial or reputation losses (e.g., information disclosure).Robustness is, therefore, a key property in REST services.In this dissertation, we present a systematic literature review on software robustness testing, and an approach and tool (named bBOXRT) for performing robustness tests over REST services.We begin by performing a comprehensive analysis, in the form of a systematic literature review, of the state of the art on software robustness testing, which we complement with an overview of the related work on REST API testing.This allows us to show that multiple techniques and tools for robustness assessment have been thoroughly studied and applied to a large diversity of domains, but REST services still lack practical approaches that specialize in robustness evaluation.We fill-in this gap by proposing bBOXRT, a tool for testing the robustness of REST services solely based on minimal information present in their interface descriptions.We used bBOXRT to evaluate an heterogeneous set of 52 REST services that comprise 1351 operations and fit in distinct categories (e.g., public, private, in-house).We were able to disclose several different types of robustness problems, including issues in services with strong reliability requirements and also a few security vulnerabilities.The results show that REST services are being deployed online holding software defects that harm service integration, and also carrying security vulnerabilities that can be exploited by malicious users.