Reliability techniques for MPP SQL database product engineering
K. T. Sridhar · 2017
Reliability of software is an area of concern for researchers, practitioners and industry. In a world that collects Big Data for informed decision making through data mining, and to monetize data using mined information, tools that deal with Big Data need to pay special attention to the gray area of software reliability. This paper focuses on software techniques for improving reliability of a parallel, MPP SQL product to handle large data volumes on cloud and on-premise platforms. We define a framework, identify and classify faults in it and propose degrees of fault tolerance mapping them to product termination levels. Paying special attention to exception handling and resource clean-up, we specify an exception termination handler with coordinated termination, fault prediction by run-time monitoring, fault level elevation, messages and interrupt processing using Linux system calls. Domain specific reliability issues and their fault tolerant solutions are also discussed for bulk loading of data and protection of data under faults.