Software Testing and Quality Assurance for Data Intensive Applications
Sonali Agarwal, Sanjay Kumar Sonbhadra, Narinder Singh Punn · 2022
Data intensive applications are one of the most critical real-time applications which are desired in most of the new-normal practices such as recommendation systems, social media analytics systems, fake news detection systems, etc. However, to deploy such solutions for real-time usage, software testing and quality assurance plays a vital role to understand the application behavior. The characteristic 4 Vs of big data adds complexities or challenges that need to be addressed for real-time applications or development. Testing of big data applications can be made efficient by designing and executing test plans; approach and strategy for all V’s of big data. This tutorial comprehensively covers both the theoretical and practical aspects of testing data-intensive applications. The tutorial discusses testing data-intensive applications built on top of modern big data frameworks such as Hadoop, Spark, Flink, NoSQL, Hive, Zookeeper, Elastic Search, Flume, and Kafka. The hands-on with MapReduce unit testing, Spark streaming testing, Kafka unit testing and related testing libraries has been covered with simple integration of testing examples and test case driven developments.