Big Data Storage and Analysis
Namrata Dhanda · 2022
With the recent advancements in information technology and the growing number of electronic devices, data is evolving at a very rapid rate. This humongous data needs to be efficiently managed and utilized. Hence, the primary concern is the storage, management, and exhaustive analysis of this enormous amount of data. Due to the increased volume of data, the term data is replaced by Big Data. Certain properties are possessed by Big Data like Volume, Variety, Velocity, Veracity, Value, and Variability. These six properties are also referred to as 6V's of Big Data. Now, unlike the simple relational database system, Big Data is in various formats as it is being generated and collected from different sources. It need not necessarily be structured in a tabular format. Most of the data that is generated these days is either semi-structured or unstructured. Hence, the traditional database systems are no longer sufficient to handle this Big Data. We need to adopt new methods to handle such varied forms of data. This chapter will give a brief introduction to the concepts of Big Data, its components, and the various forms of data that are generated by different devices. It also provides an introduction to the data processing of Big Data in a distributed fashion, which is done using Hadoop and MapReduce.