Global Expansion of Apache Hadoop/Apache Spark Activities at NTT DATA
Ravindra Sandaruwan Ranaweera, Akira Ajisaka · NTT technical review · 2018
SparkAn increasing number of companies are attempting to create new businesses or expand existing businesses based on the results of analyzing huge amounts of various types of data known as big data.Apache Hadoop was introduced in 2006 and enables massive amounts of diverse data to be stored, processed, and analyzed in a realistic period of time at a reasonable cost.Apache Spark, meanwhile, was developed by postgraduate students at the University of California, Berkeley to efficiently perform repetitive and complex processing of big data, which Hadoop is not good at doing.At present, Spark is becoming increasingly popular thanks to a number of key features.For example, it supports programming languages such as Python and R widely used in the data analysis industry, incorporates a machine-learning library, and performs various types of optimization. Hadoop/Spark activities at NTT DATANTT DATA began using Hadoop in 2008 before Hadoop began attracting attention worldwide.At that time, Hadoop was still immature, and there were no functions satisfying the strict requirements pertaining to availability and operability demanded by enterprise customers.The NTT DATA team working on Hadoop, however, felt that Hadoop could bring significant benefits to enterprise customers.With this in mind, the team conducted tests on applying NTT DATA's know-how in system integration in order to meet such strict availability/operability requirements, and published that know-how in the form of a test report released in 2010 [1].This report helped to make NTT DATA's work on Hadoop well known, and since then, NTT DATA has gone on to construct and provide systems using Hadoop/Spark in all sorts of industries including telecommunications, real estate, public administration, finance, media, and manufacturing.NTT DATA's involvement is not limited to just a portion of system development, but rather, it provides a wide range of services covering the entire system development process including planning, design, development, and support to provide its customers with systems that can lead to new business opportunities (Fig. 1).Providing versatile services in this way enables NTT DATA to satisfy the genuine needs of its customers.Furthermore, to enhance services and raise the level of customer satisfaction, NTT DATA proactively Global Expansion of Apache Hadoop/ Apache Spark Activities at NTT DATA