A novel pattern classifier approach towards the performance optimization of Big Data analysis in distributed environment
Sandeep Kumar Hegde, Krishnarajanagar G. Srinivasa · 2017
Big data is a collection of massive amount of data whose characteristics exceeds the capabilities of conventional algorithm and techniques to derive the useful value. The real power lies not just in having colossal data but in what insights can be drawn from this data to facilitate better and faster decisions. Hadoop Map Reduce is emerged as powerful and cost effective open source framework for processing large data set distributed computing environment. The load imbalance, which occurs while processing the Big Data is common problem with this framework which reduces the efficiency of the processing, may impact the QOS of the application. In this paper a new pattern classifier algorithm has been proposed to tackle these problems. The proposed algorithm integrated with the traditional environment and experimentally it has been proved that response time of Map Reduce jobs are amplified by minimizing the load imbalance factor, thereby QOS requirement of the application is achieved.