Designing Real-Time Web Mining and Analyzing System
Jong-Hwa Lee, Hyunkyu Lee · The Journal of Internet Electronic Commerce Resarch · 2018
Regardless of the population growth trend, the number of Internet users is steadily increasing due to the development of networks. And the company’s products and services evolve in the IOT environment. Due to these two factors, the speed and amount of data generated through the Internet is increasing exponentially. In addition, data generated through communication between objects is used as a material that can be continuously developed by human beings and is used as a source of ideas for innovative products and services. The big data industry is at the center of this development. Therefore, this study aims to develop a system capable of processing all the processes related to big data analysis in real time by designing a distributed multi-processing system for collecting and analyzing such huge data. In order to utilize various media data, multi - channel data collection, first refined keyword centered DB center, and text mining technique using processed data were processed in real time. In other words, it is proved that corporate decision making and consumer needs analysis can be done in real time by collecting, storing and analyzing data at the same time.