Design and implementation of public opinion analysis system for network based on MapReduce
Huang We · Information technology newsletter · 2014
This paper designed a network public opinion analysis system based on MapReduce. Dual storage mechanism composed of HDFS and HBase used for storing data. MMSeg4j was selected for Chinese word segmentation by comparing the experimental results and word segmentation efficiency. In order to improve the accuracy and efficiency of the clusters,the Canopy-Kmeans algorithm was improved.Currently,the system has been applied to a public opinion analysis system in an army,the system can detect hot topics in real time and grasp the trend of public opinion accurately. It offered scientific and systematic support for dealing with public opinion crises and formulating public policy.