Research On The Bid Data Technique In Electric Power Industry
Chunying Wang, Wen‐Cui Li, Wu Lijie, Yi Yang, Xinjian Shu, Yong Zhang · 2016
With the rapid development of our country electric power industry, as well as the construction of the smart grid, the electric power industry are increasingly demanding data management and analysis, etc；The big data technology is introduced into the electric power industry applications, not only for the electric power enterprise data information analysis and management to provide the necessary technical support, also can promote the electric power enterprises to further improve the service level.This article mainly expounds the technology of data, and the practical application of electric power industry are further analyzed. Big Data Research Basic Theory and Technical RouteBig data is not only refers to the vast amounts of information, more emphasis on human screening and processing of information.Big data processing method has a lot of, big data processing procedure of general application, can be summarized as four steps, are gathering, import and pretreatment, statistics and analysis, finally the data mining.Gathering.Big data acquisition refers to the use of multiple databases to receive from the client (Web, App or sensor forms, etc.), and the user can through these database for simple queries and processing work.For example, electric chamber of commerce use the traditional relational database MySQL and Oracle to store every transaction data, in addition, NoSQL database such as Redis and MongoDB is often used for data collection.In large data collection process, its main characteristic and challenge is high concurrency, because at the same time may have tens of thousands of users to access and manipulate, such as train ticketing website and Taobao, they reached millions of concurrent traffic during peak, so need to deploy a large number of database on the acquisition end to support.And how in between these database load balancing and subdivision is the need for in-depth thinking and design.Import/pretreatment. Collection side, although there will be a lot of the database itself, but to the analysis of these huge amounts of data efficiently, or should the data import from front end to a centralized large distributed database, or distributed storage cluster, and can be based on the import do some simple cleaning and pretreatment.There are some users will use when importing data from Twitter Storm to flow calculation, to meet the demand of real-time computing in the business.Import and challenges to the characteristics of the pretreatment process is mainly import large amount of data, often correlates to the import amount of every second, even gigabit levels.