Research on Performance Optimization for Power Big Data Storage based on HBase
Li Peng Zhu, Xiang Cai, Yuanxin Le · Journal of Physics Conference Series · 2021
With the development of smart grid, the online monitoring of power equipment has become the focus. Due to the increasing of power data, data storage in power grid is faced with great challenge. HBase is widely used for storing and managing power data due to its excellent performance in distributed access to data. However, HBase is prone to write a hot spot problem when storing data. To address this problem, this paper proposes an optimization method for data storage performance based on a dynamic load balancing strategy, which is implemented by pre-partitioning processing, RowKey substitution, and WLC (Weighted Least Connection) algorithm adapted to HBase storage mechanism. Through the experimental analysis of insulator current data storage, the results show that the method improves the performance of HBase cluster parallel data storage.