Dimensionality Reduction of Massive I/O Log Data Flow in Power System
Bo Zhang, Zesheng Xi, Tao Zhang, Yuanyuan Ma, Zhipeng Shao, Hongfa Li · 2020
Every day, the power system receives massive I/O logs. The amount of data in these logs is so large that it takes huge computational resources to analyze. Therefore, it is necessary to reduce the size of the massive I/O logs and only analyze the key log data, thereby reducing the workload of invalid analysis. This paper takes the I/O log of the substation as the research object, and studies the dimension reduction method of the massive I/O data flow log, which reduces the computational load brought by the high-dimensional I/O data flow log data and reduces the massive I/O data flow log. This paper proposes a method of secondary dimensionality reduction. Firstly, the high dimensional I/O log data stream is classified, so that the data is transformed from high-dimensional to low-dimensional. Then, the dimension is reduced again in each category, so that the most simplified massive I/O logs are achieved. Through theoretical analysis, we can come to the conclusion that the computational time complexity of the data after dimension reduction is reduced by more than 80%.