Design and implementation of a lightweight ETL system for power real-time data
Duan Cheng, Zengping Wang · Power System Protection and Control · 2010
Using the data center construction project for a power real-time monitoring system as the background,this paper proposes a novel lightweight ETL solution for the migration and synchronization of power real-time data.The solution not only supports a variety of relational database and text-based data sources,but also provides total,incremental,main-sub table and several other custom mode data migration schemes.By combining JDBC database access technology,Util.Concurrent package of JSR-166 and SWT technology with the management of metadata,our system solves the problems of heterogeneous data extraction,cleaning, transformation and loading,task scheduling and cross-platform operation.Practical applications demonstrate the practicality and stability of this system.