Research of error data detection algorithm based on rules
Zhongbin Zhang, Yuhua Zhou, Yongzhi Liu · 2011
Data entry errors, improper integration, data environment changes, etc., will affect the quality of the data. Among them, the error data is the most serious data quality problems. To clean up the error data, to play the role of information systems and improve the quality of the data, the detection method of error data based on rules is studied, the detection process is analyzed, a common set of detection rules is established, how the SQL statements into the rules is discussed, the detection algorithm is achieved and carried out a series of optimization. This method is easy, its rules are simple, and the efficiency and the false discovery rate are high after optimization. Therefore, this approach may well be a good method of data cleaning.