Data Profiling and Scrubbing
Francis Rodrigues, MICHAEL G. H. COLES, David Dye · Apress eBooks · 2012
Projects that require bringing together data from multiple sources—for example, data warehouse, data mart, or operational data store (ODS) projects—are extremely common. You could spend months gathering business requirements, putting together technical specifications, designing target databases, and coding and testing your ETL process. You could spend an eternity in “ad hoc maintenance mode” rewriting large sections of code that don’t handle unanticipated bad or nonconforming data. This scenario is the result of a failure to properly plan and execute data intergration projects—a phenomenon known as code, load, and explode . These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.