Improving the quality of geotechnical datasets: A rules-based approach
Asitha Senanayake, X Peng, E Lewis · 2023
A key challenge as we progress through this age of abundant data is to ensure the quality of vast amounts of data being collected, stored, and shared. Though geotechnical engineering is not typically considered a data-intensive discipline, large amounts of geodata are frequently generated in site characterization programs for projects with large spatial footprints. Traditional quality assurance processes that rely on manual checking do not scale well to these large data sets as they become expensive, slow, and unreliable. Automated testing and validation of data using a rules-based approach is a better alternative. This paper presents two testing frameworks that leverage AGS4 and DIGGS data interchange formats together with opensource software libraries that have been developed around them. They allow custom data rules to be easily defined in either Python or Schematron. Results from such scripts are repeatable and reproducible, therefore can be shared between various parties in a project or published for general use. Increases in speed, efficiency, and reliability that this approach brings can produce significant economic benefits to both data producers and data consumers.