An extensible infrastructure for querying and mining event-level metadata in ATLAS
David M. Malon, J. Cranshaw, Qi Zhang · Journal of Physics Conference Series · 2012
The ATLAS event-level metadata infrastructure supports applications that range from data quality monitoring, anomaly detection, and fast physics monitoring to event-level selection and navigation to file-resident event data at any processing stage, from raw through analysis object data, in globally distributed analysis. A central component of the infrastructure is a distributed TAG database, which contains event-level metadata records for all ATLAS events, real and simulated. This resource offers a unique global view of ATLAS data, and provides an opportunity, not only for stream-style mining of event data, but also for an examination of data across streams, across runs, and across (re)processings. The TAG database serves as a natural locus for run-level and processing-level integrity checks, for investigations of event duplication and other issues in the trigger and offline systems, for questions about stream overlap, for queries about interesting but out-of-stream events, for statistics, and more. In early ATLAS running, such database queries were largely ad hoc , and were handled manually. In this paper, we describe an extensible infrastructure for addressing these and other use cases during upload and post-upload processing, and discuss some of the uses to which this infrastructure has been applied.