Don Quijote - Data Management for the ATLAS Automatic Production System
Branco, M · CERN Document Server (European Organization for Nuclear Research) · 2005
As part of the ATLAS Data Challenges 2 (DC2), an automatic production system was introduced along with a new data management component.Previously data management tools used for the Data Challenges were built as separate components from the Grid middleware.These tools relied on a database of its own which acted as a replica catalog.With the extensive use of Grid for the entire DC2 production it is no longer possible to have a data management tool that is designed to be independent of the Grid middleware.Each Grid relies on a replica catalog of its own along with Grid-specific data management software.ATLAS Data Challenges 2 has used uniformly the resources provided by three Grids: NorduGrid, US Grid3 and LCG-2.The solution we present was to build a data management proxy service -Don Quijote -which consists of a common high-level interface, whose implementation depends on each Grid's replica and metadata catalog as well as the storage backend.Don Quijote provides management of replicas in a services oriented architecture, across the several "flavours" of Grid middleware used by ATLAS.With a higher-level interface common across several Grids a user such as the new automatic production system can seamlessly manage replicas independently of their hosting environment.Given the services-based architecture a lightweight set of command line tools is available for users to interact uniformly with ATLAS filebased data.Users can therefore uniformly manage files within or across Grid flavours, e.g: moving files from LCG-2 to US Grid 3 while maintaining attributes such as the Global Unique Identifiers.