APEL: An implementation of Grid accounting using R-GMA
Rob Byroma, Roney Cordenonsi, Linda Cornwalla, Martin Craiga, Abdeslem Djaouia, Alastair Duncana, Steve Fishera, John Gordona, Steve Hicksa, Dave Kanta, J. Leake, Robin Middletona, Matt Thorpea, John Walka, Antony Wilsona · 2005
This article describes the implementation of an accounting tool in the LHC Computing Grid (LCG): a distributed computing grid project consisting of over 100 resource centres and more than 10,000 CPUs. APEL (Accounting Processor for Event Logs) parses batch, system and gatekeeper logs generated by a site and builds accounting records, which provide a summary of the resources consumed based on attributes such as CPU time, Wall Clock Time, Memory and grid user DN. The accounting data is published into the R-GMA information and monitoring system, and archived for processing by a graphical front-end utilised by the accounting web tool. 1. LCG Accounting Overview In the LCG [1] grid environment, the computing resources, the application data and the grid users belonging to Virtual Organisations (VO) are distributed. Jobs submitted by these users may be sent to computing resources where the data is stored locally, or may go to remote resources where there are available job slots to reduce queue times. Jobs that run on LCG resources must be properly accounted for, so that the resources consumed by VOs and the resources provided by sites (resource centres), as a function of time can be determined. In this paper we distinguish between two flavours of grid accounting; “Grid Job accounting” in which a job usage record provides a complete description of resource consumption and, “Real-time Grid accounting”, which is based on an incremental determination of resource value while the job is being executed. In the former, the cost of computing is determined on the basis of what was done (after job execution), whilst in the latter, data feeds into a pricing model on which a cost for consumption is determined in real time. An implementation of real-time accounting (DGAS [2]) is under development within the gLite [3] middleware framework of the EGEE [4] grid-computing project. Below we summarise the implementation of a web-based job accounting tool that collects usage records from distributed resources using R-GMA [5] to a central point, and consolidates the information for presentation on the web. 2. Data Collection Mechanism and