ATLAS Eventlndex monitoring system using the Kibana analytics and visualization platform
Dario Barberis, S.E. Cárdenas Zárate, A. Favareto, Álvaro Fernández Casaní, E. J. Gallas, Carlos García Montoro, S. González de la Hoz, J. Hřivnáč, D. Malon, F. Prokoshin, J. Salt, Javier Sanchez, Rainer Töebbicke, R. Yuan · Journal of Physics Conference Series · 2016
The ATLAS EventIndex is a data catalogue system that stores event-related metadata for all (real and simulated) ATLAS events, on all processing stages. As it consists of different components that depend on other applications (such as distributed storage, and different sources of information) we need to monitor the conditions of many heterogeneous subsystems, to make sure everything is working correctly. This paper describes how we gather information about the EventIndex components and related subsystems: the Producer-Consumer architecture for data collection, health parameters from the servers that run EventIndex components, EventIndex web interface status, and the Hadoop infrastructure that stores EventIndex data. This information is collected, processed, and then displayed using CERN service monitoring software based on the Kibana analytic and visualization package, provided by CERN IT Department. EventIndex monitoring is used both by the EventIndex team and ATLAS Distributed Computing shifts crew.