Data Analysis Support in Karabo at European XFEL
Hans Fangohr, Steve Aplin, Anton Barty, Marijan Beg, Valerii Bondar, Djelloul Boukhelef, Sándor Brockhauser, Cyril Danilevski, Wajid Ehsan, Sergey Esenov, Gero Flucke, Gabriele Giovanetti, Dennis Goeries, Steffen Hauf, Burkhard C. Heisen, David Hickin, Dmitry Khakhulin, Anna Klimovskaia, Thomas A. Kluyver, Manuela Kuhn · ePrints Soton (University of Southampton) · 2018
We describe the data analysis structure that is integrated into the Karabo framework [1] to support scientific experiments and data analysis at European XFEL GmbH. The photon science experiments have a range of data analysis requirements, including online (i.e. near real-time during the actual measurement) and offline data analysis. The Karabo data analysis framework supports execution of automatic data analysis for routine tasks, supports complex experiment protocols including data analysis feedback integration to instrument control, and supports integration of external applications. The online data analysis is carried out using distributed and accelerator hardware (such as GPUs) where required to balance load and achieve near real-time data analysis throughput. Analysis routines provided by Karabo are implemented in C++ and Python, and make use of established scientific libraries. The XFEL control and analysis software team collaborates with users to integrate experiment specific analysis codes, protocols and requirements into this framework, and to make it available for the experiments and subsequent offline data analysis.