Opportunistic data locality for end user data analysis
Max Fischer, Christoph Heidecker, Eileen Kuehn, Gunter Quast, M. Giffels, M. Schnepf, A Heiss, Andreas Petzold · Journal of Physics Conference Series · 2017
With the increasing data volume of LHC Run2, user analyses are evolving towards increasing data throughput. This evolution translates to higher requirements for efficiency and scalability of the underlying analysis infrastructure. We approach this issue with a new middleware to optimise data access: a layer of coordinated caches transparently provides data locality for high-throughput analyses. We demonstrated the feasibility of this approach with a prototype used for analyses of the CMS working groups at KIT. In this paper, we present our experience both with the approach in general, and our prototype in specific.