Data-intensive analysis for scientific experiments at the Large Scale Data Facility
A.O. Garcia, Serguei Bourov, A. Hammad, Volker Hartmann, Thomas Jejkal, Jens C. Otte, Sebastian Pfeiffer, T. Schenker, Christian Schmidt, Patrick Neuberger, Rainer Stotzka, Jos van Wezel, Bernhard Neumair, Achim Streit · 2011
The Large Scale Data Facility (LSDF) was conceived and launched at the Karlsruhe Institute of Technology (KIT) end of 2009 to address the growing need for value-added storage services for data intensive experiments. The LSDF main focus is to support scientific experiments producing large data sets reaching into the petabyte range with adequate storage, support and value added services for data management, processing and preservation. In this work we describe the approach taken to perform data analysis in LSDF, as well as for data management of the scientific datasets.