Management of Physics Simulations with Databases
Viktoria Maike Paula Pauw, David Číž, Vojtěch Mlýnský, Pavel Banáš, Michal Otyepka, Stephan Hachinger, Jan Martinovič · 2024
The on-going work presented in this article explores different technical approaches and systems for management and analysis of data obtained from large physics simulations, optimising the respective data-driven workflows across Cloud-Computing (IaaS) and HPC systems. The work is carried out in the context of the EXA4MIND Horizon Europe project, which produces an Extreme Data processing platform, bringing together specialised data management systems and powerful computing infrastructures. We evaluate two typical use cases with physics simulations carried out on supercomputing systems at LRZ (Garching b.M./DE) and IT4Innovations (Ostrava/CZ). These use cases come from different areas of physics – they focus on the treatment of low energy many-body systems of molecules, and of high-energy (relativistic) elementary particles, respectively. Accordingly, molecular dynamics (MD, low energy) and plasma simulation methods (FDTD, Particle-in-Cell, high energy) are used. As often in computationally supported, data-driven science, a large fraction of the work then goes into postprocessing, visualising and re-assessing the data, often several times in an iterative process. A well-managed, integrated and efficient “next-generation” methodology to facilitate and manage such a (re-)use of the valuable data – in particular in the context of parameter studies – with an eye on FAIR research data management, is one of our final objectives.