Efficient Visualization of Large-Scale Metal Melt Flow Simulations Using Lossy In-Situ Tabular Encoding for Query-Driven Analytics
Henry Lehmann, Eric Werzner, Cornelius Demuth, Subhashis Ray, Bernhard Jung · 2018
Numerical simulations carried out in powerful High-Perfomance-Computing (HPC) environments are becoming increasingly important for the design of improved filters for metal melts. However, the massive amount of data generated by such simulations impose challenges on data management and analysis. A particularly limiting factor is file system access, i.e. the so-called I/O bottleneck, that affects both data storage in the HPC environment and, more frequently and arguably more critically, the loading of data for analysis and visualization purposes in less powerful local workstations or visualization clusters. This article introduces LITE-QA, a method for reducing the amount of data in large-scale scientific simulations. During a simulation in the HPC environment, it supports both in-situ data compression and additional data indexing in an integrated fashion. During the analysis phase, it supports efficient query-based data retrieval where only data of interest to the user is loaded from the file system. The proposed approach is evaluated in a simulation of metal melt filtration using the lattice-Boltzmann method. As compared to conventional data storage methods, the amounts of data generated by the simulation are significantly reduced even including the additional indices. For exemplary visualization tasks, the amounts of data to be read from the file system are reduced to ~1.8-23.9% of the original data size, while yielding an overall speed-up of loading times by ~4.9-16.5×.