In-situ multi-resolution and temporal data compression for visual exploration of large-scale scientific simulations
Henry Lehmann, Bernhard Jung · 2014
Today's large-scale scientific simulations generate massive data sets that pose challenges both for data storage in HPC environments during the simulation phase and the subsequent data analysis phase. A promising approach for reducing the amount of data written out during simulation run is in-situ compression. However, even the compressed data sets are typically still too large for interactive visual data exploration which calls for multi-resolution data layouts. The recently proposed ISABELA method for lossy in-situ compression was shown to outperform other compression methods for scientific data sets. In this paper, we propose two main extensions to the ISABELA method: (1) an interlaced data layout that supports decompression of multi-resolution views of the data without overhead in the compressed format; (2) a new temporal compression scheme for improving the compression rate by exploiting temporal coherence in the data set. The compressed multi-resolution data can easily be transformed to the VTK AMR (adaptive multi-resolution) data format to support interactive exploration in ParaView and other visualization tools based on VTK. During the simulation phase, there is no significant increase of computational demands for the generation of complete multi-resolution compressed data sets as compared to flat ISABELA compression. During the analysis phase, due to the AMR data layout, our method supports selective loading of regions of interests as well as progressive loading of data sets, thus enabling interactive visualizations of large-scale scientific simulations.