Progressive Retrieval and Hierarchical Visualization of Large Remote Data
Hans‐Christian Hege, Andrei Hutanu, Ralf Kähler, André Merzky, Thomas Radke, Edward Seidel, Brygg Ullmer · 2001
The size of data sets produ ed on remote super omputer fa ilities frequently ex eeds the pro essing apabilities of lo al visualization workstations. This phenomenon in reasingly limits s ientists when analyzing results of large-s ale s ienti simulations. That problem gets even more prominent in s ienti ollaborations, spanning large virtual organizations, working on ommon shared sets of data distributed in Grid environments. In the visualization ommunity, this problem is addressed by distributing the visualization pipeline. In parti ular, early stages of the pipeline are exe uted on resour es loser to the initial (remote) lo ations of the data sets. This paper presents an e ient te hnique for pla ing the rst two stages of the visualization pipeline (data a ess and data lter) onto remote resour es. This is realized by exploiting the extended retrieve feature of GridFTP for exible, high performan e a ess to very large HDF5 les. We redu e the number of network transa tions for ltering operations by utilizing a server side data pro essing plugin, and hen e redu e laten y overhead ompared to GridFTP partial le a ess. The paper further des ribes the appli ation of hierar hi al rendering te hniques on remote uniform data sets, whi h make use of the remote data ltering stage. 1. Introdu tion. The amount of data produ ed by numeri al simulations on super omputing fa ilities ontinues to in rease rapidly in parallel with the in reasing ompute power, main memory, storage spa e, and I/O transfer rates available to resear hers. These developments in super omputing have been observed to ex eed the growth of ommodity network bandwith and visualization workstation memory/performan e by a fa tor of 4 [11℄. Hen e, it is in reasingly riti al to use remote data a ess te hniques for analyzing this data. Among