Cluster versus grid for large-volume hyperspectral image preprocessing
Jason Brazile, Michael E. Schaepman, Daniel R. Schlapfer, Johannes W. Kaiser, Jens Nieke, K.I. Itten · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 2004
The handling of satellite or airborne earth observation data for scientific applications minimally requires pre-processing to convert raw digital numbers into scientific units. However depending on sensor characteristics and architecture, additional work may be needed to achieve spatial and/or spectral uniformity. Standard higher level processing also typically involves providing orthorectification and atmospheric correction. Fortunately some of the computational tasks required to perform radiometric and geometric calibration can be decomposed into highly independent subtasks making this processing highly parallelizable. Such "embarrassingly parallel" problems provide the luxury of being able to choose between cluster or grid based solutions to perform these functions. Perhaps the most convenient solutions are grid-based, since most research groups making these kinds of measurements are likely to have access to a LAN whose spare computing resources could be non-obtrusively employed in a grid. However, since many higher level scientific applications of earth observation data might be composed of more highly interdependent subtasks, the parallel computing resources allocated for these tasks might also be made available for low level pre-processing as well. We look at two modules developed for our prototype data calibration processor for APEX, an airborne imaging spectrometer, which have been implemented on both a cluster and a grid leading us to be able to make observations and comparisons of the two approaches.