A Large Scale File Processing Application on a Hypercube
Chuck H. Baldwin, W.C. Nestlerode · 2005
With the introduction of disk subsystems on hypercube multiprocessors researchers have begun to study how applications which have large data requirements, either as input or output, can efficiently use extra storage capacity. We have been exploring the performance of the NCUBE/IO and an attached disk subsystem which provides 6.4 gigabytes of storage on several applications. An application which we have recently developed for use with the “disk farm” is a census data analysis program that performs statistical analysis on the 1970 census data, approximately 233 megabytes of data. We have achieved traditional speedups of over 200 on 512 nodes, with a total host execution time of under 15 seconds. In this paper we will outline the procedure used to achieve this speedup and present a technical analysis of the disk subsystem on this particular application, as well as some summary comments on the uses for such the disk subsystem.