Datamation 2001: A Sorting Odyssey
Florentina I. Popovici, John Bent, Brian C. Forney, Andrea Carol Arpaci-Dusseau, Remzi H. Arpaci-Dusseau · Minds at UW (University of Wisconsin) · 2001
We present our experience of turning a Linux cluster into a high-performance parallel sorting system. Our implementation, WIND-SORT, broke the Datamation record by roughly a factor of two, sorting 1 million 100-byte records in 0.48 seconds. We have identied three keys to our success: developing a fast remote execution service, conguring the cluster properly, and avoiding the potential ill-effects of occasionally faulty hardware.