Setting up an experimental high performance computing cluster
Brian M. Henehan · Journal of computing sciences in colleges · 2013
High Performance Computing (HPC) clusters have entered mainstream computing. While government and academic natural sciences (and Hedge funds) still dominate the space, every academic has the opportunity to join one or more consortiums that promise access to HPC clusters with thousands of available computing cores. The move to parallelism is ongoing and the ability to scale-up beyond the 2-8 cores available within a typical processor package is rapidly developing from an esoteric niche skill to a mainstream business skill. But it can be an intimidating experience to take your first on a giant NSF-funded HPC cluster. This tutorial is geared toward getting a small-scale HPC cluster designed and installed so that the baby steps of the HPC learning process can be performed away from mega-scale operations. Utilizing 4 64-bit computers available for less than $160 each (our University surpluses them), we will create a small cluster using the software available from a Dreamspark Premium (formerly known as MSDNAA) subscription in about 75 minutes. While the installation operations are ongoing, we will discuss basic HPC architecture and the architectural design decisions that must be made prior to an installation. After installation, we will take a tour through the administrative interface, and then install a benchmarking software suite. In addition, the presenter will share lessoned learned from his own HPC installations to head off potential frustrations when you design and install with your own HPC.