Affordable supercomputing using open source software
D. Trejo, Iyad Obeid, Joseph Picone · 2015
Big data and machine learning require powerful centralized computing systems. Small research groups cannot afford to support large, expensive computing infrastructure. Cloud computing options, such as renting cycles from Amazon AWS, can often end up costing more than hosting hardware locally, and pose challenges when attempting to move big data resources across the network (or staging them remotely on the server). Open source projects are enabling the development of low cost scalable clusters and are significantly lowering the barrier for administrating and maintaining these clusters. In this poster, we explore the tradeoffs a small research group faces in constructing a cost-effective cluster. We present an affordable approach to cluster computing that uses commodity processors and open source software. Though the overall system is not novel, we believe the lessons learned in this project can be a valuable guide for small research groups interested in building such clusters.