Containers in Research
Spencer Julian, Michael Shuey, Seth L. Cook · 2016
HPC environments have traditionally existed installed directly on hardware or through virtual machine environments. Linux Containers, and Docker specifically, have gained extensive popularity; we believe this current trend toward containers and microservices can be applied to HPC to improve efficiency and quality of development and deployment. User interest in Docker is rising, with several communities planning production deployments. We describe some of our site's experiences, along with an autoscaling web cluster and an autoscaling PBS-based computational cluster we have developed that are currently in a pre-production testing phase. Some basic performance tests are covered, comparing network and filesystem performance between a native Docker environment and a traditional Red Hat-based environment. In our tests, we noticed negligible differences in computational performance when run out of the box, approximately 0.4%, but we required some minor tweaking in the form of additional docker plugins to achieve similar or better performance in the network and filesystem tests. While additional testing is needed for some aspects of computational clusters, particularly RDMA performance, we believe initial testing indicates Docker containers are ready for broader adoption at larger-scale production environments.