Innovations from the early user phase on the Jetstream Research Cloud
Richard Knepper, Jeremy Fischer, Craig A. Stewart, David Y. Hancock, Matthew R. Link · Annals of Computer Science and Information Systems · 2016
We describe the Jetstream cyberinfrastructure for research, a purpose-built system with the goal of supporting "long-tail" research by providing a flexible infrastructure that can provide a set cloud services tuned for research applications, whether they be traditional HPC applications, science gateways, or desktop applications.Jetstream offers a library of virtual machines and allows the user to create their own virtual machines in order to provide an open cloud for science that allows both on-demand and persistent instances.The system is currently in early-user mode and a number of users at partner institutions are already creating and using images in the system.This paper details some of the early work being done with the system to create high performance clusters in an on-demand fashion to support scientific work directly as well as serve as capability backend to scientific gateways such as CyVerse and Galaxy.Within the cyberinfrastructure ecosystem, the Open Science Grid, the Extreme Science and Engineering Discovery Environment (XSEDE), and other projects provide a broad range of resources for computational support of research: high-performance computing at large scape, high-memory resources, big data, and visualization systems, but to date no system provides a highly configurable virtualized environment with the capabilities described above.The Jetstream system is designed to provide resources for the "long tail" of science [8], who frequently need more access to interactive computational resources than they have locally available, and provide interactive access to a handful of systems as needed in an on-demand fashion, rather than forcing them to work with an allocation system, as in XSEDE, or with a virtual organization, as in the Open Science Grid. B. Hardware ConfigurationThe Jetstream system hardware architecture follows commercial cloud offerings in terms of uptime and availability.Two production systems, one at IU and one at TACC, provide a 100Gbps-linked, distributed cluster infrastructure.A third development and testing system resides at the University of Arizona.This configuration offers "zoning" between the two production centers, similar to that offered in Amazon EC2