PDIO: High-Performance Remote File I/O for Portals-Enabled Compute Nodes.
N. T. B. Stone, Douglas A. Balog, Bryon Gill, Brian Johanson, Jim Marsteller, Paul Nowoczynski, David H. Porter, Raghurama Reddy, JAMES R. SCOTT, Derek Simmel, Jason Sommerfield, Katie Vargo, Chad Vizino · 2006
middleware infrastructure for writing data from compute processor memory on Portals-enabled compute nodes to remote agents anywhere on the WAN in realtime. The prototype implementation provided a means for aggregation of outgoing data through multiple loadbalanced routing daemons, end-to-end parallel data streams through externally connected “I/O nodes”, and a bandwidth feedback mechanism for stability and robustness. It was used by one research group, demonstrated live at several conferences, and shown to deliver bandwidths of up to 800 Mbit/sec. Although the prototype met the initial design requirements for the target application, it had some limitations due to the special-purpose nature of that design. Based on experiences with that implementation, the beta version now under development has a number of interface, functionality and performance enhancements. We present the motivations for this infrastructure and the revisions that should make it a general purpose solution for users on PSC’s Cray XT3 and other compute platforms. Index Terms—computer input-output, data communication, data handling