Design and Analysis of a Network Transfer Layer for Parallel File Systems
Philip Carns · 2001
This document describes the design and analysis of a network transfer layer for the Parallel Virtual File System (PVFS). PVFS [6] is a parallel le system developed at Clemson University for use on Linux based Beowulf [23, 2] clusters. It is intended to serve as a platform for high performance I/O research while also meeting the need for a production parallel le system for the scientific computing community. Over time, computational capability has improved so rapidly that there is now typically a large gap between I/O performance and processing power, even in the most powerful computer systems. This has led to a situation in which le system performance is the primary bottleneck for a variety of applications. PVFS seeks to provide a solution to this problem. Many components must be brought together in order to do this, including network communications, data storage, application or system interfaces, and scheduling. This document will focus on network communications. We intend to demonstrate and analyze a system for improving network efficiency and usability for the purpose of high performance I/O. This system will take advantage of lessons learned from current le system implementations, as well as advances in cluster communication technology, to help achieve the specialized goals of parallel le systems. Dedication To all of my family and friends who have supported me through the years.