NFS Tricks and Benchmarking Traps
Daniel Ellard, Margo Seltzer · 2003
Abstract We describe two modifications to the FreeBSD 4.6 NFSserver to increase read throughput by improving the read-ahead heuristic to deal with reordered requests andstride access patterns. We show that for some stride access patterns, our new heuristics improve end-to-endNFS throughput by nearly a factor of two. We also show that benchmarking and experimenting with changes toan NFS server can be a subtle and challenging task, and that it is often difficult to distinguish the impact of a newalgorithm or heuristic from the quirks of the underlying software and hardware with which they interact. We dis-cuss these quirks and their potential effects. 1 Introduction Despite many innovations, file system performance issteadily losing ground relative to CPU, memory, and even network performance. This is due primarily to theimprovement rates of the underlying hardware. CPU speed and memory density typically double every 18months, while similar improvements in disk latency have taken the better part of a decade. Disks do keep pace interms of total storage capacity, and to a lesser extent in total bandwidth, but disk latency has become the primaryimpediment to total system performance. To avoid paying the full cost of disk latency, mod-ern file systems leverage the relatively high bandwidth of the disk to perform long sequential operations asyn-chronously and amortize the cost of these operations over the set of synchronous operations that would oth-erwise be necessary. For write operations, some techniques for doing this are log-structured file systems [18],journalling, and soft updates [21]. For reading, the primary mechanism is read-ahead or prefetching. Whenthe file system detects that a process is reading blocks from a file in a predictable pattern, it may optimisticallyread blocks that it anticipates will be requested soon. If the blocks are arranged sequentially on disk, then these"extra " reads can be performed relatively efficiently because the incremental cost of reading additional contiguous blocks is small. This technique can be beneficialeven when the disk blocks are not adjacent, as shown by Shriver et al. [23].