Measuring the Compressibility of Metadata and Small Files for Disk/NVRAM Hybrid Storage Systems
Nathan K. Edel, Ethan L. Miller, Karl S. Brandt · 2004
File systems combining disk storage with non-volatile RAM (NVRAM) promise large improvements in file system performance. However, current technology allows for a relatively limited amount of NVRAM, limiting the effectiveness of such an approach. We are examining in-memory compression techniques that allow for significantly more efficient utilization of this limited resource. We focus on small objects- metadata and small files- and we have measured the compressibility of these objects for a set of representative file systems. Our results show that inodes are compressible by at least 76–90 % at a rate of 270–900 thousand inodes per second for the best algorithms. For files in the range of 4– 128 KB, we achieved an average compressibility of 40–60% at rates of 20–40 megabytes per second. Based on these measurements, we believe that compression of both metadata and small files should be included in any disk/NVRAM hybrid file system. 1