Cooperating with high available write cache and local read only buffer to reduces the latency of small read and write
Hu Xu, Xiaopeng Liu · 2018
Distributed file system is the basic platform for big data analysis. For VM, database, log generation and other file processing, the traditional distributed file system for big data analysis cannot meet the IOPS requirements. Flash storage is becoming more common and costs are slowly coming down but it is still not practical to store a large quantity of your data on flash. Making the best use of flash storage requires automation. The storage needs to be smart enough to handle the movement of data between flash and slower storage so applications don't need to understand the underlying architecture. In addition to smart storage sometimes it makes the most sense to allow an application to provide hints or direction to the storage so data can be place most efficiently. This paper presents a method for constructing a set of highly available write cache and Local read-only buffer using SSD. By configuring recovery log and meta-data separation management, the method collaborates with the cache and reduces the read-write delay of massive small files to meet the requirement of high IOPS application.