Exploiting FastDFS Client-based Small File Merging
Haimeng Chen, Hua Zhang · 2016
How to store masses of small files is a generally acknowledged problem in industry and academia.Small file merging is the most successful strategy to solve this problem, which has been supported by many distributed filesystems, such as FastDFS.However, in FastDFS, our experiments indicate that excessively wide striping causes performance degradation, and pre-allocated space causes data loss or error.Therefore, this paper presents a client-based small file merging scheme based on the characteristics of online web application.This scheme makes full use of the resources of the web server and optimizes the small file merging with the annealing algorithm to improve the throughput.Experimental results show that the write performance are improved by 52.03% and query performance by 27.69% compared with the original FastDFS small file merging scheme.