Exploiting FastDFS Client-based Small File Merging
- 10.2991/aiea-16.2016.40How to use a DOI?
- Masses of small files; small file merging; Client-based.
How to store masses of small files is a generally acknowledged problem in industry and academia. Small file merging is the most successful strategy to solve this problem, which has been supported by many distributed filesystems, such as FastDFS. However, in FastDFS, our experiments indicate that excessively wide striping causes performance degradation, and pre-allocated space causes data loss or error. Therefore, this paper presents a client-based small file merging scheme based on the characteristics of online web application. This scheme makes full use of the resources of the web server and optimizes the small file merging with the annealing algorithm to improve the throughput. Experimental results show that the write performance are improved by 52.03% and query performance by 27.69% compared with the original FastDFS small file merging scheme.
- © 2016, the Authors. Published by Atlantis Press.
- Open Access
- This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Cite this article
TY - CONF AU - Haimeng Chen AU - Hua Zhang PY - 2016/11 DA - 2016/11 TI - Exploiting FastDFS Client-based Small File Merging BT - Proceedings of the 2016 International Conference on Artificial Intelligence and Engineering Applications PB - Atlantis Press SP - 213 EP - 217 SN - 2352-538X UR - https://doi.org/10.2991/aiea-16.2016.40 DO - 10.2991/aiea-16.2016.40 ID - Chen2016/11 ER -