Distributed and High Performance Big-File Cloud Storage Based On Key-Value Store

DOI

10.2991/ijndc.2016.4.3.3How to use a DOI?

Abstract

This research proposes a new Big File Cloud (BFC) with its architecture and algorithms to solve difficult problems of cloud-based storage using the advantages of key-value stores. There are many problems when designing an efficient storage engine for cloud-based storage systems with strict requirements such as big-file processing, lightweight meta-data, low latency, parallel I/O, deduplication, distributed, high scalability. Keyvalue stores have many advantages and outperform traditional relational database in storing data for heavy load systems. This paper contributes a low-complicated, fixed-size meta-data design, which supports fast and highly-concurrent, distributed file I/O, several algorithms for resumable upload, download and simple data deduplication method for static data. This research applies the advantages of ZDB - an in-house key-value store which was optimized with auto-increment integer keys for solving big-file storage problems efficiently. The results can be used for building scalable distributed data cloud storage that support big-files with sizes up to several terabytes.

Copyright

© 2017, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - JOUR
AU  - Thanh Trung Nguyen
AU  - Minh Hieu Nguyen
PY  - 2016
DA  - 2016/07/01
TI  - Distributed and High Performance Big-File Cloud Storage Based On Key-Value Store
JO  - International Journal of Networked and Distributed Computing
SP  - 159
EP  - 172
VL  - 4
IS  - 3
SN  - 2211-7946
UR  - https://doi.org/10.2991/ijndc.2016.4.3.3
DO  - 10.2991/ijndc.2016.4.3.3
ID  - Nguyen2016
ER  -