Proceedings of the 2016 4th International Conference on Electrical & Electronics Engineering and Computer Science (ICEEECS 2016)

Parallel computation method for incremental cloud services based on Hadoop

Authors
Yonghong Li, Na Zhou, Guofeng Zhao
Corresponding Author
Yonghong Li
Available Online December 2016.
DOI
10.2991/iceeecs-16.2016.220How to use a DOI?
Keywords
Parallel computation; incremental cloud services; Hadoop
Abstract

Performance of Hadoop platform is closely related to the task scheduler. Based on the analysis of the existing Hadoop platform's job scheduling algorithm, I propose a multi-queue job scheduling optimization algorithm in the paper. Based on the actual test and analysis of Hadoop platform, it can be seen that the optimization algorithm proposed in this paper can effectively allocate the node resources according to the degree of demand for the node resource. It can achieve the resource sharing among multiple queues. And at the same time, it can also effectively avoid the ping-pong effect brought by resource competition at the same time. Optimization algorithm has greater improvements in execution efficiency than Hadoop default algorithm.

Copyright
© 2016, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

Volume Title
Proceedings of the 2016 4th International Conference on Electrical & Electronics Engineering and Computer Science (ICEEECS 2016)
Series
Advances in Computer Science Research
Publication Date
December 2016
ISBN
10.2991/iceeecs-16.2016.220
ISSN
2352-538X
DOI
10.2991/iceeecs-16.2016.220How to use a DOI?
Copyright
© 2016, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - CONF
AU  - Yonghong Li
AU  - Na Zhou
AU  - Guofeng Zhao
PY  - 2016/12
DA  - 2016/12
TI  - Parallel computation method for incremental cloud services based on Hadoop
BT  - Proceedings of the 2016 4th International Conference on Electrical & Electronics Engineering and Computer Science (ICEEECS 2016)
PB  - Atlantis Press
SP  - 1146
EP  - 1149
SN  - 2352-538X
UR  - https://doi.org/10.2991/iceeecs-16.2016.220
DO  - 10.2991/iceeecs-16.2016.220
ID  - Li2016/12
ER  -