Proceedings of the 2018 International Conference on Network, Communication, Computer Engineering (NCCE 2018)

Physical Computing Resource Scheduling on Distributed Stream Processing System

Authors
Kailin Tang
Corresponding Author
Kailin Tang
Available Online May 2018.
DOI
10.2991/ncce-18.2018.62How to use a DOI?
Keywords
real-time data; workload; system-tier; stream processing.; elastic scheduling.
Abstract

Recently, distributed stream computing has emerged as a popular model for real-time data stream analytics. This paper focuses on computing resources scheduling of distributed stream processing. To adapt to the fluctuation of data stream, distributed stream processing system must elastically provision the computing resources based on the observed workload. However, there is no system that satisfies all these requirements. Motivated by this, this paper presents a physical resource scheduling algorithm for elastic scheduling computing resources of stream processing task. Base on existing task tier elastic resource scheduling and system-tier resource scheduling, this paper proposes a collaborative algorithm. According to the fluctuant workload, task-tier dynamic resource scheduler config computing resource of stream processing task. Meanwhile, based on the scheduling decision of task-tier scheduler, the system-tier resource scheduler is assign physical computing resources to stream processing tasks.

Copyright
© 2018, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

Volume Title
Proceedings of the 2018 International Conference on Network, Communication, Computer Engineering (NCCE 2018)
Series
Advances in Intelligent Systems Research
Publication Date
May 2018
ISBN
978-94-6252-517-7
ISSN
1951-6851
DOI
10.2991/ncce-18.2018.62How to use a DOI?
Copyright
© 2018, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - CONF
AU  - Kailin Tang
PY  - 2018/05
DA  - 2018/05
TI  - Physical Computing Resource Scheduling on Distributed Stream Processing System
BT  - Proceedings of the 2018 International Conference on Network, Communication, Computer Engineering (NCCE 2018)
PB  - Atlantis Press
SP  - 385
EP  - 388
SN  - 1951-6851
UR  - https://doi.org/10.2991/ncce-18.2018.62
DO  - 10.2991/ncce-18.2018.62
ID  - Tang2018/05
ER  -