Proceedings of the 2015 Joint International Mechanical, Electronic and Information Technology Conference

Research on the reliability of network traffic data collection based on Hadoop

Authors
Zong Feng
Corresponding Author
Zong Feng
Available Online December 2015.
DOI
10.2991/jimet-15.2015.85How to use a DOI?
Keywords
Hadoop; Big data; network traffic; data collection; reliability transmission
Abstract

In this paper, the problem of how to improve the integrity and reliability of the data acquisition system based on Hadoop is introduced, and the distributed network fault detection technology is introduced; And based on this technology, a distributed node monitoring framework (DNMF) is designed, which is suitable for the data acquisition system ,the node fault detection and processing algorithm and node load balancing algorithm are proposed in this framework; At the same time, the load of DNMF is used to balance the load, and the load of some nodes is prevented.

Copyright
© 2015, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

Volume Title
Proceedings of the 2015 Joint International Mechanical, Electronic and Information Technology Conference
Series
Advances in Computer Science Research
Publication Date
December 2015
ISBN
10.2991/jimet-15.2015.85
ISSN
2352-538X
DOI
10.2991/jimet-15.2015.85How to use a DOI?
Copyright
© 2015, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - CONF
AU  - Zong Feng
PY  - 2015/12
DA  - 2015/12
TI  - Research on the reliability of network traffic data collection based on Hadoop
BT  - Proceedings of the 2015 Joint International Mechanical, Electronic and Information Technology Conference
PB  - Atlantis Press
SP  - 454
EP  - 457
SN  - 2352-538X
UR  - https://doi.org/10.2991/jimet-15.2015.85
DO  - 10.2991/jimet-15.2015.85
ID  - Feng2015/12
ER  -