Research on the reliability of network traffic data collection based on Hadoop

DOI

10.2991/jimet-15.2015.85How to use a DOI?

Keywords

Hadoop; Big data; network traffic; data collection; reliability transmission

Abstract

In this paper, the problem of how to improve the integrity and reliability of the data acquisition system based on Hadoop is introduced, and the distributed network fault detection technology is introduced; And based on this technology, a distributed node monitoring framework (DNMF) is designed, which is suitable for the data acquisition system ,the node fault detection and processing algorithm and node load balancing algorithm are proposed in this framework; At the same time, the load of DNMF is used to balance the load, and the load of some nodes is prevented.

Copyright

© 2015, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - Zong Feng
PY  - 2015/12
DA  - 2015/12
TI  - Research on the reliability of network traffic data collection based on Hadoop
BT  - Proceedings of the 2015 Joint International Mechanical, Electronic and Information Technology Conference
PB  - Atlantis Press
SP  - 454
EP  - 457
SN  - 2352-538X
UR  - https://doi.org/10.2991/jimet-15.2015.85
DO  - 10.2991/jimet-15.2015.85
ID  - Feng2015/12
ER  -