Research on the reliability of network traffic data collection based on Hadoop
- 10.2991/jimet-15.2015.85How to use a DOI?
- Hadoop; Big data; network traffic; data collection; reliability transmission
In this paper, the problem of how to improve the integrity and reliability of the data acquisition system based on Hadoop is introduced, and the distributed network fault detection technology is introduced; And based on this technology, a distributed node monitoring framework (DNMF) is designed, which is suitable for the data acquisition system ,the node fault detection and processing algorithm and node load balancing algorithm are proposed in this framework; At the same time, the load of DNMF is used to balance the load, and the load of some nodes is prevented.
- © 2015, the Authors. Published by Atlantis Press.
- Open Access
- This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Cite this article
TY - CONF AU - Zong Feng PY - 2015/12 DA - 2015/12 TI - Research on the reliability of network traffic data collection based on Hadoop BT - Proceedings of the 2015 Joint International Mechanical, Electronic and Information Technology Conference PB - Atlantis Press SP - 454 EP - 457 SN - 2352-538X UR - https://doi.org/10.2991/jimet-15.2015.85 DO - 10.2991/jimet-15.2015.85 ID - Feng2015/12 ER -