High Performance Data Processing Pipeline Of Chinese Solar Radio Heliograph
- 10.2991/isccca.2013.134How to use a DOI?
- CSRH, data processing, HPC
The Chinese Solar Radio Heliograph (CSRH) is a new generation radio heliograph would produce more than 4 terabytes data every day. As a aperture synthesis telescope, CSRH is facing the challenge of processing and storing such a vast data. Pipeline system is the key issue of data automatical processing for CSRH. In this study, to push the development of CSRH, we present a framework of high performance data processing pipeline system for saving and processing real-time observation data. The related techniques of pipeline software are presented in detail including raw data acquisition, UVFITS file, data calibration, parallel computing and data publication. The pipeline has been deployed and has played an important role for the development of CSRH.
- © 2013, the Authors. Published by Atlantis Press.
- Open Access
- This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Cite this article
TY - CONF AU - Fan Zhang AU - Feng Wang AU - Wei Wang AU - Wei Dai AU - Hui Deng AU - Kaifan Ji AU - Yihua Yan PY - 2013/02 DA - 2013/02 TI - High Performance Data Processing Pipeline Of Chinese Solar Radio Heliograph BT - Proceedings of the 2nd International Symposium on Computer, Communication, Control and Automation (ISCCCA 2013) PB - Atlantis Press SP - 535 EP - 538 SN - 1951-6851 UR - https://doi.org/10.2991/isccca.2013.134 DO - 10.2991/isccca.2013.134 ID - Zhang2013/02 ER -