Proceedings of the 2015 3rd International Conference on Machinery, Materials and Information Technology Applications

A Scheme of Implementing PCA Alogrithm on Storm Platform

Authors
Yan Shan, Lingjuan Li, Yimu Ji
Corresponding Author
Yan Shan
Available Online November 2015.
DOI
https://doi.org/10.2991/icmmita-15.2015.125How to use a DOI?
Keywords
Data Stream; Dimensional Reduction; Storm; PCA; Distribution and Parallelization
Abstract

In order to sovle the problem of dimension disaster when mining the high-dimensional data in the data stream and the problem of poor real-time response and insufficient system throughput of dimensional reduction algorithms, a scheme of implementing PCA algorithm on Storm platform is designed. This scheme programs each branch of PCA algorithm by using Storm’s own components, and each component forms the task entity through data flow communication. The scheme realizes the alogrithm distribution and parallelization by setting the threads number and the process number of task entity. Experimental results of running PCA algorithm on Storm and computer cluster according to the scheme show that the PCA algorithm on Storm platform can meet the requirement of real-time dimensional reduction of data stream.

Copyright
© 2015, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

Volume Title
Proceedings of the 2015 3rd International Conference on Machinery, Materials and Information Technology Applications
Series
Advances in Computer Science Research
Publication Date
November 2015
ISBN
978-94-6252-120-9
ISSN
2352-538X
DOI
https://doi.org/10.2991/icmmita-15.2015.125How to use a DOI?
Copyright
© 2015, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - CONF
AU  - Yan Shan
AU  - Lingjuan Li
AU  - Yimu Ji
PY  - 2015/11
DA  - 2015/11
TI  - A Scheme of Implementing PCA Alogrithm on Storm Platform
BT  - Proceedings of the 2015 3rd International Conference on Machinery, Materials and Information Technology Applications
PB  - Atlantis Press
SP  - 644
EP  - 648
SN  - 2352-538X
UR  - https://doi.org/10.2991/icmmita-15.2015.125
DO  - https://doi.org/10.2991/icmmita-15.2015.125
ID  - Shan2015/11
ER  -