Machine learning for astronomical big data processing
Long Yan, Long Xu · 2017
Recently, more and more high resolution, high precision telescopes have been developing in the world, such as SKA[1], Arecibo[2], ALMA[2], Muser[4][5]. By aid of these modern telescopes, human acquired more and in-depth knowledge about the universe; meanwhile, a “big data” challenge was raised for astronomical big data processing. For example, the MingantU SpEctral Radioheliograph (Muser) records about 100TB raw data per month for solar radio observation. The big data firstly causes a big challenge for archiving and classifying recorded data, especially as we need the fast processing of daily recorded data. The traditional data processing was usually implemented manually before the emergence of big data, so it is no longer applicable to current big data. It is urgent to develop automatic algorithms for processing the daily recorded data efficiently and timely. Secondly, the big data also cause great difficulty in the storage and transmission of data, so data compression is highly desirable. This paper reports our efforts on big data archiving, classification and activity forecast by using machine learning, especially deep learning.