<abstract xmlns="http://www.w3.org/1999/xhtml"><p>Extracting useful information from astronomical observations represents one of the most challenging tasks of data exploration. This is largely due to the volume of the data acquired using advanced observational tools. While other challenges typical for the class of big data problems (like data variety) are also present, the size of datasets represents the most significant obstacle in visualization and subsequent analysis. This paper studies an efficient data condensation algorithm aimed at providing its compact representation. It is based on fast nearest neighbor calculation using tree structures and parallel processing. In addition to that, the possibility of using approximate identification of neighbors, to even further improve the algorithm time performance, is also evaluated. The properties of the proposed approach, both in terms of performance and condensation quality, are experimentally assessed on astronomical datasets related to the GAIA mission. It is concluded that the introduced technique might serve as a scalable method of alleviating the problem of the dataset size.</p></abstract>

Extracting useful information from astronomical observations represents one of the most challenging tasks of data exploration. This is largely due to the volume of the data acquired using advanced observational tools. While other challenges typical for the class of big data problems (like data variety) are also present, the size of datasets represents the most significant obstacle in visualization and subsequent analysis. This paper studies an efficient data condensation algorithm aimed at providing its compact representation. It is based on fast nearest neighbor calculation using tree structures and parallel processing. In addition to that, the possibility of using approximate identification of neighbors, to even further improve the algorithm time performance, is also evaluated. The properties of the proposed approach, both in terms of performance and condensation quality, are experimentally assessed on astronomical datasets related to the GAIA mission. It is concluded that the introduced technique might serve as a scalable method of alleviating the problem of the dataset size.

Efficient Astronomical Data Condensation Using Approximate Nearest Neighbors

Faculty of Physics and Applied Computer Science

Faculty of Electrical Engineering and Computer Science

International Journal of Applied Mathematics and Computer Science

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

{"article-title":"Efficient Astronomical Data Condensation Using Approximate Nearest Neighbors"}

Extracting useful information from astronomical observations represents one of the most challenging tasks of data exploration. This is largely due to the...

Efficient Astronomical Data Condensation Using Approximate Nearest Neighbors

Published Online: Sep 28, 2019

Page range: 467 - 476

Received: Nov 11, 2018

Accepted: Mar 18, 2019

DOI: https://doi.org/10.2478/amcs-2019-0034

Keywords
big data, astronomy, data reduction, nearest neighbor search, kd-trees

© 2019 Szymon Łukasik et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Efficient Astronomical Data Condensation Using Approximate Nearest Neighbors

Published Online: Sep 28, 2019

Page range: 467 - 476

Received: Nov 11, 2018

Accepted: Mar 18, 2019

DOI: https://doi.org/10.2478/amcs-2019-0034

Keywordsbig data, astronomy, data reduction, nearest neighbor search, kd-trees

© 2019 Szymon Łukasik et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Keywords
big data, astronomy, data reduction, nearest neighbor search, kd-trees