Beyond the Data Explosion (eAstronomy)

An eScience infrastructure for huge interferometric datasets

Image: SKA Organization

People are used to the stunning visual images taken by telescopes like the Hubble or the great telescopes in Hawaii and Chile, but maybe only have heard of radio astronomy through movies like “Contact”. But radio telescopes can “see” regions beyond the optical view of a galaxy. They can uncover the mysteries as to how the early galaxies, in the millions of years following the Big Bang, began to evolve: Where did they get their material? What drives their rotation? What has shaped them?

Since the 1930s astronomers have used radio telescopes to explore the Universe by detecting radio waves emitted by a wide range of objects. Our Sun, the nearest star to Earth is a powerful radio emission source, mainly due to its proximity to our planet, but some radio sources, which are millions of even billions of light years away, are truly colossal in terms of their radio output.

Radio telescopes provide alternative views to optical telescopes. They can detect invisible gas, and can reveal areas of space that may be obscured with cosmic dust. And unlike optical telescopes, which can be hampered by cloud or poor weather conditions on Earth, radio telescopes, working with signals at a longer wavelength, can be used even in cloudy skies.Radio telescopes can, for example, indirectly measure the effects of gravity on objects in the Universe. Objects like Pulsars, the fast spinning remnants of supernova explosions, and their more exotic cousins, the Black Holes, which are the densest objects in the Universe, exert huge gravitational effects on nearby objects.

However, to discover these pulsars you need a high resolution radio telescope; it would be kilometers across. For a single dish this is clearly not possible or practical. To get around this limitation in size radio astronomers use astronomical interferometers; an array of radio telescopes linked together. In the Netherlands, the LOFAR radio interferometric array is distributed over an area of about one hundred kilometers in diameter.

The brute-force search for pulsars takes place over many parameter combinations, because we do not know the position, distance and period of the pulsars. Searching for pulsars is a Big Data problem: typical observations produce hundreds of terabytes, and petabytes of results. Moreover, the pulsar signal is faint and can be completely covered by Radio Frequency Interference (RFI), which has to be analyzed and removed from the signal. In this project we aim to greatly speed up the search for new pulsars by using Graphics Processing Units (GPUs) for the complex computations. We are developing a complete real-time pulsar searching pipeline using a GPU cluster, and are testing it with two Dutch radio telescopes: LOFAR and Apertif. LOFAR currently is the largest radio telescope in the world.

The results of this work will be extremely important in developing the hardware and software for what will be the most powerful radio telescope ever built, the Square Kilometer Array (SKA). The SKA will be three to four orders of magnitude larger than LOFAR; requiring exascale computing and networking, and the tools developed in this project, to run efficiently.

This project explores a new development model for astronomical processing software where expertise from a variety of disciplines is combined. This includes for example mathematics for the foundations of new algorithms, and computer science to optimize for high-performance platforms. Optimized software and demonstrators will be developed that can be re-used in a variety of contexts, not just for radio/mm-astronomy, but also in other areas where large data-streams are combined.

Participating organisations

Natural Sciences & Engineering
Natural Sciences & Engineering
ASTRON
Atacama Large Millimeter/Submillimeter Array
CWI
JIVE
Leiden University
Netherlands eScience Center
Vrije Universiteit Amsterdam

Impact

Output

Team

MdV
Marco de Vos
Principle Investigator
ASTRON
Jason Maassen
Senior eScience Research Engineer
Netherlands eScience Center

Related projects

CORTEX

Self-learning machines hunt for explosions in the universe and speed up innovations in industry and...

Updated 25 months ago
In progress

PADRE - The PetaFLOP AARTFAAC Data-Reduction Engine

Improving the AARTFAAC processing pipeline

Updated 1 month ago
Finished

Efficient Deep Learning

Improving data training and computational efficiency for deep learning

Updated 7 months ago
Finished

EOSCpilot LOFAR

Unlocking the LOFAR Long Term Archive

Updated 21 months ago
Finished

DIRAC

Distributed radio astronomical computing

Updated 1 month ago
Finished

Parallelisation of multi point-cloud registration via multi-core and GPU for localization microscopy

Studying subcellular structures and functions

Updated 20 months ago
Finished

Compressing the sky into a large collection of statistical models

Optimized data handling for observations in astronomy

Updated 21 months ago
Finished

Related software

AMBER

AM

A real-time pipeline to search for Fast Radio Bursts and other transient radio sources.

Updated 14 months ago
25 3

eAstroViz

EA

This tool can convert and visualize radio astronomy measurement sets, as well as most LOFAR intermediate data products. It also performs RFI mitigation.

Updated 5 months ago
4 1