Sign in

Beyond the Data Explosion (eAstronomy)

An eScience infrastructure for huge interferometric datasets

Image: SKA Organization

People are used to the stunning visual images taken by telescopes like the Hubble or the great telescopes in Hawaii and Chile, but maybe only have heard of radio astronomy through movies like “Contact”. But radio telescopes can “see” regions beyond the optical view of a galaxy. They can uncover the mysteries as to how the early galaxies, in the millions of years following the Big Bang, began to evolve: Where did they get their material? What drives their rotation? What has shaped them?

Since the 1930s astronomers have used radio telescopes to explore the Universe by detecting radio waves emitted by a wide range of objects. Our Sun, the nearest star to Earth is a powerful radio emission source, mainly due to its proximity to our planet, but some radio sources, which are millions of even billions of light years away, are truly colossal in terms of their radio output.

Radio telescopes provide alternative views to optical telescopes. They can detect invisible gas, and can reveal areas of space that may be obscured with cosmic dust. And unlike optical telescopes, which can be hampered by cloud or poor weather conditions on Earth, radio telescopes, working with signals at a longer wavelength, can be used even in cloudy skies.Radio telescopes can, for example, indirectly measure the effects of gravity on objects in the Universe. Objects like Pulsars, the fast spinning remnants of supernova explosions, and their more exotic cousins, the Black Holes, which are the densest objects in the Universe, exert huge gravitational effects on nearby objects.

However, to discover these pulsars you need a high resolution radio telescope; it would be kilometers across. For a single dish this is clearly not possible or practical. To get around this limitation in size radio astronomers use astronomical interferometers; an array of radio telescopes linked together. In the Netherlands, the LOFAR radio interferometric array is distributed over an area of about one hundred kilometers in diameter.

The brute-force search for pulsars takes place over many parameter combinations, because we do not know the position, distance and period of the pulsars. Searching for pulsars is a Big Data problem: typical observations produce hundreds of terabytes, and petabytes of results. Moreover, the pulsar signal is faint and can be completely covered by Radio Frequency Interference (RFI), which has to be analyzed and removed from the signal. In this project we aim to greatly speed up the search for new pulsars by using Graphics Processing Units (GPUs) for the complex computations. We are developing a complete real-time pulsar searching pipeline using a GPU cluster, and are testing it with two Dutch radio telescopes: LOFAR and Apertif. LOFAR currently is the largest radio telescope in the world.

The results of this work will be extremely important in developing the hardware and software for what will be the most powerful radio telescope ever built, the Square Kilometer Array (SKA). The SKA will be three to four orders of magnitude larger than LOFAR; requiring exascale computing and networking, and the tools developed in this project, to run efficiently.

This project explores a new development model for astronomical processing software where expertise from a variety of disciplines is combined. This includes for example mathematics for the foundations of new algorithms, and computer science to optimize for high-performance platforms. Optimized software and demonstrators will be developed that can be re-used in a variety of contexts, not just for radio/mm-astronomy, but also in other areas where large data-streams are combined.

Participating organisations

Natural Sciences & Engineering
Atacama Large Millimeter/Submillimeter Array
Leiden University
Netherlands eScience Center
Vrije Universiteit Amsterdam


Marco de Vos
Principle Investigator
Jason Maassen
Jason Maassen
Senior eScience Research Engineer
Netherlands eScience Center
Rob van Nieuwpoort
Rob van Nieuwpoort
Director of Technology
Netherlands eScience Center

Related projects

PADRE - The PetaFLOP AARTFAAC Data-Reduction Engine

Improving the AARTFAAC processing pipeline

Updated 1 week ago


Unlocking the LOFAR Long Term Archive

Updated 1 week ago


Distributed radio astronomical computing

Updated 23 hours ago

Compressing the sky into a large collection of statistical models

Optimized data handling for observations in astronomy

Updated 4 days ago

Related tools



This tool can convert and visualize radio astronomy measurement sets, as well as most LOFAR intermediate data products. It also performs RFI mitigation.

Updated 8 months ago
3 1