DIRAC

Distributed radio astronomical computing

Image by: Adolf Schaller NASA-MSFC

The infant Universe (its first-billion years) remains its least explored era. Although sparse observations are available, the red-shifted 21-cm emission of neutral hydrogen (HI) –seen as spectral fluctuations at several meter wavelengths– allows this era to be opened up for much more detailed study. The HI signal, however, is orders of magnitude fainter than most contaminating signals (e.g. (extra)Galactic foregrounds). Nonetheless, it is possible to detect and study this weak HI signal using the latest generation low-frequency radio interferometers (e.g. LOFAR, MWA),provided that all systematic (instrumental, ionospheric, etc.) errors are eliminated to sufficient levels (i.e. “calibrated”). These errors are determined and removed by solving a complex non-linear optimization problem with millions of unknown parameters, constrained by many terabytes of data.

Data parallelism is inherently exploited in calibration of radio-interferometric observations, where calibration is done in parallel on data at different frequencies. However, to achieve the highest accuracy and precision in calibration, without biasing the weak HI signal, a global calibration scheme is needed. We have demonstrated that this can be done using consensus optimization. In this project we will develop this from a proof-of-concept to a fully capable, computationally efficient and scalable software system. Ensuing orders-of-magnitude improvements in accuracy and computational speed not only enable the detection of this weak HI signal but also benefit a wider astronomical community (those using e.g. LOFAR, MWA, MeerKAT, ASKAP, APERTIF, SKA). The software developed in this project will be made publicly available for many other distributed optimization applications.

Participating organisations

ASTRON
Netherlands eScience Center
University of Groningen

Team

AM
Adriënne Mendrik
eScience Coordinator
Netherlands eScience Center
Faruk Diblen
Faruk Diblen
eScience Research Engineer
Netherlands eScience Center
HS
Hanno Spreeuw
eScience Research Engineer
Netherlands eScience Center
SY
Sarod Yatawatta
Principal investigator

Related projects

PADRE - The PetaFLOP AARTFAAC Data-Reduction Engine

Improving the AARTFAAC processing pipeline

Updated 1 month ago
Running

EOSCpilot LOFAR

Unlocking the LOFAR Long Term Archive

Updated 2 months ago
Finished

AA-ALERT

Access and acceleration of the Apertif Legacy exploration of the radio transient sky

Updated 2 months ago
Finished

Compressing the sky into a large collection of statistical models

Optimized data handling for observations in astronomy

Updated 2 months ago
Finished

Related tools

Kernel Tuner

KE

Kernel Tuner greatly simplifies the development of highly-optimized and auto-tuned CUDA, OpenCL, and C code, supporting many advanced use-cases and optimization strategies that speed up the auto-tuning process.

Updated 3 weeks ago
29 mentions, 13 contributors

Sagecal

SA

Fast, memory efficient and GPU accelerated radio interferometric calibration program

Updated 2 months ago
1 mention, 4 contributors