DIRAC

The infant Universe (its first-billion years) remains its least explored era. Although sparse observations are available, the red-shifted 21-cm emission of neutral hydrogen (HI) –seen as spectral fluctuations at several meter wavelengths– allows this era to be opened up for much more detailed study. The HI signal, however, is orders of magnitude fainter than most contaminating signals (e.g. (extra)Galactic foregrounds). Nonetheless, it is possible to detect and study this weak HI signal using the latest generation low-frequency radio interferometers (e.g. LOFAR, MWA),provided that all systematic (instrumental, ionospheric, etc.) errors are eliminated to sufficient levels (i.e. “calibrated”). These errors are determined and removed by solving a complex non-linear optimization problem with millions of unknown parameters, constrained by many terabytes of data.

Data parallelism is inherently exploited in calibration of radio-interferometric observations, where calibration is done in parallel on data at different frequencies. However, to achieve the highest accuracy and precision in calibration, without biasing the weak HI signal, a global calibration scheme is needed. We have demonstrated that this can be done using consensus optimization. In this project we will develop this from a proof-of-concept to a fully capable, computationally efficient and scalable software system. Ensuing orders-of-magnitude improvements in accuracy and computational speed not only enable the detection of this weak HI signal but also benefit a wider astronomical community (those using e.g. LOFAR, MWA, MeerKAT, ASKAP, APERTIF, SKA). The software developed in this project will be made publicly available for many other distributed optimization applications.