Radio Aperture Synthesis

Radio Aperture Synthesis

Join the Competition Contribute to Benchmark

Standard reconstruction benchmark — forward model perfectly known, no calibration needed. Score = 0.5 × clip((PSNR−15)/30, 0, 1) + 0.5 × SSIM

# Method Score PSNR (dB) SSIM Source
🥇 PRIMO 0.722 31.2 0.905 ✓ Certified Medeiros et al., ApJL 2023
🥈 R2D2 0.684 29.8 0.875 ✓ Certified Aghabiglou et al., ApJS 2024
🥉 AIRI 0.573 26.3 0.770 ✓ Certified Terris et al., MNRAS 2022
4 CLEAN 0.425 22.5 0.600 ✓ Certified Hogbom, A&AS 1974

Dataset: PWM Benchmark (4 algorithms)

Blind Reconstruction Challenge — forward model has unknown mismatch, must calibrate from data. Score = 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖)

# Method Overall Score Public
PSNR / SSIM
 Dev
PSNR / SSIM
 Hidden
PSNR / SSIM
 Trust Source
🥇 PRIMO + gradient 0.652
0.729
29.42 dB / 0.908
0.644
25.42 dB / 0.815
0.582
23.27 dB / 0.742
✓ Certified Medeiros et al., ApJL 2023
🥈 R2D2 + gradient 0.618
0.725
28.37 dB / 0.888
0.604
23.8 dB / 0.761
0.526
21.36 dB / 0.662
✓ Certified Aghabiglou et al., ApJS 2024
🥉 AIRI + gradient 0.586
0.627
24.23 dB / 0.777
0.602
23.54 dB / 0.752
0.528
20.63 dB / 0.629
✓ Certified Terris et al., MNRAS 2022
4 CLEAN + gradient 0.494
0.561
21.24 dB / 0.657
0.497
19.77 dB / 0.588
0.423
17.23 dB / 0.462
✓ Certified Hogbom, A&AS 1974

Complete score requires all 3 tiers (Public + Dev + Hidden).

Join the competition →
Scoring: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖) PSNR 40% · SSIM 40% · Consistency 20%
Public 5 scenes

Full-access development tier with all data visible.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), spec ranges, ground truth (x_true), and true mismatch spec.

How to use: Load HDF5 → compare reconstruction vs x_true → check consistency → iterate.

What to submit: Reconstructed signals (x_hat) and corrected spec as HDF5.

Public Leaderboard
# Method Score PSNR SSIM
1 PRIMO + gradient 0.729 29.42 0.908
2 R2D2 + gradient 0.725 28.37 0.888
3 AIRI + gradient 0.627 24.23 0.777
4 CLEAN + gradient 0.561 21.24 0.657
Spec Ranges (4 parameters)
Parameter Min Max Unit
antenna_gain_error 0.99 1.02 -
phase_calibration_error -1.0 2.0 deg
bandpass_slope -0.002 0.004 1/MHz
pointing_offset -1.0 2.0 arcsec
View Details →
Dev 5 scenes

Blind evaluation tier — no ground truth available.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), and spec ranges only.

How to use: Apply your pipeline from the Public tier. Use consistency as self-check.

What to submit: Reconstructed signals and corrected spec. Scored server-side.

Dev Leaderboard
# Method Score PSNR SSIM
1 PRIMO + gradient 0.644 25.42 0.815
2 R2D2 + gradient 0.604 23.8 0.761
3 AIRI + gradient 0.602 23.54 0.752
4 CLEAN + gradient 0.497 19.77 0.588
Spec Ranges (4 parameters)
Parameter Min Max Unit
antenna_gain_error 0.988 1.018 -
phase_calibration_error -1.2 1.8 deg
bandpass_slope -0.0024 0.0036 1/MHz
pointing_offset -1.2 1.8 arcsec
Submit Reconstruction View Details →
Hidden 5 scenes

Fully blind server-side evaluation — no data download.

What you get & how to use

What you get: No data downloadable. Algorithm runs server-side on hidden measurements.

How to use: Package algorithm as Docker container / Python script. Submit via link.

What to submit: Containerized algorithm accepting y + H, outputting x_hat + corrected spec.

Hidden Leaderboard
# Method Score PSNR SSIM
1 PRIMO + gradient 0.582 23.27 0.742
2 AIRI + gradient 0.528 20.63 0.629
3 R2D2 + gradient 0.526 21.36 0.662
4 CLEAN + gradient 0.423 17.23 0.462
Spec Ranges (4 parameters)
Parameter Min Max Unit
antenna_gain_error 0.993 1.023 -
phase_calibration_error -0.7 2.3 deg
bandpass_slope -0.0014 0.0046 1/MHz
pointing_offset -0.7 2.3 arcsec
Submit Algorithm View Details →

Blind Reconstruction Challenge

Challenge

Given measurements with unknown mismatch and spec ranges (not exact params), reconstruct the original signal. A method must be evaluated on all three tiers for a complete score. Scored on a composite metric: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖).

Input

Measurements y, ideal forward model H, spec ranges

Output

Reconstructed signal x̂

Spec DAG — Forward Model Pipeline

F → S → D

F Fourier
S Sampling
D Detector

Mismatch Parameters

Symbol Parameter Description Nominal Perturbed
a_g antenna_gain_error Antenna gain error (-) 1.0 1.01
p_c phase_calibration_error Phase calibration error (deg) 0.0 1.0
b_s bandpass_slope Bandpass slope (1/MHz) 0.0 0.002
p_o pointing_offset Pointing offset (arcsec) 0.0 1.0

Credits System

40%
Platform Profit Pool
Revenue allocated to benchmark rewards
30%
Winner Share
Top algorithm receives from pool
$100
Min Withdrawal
Minimum payout threshold
Spec Primitives Reference (11 primitives)
P Propagation

Free-space or medium propagation kernel (Fresnel, Rayleigh-Sommerfeld).

M Mask / Modulation

Spatial or spatio-temporal amplitude modulation (coded aperture, SLM pattern).

Π Projection

Geometric projection operator (Radon transform, fan-beam, cone-beam).

F Fourier Sampling

Sampling in the Fourier / k-space domain (MRI, ptychography).

C Convolution

Shift-invariant convolution with a point-spread function (PSF).

Σ Summation / Integration

Summation along a physical dimension (spectral, temporal, angular).

D Detector

Sensor readout with gain g and noise model η (Gaussian, Poisson, mixed).

S Structured Illumination

Patterned illumination (block, Hadamard, random) applied to the scene.

W Wavelength Dispersion

Spectral dispersion element (prism, grating) with shift α and aperture a.

R Rotation / Motion

Sample or gantry rotation (CT, electron tomography).

Λ Wavelength Selection

Spectral filter or monochromator selecting a wavelength band.