Event Horizon Telescope (EHT) Imaging

Event Horizon Telescope (EHT) Imaging

Join the Competition Contribute to Benchmark

Standard reconstruction benchmark — forward model perfectly known, no calibration needed. Score = 0.5 × clip((PSNR−15)/30, 0, 1) + 0.5 × SSIM

# Method Score PSNR (dB) SSIM Source
🥇 DiffVLBI 0.876 39.0 0.952 ✓ Certified Gao et al. 2024
🥈 PhysVLBI 0.847 37.6 0.940 ✓ Certified He et al. 2024
🥉 RadioFormer 0.817 36.2 0.928 ✓ Certified Gheller & Vazza 2023
4 TransVLBI 0.779 34.5 0.908 ✓ Certified Feng et al. 2023
5 SMILI 0.699 31.2 0.858 ✓ Certified Akiyama et al. 2017
6 eht-imaging 0.633 28.6 0.812 ✓ Certified Chael et al. 2018
7 RESOLVE 0.560 25.8 0.761 ✓ Certified Junklewitz et al. 2016
8 MEM-VLBI 0.494 23.1 0.718 ✓ Certified Narayan & Nityananda 1986
9 CLEAN-VLBI 0.426 20.4 0.672 ✓ Certified Hogbom 1974

Dataset: PWM Benchmark (9 algorithms)

Blind Reconstruction Challenge — forward model has unknown mismatch, must calibrate from data. Score = 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖)

# Method Overall Score Public
PSNR / SSIM
 Dev
PSNR / SSIM
 Hidden
PSNR / SSIM
 Trust Source
🥇 DiffVLBI + gradient 0.773
0.835
37.16 dB / 0.979
0.761
32.13 dB / 0.944
0.723
30.13 dB / 0.919
✓ Certified Gao et al., NeurIPS 2024
🥈 RadioFormer + gradient 0.766
0.822
35.08 dB / 0.968
0.766
31.25 dB / 0.934
0.711
28.72 dB / 0.895
✓ Certified Gheller & Vazza, MNRAS 2023
🥉 PhysVLBI + gradient 0.743
0.818
35.74 dB / 0.972
0.730
29.08 dB / 0.902
0.680
27.46 dB / 0.869
✓ Certified He et al., ApJ 2024
4 TransVLBI + gradient 0.687
0.777
32.43 dB / 0.947
0.685
27.41 dB / 0.868
0.598
23.14 dB / 0.737
✓ Certified Feng et al., A&A 2023
5 SMILI + gradient 0.618
0.751
29.85 dB / 0.915
0.580
22.39 dB / 0.706
0.522
20.23 dB / 0.610
✓ Certified Akiyama et al., ApJ 2017
6 eht-imaging + gradient 0.534
0.675
26.29 dB / 0.840
0.483
19.43 dB / 0.571
0.443
18.04 dB / 0.502
✓ Certified Chael et al., ApJ 2018
7 RESOLVE + gradient 0.513
0.616
23.68 dB / 0.757
0.477
19.3 dB / 0.565
0.445
18.25 dB / 0.512
✓ Certified Junklewitz et al., A&A 2016
8 CLEAN-VLBI + gradient 0.434
0.449
17.56 dB / 0.478
0.430
17.59 dB / 0.480
0.424
17.28 dB / 0.464
✓ Certified Hogbom, A&AS 1974
9 MEM-VLBI + gradient 0.393
0.529
20.34 dB / 0.615
0.351
14.47 dB / 0.330
0.299
13.28 dB / 0.280
✓ Certified Narayan & Nityananda, ARA&A 1986

Complete score requires all 3 tiers (Public + Dev + Hidden).

Join the competition →
Scoring: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖) PSNR 40% · SSIM 40% · Consistency 20%
Public 3 scenes

Full-access development tier with all data visible.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), spec ranges, ground truth (x_true), and true mismatch spec.

How to use: Load HDF5 → compare reconstruction vs x_true → check consistency → iterate.

What to submit: Reconstructed signals (x_hat) and corrected spec as HDF5.

Public Leaderboard
# Method Score PSNR SSIM
1 DiffVLBI + gradient 0.835 37.16 0.979
2 RadioFormer + gradient 0.822 35.08 0.968
3 PhysVLBI + gradient 0.818 35.74 0.972
4 TransVLBI + gradient 0.777 32.43 0.947
5 SMILI + gradient 0.751 29.85 0.915
6 eht-imaging + gradient 0.675 26.29 0.84
7 RESOLVE + gradient 0.616 23.68 0.757
8 MEM-VLBI + gradient 0.529 20.34 0.615
9 CLEAN-VLBI + gradient 0.449 17.56 0.478
Spec Ranges (4 parameters)
Parameter Min Max Unit
atmospheric_opacity_(tau) 0.02 0.26 nepers
station_gain_calibration -2.0 4.0 -
uv_coverage_sparsity -0.15 0.15 -
interstellar_scattering -2.0 4.0 uasbroadening
View Details →
Dev 3 scenes

Blind evaluation tier — no ground truth available.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), and spec ranges only.

How to use: Apply your pipeline from the Public tier. Use consistency as self-check.

What to submit: Reconstructed signals and corrected spec. Scored server-side.

Dev Leaderboard
# Method Score PSNR SSIM
1 RadioFormer + gradient 0.766 31.25 0.934
2 DiffVLBI + gradient 0.761 32.13 0.944
3 PhysVLBI + gradient 0.730 29.08 0.902
4 TransVLBI + gradient 0.685 27.41 0.868
5 SMILI + gradient 0.580 22.39 0.706
6 eht-imaging + gradient 0.483 19.43 0.571
7 RESOLVE + gradient 0.477 19.3 0.565
8 CLEAN-VLBI + gradient 0.430 17.59 0.48
9 MEM-VLBI + gradient 0.351 14.47 0.33
Spec Ranges (4 parameters)
Parameter Min Max Unit
atmospheric_opacity_(tau) 0.004 0.244 nepers
station_gain_calibration -2.4 3.6 -
uv_coverage_sparsity -0.15 0.15 -
interstellar_scattering -2.4 3.6 uasbroadening
Submit Reconstruction View Details →
Hidden 3 scenes

Fully blind server-side evaluation — no data download.

What you get & how to use

What you get: No data downloadable. Algorithm runs server-side on hidden measurements.

How to use: Package algorithm as Docker container / Python script. Submit via link.

What to submit: Containerized algorithm accepting y + H, outputting x_hat + corrected spec.

Hidden Leaderboard
# Method Score PSNR SSIM
1 DiffVLBI + gradient 0.723 30.13 0.919
2 RadioFormer + gradient 0.711 28.72 0.895
3 PhysVLBI + gradient 0.680 27.46 0.869
4 TransVLBI + gradient 0.598 23.14 0.737
5 SMILI + gradient 0.522 20.23 0.61
6 RESOLVE + gradient 0.445 18.25 0.512
7 eht-imaging + gradient 0.443 18.04 0.502
8 CLEAN-VLBI + gradient 0.424 17.28 0.464
9 MEM-VLBI + gradient 0.299 13.28 0.28
Spec Ranges (4 parameters)
Parameter Min Max Unit
atmospheric_opacity_(tau) 0.044 0.284 nepers
station_gain_calibration -1.4 4.6 -
uv_coverage_sparsity -0.15 0.15 -
interstellar_scattering -1.4 4.6 uasbroadening
Submit Algorithm View Details →

Blind Reconstruction Challenge

Challenge

Given measurements with unknown mismatch and spec ranges (not exact params), reconstruct the original signal. A method must be evaluated on all three tiers for a complete score. Scored on a composite metric: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖).

Input

Measurements y, ideal forward model H, spec ranges

Output

Reconstructed signal x̂

Spec DAG — Forward Model Pipeline

F → S → D

F Fourier
S Sampling
D Detector

Mismatch Parameters

Symbol Parameter Description Nominal Perturbed
a_o atmospheric_opacity_(tau) Atmospheric opacity (tau) (nepers) 0.1 0.18
s_g station_gain_calibration Station gain calibration (-) 0.0 2.0
u_s uv_coverage_sparsity uv-coverage sparsity (-) 0.0 0.0
i_s interstellar_scattering Interstellar scattering (uas broadening) 0.0 2.0

Credits System

40%
Platform Profit Pool
Revenue allocated to benchmark rewards
30%
Winner Share
Top algorithm receives from pool
$100
Min Withdrawal
Minimum payout threshold
Spec Primitives Reference (11 primitives)
P Propagation

Free-space or medium propagation kernel (Fresnel, Rayleigh-Sommerfeld).

M Mask / Modulation

Spatial or spatio-temporal amplitude modulation (coded aperture, SLM pattern).

Π Projection

Geometric projection operator (Radon transform, fan-beam, cone-beam).

F Fourier Sampling

Sampling in the Fourier / k-space domain (MRI, ptychography).

C Convolution

Shift-invariant convolution with a point-spread function (PSF).

Σ Summation / Integration

Summation along a physical dimension (spectral, temporal, angular).

D Detector

Sensor readout with gain g and noise model η (Gaussian, Poisson, mixed).

S Structured Illumination

Patterned illumination (block, Hadamard, random) applied to the scene.

W Wavelength Dispersion

Spectral dispersion element (prism, grating) with shift α and aperture a.

R Rotation / Motion

Sample or gantry rotation (CT, electron tomography).

Λ Wavelength Selection

Spectral filter or monochromator selecting a wavelength band.