Weather / Doppler Radar

Weather / Doppler Radar

Join the Competition Contribute to Benchmark

Standard reconstruction benchmark — forward model perfectly known, no calibration needed. Score = 0.5 × clip((PSNR−15)/30, 0, 1) + 0.5 × SSIM

# Method Score PSNR (dB) SSIM Source
🥇 Earthformer 0.776 33.5 0.935 ✓ Certified Gao et al., NeurIPS 2022
🥈 RainNet 0.730 31.8 0.900 ✓ Certified Ayzel et al., GMD 2020
🥉 CLEAN-AP 0.603 27.5 0.790 ✓ Certified Torres & Zrnic, IEEE TGRS 1999
4 Pulse-Pair Doppler 0.485 24.0 0.670 ✓ Certified Zrnic, IEEE TAES 1977

Dataset: PWM Benchmark (4 algorithms)

Blind Reconstruction Challenge — forward model has unknown mismatch, must calibrate from data. Score = 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖)

# Method Overall Score Public
PSNR / SSIM
 Dev
PSNR / SSIM
 Hidden
PSNR / SSIM
 Trust Source
🥇 Earthformer + gradient 0.708
0.764
31.37 dB / 0.935
0.698
28.4 dB / 0.889
0.661
26.67 dB / 0.850
✓ Certified Gao et al., NeurIPS 2022
🥈 RainNet + gradient 0.618
0.761
30.49 dB / 0.924
0.601
23.33 dB / 0.744
0.493
19.22 dB / 0.561
✓ Certified Ayzel et al., GMD 2020
🥉 CLEAN-AP + gradient 0.613
0.644
24.62 dB / 0.790
0.616
23.61 dB / 0.754
0.578
22.22 dB / 0.699
✓ Certified Torres & Zrnic, IEEE TGRS 1999
4 Pulse-Pair Doppler + gradient 0.527
0.554
21.13 dB / 0.652
0.544
21.27 dB / 0.658
0.484
19.08 dB / 0.554
✓ Certified Zrnic, IEEE TAES 1977

Complete score requires all 3 tiers (Public + Dev + Hidden).

Join the competition →
Scoring: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖) PSNR 40% · SSIM 40% · Consistency 20%
Public 3 scenes

Full-access development tier with all data visible.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), spec ranges, ground truth (x_true), and true mismatch spec.

How to use: Load HDF5 → compare reconstruction vs x_true → check consistency → iterate.

What to submit: Reconstructed signals (x_hat) and corrected spec as HDF5.

Public Leaderboard
# Method Score PSNR SSIM
1 Earthformer + gradient 0.764 31.37 0.935
2 RainNet + gradient 0.761 30.49 0.924
3 CLEAN-AP + gradient 0.644 24.62 0.79
4 Pulse-Pair Doppler + gradient 0.554 21.13 0.652
Spec Ranges (4 parameters)
Parameter Min Max Unit
calibration_bias -0.2 0.4 dBZ
beam_elevation_error -0.04 0.08 deg
attenuation_correction 0.98 1.04 -
ground_clutter_leakage -0.01 0.02 -
View Details →
Dev 3 scenes

Blind evaluation tier — no ground truth available.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), and spec ranges only.

How to use: Apply your pipeline from the Public tier. Use consistency as self-check.

What to submit: Reconstructed signals and corrected spec. Scored server-side.

Dev Leaderboard
# Method Score PSNR SSIM
1 Earthformer + gradient 0.698 28.4 0.889
2 CLEAN-AP + gradient 0.616 23.61 0.754
3 RainNet + gradient 0.601 23.33 0.744
4 Pulse-Pair Doppler + gradient 0.544 21.27 0.658
Spec Ranges (4 parameters)
Parameter Min Max Unit
calibration_bias -0.24 0.36 dBZ
beam_elevation_error -0.048 0.072 deg
attenuation_correction 0.976 1.036 -
ground_clutter_leakage -0.012 0.018 -
Submit Reconstruction View Details →
Hidden 3 scenes

Fully blind server-side evaluation — no data download.

What you get & how to use

What you get: No data downloadable. Algorithm runs server-side on hidden measurements.

How to use: Package algorithm as Docker container / Python script. Submit via link.

What to submit: Containerized algorithm accepting y + H, outputting x_hat + corrected spec.

Hidden Leaderboard
# Method Score PSNR SSIM
1 Earthformer + gradient 0.661 26.67 0.85
2 CLEAN-AP + gradient 0.578 22.22 0.699
3 RainNet + gradient 0.493 19.22 0.561
4 Pulse-Pair Doppler + gradient 0.484 19.08 0.554
Spec Ranges (4 parameters)
Parameter Min Max Unit
calibration_bias -0.14 0.46 dBZ
beam_elevation_error -0.028 0.092 deg
attenuation_correction 0.986 1.046 -
ground_clutter_leakage -0.007 0.023 -
Submit Algorithm View Details →

Blind Reconstruction Challenge

Challenge

Given measurements with unknown mismatch and spec ranges (not exact params), reconstruct the original signal. A method must be evaluated on all three tiers for a complete score. Scored on a composite metric: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖).

Input

Measurements y, ideal forward model H, spec ranges

Output

Reconstructed signal x̂

Spec DAG — Forward Model Pipeline

P → R → D

P Propagation
R Rotation
D Detector

Mismatch Parameters

Symbol Parameter Description Nominal Perturbed
c_b calibration_bias Calibration bias (dBZ) 0.0 0.2
b_e beam_elevation_error Beam elevation error (deg) 0.0 0.04
a_c attenuation_correction Attenuation correction (-) 1.0 1.02
g_c ground_clutter_leakage Ground clutter leakage (-) 0.0 0.01

Credits System

40%
Platform Profit Pool
Revenue allocated to benchmark rewards
30%
Winner Share
Top algorithm receives from pool
$100
Min Withdrawal
Minimum payout threshold
Spec Primitives Reference (11 primitives)
P Propagation

Free-space or medium propagation kernel (Fresnel, Rayleigh-Sommerfeld).

M Mask / Modulation

Spatial or spatio-temporal amplitude modulation (coded aperture, SLM pattern).

Π Projection

Geometric projection operator (Radon transform, fan-beam, cone-beam).

F Fourier Sampling

Sampling in the Fourier / k-space domain (MRI, ptychography).

C Convolution

Shift-invariant convolution with a point-spread function (PSF).

Σ Summation / Integration

Summation along a physical dimension (spectral, temporal, angular).

D Detector

Sensor readout with gain g and noise model η (Gaussian, Poisson, mixed).

S Structured Illumination

Patterned illumination (block, Hadamard, random) applied to the scene.

W Wavelength Dispersion

Spectral dispersion element (prism, grating) with shift α and aperture a.

R Rotation / Motion

Sample or gantry rotation (CT, electron tomography).

Λ Wavelength Selection

Spectral filter or monochromator selecting a wavelength band.