X-ray NDT (Radiography)

X-ray NDT (Radiography)

Join the Competition Contribute to Benchmark

Standard reconstruction benchmark — forward model perfectly known, no calibration needed. Score = 0.5 × clip((PSNR−15)/30, 0, 1) + 0.5 × SSIM

# Method Score PSNR (dB) SSIM Source
🥇 FBPConvNet 0.816 35.81 0.939 ✓ Certified Jin et al., IEEE TIP 2017
🥈 DR-GAN 0.798 34.07 0.961 ✓ Certified Zhang et al., NDT&E Int. 2021
🥉 PnP-ADMM 0.740 32.64 0.891 ✓ Certified Venkatakrishnan et al., 2013
4 FBP 0.601 27.38 0.790 ✓ Certified Kak & Slaney, 1988

Dataset: PWM Benchmark (4 algorithms)

Blind Reconstruction Challenge — forward model has unknown mismatch, must calibrate from data. Score = 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖)

# Method Overall Score Public
PSNR / SSIM
 Dev
PSNR / SSIM
 Hidden
PSNR / SSIM
 Trust Source
🥇 PnP-ADMM + gradient 0.708
0.751
30.51 dB / 0.924
0.712
28.64 dB / 0.894
0.660
26.54 dB / 0.847
✓ Certified Venkatakrishnan et al., 2013
🥈 FBPConvNet + gradient 0.704
0.797
34.03 dB / 0.961
0.684
26.74 dB / 0.852
0.632
25.24 dB / 0.810
✓ Certified Jin et al., IEEE TIP 2017
🥉 DR-GAN + gradient 0.666
0.771
31.71 dB / 0.939
0.638
25.41 dB / 0.815
0.590
23.4 dB / 0.746
✓ Certified Zhang et al., NDT&E Int. 2021
4 FBP + gradient 0.648
0.655
25.45 dB / 0.816
0.658
25.96 dB / 0.831
0.630
25.16 dB / 0.807
✓ Certified Kak & Slaney, IEEE Press 1988

Complete score requires all 3 tiers (Public + Dev + Hidden).

Join the competition →
Scoring: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖) PSNR 40% · SSIM 40% · Consistency 20%
Public 3 scenes

Full-access development tier with all data visible.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), spec ranges, ground truth (x_true), and true mismatch spec.

How to use: Load HDF5 → compare reconstruction vs x_true → check consistency → iterate.

What to submit: Reconstructed signals (x_hat) and corrected spec as HDF5.

Public Leaderboard
# Method Score PSNR SSIM
1 FBPConvNet + gradient 0.797 34.03 0.961
2 DR-GAN + gradient 0.771 31.71 0.939
3 PnP-ADMM + gradient 0.751 30.51 0.924
4 FBP + gradient 0.655 25.45 0.816
Spec Ranges (4 parameters)
Parameter Min Max Unit
source_position_error -0.2 0.4 mm
beam_hardening -0.02 0.04 -
detector_gain_drift 0.99 1.02 -
geometric_magnification 1.96 2.08 -
View Details →
Dev 3 scenes

Blind evaluation tier — no ground truth available.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), and spec ranges only.

How to use: Apply your pipeline from the Public tier. Use consistency as self-check.

What to submit: Reconstructed signals and corrected spec. Scored server-side.

Dev Leaderboard
# Method Score PSNR SSIM
1 PnP-ADMM + gradient 0.712 28.64 0.894
2 FBPConvNet + gradient 0.684 26.74 0.852
3 FBP + gradient 0.658 25.96 0.831
4 DR-GAN + gradient 0.638 25.41 0.815
Spec Ranges (4 parameters)
Parameter Min Max Unit
source_position_error -0.24 0.36 mm
beam_hardening -0.024 0.036 -
detector_gain_drift 0.988 1.018 -
geometric_magnification 1.952 2.072 -
Submit Reconstruction View Details →
Hidden 3 scenes

Fully blind server-side evaluation — no data download.

What you get & how to use

What you get: No data downloadable. Algorithm runs server-side on hidden measurements.

How to use: Package algorithm as Docker container / Python script. Submit via link.

What to submit: Containerized algorithm accepting y + H, outputting x_hat + corrected spec.

Hidden Leaderboard
# Method Score PSNR SSIM
1 PnP-ADMM + gradient 0.660 26.54 0.847
2 FBPConvNet + gradient 0.632 25.24 0.81
3 FBP + gradient 0.630 25.16 0.807
4 DR-GAN + gradient 0.590 23.4 0.746
Spec Ranges (4 parameters)
Parameter Min Max Unit
source_position_error -0.14 0.46 mm
beam_hardening -0.014 0.046 -
detector_gain_drift 0.993 1.023 -
geometric_magnification 1.972 2.092 -
Submit Algorithm View Details →

Blind Reconstruction Challenge

Challenge

Given measurements with unknown mismatch and spec ranges (not exact params), reconstruct the original signal. A method must be evaluated on all three tiers for a complete score. Scored on a composite metric: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖).

Input

Measurements y, ideal forward model H, spec ranges

Output

Reconstructed signal x̂

Spec DAG — Forward Model Pipeline

Π → D

Π Projection
D Detector

Mismatch Parameters

Symbol Parameter Description Nominal Perturbed
s_p source_position_error Source position error (mm) 0.0 0.2
b_h beam_hardening Beam hardening (-) 0.0 0.02
d_g detector_gain_drift Detector gain drift (-) 1.0 1.01
g_m geometric_magnification Geometric magnification (-) 2.0 2.04

Credits System

40%
Platform Profit Pool
Revenue allocated to benchmark rewards
30%
Winner Share
Top algorithm receives from pool
$100
Min Withdrawal
Minimum payout threshold
Spec Primitives Reference (11 primitives)
P Propagation

Free-space or medium propagation kernel (Fresnel, Rayleigh-Sommerfeld).

M Mask / Modulation

Spatial or spatio-temporal amplitude modulation (coded aperture, SLM pattern).

Π Projection

Geometric projection operator (Radon transform, fan-beam, cone-beam).

F Fourier Sampling

Sampling in the Fourier / k-space domain (MRI, ptychography).

C Convolution

Shift-invariant convolution with a point-spread function (PSF).

Σ Summation / Integration

Summation along a physical dimension (spectral, temporal, angular).

D Detector

Sensor readout with gain g and noise model η (Gaussian, Poisson, mixed).

S Structured Illumination

Patterned illumination (block, Hadamard, random) applied to the scene.

W Wavelength Dispersion

Spectral dispersion element (prism, grating) with shift α and aperture a.

R Rotation / Motion

Sample or gantry rotation (CT, electron tomography).

Λ Wavelength Selection

Spectral filter or monochromator selecting a wavelength band.