Machine Vision / AOI

Machine Vision / AOI

Join the Competition Contribute to Benchmark

Standard reconstruction benchmark — forward model perfectly known, no calibration needed. Score = 0.5 × clip((PSNR−15)/30, 0, 1) + 0.5 × SSIM

# Method Score PSNR (dB) SSIM Source
🥇 UniAD 0.809 34.61 0.965 ✓ Certified You et al., NeurIPS 2022
🥈 PatchCore 0.738 31.24 0.934 ✓ Certified Roth et al., CVPR 2022
🥉 PnP-ADMM 0.672 29.7 0.855 ✓ Certified ADMM + denoiser prior
4 Template Match 0.646 27.59 0.872 ✓ Certified Brunelli, Template Matching, 2009

Dataset: PWM Benchmark (4 algorithms)

Blind Reconstruction Challenge — forward model has unknown mismatch, must calibrate from data. Score = 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖)

# Method Overall Score Public
PSNR / SSIM
 Dev
PSNR / SSIM
 Hidden
PSNR / SSIM
 Trust Source
🥇 UniAD + gradient 0.746
0.800
33.21 dB / 0.954
0.742
30.05 dB / 0.918
0.695
27.7 dB / 0.874
✓ Certified You et al., NeurIPS 2022
🥈 PatchCore + gradient 0.650
0.728
29.11 dB / 0.902
0.620
23.78 dB / 0.761
0.601
23.58 dB / 0.753
✓ Certified Roth et al., CVPR 2022
🥉 Template Match + gradient 0.631
0.680
25.88 dB / 0.829
0.631
24.82 dB / 0.796
0.581
22.66 dB / 0.717
✓ Certified Brunelli, Template Matching, 2009
4 PnP-ADMM + gradient 0.622
0.696
27.11 dB / 0.861
0.601
23.73 dB / 0.759
0.568
22.39 dB / 0.706
✓ Certified Venkatakrishnan et al., 2013

Complete score requires all 3 tiers (Public + Dev + Hidden).

Join the competition →
Scoring: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖) PSNR 40% · SSIM 40% · Consistency 20%
Public 3 scenes

Full-access development tier with all data visible.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), spec ranges, ground truth (x_true), and true mismatch spec.

How to use: Load HDF5 → compare reconstruction vs x_true → check consistency → iterate.

What to submit: Reconstructed signals (x_hat) and corrected spec as HDF5.

Public Leaderboard
# Method Score PSNR SSIM
1 UniAD + gradient 0.800 33.21 0.954
2 PatchCore + gradient 0.728 29.11 0.902
3 PnP-ADMM + gradient 0.696 27.11 0.861
4 Template Match + gradient 0.680 25.88 0.829
Spec Ranges (4 parameters)
Parameter Min Max Unit
focus_distance_error -1.0 2.0 mm
lens_distortion_k1 -0.02 0.04 -
exposure_time_drift 9.6 10.8 ms
white_balance_gain 0.98 1.04 -
View Details →
Dev 3 scenes

Blind evaluation tier — no ground truth available.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), and spec ranges only.

How to use: Apply your pipeline from the Public tier. Use consistency as self-check.

What to submit: Reconstructed signals and corrected spec. Scored server-side.

Dev Leaderboard
# Method Score PSNR SSIM
1 UniAD + gradient 0.742 30.05 0.918
2 Template Match + gradient 0.631 24.82 0.796
3 PatchCore + gradient 0.620 23.78 0.761
4 PnP-ADMM + gradient 0.601 23.73 0.759
Spec Ranges (4 parameters)
Parameter Min Max Unit
focus_distance_error -1.2 1.8 mm
lens_distortion_k1 -0.024 0.036 -
exposure_time_drift 9.52 10.72 ms
white_balance_gain 0.976 1.036 -
Submit Reconstruction View Details →
Hidden 3 scenes

Fully blind server-side evaluation — no data download.

What you get & how to use

What you get: No data downloadable. Algorithm runs server-side on hidden measurements.

How to use: Package algorithm as Docker container / Python script. Submit via link.

What to submit: Containerized algorithm accepting y + H, outputting x_hat + corrected spec.

Hidden Leaderboard
# Method Score PSNR SSIM
1 UniAD + gradient 0.695 27.7 0.874
2 PatchCore + gradient 0.601 23.58 0.753
3 Template Match + gradient 0.581 22.66 0.717
4 PnP-ADMM + gradient 0.568 22.39 0.706
Spec Ranges (4 parameters)
Parameter Min Max Unit
focus_distance_error -0.7 2.3 mm
lens_distortion_k1 -0.014 0.046 -
exposure_time_drift 9.72 10.92 ms
white_balance_gain 0.986 1.046 -
Submit Algorithm View Details →

Blind Reconstruction Challenge

Challenge

Given measurements with unknown mismatch and spec ranges (not exact params), reconstruct the original signal. A method must be evaluated on all three tiers for a complete score. Scored on a composite metric: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖).

Input

Measurements y, ideal forward model H, spec ranges

Output

Reconstructed signal x̂

Spec DAG — Forward Model Pipeline

C → D

C Convolution
D Detector

Mismatch Parameters

Symbol Parameter Description Nominal Perturbed
f_d focus_distance_error Focus distance error (mm) 0.0 1.0
l_d lens_distortion_k1 Lens distortion k1 (-) 0.0 0.02
e_t exposure_time_drift Exposure time drift (ms) 10.0 10.4
w_b white_balance_gain White balance gain (-) 1.0 1.02

Credits System

40%
Platform Profit Pool
Revenue allocated to benchmark rewards
30%
Winner Share
Top algorithm receives from pool
$100
Min Withdrawal
Minimum payout threshold
Spec Primitives Reference (11 primitives)
P Propagation

Free-space or medium propagation kernel (Fresnel, Rayleigh-Sommerfeld).

M Mask / Modulation

Spatial or spatio-temporal amplitude modulation (coded aperture, SLM pattern).

Π Projection

Geometric projection operator (Radon transform, fan-beam, cone-beam).

F Fourier Sampling

Sampling in the Fourier / k-space domain (MRI, ptychography).

C Convolution

Shift-invariant convolution with a point-spread function (PSF).

Σ Summation / Integration

Summation along a physical dimension (spectral, temporal, angular).

D Detector

Sensor readout with gain g and noise model η (Gaussian, Poisson, mixed).

S Structured Illumination

Patterned illumination (block, Hadamard, random) applied to the scene.

W Wavelength Dispersion

Spectral dispersion element (prism, grating) with shift α and aperture a.

R Rotation / Motion

Sample or gantry rotation (CT, electron tomography).

Λ Wavelength Selection

Spectral filter or monochromator selecting a wavelength band.