Photometric Stereo

Photometric Stereo

Join the Competition Contribute to Benchmark

Standard reconstruction benchmark — forward model perfectly known, no calibration needed. Score = 0.5 × clip((PSNR−15)/30, 0, 1) + 0.5 × SSIM

# Method Score PSNR (dB) SSIM Source
🥇 PS-Transformer 0.792 34.2 0.945 ✓ Certified Ikehata, ICCV 2023
🥈 CNN-PS 0.749 32.5 0.915 ✓ Certified Ikehata, ECCV 2018
🥉 Robust PCA 0.635 28.5 0.820 ✓ Certified Wu et al., ECCV 2010
4 LS Normal Est. 0.517 25.0 0.700 ✓ Certified Woodham, Opt. Eng. 1980

Dataset: PWM Benchmark (4 algorithms)

Blind Reconstruction Challenge — forward model has unknown mismatch, must calibrate from data. Score = 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖)

# Method Overall Score Public
PSNR / SSIM
 Dev
PSNR / SSIM
 Hidden
PSNR / SSIM
 Trust Source
🥇 PS-Transformer + gradient 0.715
0.795
32.82 dB / 0.951
0.707
28.93 dB / 0.899
0.643
24.98 dB / 0.802
✓ Certified Ikehata, ICCV 2023
🥈 CNN-PS + gradient 0.683
0.771
31.04 dB / 0.931
0.647
25.72 dB / 0.824
0.631
24.83 dB / 0.797
✓ Certified Ikehata, ECCV 2018
🥉 Robust PCA + gradient 0.643
0.679
26.58 dB / 0.848
0.647
24.93 dB / 0.800
0.604
24.11 dB / 0.772
✓ Certified Wu et al., ECCV 2010
4 LS Normal Est. + gradient 0.548
0.591
22.73 dB / 0.720
0.537
20.75 dB / 0.634
0.516
20.49 dB / 0.622
✓ Certified Woodham, Opt. Eng. 1980

Complete score requires all 3 tiers (Public + Dev + Hidden).

Join the competition →
Scoring: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖) PSNR 40% · SSIM 40% · Consistency 20%
Public 5 scenes

Full-access development tier with all data visible.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), spec ranges, ground truth (x_true), and true mismatch spec.

How to use: Load HDF5 → compare reconstruction vs x_true → check consistency → iterate.

What to submit: Reconstructed signals (x_hat) and corrected spec as HDF5.

Public Leaderboard
# Method Score PSNR SSIM
1 PS-Transformer + gradient 0.795 32.82 0.951
2 CNN-PS + gradient 0.771 31.04 0.931
3 Robust PCA + gradient 0.679 26.58 0.848
4 LS Normal Est. + gradient 0.591 22.73 0.72
Spec Ranges (4 parameters)
Parameter Min Max Unit
light_direction_error -1.0 2.0 degpersource
light_intensity_calibration 0.96 1.08 -
non_lambertian_surface_fraction -6.0 12.0 -
cast_shadow_fraction -3.0 6.0 ofpixels
View Details →
Dev 5 scenes

Blind evaluation tier — no ground truth available.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), and spec ranges only.

How to use: Apply your pipeline from the Public tier. Use consistency as self-check.

What to submit: Reconstructed signals and corrected spec. Scored server-side.

Dev Leaderboard
# Method Score PSNR SSIM
1 PS-Transformer + gradient 0.707 28.93 0.899
2 CNN-PS + gradient 0.647 25.72 0.824
3 Robust PCA + gradient 0.647 24.93 0.8
4 LS Normal Est. + gradient 0.537 20.75 0.634
Spec Ranges (4 parameters)
Parameter Min Max Unit
light_direction_error -1.2 1.8 degpersource
light_intensity_calibration 0.952 1.072 -
non_lambertian_surface_fraction -7.2 10.8 -
cast_shadow_fraction -3.6 5.4 ofpixels
Submit Reconstruction View Details →
Hidden 5 scenes

Fully blind server-side evaluation — no data download.

What you get & how to use

What you get: No data downloadable. Algorithm runs server-side on hidden measurements.

How to use: Package algorithm as Docker container / Python script. Submit via link.

What to submit: Containerized algorithm accepting y + H, outputting x_hat + corrected spec.

Hidden Leaderboard
# Method Score PSNR SSIM
1 PS-Transformer + gradient 0.643 24.98 0.802
2 CNN-PS + gradient 0.631 24.83 0.797
3 Robust PCA + gradient 0.604 24.11 0.772
4 LS Normal Est. + gradient 0.516 20.49 0.622
Spec Ranges (4 parameters)
Parameter Min Max Unit
light_direction_error -0.7 2.3 degpersource
light_intensity_calibration 0.972 1.092 -
non_lambertian_surface_fraction -4.2 13.8 -
cast_shadow_fraction -2.1 6.9 ofpixels
Submit Algorithm View Details →

Blind Reconstruction Challenge

Challenge

Given measurements with unknown mismatch and spec ranges (not exact params), reconstruct the original signal. A method must be evaluated on all three tiers for a complete score. Scored on a composite metric: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖).

Input

Measurements y, ideal forward model H, spec ranges

Output

Reconstructed signal x̂

Spec DAG — Forward Model Pipeline

M → C → D

M Modulation
C Convolution
D Detector

Mismatch Parameters

Symbol Parameter Description Nominal Perturbed
l_d light_direction_error Light direction error (deg per source) 0.0 1.0
l_i light_intensity_calibration Light intensity calibration (-) 1.0 1.04
n_s non_lambertian_surface_fraction Non-Lambertian surface fraction (-) 0.0 6.0
c_s cast_shadow_fraction Cast shadow fraction (of pixels) 0.0 3.0

Credits System

40%
Platform Profit Pool
Revenue allocated to benchmark rewards
30%
Winner Share
Top algorithm receives from pool
$100
Min Withdrawal
Minimum payout threshold
Spec Primitives Reference (11 primitives)
P Propagation

Free-space or medium propagation kernel (Fresnel, Rayleigh-Sommerfeld).

M Mask / Modulation

Spatial or spatio-temporal amplitude modulation (coded aperture, SLM pattern).

Π Projection

Geometric projection operator (Radon transform, fan-beam, cone-beam).

F Fourier Sampling

Sampling in the Fourier / k-space domain (MRI, ptychography).

C Convolution

Shift-invariant convolution with a point-spread function (PSF).

Σ Summation / Integration

Summation along a physical dimension (spectral, temporal, angular).

D Detector

Sensor readout with gain g and noise model η (Gaussian, Poisson, mixed).

S Structured Illumination

Patterned illumination (block, Hadamard, random) applied to the scene.

W Wavelength Dispersion

Spectral dispersion element (prism, grating) with shift α and aperture a.

R Rotation / Motion

Sample or gantry rotation (CT, electron tomography).

Λ Wavelength Selection

Spectral filter or monochromator selecting a wavelength band.