X-ray Crystallography

X-ray Crystallography

Join the Competition Contribute to Benchmark

Standard reconstruction benchmark — forward model perfectly known, no calibration needed. Score = 0.5 × clip((PSNR−15)/30, 0, 1) + 0.5 × SSIM

# Method Score PSNR (dB) SSIM Source
🥇 CrystFormer 0.749 32.5 0.915 ✓ Certified Crystallographic transformer, 2024
🥈 DL-Phase 0.698 30.5 0.880 ✓ Certified Jumper et al., 2021
🥉 SHELXD 0.553 26.0 0.740 ✓ Certified Sheldrick, 2010
4 Molecular Replacement 0.412 22.0 0.590 ✓ Certified McCoy et al., 2007

Dataset: PWM Benchmark (4 algorithms)

Blind Reconstruction Challenge — forward model has unknown mismatch, must calibrate from data. Score = 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖)

# Method Overall Score Public
PSNR / SSIM
 Dev
PSNR / SSIM
 Hidden
PSNR / SSIM
 Trust Source
🥇 CrystFormer + gradient 0.676
0.745
29.87 dB / 0.915
0.670
26.75 dB / 0.852
0.613
24.22 dB / 0.776
✓ Certified Crystallographic phasing transformer, 2024
🥈 DL-Phase + gradient 0.630
0.710
27.99 dB / 0.881
0.618
24.37 dB / 0.781
0.563
22.47 dB / 0.710
✓ Certified Jumper et al., Nature 2021
🥉 SHELXD + gradient 0.607
0.651
24.74 dB / 0.794
0.610
23.67 dB / 0.757
0.559
22.04 dB / 0.692
✓ Certified Sheldrick, Acta Cryst. D 2010
4 Molecular Replacement + gradient 0.455
0.497
19.24 dB / 0.562
0.449
17.9 dB / 0.495
0.418
17.47 dB / 0.474
✓ Certified McCoy et al., J. Appl. Cryst. 2007

Complete score requires all 3 tiers (Public + Dev + Hidden).

Join the competition →
Scoring: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖) PSNR 40% · SSIM 40% · Consistency 20%
Public 5 scenes

Full-access development tier with all data visible.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), spec ranges, ground truth (x_true), and true mismatch spec.

How to use: Load HDF5 → compare reconstruction vs x_true → check consistency → iterate.

What to submit: Reconstructed signals (x_hat) and corrected spec as HDF5.

Public Leaderboard
# Method Score PSNR SSIM
1 CrystFormer + gradient 0.745 29.87 0.915
2 DL-Phase + gradient 0.710 27.99 0.881
3 SHELXD + gradient 0.651 24.74 0.794
4 Molecular Replacement + gradient 0.497 19.24 0.562
Spec Ranges (4 parameters)
Parameter Min Max Unit
crystal_orientation_error -0.1 0.2 deg
beam_divergence 0.08 0.14 mrad
absorption_correction 0.98 1.04 -
scale_factor 0.99 1.02 -
View Details →
Dev 5 scenes

Blind evaluation tier — no ground truth available.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), and spec ranges only.

How to use: Apply your pipeline from the Public tier. Use consistency as self-check.

What to submit: Reconstructed signals and corrected spec. Scored server-side.

Dev Leaderboard
# Method Score PSNR SSIM
1 CrystFormer + gradient 0.670 26.75 0.852
2 DL-Phase + gradient 0.618 24.37 0.781
3 SHELXD + gradient 0.610 23.67 0.757
4 Molecular Replacement + gradient 0.449 17.9 0.495
Spec Ranges (4 parameters)
Parameter Min Max Unit
crystal_orientation_error -0.12 0.18 deg
beam_divergence 0.076 0.136 mrad
absorption_correction 0.976 1.036 -
scale_factor 0.988 1.018 -
Submit Reconstruction View Details →
Hidden 5 scenes

Fully blind server-side evaluation — no data download.

What you get & how to use

What you get: No data downloadable. Algorithm runs server-side on hidden measurements.

How to use: Package algorithm as Docker container / Python script. Submit via link.

What to submit: Containerized algorithm accepting y + H, outputting x_hat + corrected spec.

Hidden Leaderboard
# Method Score PSNR SSIM
1 CrystFormer + gradient 0.613 24.22 0.776
2 DL-Phase + gradient 0.563 22.47 0.71
3 SHELXD + gradient 0.559 22.04 0.692
4 Molecular Replacement + gradient 0.418 17.47 0.474
Spec Ranges (4 parameters)
Parameter Min Max Unit
crystal_orientation_error -0.07 0.23 deg
beam_divergence 0.086 0.146 mrad
absorption_correction 0.986 1.046 -
scale_factor 0.993 1.023 -
Submit Algorithm View Details →

Blind Reconstruction Challenge

Challenge

Given measurements with unknown mismatch and spec ranges (not exact params), reconstruct the original signal. A method must be evaluated on all three tiers for a complete score. Scored on a composite metric: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖).

Input

Measurements y, ideal forward model H, spec ranges

Output

Reconstructed signal x̂

Spec DAG — Forward Model Pipeline

F → S → D

F Fourier
S Sampling
D Detector

Mismatch Parameters

Symbol Parameter Description Nominal Perturbed
c_o crystal_orientation_error Crystal orientation error (deg) 0.0 0.1
b_d beam_divergence Beam divergence (mrad) 0.1 0.12
a_c absorption_correction Absorption correction (-) 1.0 1.02
s_f scale_factor Scale factor (-) 1.0 1.01

Credits System

40%
Platform Profit Pool
Revenue allocated to benchmark rewards
30%
Winner Share
Top algorithm receives from pool
$100
Min Withdrawal
Minimum payout threshold
Spec Primitives Reference (11 primitives)
P Propagation

Free-space or medium propagation kernel (Fresnel, Rayleigh-Sommerfeld).

M Mask / Modulation

Spatial or spatio-temporal amplitude modulation (coded aperture, SLM pattern).

Π Projection

Geometric projection operator (Radon transform, fan-beam, cone-beam).

F Fourier Sampling

Sampling in the Fourier / k-space domain (MRI, ptychography).

C Convolution

Shift-invariant convolution with a point-spread function (PSF).

Σ Summation / Integration

Summation along a physical dimension (spectral, temporal, angular).

D Detector

Sensor readout with gain g and noise model η (Gaussian, Poisson, mixed).

S Structured Illumination

Patterned illumination (block, Hadamard, random) applied to the scene.

W Wavelength Dispersion

Spectral dispersion element (prism, grating) with shift α and aperture a.

R Rotation / Motion

Sample or gantry rotation (CT, electron tomography).

Λ Wavelength Selection

Spectral filter or monochromator selecting a wavelength band.