XFEL Serial Femtosecond Crystallography (SFX)

XFEL Serial Femtosecond Crystallography (SFX)

Join the Competition Contribute to Benchmark

Standard reconstruction benchmark — forward model perfectly known, no calibration needed. Score = 0.5 × clip((PSNR−15)/30, 0, 1) + 0.5 × SSIM

# Method Score PSNR (dB) SSIM Source
🥇 CNN Hit-Finder 0.775 32.92 0.952 ✓ Certified Ke et al., J. Synchrotron Rad. 2018
🥈 CrysFormer 0.745 31.56 0.938 ✓ Certified Crystallographic transformer, 2024
🥉 CrystFEL 0.547 24.38 0.782 ✓ Certified White et al., J. Appl. Cryst. 2012
4 EMC 0.546 24.36 0.781 ✓ Certified Loh & Elser, Phys. Rev. E 2009

Dataset: PWM Benchmark (4 algorithms)

Blind Reconstruction Challenge — forward model has unknown mismatch, must calibrate from data. Score = 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖)

# Method Overall Score Public
PSNR / SSIM
 Dev
PSNR / SSIM
 Hidden
PSNR / SSIM
 Trust Source
🥇 CrysFormer + gradient 0.653
0.727
28.72 dB / 0.895
0.663
26.2 dB / 0.838
0.569
22.47 dB / 0.710
✓ Certified Crystallographic transformer, 2024
🥈 CNN Hit-Finder + gradient 0.642
0.753
30.66 dB / 0.926
0.619
24.49 dB / 0.785
0.553
21.98 dB / 0.689
✓ Certified Ke et al., J. Synchrotron Rad. 2018
🥉 CrystFEL + gradient 0.534
0.612
23.14 dB / 0.737
0.533
20.95 dB / 0.643
0.457
18.89 dB / 0.544
✓ Certified White et al., J. Appl. Cryst. 2012
4 EMC + gradient 0.518
0.578
22.31 dB / 0.703
0.505
20.25 dB / 0.611
0.470
19.21 dB / 0.560
✓ Certified Loh & Elser, Phys. Rev. E 2009

Complete score requires all 3 tiers (Public + Dev + Hidden).

Join the competition →
Scoring: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖) PSNR 40% · SSIM 40% · Consistency 20%
Public 5 scenes

Full-access development tier with all data visible.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), spec ranges, ground truth (x_true), and true mismatch spec.

How to use: Load HDF5 → compare reconstruction vs x_true → check consistency → iterate.

What to submit: Reconstructed signals (x_hat) and corrected spec as HDF5.

Public Leaderboard
# Method Score PSNR SSIM
1 CNN Hit-Finder + gradient 0.753 30.66 0.926
2 CrysFormer + gradient 0.727 28.72 0.895
3 CrystFEL + gradient 0.612 23.14 0.737
4 EMC + gradient 0.578 22.31 0.703
Spec Ranges (4 parameters)
Parameter Min Max Unit
hit_rate 6.0 18.0 -
indexing_ambiguity -2.0 4.0 -
partiality_model_error -4.0 8.0 -
background_from_jet/carrier -6.0 12.0 -
View Details →
Dev 5 scenes

Blind evaluation tier — no ground truth available.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), and spec ranges only.

How to use: Apply your pipeline from the Public tier. Use consistency as self-check.

What to submit: Reconstructed signals and corrected spec. Scored server-side.

Dev Leaderboard
# Method Score PSNR SSIM
1 CrysFormer + gradient 0.663 26.2 0.838
2 CNN Hit-Finder + gradient 0.619 24.49 0.785
3 CrystFEL + gradient 0.533 20.95 0.643
4 EMC + gradient 0.505 20.25 0.611
Spec Ranges (4 parameters)
Parameter Min Max Unit
hit_rate 5.2 17.2 -
indexing_ambiguity -2.4 3.6 -
partiality_model_error -4.8 7.2 -
background_from_jet/carrier -7.2 10.8 -
Submit Reconstruction View Details →
Hidden 5 scenes

Fully blind server-side evaluation — no data download.

What you get & how to use

What you get: No data downloadable. Algorithm runs server-side on hidden measurements.

How to use: Package algorithm as Docker container / Python script. Submit via link.

What to submit: Containerized algorithm accepting y + H, outputting x_hat + corrected spec.

Hidden Leaderboard
# Method Score PSNR SSIM
1 CrysFormer + gradient 0.569 22.47 0.71
2 CNN Hit-Finder + gradient 0.553 21.98 0.689
3 EMC + gradient 0.470 19.21 0.56
4 CrystFEL + gradient 0.457 18.89 0.544
Spec Ranges (4 parameters)
Parameter Min Max Unit
hit_rate 7.2 19.2 -
indexing_ambiguity -1.4 4.6 -
partiality_model_error -2.8 9.2 -
background_from_jet/carrier -4.2 13.8 -
Submit Algorithm View Details →

Blind Reconstruction Challenge

Challenge

Given measurements with unknown mismatch and spec ranges (not exact params), reconstruct the original signal. A method must be evaluated on all three tiers for a complete score. Scored on a composite metric: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖).

Input

Measurements y, ideal forward model H, spec ranges

Output

Reconstructed signal x̂

Spec DAG — Forward Model Pipeline

M → R → D

M Modulation
R Rotation
D Detector

Mismatch Parameters

Symbol Parameter Description Nominal Perturbed
h_r hit_rate Hit rate (-) 10.0 14.0
i_a indexing_ambiguity Indexing ambiguity (-) 0.0 2.0
p_m partiality_model_error Partiality model error (-) 0.0 4.0
b_f background_from_jet/carrier Background from jet/carrier (-) 0.0 6.0

Credits System

40%
Platform Profit Pool
Revenue allocated to benchmark rewards
30%
Winner Share
Top algorithm receives from pool
$100
Min Withdrawal
Minimum payout threshold
Spec Primitives Reference (11 primitives)
P Propagation

Free-space or medium propagation kernel (Fresnel, Rayleigh-Sommerfeld).

M Mask / Modulation

Spatial or spatio-temporal amplitude modulation (coded aperture, SLM pattern).

Π Projection

Geometric projection operator (Radon transform, fan-beam, cone-beam).

F Fourier Sampling

Sampling in the Fourier / k-space domain (MRI, ptychography).

C Convolution

Shift-invariant convolution with a point-spread function (PSF).

Σ Summation / Integration

Summation along a physical dimension (spectral, temporal, angular).

D Detector

Sensor readout with gain g and noise model η (Gaussian, Poisson, mixed).

S Structured Illumination

Patterned illumination (block, Hadamard, random) applied to the scene.

W Wavelength Dispersion

Spectral dispersion element (prism, grating) with shift α and aperture a.

R Rotation / Motion

Sample or gantry rotation (CT, electron tomography).

Λ Wavelength Selection

Spectral filter or monochromator selecting a wavelength band.