Cryo-EM Single Particle Analysis

Cryo-EM Single Particle Analysis

Join the Competition Contribute to Benchmark

Standard reconstruction benchmark — forward model perfectly known, no calibration needed. Score = 0.5 × clip((PSNR−15)/30, 0, 1) + 0.5 × SSIM

# Method Score PSNR (dB) SSIM Source
🥇 DiffusionCryo 0.895 39.8 0.963 ✓ Certified Luo 2024
🥈 CryoSTAR 0.866 38.4 0.952 ✓ Certified Yang 2024
🥉 CryoFormer 0.839 37.1 0.941 ✓ Certified Gao 2024
4 CryoGEM 0.799 35.2 0.924 ✓ Certified He 2023
5 cryoDRGN 0.762 33.7 0.901 ✓ Certified Zhong 2021
6 IsoNet 0.709 31.4 0.871 ✓ Certified Liu 2021
7 cryoSPARC 0.630 28.1 0.823 ✓ Certified Punjani 2017
8 RELION-3D 0.571 25.8 0.782 ✓ Certified Scheres 2012
9 CTFFIND4 0.479 22.3 0.714 ✓ Certified Rohou 2015

Dataset: PWM Benchmark (9 algorithms)

Blind Reconstruction Challenge — forward model has unknown mismatch, must calibrate from data. Score = 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖)

# Method Overall Score Public
PSNR / SSIM
 Dev
PSNR / SSIM
 Hidden
PSNR / SSIM
 Trust Source
🥇 CryoSTAR + gradient 0.770
0.825
35.48 dB / 0.971
0.750
31.54 dB / 0.937
0.736
30.58 dB / 0.925
✓ Certified Yang et al., Nat. Methods 2024
🥈 CryoFormer + gradient 0.751
0.811
34.54 dB / 0.965
0.755
31.13 dB / 0.933
0.687
27.49 dB / 0.870
✓ Certified Gao et al., CVPR 2024
🥉 DiffusionCryo + gradient 0.743
0.842
37.04 dB / 0.978
0.721
29.65 dB / 0.911
0.667
26.86 dB / 0.855
✓ Certified Luo et al., arXiv 2024
4 cryoDRGN + gradient 0.701
0.789
32.27 dB / 0.946
0.668
26.73 dB / 0.851
0.645
24.93 dB / 0.800
✓ Certified Zhong et al., Nat. Methods 2021
5 CryoGEM + gradient 0.683
0.788
33.21 dB / 0.954
0.656
26.13 dB / 0.836
0.606
23.93 dB / 0.766
✓ Certified He et al., NeurIPS 2023
6 cryoSPARC + gradient 0.646
0.667
25.92 dB / 0.830
0.644
25.76 dB / 0.825
0.626
24.53 dB / 0.787
✓ Certified Punjani et al., Nat. Methods 2017
7 RELION-3D + gradient 0.580
0.640
24.18 dB / 0.775
0.559
22.27 dB / 0.701
0.540
20.92 dB / 0.642
✓ Certified Scheres, J. Struct. Biol. 2012
8 IsoNet + gradient 0.574
0.723
28.48 dB / 0.891
0.547
21.67 dB / 0.676
0.453
18.63 dB / 0.531
✓ Certified Liu et al., Nat. Commun. 2021
9 CTFFIND4 + gradient 0.495
0.556
21.11 dB / 0.651
0.490
19.33 dB / 0.566
0.440
17.93 dB / 0.497
✓ Certified Rohou & Grigorieff, J. Struct. Biol. 2015

Complete score requires all 3 tiers (Public + Dev + Hidden).

Join the competition →
Scoring: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖) PSNR 40% · SSIM 40% · Consistency 20%
Public 5 scenes

Full-access development tier with all data visible.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), spec ranges, ground truth (x_true), and true mismatch spec.

How to use: Load HDF5 → compare reconstruction vs x_true → check consistency → iterate.

What to submit: Reconstructed signals (x_hat) and corrected spec as HDF5.

Public Leaderboard
# Method Score PSNR SSIM
1 DiffusionCryo + gradient 0.842 37.04 0.978
2 CryoSTAR + gradient 0.825 35.48 0.971
3 CryoFormer + gradient 0.811 34.54 0.965
4 cryoDRGN + gradient 0.789 32.27 0.946
5 CryoGEM + gradient 0.788 33.21 0.954
6 IsoNet + gradient 0.723 28.48 0.891
7 cryoSPARC + gradient 0.667 25.92 0.83
8 RELION-3D + gradient 0.640 24.18 0.775
9 CTFFIND4 + gradient 0.556 21.11 0.651
Spec Ranges (4 parameters)
Parameter Min Max Unit
defocus_error -100.0 200.0 nm
astigmatism -20.0 40.0 nm
beam_tilt -0.2 0.4 mrad
ice_thickness_variation 44.0 62.0 nm
View Details →
Dev 5 scenes

Blind evaluation tier — no ground truth available.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), and spec ranges only.

How to use: Apply your pipeline from the Public tier. Use consistency as self-check.

What to submit: Reconstructed signals and corrected spec. Scored server-side.

Dev Leaderboard
# Method Score PSNR SSIM
1 CryoFormer + gradient 0.755 31.13 0.933
2 CryoSTAR + gradient 0.750 31.54 0.937
3 DiffusionCryo + gradient 0.721 29.65 0.911
4 cryoDRGN + gradient 0.668 26.73 0.851
5 CryoGEM + gradient 0.656 26.13 0.836
6 cryoSPARC + gradient 0.644 25.76 0.825
7 RELION-3D + gradient 0.559 22.27 0.701
8 IsoNet + gradient 0.547 21.67 0.676
9 CTFFIND4 + gradient 0.490 19.33 0.566
Spec Ranges (4 parameters)
Parameter Min Max Unit
defocus_error -120.0 180.0 nm
astigmatism -24.0 36.0 nm
beam_tilt -0.24 0.36 mrad
ice_thickness_variation 42.8 60.8 nm
Submit Reconstruction View Details →
Hidden 5 scenes

Fully blind server-side evaluation — no data download.

What you get & how to use

What you get: No data downloadable. Algorithm runs server-side on hidden measurements.

How to use: Package algorithm as Docker container / Python script. Submit via link.

What to submit: Containerized algorithm accepting y + H, outputting x_hat + corrected spec.

Hidden Leaderboard
# Method Score PSNR SSIM
1 CryoSTAR + gradient 0.736 30.58 0.925
2 CryoFormer + gradient 0.687 27.49 0.87
3 DiffusionCryo + gradient 0.667 26.86 0.855
4 cryoDRGN + gradient 0.645 24.93 0.8
5 cryoSPARC + gradient 0.626 24.53 0.787
6 CryoGEM + gradient 0.606 23.93 0.766
7 RELION-3D + gradient 0.540 20.92 0.642
8 IsoNet + gradient 0.453 18.63 0.531
9 CTFFIND4 + gradient 0.440 17.93 0.497
Spec Ranges (4 parameters)
Parameter Min Max Unit
defocus_error -70.0 230.0 nm
astigmatism -14.0 46.0 nm
beam_tilt -0.14 0.46 mrad
ice_thickness_variation 45.8 63.8 nm
Submit Algorithm View Details →

Blind Reconstruction Challenge

Challenge

Given measurements with unknown mismatch and spec ranges (not exact params), reconstruct the original signal. A method must be evaluated on all three tiers for a complete score. Scored on a composite metric: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖).

Input

Measurements y, ideal forward model H, spec ranges

Output

Reconstructed signal x̂

Spec DAG — Forward Model Pipeline

C → D

C Convolution
D Detector

Mismatch Parameters

Symbol Parameter Description Nominal Perturbed
d_e defocus_error Defocus error (nm) 0.0 100.0
a astigmatism Astigmatism (nm) 0.0 20.0
b_t beam_tilt Beam tilt (mrad) 0.0 0.2
i_t ice_thickness_variation Ice thickness variation (nm) 50.0 56.0

Credits System

40%
Platform Profit Pool
Revenue allocated to benchmark rewards
30%
Winner Share
Top algorithm receives from pool
$100
Min Withdrawal
Minimum payout threshold
Spec Primitives Reference (11 primitives)
P Propagation

Free-space or medium propagation kernel (Fresnel, Rayleigh-Sommerfeld).

M Mask / Modulation

Spatial or spatio-temporal amplitude modulation (coded aperture, SLM pattern).

Π Projection

Geometric projection operator (Radon transform, fan-beam, cone-beam).

F Fourier Sampling

Sampling in the Fourier / k-space domain (MRI, ptychography).

C Convolution

Shift-invariant convolution with a point-spread function (PSF).

Σ Summation / Integration

Summation along a physical dimension (spectral, temporal, angular).

D Detector

Sensor readout with gain g and noise model η (Gaussian, Poisson, mixed).

S Structured Illumination

Patterned illumination (block, Hadamard, random) applied to the scene.

W Wavelength Dispersion

Spectral dispersion element (prism, grating) with shift α and aperture a.

R Rotation / Motion

Sample or gantry rotation (CT, electron tomography).

Λ Wavelength Selection

Spectral filter or monochromator selecting a wavelength band.