Coherent Anti-Stokes Raman (CARS) Microscopy

Join the Competition Contribute to Benchmark

Standard reconstruction benchmark — forward model perfectly known, no calibration needed. Score = 0.5 × clip((PSNR−15)/30, 0, 1) + 0.5 × SSIM

#	Method	Score	PSNR (dB)	SSIM	Source
🥇	FMDiff-CARS FMDiff-CARS Li et al., NeurIPS 2024 40.2 dB SSIM 0.966 Checkpoint unavailable	0.903	40.2	0.966	✓ Certified	Li et al., NeurIPS 2024
🥈	Diff-CARS Diff-CARS Zhang et al., Nat. Methods 2024 39.1 dB SSIM 0.958 Checkpoint unavailable	0.881	39.1	0.958	✓ Certified	Zhang et al., Nat. Methods 2024
🥉	SpecFormer-CARS SpecFormer-CARS Liao et al., Light Sci. Appl. 2023 37.8 dB SSIM 0.947 Checkpoint unavailable	0.853	37.8	0.947	✓ Certified	Liao et al., Light Sci. Appl. 2023
4	ResNet-CARS ResNet-CARS Ying et al., Optica 2022 36.2 dB SSIM 0.933 Checkpoint unavailable	0.820	36.2	0.933	✓ Certified	Ying et al., Optica 2022
5	PINN-CARS PINN-CARS Bae et al., ACS Photonics 2021 34.8 dB SSIM 0.918 Checkpoint unavailable	0.789	34.8	0.918	✓ Certified	Bae et al., ACS Photonics 2021
6	U-Net-CARS U-Net-CARS Manifold et al., Nat. Mach. Intell. 2021 33.5 dB SSIM 0.902 Checkpoint unavailable	0.759	33.5	0.902	✓ Certified	Manifold et al., Nat. Mach. Intell. 2021
7	CNN-NRB CNN-NRB Houhou et al., Chem. Sci. 2020 30.8 dB SSIM 0.865 Checkpoint unavailable	0.696	30.8	0.865	✓ Certified	Houhou et al., Chem. Sci. 2020
8	MEM-CARS MEM-CARS Rinia et al., J. Raman Spectrosc. 2008 26.2 dB SSIM 0.798 Try in SpecLab →	0.586	26.2	0.798	✓ Certified	Rinia et al., J. Raman Spectrosc. 2008
9	KK-Retrieval KK-Retrieval Liu et al., Opt. Express 2009 24.5 dB SSIM 0.762 Try in SpecLab →	0.539	24.5	0.762	✓ Certified	Liu et al., Opt. Express 2009

Dataset: PWM Benchmark (9 algorithms)

Blind Reconstruction Challenge — forward model has unknown mismatch, must calibrate from data. Score = 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖)

#	Method	Overall Score	Public PSNR / SSIM	Dev PSNR / SSIM	Hidden PSNR / SSIM	Trust	Source
🥇	FMDiff-CARS + gradient FMDiff-CARS + gradient Li et al., NeurIPS 2024 Score 0.789 Correct & Reconstruct →	0.789	0.869 39.09 dB / 0.985	0.764 31.65 dB / 0.939	0.733 29.86 dB / 0.915	✓ Certified	Li et al., NeurIPS 2024
🥈	SpecFormer-CARS + gradient SpecFormer-CARS + gradient Liao et al., Light Sci. Appl. 2023 Score 0.762 Correct & Reconstruct →	0.762	0.841 36.47 dB / 0.976	0.751 31.82 dB / 0.941	0.695 27.8 dB / 0.877	✓ Certified	Liao et al., Light Sci. Appl. 2023
🥉	ResNet-CARS + gradient ResNet-CARS + gradient Ying et al., Optica 2022 Score 0.733 Correct & Reconstruct →	0.733	0.799 33.94 dB / 0.960	0.731 29.74 dB / 0.913	0.669 27.38 dB / 0.867	✓ Certified	Ying et al., Optica 2022
4	Diff-CARS + gradient Diff-CARS + gradient Zhang et al., Nat. Methods 2024 Score 0.728 Correct & Reconstruct →	0.728	0.836 36.75 dB / 0.977	0.703 28.49 dB / 0.891	0.646 25.21 dB / 0.809	✓ Certified	Zhang et al., Nat. Methods 2024
5	PINN-CARS + gradient PINN-CARS + gradient Bae et al., ACS Photonics 2021 Score 0.703 Correct & Reconstruct →	0.703	0.801 33.17 dB / 0.954	0.665 26.66 dB / 0.850	0.644 25.7 dB / 0.823	✓ Certified	Bae et al., ACS Photonics 2021
6	U-Net-CARS + gradient U-Net-CARS + gradient Manifold et al., Nat. Mach. Intell. 2021 Score 0.646 Correct & Reconstruct →	0.646	0.786 32.19 dB / 0.945	0.608 23.22 dB / 0.740	0.545 21.44 dB / 0.666	✓ Certified	Manifold et al., Nat. Mach. Intell. 2021
7	CNN-NRB + gradient CNN-NRB + gradient Houhou et al., Chem. Sci. 2020 Score 0.642 Correct & Reconstruct →	0.642	0.743 29.29 dB / 0.905	0.626 24.59 dB / 0.789	0.558 22.36 dB / 0.705	✓ Certified	Houhou et al., Chem. Sci. 2020
8	MEM-CARS + gradient MEM-CARS + gradient Rinia et al., J. Raman Spectrosc. 2008 Score 0.604 Correct & Reconstruct →	0.604	0.626 24.18 dB / 0.775	0.601 23.25 dB / 0.741	0.586 22.82 dB / 0.724	✓ Certified	Rinia et al., J. Raman Spectrosc. 2008
9	KK-Retrieval + gradient KK-Retrieval + gradient Liu et al., Opt. Express 2009 Score 0.575 Correct & Reconstruct →	0.575	0.610 23.07 dB / 0.734	0.575 22.64 dB / 0.717	0.540 21.77 dB / 0.680	✓ Certified	Liu et al., Opt. Express 2009

Complete score requires all 3 tiers (Public + Dev + Hidden).

Join the competition →

Scoring: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖) PSNR 40% · SSIM 40% · Consistency 20%

Public 5 scenes

Full-access development tier with all data visible.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), spec ranges, ground truth (x_true), and true mismatch spec.

How to use: Load HDF5 → compare reconstruction vs x_true → check consistency → iterate.

What to submit: Reconstructed signals (x_hat) and corrected spec as HDF5.

Public Leaderboard

#	Method	Score	PSNR	SSIM
1	FMDiff-CARS + gradient	0.869	39.09	0.985
2	SpecFormer-CARS + gradient	0.841	36.47	0.976
3	Diff-CARS + gradient	0.836	36.75	0.977
4	PINN-CARS + gradient	0.801	33.17	0.954
5	ResNet-CARS + gradient	0.799	33.94	0.96
6	U-Net-CARS + gradient	0.786	32.19	0.945
7	CNN-NRB + gradient	0.743	29.29	0.905
8	MEM-CARS + gradient	0.626	24.18	0.775
9	KK-Retrieval + gradient	0.610	23.07	0.734

Spec Ranges (3 parameters)

Parameter	Min	Max	Unit
pump_stokes_frequency_offset	-1.0	2.0	cm^-1
non_resonant_background	-10.0	20.0	-
chirp_mismatch	-100.0	200.0	fs^2

View Details →

Dev 5 scenes

Blind evaluation tier — no ground truth available.

What you get & how to use

What you get: Measurements (y), ideal forward operator (H), and spec ranges only.

How to use: Apply your pipeline from the Public tier. Use consistency as self-check.

What to submit: Reconstructed signals and corrected spec. Scored server-side.

Dev Leaderboard

#	Method	Score	PSNR	SSIM
1	FMDiff-CARS + gradient	0.764	31.65	0.939
2	SpecFormer-CARS + gradient	0.751	31.82	0.941
3	ResNet-CARS + gradient	0.731	29.74	0.913
4	Diff-CARS + gradient	0.703	28.49	0.891
5	PINN-CARS + gradient	0.665	26.66	0.85
6	CNN-NRB + gradient	0.626	24.59	0.789
7	U-Net-CARS + gradient	0.608	23.22	0.74
8	MEM-CARS + gradient	0.601	23.25	0.741
9	KK-Retrieval + gradient	0.575	22.64	0.717

Spec Ranges (3 parameters)

Parameter	Min	Max	Unit
pump_stokes_frequency_offset	-1.2	1.8	cm^-1
non_resonant_background	-12.0	18.0	-
chirp_mismatch	-120.0	180.0	fs^2

Submit Reconstruction View Details →

Hidden 5 scenes

Fully blind server-side evaluation — no data download.

What you get & how to use

What you get: No data downloadable. Algorithm runs server-side on hidden measurements.

How to use: Package algorithm as Docker container / Python script. Submit via link.

What to submit: Containerized algorithm accepting y + H, outputting x_hat + corrected spec.

Hidden Leaderboard

#	Method	Score	PSNR	SSIM
1	FMDiff-CARS + gradient	0.733	29.86	0.915
2	SpecFormer-CARS + gradient	0.695	27.8	0.877
3	ResNet-CARS + gradient	0.669	27.38	0.867
4	Diff-CARS + gradient	0.646	25.21	0.809
5	PINN-CARS + gradient	0.644	25.7	0.823
6	MEM-CARS + gradient	0.586	22.82	0.724
7	CNN-NRB + gradient	0.558	22.36	0.705
8	U-Net-CARS + gradient	0.545	21.44	0.666
9	KK-Retrieval + gradient	0.540	21.77	0.68

Spec Ranges (3 parameters)

Parameter	Min	Max	Unit
pump_stokes_frequency_offset	-0.7	2.3	cm^-1
non_resonant_background	-7.0	23.0	-
chirp_mismatch	-70.0	230.0	fs^2

Submit Algorithm View Details →

Blind Reconstruction Challenge

Challenge

Given measurements with unknown mismatch and spec ranges (not exact params), reconstruct the original signal. A method must be evaluated on all three tiers for a complete score. Scored on a composite metric: 0.4 × PSNR_norm + 0.4 × SSIM + 0.2 × (1 − ‖y − Ĥx̂‖/‖y‖).

Input

Measurements y, ideal forward model H, spec ranges

Output

Reconstructed signal x̂

Spec DAG — Forward Model Pipeline

M → R → D

M Modulation

R Rotation

D Detector

Mismatch Parameters

Symbol	Parameter	Description	Perturbed
p_f	pump_stokes_frequency_offset	Pump-Stokes frequency offset (cm^-1)	1.0
n_b	non_resonant_background	Non-resonant background (-)	10.0
c_m	chirp_mismatch	Chirp mismatch (fs^2)	100.0

Credits System

40%

Platform Profit Pool

Revenue allocated to benchmark rewards

30%

Winner Share

Top algorithm receives from pool

$100

Min Withdrawal

Minimum payout threshold

Spec Primitives Reference (11 primitives)

P Propagation

Free-space or medium propagation kernel (Fresnel, Rayleigh-Sommerfeld).

M Mask / Modulation

Spatial or spatio-temporal amplitude modulation (coded aperture, SLM pattern).

Π Projection

Geometric projection operator (Radon transform, fan-beam, cone-beam).

F Fourier Sampling

Sampling in the Fourier / k-space domain (MRI, ptychography).

C Convolution

Shift-invariant convolution with a point-spread function (PSF).

Σ Summation / Integration

Summation along a physical dimension (spectral, temporal, angular).

D Detector

Sensor readout with gain g and noise model η (Gaussian, Poisson, mixed).

S Structured Illumination

Patterned illumination (block, Hadamard, random) applied to the scene.

W Wavelength Dispersion

Spectral dispersion element (prism, grating) with shift α and aperture a.

R Rotation / Motion

Sample or gantry rotation (CT, electron tomography).

Λ Wavelength Selection

Spectral filter or monochromator selecting a wavelength band.