Contribute to Benchmark

STEM — Help grow the evaluation suite

Why Contribute?

The benchmark grows stronger with community data. More diverse datasets lead to fairer evaluation and more robust algorithms. Your contribution helps the entire computational imaging community.

All contributed datasets are reviewed by the PWM team (admin: platformaigpt@gmail.com) before being added to the benchmark. We will provide feedback on your submission.

Submit a Dataset

Choose which tier you want to contribute data to:

Public

Submit datasets with measurements, ground truth, and full spec parameters. Public data helps contestants develop and debug their algorithms.

Required: Measurements (y), forward model (H), ground truth (x), true spec parameters.
Dev

Submit datasets with measurements and spec ranges but without ground truth. Dev data is used to score reconstruction submissions.

Required: Measurements (y), forward model (H), spec ranges. Ground truth held server-side for scoring.
Hidden

Submit datasets kept entirely server-side for blind evaluation. Hidden data ensures fair competition with fully unknown parameters.

Required: Complete dataset with ground truth. All data remains private and is only used for blind evaluation.

Dataset Format Reference

Challenge datasets use HDF5 format with the following structure. Download the example files below to see the exact schema your contributed data should follow.

HDF5 Structure

/sample_00/
  ├── y              — measurements array (sinogram, k-space, etc.)
  ├── H_ideal        — ideal forward model (angles, mask, PSF, etc.)
  ├── x_true         — ground truth signal (public + hidden tiers only)
  ├── spec_ranges    — JSON attr: [{"name", "min", "max", "unit"}, ...]
  ├── metadata       — JSON attr: {"scene", "shape", "noise_model"}
  └── true_spec      — JSON attr: {"param": value} (public + hidden only)

/sample_01/
  └── ...

File attributes:
  variant       — variant key (e.g., "ct", "mri", "cassi")
  tier          — "public", "dev", or "hidden"
  version       — schema version ("1.0")
  runner_type   — forward model type ("radon", "kspace", "psf", ...)

Submission HDF5 Structure

/sample_00/
  ├── x_hat          — reconstructed signal (same shape as x_true)
  └── corrected_spec — JSON attr: {"param": estimated_value, ...}

Example Downloads

CT Examples
ct_example_public.h5 (with ground truth) ct_example_hidden.h5 (server-side format)
MRI Examples
mri_example_public.h5 (with ground truth) mri_example_hidden.h5 (server-side format)

Quick Upload

Upload your dataset file directly. For detailed submission options, use the tier-specific buttons above.

Accepted formats: .h5, .hdf5, .npy, .npz, .zip (max 500 MB)

Submission Guidelines

  • Datasets should be in HDF5 format (.h5 / .hdf5) or NumPy archives (.npy / .npz).
  • Maximum file size: 50 MB. For larger datasets, contact the team.
  • Include a clear description of the imaging setup, number of samples, and data format.
  • If you have an associated paper or code repository, include the URLs.
  • All contributions will be reviewed by the PWM team and you will receive feedback.
Back to STEM