Physics World Model — Modality Catalog

Principle

3-D Gaussian Splatting represents a scene as a set of anisotropic 3-D Gaussians, each with position, covariance, opacity, and spherical harmonics color coefficients. Novel views are rendered by projecting (splatting) these Gaussians onto the image plane and alpha-compositing them in depth order. Unlike NeRF, rendering is rasterization-based and achieves real-time frame rates (≥100 fps) with high visual quality.

How to Build the System

Start with the same multi-view image dataset as NeRF (50-200 posed images via COLMAP). Initialize 3-D Gaussians from the SfM point cloud. Train by differentiable rasterization: project Gaussians to each training view, compute photometric loss (L1 + SSIM), and optimize positions, covariances, colors, and opacities via Adam. Adaptive densification (splitting/cloning Gaussians) and pruning runs periodically during training. Training takes ~15-30 minutes on a modern GPU.

Common Reconstruction Algorithms

• 3D Gaussian Splatting (original, Kerbl et al. 2023)
• Mip-Splatting (anti-aliased multi-scale Gaussian splatting)
• SuGaR (Surface-Aligned Gaussian Splatting for mesh extraction)
• Dynamic 3D Gaussians (for dynamic scenes / video)
• Compact-3DGS (compressed Gaussian representations)

Principle

4D-STEM electron diffraction scans a convergent electron beam across the specimen and records a full 2-D diffraction pattern (convergent beam electron diffraction, CBED) at each scan position. The resulting 4-D dataset (2-D scan × 2-D diffraction) enables mapping of crystal structure, orientation, strain, electric fields, and charge density with nanometer spatial resolution.

How to Build the System

Use a STEM equipped with a fast pixelated detector (Medipix3, EMPAD, or Dectris ARINA) capable of recording diffraction patterns at >1000 fps. Set a small convergence semi-angle (1-5 mrad) for nanobeam diffraction or large (20-30 mrad) for CBED. The scan step should be comparable to the probe size. Data volumes are large (tens of GB per scan), requiring efficient data pipeline and storage.

Common Reconstruction Algorithms

• Virtual detector imaging (synthesized BF, DF, iDPC from 4D data)
• Center-of-mass (COM) analysis for electric field mapping
• Ptychographic reconstruction from 4D-STEM data
• Orientation mapping (template matching against simulated patterns)
• Strain mapping via disk position analysis

Principle

Coded Aperture Compressive Temporal Imaging (CACTI) compresses multiple high-speed video frames into a single sensor exposure by modulating the scene with a dynamic coded aperture (shifting mask) during the integration time. The sensor accumulates a coded sum of B consecutive frames, and computational algorithms recover all B frames from the single compressed measurement using video sparsity priors.

How to Build the System

Build a relay optical system with a physical translating mask or use a DMD as the coded aperture at an intermediate image plane. The mask shifts by one pixel per sub-frame interval during the camera integration time, effectively encoding B temporal frames. Use a standard camera at normal frame rate (e.g., 30 fps) to capture the compressed measurement. Calibrate the mask pattern and its motion precisely.

Common Reconstruction Algorithms

• GAP-TV (Generalized Alternating Projection with Total Variation)
• DeSCI (Decompress Snapshot Compressive Imaging, GMM prior)
• PnP-FFDNet (Plug-and-Play with FFDNet denoiser)
• Deep unfolding: BIRNAT, RevSCI, EfficientSCI
• E2E-trained networks: STFormer, CST (transformer-based)

Principle

Coded Aperture Snapshot Spectral Imaging (CASSI) captures a full 3-D spectral datacube (x, y, λ) in a single 2-D snapshot by encoding the scene with a binary coded aperture and spectrally dispersing it with a prism onto the detector. Different spectral channels are shifted and superimposed on the sensor, creating a compressed measurement. Computational algorithms recover the full datacube from this single measurement using sparsity priors.

How to Build the System

Build an optical relay with an objective lens, place a binary coded aperture (lithographic chrome-on-glass mask or DMD) at an intermediate image plane, then disperse with an Amici or double-Amici prism, and re-image onto a high-resolution detector (2048× 2048+ pixels). Precisely calibrate the spectral dispersion curve (nm/pixel). The coded aperture pattern should have ~50 % transmittance and good conditioning.

Common Reconstruction Algorithms

• TwIST (Two-step Iterative Shrinkage/Thresholding)
• GAP-TV (Generalized Alternating Projection with Total Variation)
• ADMM with sparsity in DCT or wavelet domain
• Deep unfolding networks (DGSMP, TSA-Net, BIRNAT)
• Plug-and-Play ADMM with learned denoisers

Principle

Coherent Diffractive Imaging (CDI) records the far-field diffraction pattern of an isolated object illuminated by a coherent beam. Only intensity (not phase) is measured on the detector. Phase retrieval algorithms iteratively recover the lost phase by enforcing known constraints: the measured Fourier modulus and the finite support of the object in real space. CDI achieves diffraction-limited resolution without any imaging lens.

How to Build the System

Illuminate an isolated object (nanocrystal, cell, virus particle) with a coherent, quasi-plane-wave beam (X-ray from synchrotron or XFEL, or visible laser). Record the continuous diffraction pattern on a pixel detector (Eiger, Jungfrau for X-ray; CMOS for visible) placed far enough for adequate oversampling (oversampling ratio ≥ 2 in each dimension). Remove the direct beam with a beam stop. Ensure the object is isolated (no other scatterers in the beam).

Common Reconstruction Algorithms

• Hybrid Input-Output (HIO) algorithm
• Error Reduction (ER) algorithm
• Shrink-Wrap (adaptive support HIO)
• Relaxed Averaged Alternating Reflections (RAAR)
• Deep-learning phase retrieval (PhaseDNN, learned proximal operator)

Principle

Cone-Beam CT uses a divergent cone-shaped X-ray beam and a 2-D flat-panel detector to acquire a volumetric CT dataset in a single rotation. Unlike multi-slice CT with a narrow fan beam, CBCT covers the full volume simultaneously, enabling faster acquisition but with increased scatter and cone-beam artifacts compared to conventional CT.

How to Build the System

Mount a flat-panel detector (typically 30×40 cm, CsI scintillator) opposite an X-ray tube on a rotating gantry or C-arm. Common implementations: dental CBCT (small FOV, 90 kVp), image-guided radiation therapy CBCT (kV source on linac gantry), and C-arm CBCT (interventional). Calibrate: geometric parameters (source-detector distances, isocenter), detector offset corrections, and scatter correction LUTs.

Common Reconstruction Algorithms

• FDK (Feldkamp-Davis-Kress) cone-beam filtered back-projection
• Iterative CBCT (SART, SIRT with cone-beam projector)
• Scatter correction (measurement-based or Monte Carlo simulation)
• Motion-compensated CBCT (4D-CBCT for respiratory motion)
• Deep-learning CBCT-to-CT synthesis for radiation therapy planning

Principle

Same confocal principle as live-cell mode but acquiring a full z-stack by stepping the objective or sample through the focal plane. Each optical section is convolved with the 3-D confocal PSF, and the full volume is reconstructed by 3-D deconvolution to recover isotropic resolution.

How to Build the System

Use a high-NA objective (60-100x, 1.4 NA oil or 1.2 NA water) with a piezo z-stage for precise, repeatable z-steps (typ. 200-300 nm). Acquire z-stacks covering the specimen thickness with Nyquist z-sampling. For fixed samples, oil immersion is preferred; for thick tissue, use silicone oil or glycerol objectives to minimize RI mismatch deep in the sample.

Common Reconstruction Algorithms

• 3-D Richardson-Lucy deconvolution
• 3-D Wiener / Tikhonov deconvolution
• Huygens Professional iterative deconvolution
• DeconvolutionLab2 (GPU-accelerated 3-D)
• Deep-learning volumetric restoration (3-D U-Net, RCAN3D)

Principle

A focused laser spot is scanned across the specimen and a pinhole in front of the detector rejects out-of-focus fluorescence, providing optical sectioning. The image formation is modeled as a point-by-point convolution with the confocal PSF (product of excitation and detection PSFs). For live-cell work, speed and gentleness are prioritized.

How to Build the System

Equip a laser-scanning confocal head (e.g., Nikon A1R, Zeiss LSM 980 Airyscan) on an inverted microscope with an environmental enclosure. Use a resonant scanner for fast (30 fps) imaging. Set pinhole to 1 Airy unit for best sectioning or open slightly (1.2 AU) for more signal. Use 40-60x water-immersion objectives for live cells to match RI of aqueous media.

Common Reconstruction Algorithms

• Airyscan joint deconvolution (Zeiss)
• Richardson-Lucy with measured confocal PSF
• Sparse deconvolution (Hessian regularization)
• Deep-learning denoising (Noise2Fast, DnCNN)
• Pixel reassignment (ISM) for resolution doubling

Principle

Diffuse Optical Tomography reconstructs 3-D maps of tissue optical properties (absorption μₐ and reduced scattering μ'ₛ) from measurements of multiply scattered near-infrared light transmitted through tissue. Multiple source-detector pairs on the tissue surface provide overlapping sensitivity profiles. The diffusion equation models light propagation in the multiple-scattering regime.

How to Build the System

Place fiber-coupled NIR sources (670-850 nm laser diodes, CW or frequency-domain modulated at 100-300 MHz, or time-domain pulsed) and detector fibers (avalanche photodiodes or PMTs) on the tissue surface in an array. A multiplexer switches between source positions. For breast DOT, 32-128 optode positions on a cup or ring geometry. Calibrate with known optical phantoms (Intralipid + ink solutions).

Common Reconstruction Algorithms

• Normalized Born approximation (linearized diffuse optical tomography)
• Nonlinear Newton-type iterative reconstruction (Gauss-Newton, Levenberg-Marquardt)
• Finite-element method (FEM) based forward solver + Tikhonov regularization
• TOAST++ (Time-resolved Optical Absorption and Scattering Tomography)
• Deep-learning DOT (learned regularization, direct inversion networks)

Principle

Diffusion MRI sensitizes the MR signal to the Brownian motion of water molecules by applying strong magnetic field gradient pulses (Stejskal-Tanner scheme). In fibrous tissue (e.g., white matter), water diffuses preferentially along fibers, creating directional diffusion anisotropy. Diffusion Tensor Imaging (DTI) models this as a 3×3 tensor; higher-order models (HARDI, CSD) resolve crossing fibers.

How to Build the System

Acquire on a 3T scanner with high-performance gradients (80 mT/m, 200 T/m/s). Use spin-echo EPI with multiple b-values (e.g., b=0, 1000, 2000 s/mm²) and 30-300 diffusion directions uniformly distributed on the sphere. Include reverse-phase-encode b=0 images for EPI distortion correction. Multi-band (SMS) acceleration reduces scan time. Typical parameters: 2 mm isotropic, TE 60-90 ms, TR 3-5 s.

Common Reconstruction Algorithms

• DTI tensor fitting (least-squares or weighted least-squares)
• CSD (Constrained Spherical Deconvolution) for fiber orientation distribution
• NODDI (Neurite Orientation Dispersion and Density Imaging)
• Probabilistic tractography (FSL probtrackx, MRtrix3 iFOD2)
• Deep-learning tract segmentation (TractSeg, DeepBundle)

Principle

Digital holographic microscopy records the interference pattern (hologram) between a reference wave and the wave scattered by the sample. The complex field (amplitude and phase) is recovered by numerical propagation of the recorded hologram to the object plane. Phase imaging reveals optical path length changes caused by refractive index or thickness variations, providing quantitative phase contrast without staining.

How to Build the System

Build an off-axis Mach-Zehnder interferometer: split a coherent source (He-Ne laser, 633 nm, or laser diode) into object and reference beams. The object beam passes through the sample via a microscope objective. The reference beam tilts at a small angle (off-axis) to create carrier fringes. Both beams interfere on a CMOS camera. The carrier frequency must be high enough to separate the twin image in Fourier space. Vibration isolation is essential.

Common Reconstruction Algorithms

• Fourier filtering (off-axis hologram: spatial filtering of +1 order)
• Angular spectrum propagation method
• Phase unwrapping (Goldstein, quality-guided, or least-squares)
• Numerical autofocusing (Tamura coefficient, Brenner gradient)
• Deep-learning phase retrieval (PhaseNet, holographic reconstruction CNN)

Principle

Doppler ultrasound measures blood flow velocity by detecting the frequency shift of echoes reflected from moving red blood cells. The Doppler equation relates the frequency shift to velocity: Δf = 2f₀·v·cos(θ)/c, where θ is the beam-flow angle. Color Doppler maps velocity spatially, spectral Doppler provides velocity-time waveforms at a sample volume, and power Doppler shows flow amplitude regardless of direction.

How to Build the System

Use a clinical ultrasound system with Doppler capability. For vascular studies, use a linear array transducer (5-12 MHz). Steer the beam to achieve a Doppler angle <60° to the vessel axis. Set the velocity scale (PRF) to match expected flow speeds (avoid aliasing). For spectral Doppler, place the sample volume within the vessel lumen and adjust the gate size. Angle correction must be applied for accurate velocity measurements.

Common Reconstruction Algorithms

• Autocorrelation-based color flow estimation (Kasai algorithm)
• FFT spectral analysis for pulsed-wave Doppler
• Clutter filtering (wall filtering) to remove tissue motion
• Power Doppler (amplitude mode) for slow flow detection
• Ultrafast Doppler (plane-wave compounding) for functional ultrasound

Principle

Dual-Energy X-ray Absorptiometry uses two X-ray beam energies to decompose the body into bone mineral and soft tissue compartments. The differential attenuation of the two energies allows separation of bone from soft tissue. Bone mineral density (BMD, g/cm²) is computed by comparing attenuation to calibration phantoms.

How to Build the System

A DEXA scanner (Hologic Discovery/Horizon or GE Lunar) uses a fan-beam or pencil-beam X-ray source with two energies (typically 70 and 140 kVp, or k-edge filtration). The detector is directly opposite the source below the patient table. Daily quality assurance with a calibration phantom (anthropomorphic spine) is mandatory. Cross-calibration is needed when changing scanners. Scan modes include AP spine, dual femur, whole body, and lateral vertebral assessment.

Common Reconstruction Algorithms

• Dual-energy decomposition (two-material model: bone + soft tissue)
• Edge detection for region-of-interest (ROI) identification
• BMD calculation relative to calibration phantom
• T-score / Z-score computation against normative databases
• Body composition analysis (lean mass, fat mass from whole-body scans)

Principle

Electron Backscatter Diffraction (EBSD) maps the crystallographic orientation of polycrystalline materials at each surface point. A focused electron beam (15-30 keV) strikes a tilted (70°) polished specimen, generating backscattered electrons that form Kikuchi diffraction patterns on a phosphor screen/CMOS camera. Automated pattern indexing determines the crystal orientation at each point with ~0.5° angular resolution.

How to Build the System

Install an EBSD detector (phosphor screen + CCD/CMOS camera, e.g., Oxford Instruments Symmetry, EDAX Velocity) in an SEM chamber. Tilt the specimen to 70° toward the detector. Polish the sample surface to remove any deformation layer (final step: colloidal silica or ion milling). Set accelerating voltage 15-30 kV, high probe current (1-20 nA). Map with step sizes of 50 nm to 5 μm depending on grain size.

Common Reconstruction Algorithms

• Hough transform band detection for Kikuchi pattern indexing
• Dictionary indexing (template matching against simulated patterns)
• Spherical indexing (GPU-accelerated orientation determination)
• Neighbor pattern averaging and reindexing (NPAR) for noisy patterns
• Deep-learning EBSD pattern indexing (faster and more robust than Hough)

Principle

Electron Energy Loss Spectroscopy measures the energy lost by transmitted electrons due to inelastic interactions with the specimen. The energy-loss spectrum contains characteristic edges corresponding to inner-shell ionization of specific elements, enabling elemental mapping with atomic spatial resolution. Near-edge fine structure (ELNES) reveals chemical bonding, and low-loss features probe band structure and optical properties.

How to Build the System

Attach a post-column energy filter (Gatan GIF Quantum/Continuum) to a TEM/STEM. For STEM-EELS spectrum imaging: scan the probe and record a full energy-loss spectrum (0-2000 eV range) at each pixel. Use a monochromated source (ΔE < 0.3 eV) for near-edge fine structure studies. Energy dispersion is typically 0.1-0.5 eV/channel. Acquire both core-loss edges (elemental maps) and low-loss region (thickness mapping, optical properties).

Common Reconstruction Algorithms

• Background subtraction (power-law fitting before edge onset)
• Multiple linear least-squares (MLLS) fitting for overlapping edges
• Principal component analysis (PCA) for denoising spectrum images
• Kramers-Kronig analysis for optical constants from low-loss EELS
• Deep-learning EELS denoising and quantification

Principle

Electron holography uses the interference between an object wave (transmitted through the specimen) and a reference wave (passing through vacuum) to record both amplitude and phase of the electron wave. An electrostatic biprism (charged wire) deflects the two waves to overlap and form interference fringes. Numerical reconstruction recovers the phase shift, which is sensitive to electrostatic potentials and magnetic fields in the specimen.

How to Build the System

Use a TEM (≥200 kV, FEG source for high coherence) equipped with an electron biprism (a thin metallized quartz fiber at adjustable voltage 50-300 V). Position the specimen so one half of the biprism overlaps the specimen edge and the other half is in vacuum. Record the hologram on a direct-electron detector. Fringe spacing should be 3-4× the desired resolution. Acquire reference holograms (empty) for normalization.

Common Reconstruction Algorithms

• Fourier filtering (sideband extraction and inverse FFT for phase/amplitude)
• Phase unwrapping for large phase shifts (>2π)
• Mean inner potential measurement from phase maps
• Magnetic induction mapping (from phase gradient of Lorentz holography)
• In-line holography (through-focus series) with transport-of-intensity equation

Principle

Electron tomography reconstructs a 3-D volume from a tilt series of 2-D TEM or STEM projections acquired at different specimen tilts (typically ±60-70°). The Radon transform (or its generalization) relates the projections to the 3-D structure. The limited tilt range causes a 'missing wedge' artifact — elongation in the beam direction — which must be addressed by regularization or dual-axis acquisition.

How to Build the System

Use a TEM/STEM with a high-tilt specimen holder (±70-80°). Acquire images at tilt increments of 1-2° across the full range. For STEM tomography, HAADF signal provides monotonic contrast (no CTF complications). Include gold nanoparticles as fiducial markers for alignment. Automated acquisition software (SerialEM, Tomography by Thermo Fisher) controls stage tilt, focus tracking, and image acquisition.

Common Reconstruction Algorithms

• Weighted back-projection (WBP)
• SIRT / SART (Simultaneous Iterative Reconstruction Techniques)
• GENFIRE (GENeralized Fourier Iterative REconstruction)
• Compressed sensing tomography for missing-wedge artifact reduction
• Deep-learning tomographic reconstruction (TomoGAN, DeepRecon)

Principle

Fiber-bundle endoscopy transmits an image through a flexible coherent fiber bundle (10,000-100,000 individual fiber cores) to visualize internal body cavities. Each fiber core acts as a single pixel, transmitting light from the distal end to the proximal end where a camera captures the image. The hexagonal fiber packing imposes a fixed pixelation pattern (comb/honeycomb structure) on the image.

How to Build the System

A medical endoscope has a flexible insertion tube containing the coherent fiber bundle (or a distal CMOS chip for video endoscopes), illumination fibers, working channels, and air/water channels. Light source: LED or Xenon lamp transmitted through illumination fibers. For fiber-bundle type: attach a high-resolution camera and relay lens at the proximal end. Calibrate fiber core positions and individual fiber transmission for computational image improvement.

Common Reconstruction Algorithms

• Fiber core mapping and interpolation (honeycomb artifact removal)
• Deep-learning super-resolution for fiber-bundle images
• Structure-from-motion for endoscopic 3-D reconstruction
• Defogging / dehazing for underwater or smoke-obscured endoscopy
• Real-time mosaicking for extended field-of-view endoscopy

Principle

Fluorescence Lifetime Imaging measures the exponential decay time of fluorophore emission (typically 1-10 ns) rather than intensity. Lifetime is sensitive to the fluorophore's local chemical environment (pH, ion concentration, FRET) but independent of concentration and photobleaching. Detection uses either time-correlated single-photon counting (TCSPC) or frequency-domain phase/modulation methods.

How to Build the System

Add a pulsed laser source (ps diode laser or Ti:Sapphire, 40-80 MHz repetition rate) to a confocal or widefield microscope. For TCSPC, install single-photon counting detectors (hybrid PMTs or SPADs) with timing electronics (Becker & Hickl SPC-150/830 or PicoQuant TimeHarp). For widefield FLIM, use a gated or modulated camera (Lambert Instruments). Synchronize laser pulses with detector timing.

Common Reconstruction Algorithms

• Mono-exponential / bi-exponential tail fitting (least-squares or MLE)
• Phasor analysis (model-free lifetime decomposition)
• Global analysis (linked lifetime fitting across pixels)
• Bayesian lifetime estimation
• Deep-learning FLIM (FLIMnet, rapid lifetime prediction from few photons)

Principle

Fluoroscopy provides real-time continuous X-ray imaging for guiding interventional procedures. A pulsed or continuous X-ray beam produces live projection images at 7.5-30 fps on a flat-panel detector. The trade-off is between frame rate, radiation dose, and image quality. Temporal filtering and dose-saving modes reduce patient exposure while maintaining diagnostic quality.

How to Build the System

A C-arm fluoroscopy unit has an X-ray tube and flat-panel detector on a C-shaped gantry that can rotate around the patient. Modern systems use pulsed fluoroscopy (variable pulse rate 3.75-30 fps) with automatic brightness control. Install last-image-hold and virtual collimation features. Calibrate geometric distortion for 3-D cone-beam reconstruction capability. Regular dosimetry checks (DAP meter calibration) are mandatory.

Common Reconstruction Algorithms

• Recursive temporal averaging (IIR filtering for noise reduction)
• Contrast-enhanced subtraction (road-mapping for angiography)
• Motion-compensated temporal filtering
• Cone-beam CT reconstruction from rotational fluoroscopy runs
• Deep-learning frame interpolation for reduced pulse-rate operation

Principle

Fourier Ptychographic Microscopy synthetically increases the NA of a low-magnification objective by illuminating the sample from multiple angles (LED array) and computationally stitching together the resulting images in Fourier space. Each LED angle shifts the sample spectrum so different spatial-frequency bands enter the objective pupil, allowing recovery of both amplitude and phase at high resolution over a large field of view.

How to Build the System

Replace the microscope condenser with a programmable LED matrix (e.g., 32×32 RGB LED array, ~4 mm pitch, placed ~80 mm above the sample). Use a low-magnification objective (4-10×, 0.1-0.3 NA) for large FOV. Acquire one image per LED (typically 100-300 images for the full matrix). Precise knowledge of LED positions is required for Fourier-space stitching.

Common Reconstruction Algorithms

• Alternating projection (Gerchberg-Saxton style in Fourier space)
• Embedded pupil function recovery (joint sample + aberration estimation)
• Wirtinger gradient descent with total-variation regularization
• Neural network-accelerated FPM (learned initialization + refinement)
• Multiplexed FPM (multiple LEDs simultaneously for faster acquisition)

Principle

Functional MRI detects brain activity indirectly through the Blood Oxygen Level Dependent (BOLD) contrast mechanism. Neural activity increases local blood flow and oxygenation, changing the ratio of diamagnetic oxyhemoglobin to paramagnetic deoxyhemoglobin. This alters the local T2* relaxation time, producing a small (~1-5 %) signal change detectable by gradient-echo EPI sequences acquired rapidly at whole-brain coverage.

How to Build the System

Use a 3T MRI scanner with a 32-64 channel head coil. Acquire multi-band (simultaneous multi-slice) gradient-echo EPI sequences (TR 0.5-1.5 s, TE ~30 ms, 2 mm isotropic voxels, multiband factor 4-8). Include a high-resolution T1w structural scan for registration. Physiological monitoring (pulse oximetry, respiratory bellows) enables noise regression. Use foam padding to minimize head motion.

Common Reconstruction Algorithms

• General Linear Model (GLM) for task-based fMRI (FSL FEAT, SPM)
• ICA (Independent Component Analysis) for resting-state networks
• Seed-based functional connectivity analysis
• Motion correction and nuisance regression (6-parameter rigid body + CompCor)
• Deep-learning denoising and parcellation (BrainNetCNN, fMRIPrep pipeline)

Principle

A fundus camera images the posterior segment of the eye (retina, optic disc, macula, vasculature) by illuminating the retina through the pupil and capturing the reflected/backscattered light. The optical path is designed to separate illumination and observation through different portions of the pupil to avoid corneal reflections. Standard fundus imaging provides 30-50° field-of-view color photographs of the retina.

How to Build the System

Use a dedicated fundus camera (e.g., Topcon TRC-NW400, Canon CR-2 AF) or a scanning laser ophthalmoscope (Optos for widefield). Dilate the patient's pupil (tropicamide 1%) for standard fundus photography. Align the camera to center on the macula or optic disc. Set appropriate flash intensity and focus. Capture color and red-free (green channel) images. For fluorescein angiography, inject sodium fluorescein IV and capture timed image series with excitation/barrier filters.

Common Reconstruction Algorithms

• Image quality assessment and auto-focus/auto-exposure
• Vessel segmentation (U-Net, DeepVessel)
• Optic disc and cup segmentation for glaucoma screening
• Diabetic retinopathy grading (deep-learning classifiers)
• Multi-frame averaging and super-resolution for fundus images

Principle

Generic matrix sensing models the forward process as y = Ax + n, where A is an arbitrary measurement matrix (not necessarily structured like a convolution or Radon transform). This is the most general compressive sensing framework, applicable to random projections, coded apertures, and any linear dimensionality reduction scheme. The key requirement is that A satisfies the Restricted Isometry Property (RIP) for successful sparse recovery.

How to Build the System

Implementation depends on the physical sensing modality. For optical random projections, use a DMD or scattering medium to implement pseudo-random measurement vectors. Calibrate the measurement matrix A by measuring the system response to a complete basis set (e.g., Hadamard patterns). Store A as a dense or structured matrix. Ensure the measurement SNR is adequate for the desired reconstruction quality.

Common Reconstruction Algorithms

• ISTA / FISTA (Iterative Shrinkage-Thresholding Algorithm)
• Basis pursuit (L1 minimization via linear programming)
• AMP (Approximate Message Passing)
• ADMM with various regularizers (TV, wavelet sparsity, low-rank)
• Learned ISTA (LISTA) and other deep unfolding networks

Principle

Integral photography (also known as integral imaging) uses a 2-D array of elemental lenses to capture multi-perspective views of a 3-D scene simultaneously. Each elemental lens records a small perspective image, and the full set encodes the 4-D light field. Computational reconstruction produces 3-D images that can be viewed from different angles or refocused without glasses.

How to Build the System

Place a 2-D microlens or lenslet array (pitch 0.5-1 mm, ~50-200 elements per side) at one focal length from a high-resolution sensor. Each lenslet forms a separate elemental image. For display: show the integral image on a high-resolution display with a matched output lenslet array. Calibrate lenslet grid alignment, individual lens focal lengths, and vignetting correction. Use telecentric imaging for uniform magnification.

Common Reconstruction Algorithms

• Computational refocusing via pixel rearrangement and summation
• Depth estimation from elemental image disparity analysis
• 3-D scene reconstruction from integral images
• Super-resolution integral imaging (combining multiple shifted captures)
• Deep-learning integral image reconstruction and view synthesis

Principle

Lensless (diffuser-cam) imaging replaces the imaging lens with a thin diffuser or coded mask placed directly before the sensor. The sensor records a multiplexed pattern (caustic or speckle) that encodes the 3-D scene. Computational reconstruction inverts the known point-spread function of the diffuser to recover the image, enabling an extremely compact, lightweight camera suitable for miniaturized or in-vivo applications.

How to Build the System

Place a thin diffuser (ground glass, engineered phase mask, or Scotch tape) at a fixed, small distance (~1-5 mm) from a bare sensor (CMOS, e.g., Sony IMX sensor). Precisely characterize the diffuser PSF by scanning a point source across the field of view. Mount rigidly to prevent any relative motion between diffuser and sensor. For 3-D reconstruction, the depth-dependent PSF must be calibrated at multiple axial planes.

Common Reconstruction Algorithms

• ADMM (alternating direction method of multipliers) with TV regularization
• Wiener deconvolution (fast, single-step but lower quality)
• Gradient descent with learned priors (DiffuserCam, neural network prior)
• Tikhonov-regularized least squares
• Unrolled optimization networks (physics-informed deep learning)

Principle

Light Detection and Ranging (LiDAR) measures distances by emitting laser pulses (905 nm or 1550 nm) and timing their return after reflection from the scene (time-of-flight: d = c·t/2). A scanning mechanism (rotating mirror, MEMS, or optical phased array) sweeps the beam to build a 3-D point cloud of the environment. Resolution depends on the beam divergence, scanning density, and pulse timing precision.

How to Build the System

Select a LiDAR sensor appropriate for the application: mechanical spinning (Velodyne VLP-16/128 for autonomous vehicles), solid-state (Livox, Ouster), or airborne (Leica ALS80 for terrain mapping). Mount rigidly and combine with an IMU and GNSS for georeferencing. Calibrate intrinsic parameters (beam angles, timing offsets, intensity response) and extrinsics (relative to vehicle coordinate frame). Process returns: first/last/full waveform for different applications.

Common Reconstruction Algorithms

• Point cloud registration (ICP, NDT for multi-scan alignment)
• Ground filtering and classification (progressive morphological filter)
• SLAM (Simultaneous Localization and Mapping) with LiDAR
• Object detection and segmentation (PointNet, PointPillars)
• Surface reconstruction from point clouds (Poisson, ball-pivoting)

Principle

Light-field imaging captures both the spatial position and direction of light rays in a scene, recording a 4-D light field L(u,v,s,t) where (u,v) parameterize the aperture and (s,t) parameterize the spatial position. This enables computational refocusing, depth estimation, and novel viewpoint synthesis from a single capture. A microlens array placed before the sensor trades spatial resolution for angular resolution.

How to Build the System

Place a microlens array (MLA) at the sensor plane of a camera, one focal length in front of the image sensor. Each microlens captures the angular distribution of light from a corresponding spatial position (Lytro-style plenoptic camera). Alternative: use a camera array (e.g., 4×4 or 8×8 synchronized cameras) for higher angular and spatial resolution. Calibrate MLA alignment, microlens pitch, and main lens parameters.

Common Reconstruction Algorithms

• Shift-and-sum refocusing (synthetic aperture)
• Depth estimation from disparity between sub-aperture images
• Fourier slice theorem for light-field refocusing
• Light-field super-resolution (recovering spatial resolution lost to MLA)
• Deep-learning view synthesis (light field reconstruction from sparse views)

Principle

A thin sheet of laser light illuminates only the focal plane of the detection objective, providing intrinsic optical sectioning with minimal out-of-plane photobleaching. The orthogonal geometry between illumination and detection decouples sectioning from resolution. Detection is widefield, enabling fast volumetric imaging of large specimens.

How to Build the System

Arrange two orthogonal objective arms: one for the excitation sheet (cylindrical lens or digitally scanned Gaussian/Bessel beam) and one for detection (high-NA water-dipping). Mount the sample in agarose or hold in a chamber compatible with the dual-objective geometry. Use a fast sCMOS camera for detection. Stage scanning or sheet scanning acquires z-stacks. Consider diSPIM (dual-view) for isotropic resolution.

Common Reconstruction Algorithms

• Multi-view fusion (weighted averaging of complementary views)
• Multi-view deconvolution (Bayesian, joint Richardson-Lucy)
• Content-based image fusion
• Deep-learning denoising for high-speed acquisitions (CARE)
• Stripe artifact removal (wavelet-FFT filtering)

Principle

Identical optical path to standard widefield but operated at very low photon budgets (short exposure or attenuated excitation) to minimize phototoxicity in live cells. The acquired images are severely photon-starved, making Poisson noise the dominant degradation rather than out-of-focus blur.

How to Build the System

Use the same widefield microscope but reduce LED power to 1-5 % and/or shorten exposure to 5-20 ms. A high-QE back-illuminated sCMOS sensor (>80 % QE) is essential for capturing the limited photon signal. Install an environmental chamber for live-cell stability (37 °C, 5 % CO₂). Validate that the camera read noise floor is well below the expected signal.

Common Reconstruction Algorithms

• CARE (Content-Aware image REstoration)
• Noise2Void / Noise2Self (self-supervised denoising)
• BM3D / VST + BM3D for Poisson-Gaussian denoising
• PURE-LET (Poisson Unbiased Risk Estimator)
• Noise2Noise paired denoising networks

Principle

Magnetic Resonance Imaging measures the precession of hydrogen nuclear spins in a strong magnetic field (1.5-7 T). Radiofrequency pulses tip spins away from equilibrium, and gradient fields spatially encode the MR signal into k-space (spatial frequency domain). The image is obtained by inverse Fourier transform of k-space data. Contrast depends on tissue T1, T2, and proton density via the pulse sequence timing parameters.

How to Build the System

A clinical MRI scanner has a superconducting magnet (1.5 T or 3 T), gradient coils (40-80 mT/m, 200 T/m/s slew rate), RF transmit body coil, and local receive coil arrays (8-128 channels). The patient lies inside the bore on a table. Key calibrations: center frequency, RF transmit calibration (B₁ mapping), shimming (B₀ homogeneity), and gradient eddy current compensation. Use pulse sequences optimized for the clinical question (T1w, T2w, FLAIR, DWI, etc.).

Common Reconstruction Algorithms

• Inverse FFT (standard Cartesian k-space reconstruction)
• GRAPPA (GeneRalized Autocalibrating Partially Parallel Acquisitions)
• SENSE (SENSitivity Encoding) parallel imaging
• Compressed sensing MRI (L1-wavelet + TV regularization)
• Deep-learning MRI reconstruction (fastMRI, variational networks, E2E-VarNet)

Principle

Mammography uses low-energy X-rays (25-35 kVp) with specialized anode/filter combinations (Mo/Mo, Mo/Rh, W/Rh) to optimize contrast between breast tissue types (adipose, glandular, calcifications). Breast compression reduces thickness and scatter, improving contrast and reducing dose. Digital mammography uses flat-panel detectors for direct or indirect X-ray detection.

How to Build the System

A dedicated mammography unit with a compression paddle, specialized X-ray tube (Mo, Rh, or W anode), and high-resolution flat-panel detector (50-100 μm pixel size, amorphous selenium for direct conversion). Automatic optimization of target/filter and kVp based on compressed breast thickness. Regular quality assurance per ACR/MQSA requirements: phantom images, SNR measurements, artifact checks, and AEC calibration.

Common Reconstruction Algorithms

• Contrast-limited adaptive histogram equalization (CLAHE) for display
• Computer-aided detection (CAD) for microcalcification and mass detection
• Digital breast tomosynthesis (DBT) reconstruction (FBP or iterative)
• Deep-learning breast density classification (BI-RADS categories)
• Synthetic 2D mammography from DBT volumes

Principle

MR Spectroscopy measures the chemical shift spectrum of nuclear spins (usually ¹H) from a localized volume in the body, providing concentrations of metabolites such as NAA, creatine, choline, lactate, myo-inositol, and glutamate/glutamine. Chemical shift differences (in ppm) arise from the varying electronic shielding of nuclei in different molecular environments.

How to Build the System

Use PRESS or STEAM single-voxel localization on a 1.5T or 3T scanner. Voxel sizes are typically 2×2×2 cm³ for brain. Suppress the dominant water signal (CHESS or VAPOR water suppression). Acquire 64-256 averages (NEX) for adequate SNR. Shimming is critical: water linewidth should be <12 Hz (3T) for the voxel. Multi-voxel CSI (Chemical Shift Imaging) maps metabolite distributions but requires longer acquisition and careful lipid suppression.

Common Reconstruction Algorithms

• LCModel (frequency-domain linear combination fitting)
• TARQUIN (open-source time-domain fitting)
• jMRUI (time-domain quantification with AMARES/QUEST)
• HSVD (Hankel SVD) for water removal and baseline correction
• Deep-learning spectral quantification (DeepSpectra, convolutional fitting)

Principle

Muon tomography uses naturally occurring cosmic-ray muons to image the internal density structure of large objects (buildings, volcanoes, cargo containers). Muons undergo multiple Coulomb scattering, with the scattering angle proportional to the areal density and atomic number of the traversed material. By measuring the incoming and outgoing muon trajectories, the density distribution inside the object can be tomographically reconstructed.

How to Build the System

Place tracking detectors (drift tubes, scintillator strips, resistive plate chambers, or GEM detectors) above and below (or around) the object to be imaged. Each detector station measures the position and angle of each cosmic-ray muon before and after it traverses the object. Typical cosmic-ray muon flux is ~10,000 muons/m²/min at sea level. Exposure times range from minutes (for dense nuclear materials) to months (for geological structures like volcanoes).

Common Reconstruction Algorithms

• Point of Closest Approach (POCA) voxel reconstruction
• Maximum Likelihood / Expectation Maximization (ML/EM) scattering tomography
• Angle Statistics Reconstruction (ASR) for material discrimination
• Binned scattering density reconstruction
• Deep-learning muon tomography for faster convergence with fewer muons

Principle

Neural Radiance Fields (NeRF) represent a 3-D scene as a continuous volumetric function F(x,y,z,θ,φ) → (RGB, σ) parameterized by a multi-layer perceptron (MLP). The network maps 3-D position and viewing direction to color and volume density. Novel views are synthesized by differentiable volume rendering along camera rays, and the network is trained by minimizing photometric loss against a set of posed 2-D images.

How to Build the System

Capture 50-200 images of a scene from diverse viewpoints using a calibrated camera (known intrinsics) or estimate camera poses with COLMAP structure-from-motion. Images should cover the scene uniformly. Train a NeRF MLP (typically 8 layers, 256 units, with positional encoding of input coordinates) on a GPU (≥12 GB VRAM). Training takes 12-48 hours on a single V100. Use mip-NeRF, Instant-NGP, or TensoRF for faster convergence.

Common Reconstruction Algorithms

• Vanilla NeRF (MLP + positional encoding)
• Instant-NGP (multi-resolution hash encoding, minutes training)
• mip-NeRF (anti-aliased cone tracing)
• Nerfacto (nerfstudio default combining multiple improvements)
• TensoRF (tensor factorization for compact radiance fields)

Principle

Neutron radiography and tomography image the transmission of a thermal or cold neutron beam through a sample. Neutrons interact with nuclei (not electrons), providing complementary contrast to X-rays: hydrogen-rich materials (water, polymers, organics) attenuate neutrons strongly, while metals like aluminum and lead are relatively transparent. Tomographic reconstruction from multiple projection angles yields 3-D maps of neutron attenuation.

How to Build the System

Access a research reactor or spallation neutron source with an imaging beamline (e.g., ICON at PSI, IMAT at ISIS, NIST BT-2). A collimated neutron beam (thermal or cold, 1-10 Å) passes through the sample, and a scintillator-camera system (⁶LiF/ZnS screen + sCMOS camera) records the transmitted intensity. Rotate the sample through 180° or 360° for tomography. Spatial resolution is typically 20-100 μm, limited by beam divergence and scintillator thickness.

Common Reconstruction Algorithms

• Filtered back-projection (FBP) adapted for neutron tomography
• Iterative reconstruction (SIRT, CGLS) for limited-angle or noisy data
• Beam hardening correction for polychromatic neutron spectra
• Scattering correction (point-scattered function approach)
• Neutron phase-contrast tomography (grating interferometry)

Principle

OCT Angiography detects blood flow non-invasively by comparing repeated OCT B-scans at the same location. Moving red blood cells cause temporal fluctuations in the OCT signal (amplitude and/or phase), while static tissue remains constant. Decorrelation, variance, or differential analysis between repeated scans produces a motion-contrast image revealing the vasculature without the need for injectable contrast agents.

How to Build the System

Use a high-speed OCT system (≥70 kHz A-scan rate, swept-source preferred) capable of repeated B-scans at the same location. Acquire 2-4 repeated B-scans at each position with inter-scan time of 3-10 ms. An eye-tracking system is essential for ophthalmic OCTA to correct microsaccades. Process with split-spectrum amplitude-decorrelation (SSADA), optical microangiography (OMAG), or phase-variance algorithms.

Common Reconstruction Algorithms

• SSADA (Split-Spectrum Amplitude-Decorrelation Angiography)
• OMAG (Optical Micro-Angiography, complex signal differential)
• Phase-variance OCTA
• Deep-learning OCTA denoising and vessel segmentation
• Projection artifact removal algorithms

Principle

Optical Coherence Tomography uses low-coherence interferometry to produce cross-sectional images of tissue microstructure. A broadband light source (superluminescent diode, ~840 nm or ~1310 nm) is split between sample and reference arms. Interference occurs only when the path lengths match within the coherence length (~5-10 μm), providing axial resolution. Spectral-domain OCT records the spectral interferogram and uses FFT for fast depth-resolved imaging.

How to Build the System

Build or acquire a spectral-domain OCT system: broadband SLD source (center 840 nm, 50 nm bandwidth for retinal; 1310 nm for dermal/cardiac), fiber-based Michelson interferometer, galvo scanner for lateral scanning, and a spectrometer with line camera (2048-4096 pixels) for spectral detection. Calibrate wavelength-to-wavenumber mapping, dispersion compensation, and reference arm delay. For swept-source OCT, use a frequency-swept laser (100-400 kHz sweep rate) and balanced detector.

Common Reconstruction Algorithms

• FFT-based spectral-domain OCT reconstruction (spectral interferogram → A-scan)
• Dispersion compensation (numerical or hardware)
• Speckle reduction (spatial/angular compounding, or deep-learning)
• Segmentation of retinal layers (graph-based, U-Net, or transformer models)
• OCT Angiography (OCTA) via decorrelation or phase-variance of repeated B-scans

Principle

Single-Molecule Localization Microscopy (PALM/STORM) achieves ~20 nm resolution by stochastically switching individual fluorophores between bright and dark states. In each frame, only a sparse subset of molecules emit, allowing their positions to be localized with sub-pixel precision by fitting 2-D Gaussians. Thousands of frames are accumulated and all localizations are plotted to form a super-resolution image.

How to Build the System

Use a TIRF microscope (100x 1.49 NA oil objective) with powerful laser excitation (200-500 mW at the sample, 647 nm for Alexa647 STORM or 561 nm for mEos PALM). TIRF geometry reduces background. An oxygen-scavenging buffer with thiol (MEA/BME) is critical for Alexa647 blinking. Use an EMCCD (Andor iXon 897) or fast sCMOS camera at 30-100 Hz frame rate. Acquire 10,000-50,000 frames.

Common Reconstruction Algorithms

• ThunderSTORM (ImageJ plugin, MLE/LSQ Gaussian fitting)
• SMLM ZOLA-3D (deep-learning 3D localization)
• DAOSTORM (multi-emitter fitting for high density)
• Drift correction (fiducial-based or cross-correlation)
• HAWK / ANNA-PALM (deep-learning for accelerated SMLM)

Principle

Panoramic multi-focus fusion captures multiple images of the same wide scene at different focal distances and combines them to produce a single all-in-focus panorama with extended depth of field. Image stitching aligns overlapping frames using feature matching and homography estimation, while focus fusion selects the sharpest pixels from each focal plane.

How to Build the System

Mount a camera on a motorized panoramic head (nodal point rotation). For each pan/tilt position, capture a focus stack (3-10 images at different focus distances). Use a medium-aperture setting (f/5.6-f/8) for each frame. Stitch overlapping views (30 % horizontal overlap) and fuse focus stacks per view tile. Calibrate the panoramic head to rotate around the lens entrance pupil to minimize parallax.

Common Reconstruction Algorithms

• Laplacian pyramid focus fusion (weighted blending by local contrast)
• SIFT/SURF feature matching + RANSAC homography estimation
• Multi-band blending (Burt-Adelson) for seamless stitching
• Exposure fusion (Mertens et al.) for HDR panoramas
• Deep-learning focus stacking (DFDF, DeepFocus)

Principle

Photoacoustic imaging converts absorbed pulsed laser light into ultrasound via thermoelastic expansion. Short laser pulses (<10 ns) are absorbed by tissue chromophores (hemoglobin, melanin), causing rapid thermal expansion that generates broadband acoustic waves. These waves are detected by ultrasound transducers and reconstructed to form images reflecting optical absorption contrast at ultrasonic spatial resolution.

How to Build the System

Combine a tunable pulsed laser (Nd:YAG pumped OPO, 680-1100 nm, 5-20 ns pulses, 10-20 Hz) with an ultrasound transducer array (linear or curved, 5-40 MHz). Deliver light via fiber bundle to the tissue surface adjacent to the transducer. Use a multi-channel DAQ (12-14 bit, 40-100 MS/s) to record acoustic signals. For tomographic PAT, surround the sample with a ring or spherical array of transducers.

Common Reconstruction Algorithms

• Universal back-projection for photoacoustic tomography
• Time-reversal reconstruction
• Model-based iterative reconstruction with acoustic heterogeneity
• Spectral unmixing for multi-wavelength functional PA imaging
• Deep-learning PA image reconstruction (U-Net, pixel-wise inversion)

Principle

Polarization microscopy exploits the birefringence (orientation-dependent refractive index) of ordered biological structures such as collagen fibers, spindle microtubules, and crystalline inclusions. By analyzing the polarization state of transmitted or reflected light, structural anisotropy can be measured without fluorescent labeling. Quantitative techniques (LC-PolScope) measure both retardance magnitude and slow-axis orientation.

How to Build the System

Mount a liquid-crystal universal compensator (LC-PolScope by OpenPolScope, or Abrio system) on a standard brightfield microscope. Use strain-free optics and rotate the analyzer while keeping the polarizer fixed (or use a rotating stage). For quantitative imaging, acquire 4-5 images at different compensator settings. A monochromatic light source (546 nm green filter) minimizes chromatic effects.

Common Reconstruction Algorithms

• Mueller matrix decomposition (full polarimetric imaging)
• Jones calculus for coherent polarization analysis
• Background retardance subtraction
• Stokes parameter reconstruction from intensity measurements
• Deep-learning retardance estimation from fewer raw frames

Principle

Positron Emission Tomography detects pairs of 511 keV gamma rays emitted in opposite directions when a positron from a radiotracer annihilates with an electron. Coincidence detection of the two photons defines a line of response (LOR). Many LORs from different angles are reconstructed into a 3-D activity distribution map, providing functional and metabolic information.

How to Build the System

A PET scanner consists of a ring of scintillation detector blocks (LYSO or LSO crystals coupled to SiPMs) surrounding the patient. Each detector block has a matrix of small crystals (3-4 mm pitch). Coincidence electronics pair detected events within a timing window (4-6 ns for TOF-PET). Modern digital PET systems achieve 200-300 ps timing resolution for time-of-flight. Daily quality checks include detector normalization, timing calibration, and sensitivity phantom scans.

Common Reconstruction Algorithms

• OSEM (Ordered Subset Expectation Maximization)
• 3D OSEM with resolution modeling (PSF reconstruction)
• TOF-OSEM (time-of-flight enhanced OSEM)
• Attenuation correction from CT (PET/CT) or Dixon MR (PET/MR)
• Deep-learning PET denoising (low-count to full-count prediction)

Principle

Proton radiography images the transmission and scattering of high-energy protons (50-800 MeV) through dense objects. Unlike X-rays, protons undergo significant multiple Coulomb scattering (MCS) in matter, which provides density and compositional contrast. Both transmission (energy loss) and scattering angle measurements contribute to image formation. Proton radiography can penetrate very dense materials (steel, depleted uranium) that are opaque to X-rays.

How to Build the System

Requires a high-energy proton accelerator facility (synchrotron or cyclotron delivering 200-800 MeV protons). The object is placed in the beam path between tracking detectors (silicon strip or GEM detectors) that measure each proton's position and angle before and after the object. A magnetic spectrometer (quadrupole lens system, e.g., at LANL pRad facility) focuses transmitted protons onto a scintillator + camera detector.

Common Reconstruction Algorithms

• Most Likely Path (MLP) estimation for proton CT reconstruction
• Filtered back-projection with scattering-angle weighting
• Algebraic reconstruction (ART) with MCS forward model
• Material discrimination from dual-parameter (transmission + scattering) analysis
• Deep-learning proton CT reconstruction for reduced view angles

Principle

Ptychography is a scanning coherent diffractive imaging technique where a coherent beam (visible, X-ray, or electron) illuminates overlapping regions of the sample. At each scan position, a far-field diffraction pattern is recorded. The redundancy from overlapping illumination positions constrains the phase-retrieval problem, enabling simultaneous recovery of both the complex sample transmittance and the illumination probe function.

How to Build the System

For X-ray ptychography at a synchrotron: focus the beam to a defined spot (0.1-1 μm) using a Fresnel zone plate or KB mirrors. Mount the sample on a precision piezo scanning stage. Place a photon-counting area detector (Eiger, Pilatus) in the far field (1-5 m downstream). Scan positions should overlap by 60-70 %. For visible-light or electron ptychography, adapt the geometry but maintain the overlap requirement.

Common Reconstruction Algorithms

• ePIE (extended Ptychographic Iterative Engine)
• Difference Map algorithm
• Maximum Likelihood refinement (MLR)
• PtychoShelves (modular framework for ptychographic reconstruction)
• Deep-learning ptychography (PtychoNN, learned phase retrieval)

Principle

Scanning Electron Microscopy rasters a focused electron beam (0.1-30 keV) across the sample surface. Secondary electrons (SE) emitted from the top few nanometers provide topographic contrast, while backscattered electrons (BSE) from deeper interactions reveal compositional contrast (higher Z → more BSE). The image is formed point-by-point, with resolution down to 1-5 nm determined by the probe size.

How to Build the System

Operate a field-emission SEM (FEG-SEM, e.g., Zeiss GeminiSEM, JEOL JSM-7800F) under high vacuum (< 10⁻⁴ Pa). Mount samples on conductive stubs with carbon tape or silver paint. Non-conductive samples must be sputter-coated (5-10 nm Au/Pd or C) to prevent charging. Set accelerating voltage (1-5 kV for surface detail, 10-20 kV for BSE compositional contrast). Select appropriate detectors (Everhart-Thornley for SE, solid-state for BSE). Align the column and perform astigmatism correction.

Common Reconstruction Algorithms

• Noise reduction by frame averaging or Kalman filtering
• Charging artifact compensation (dynamic focus, low-kV imaging)
• 3-D surface reconstruction from stereo-pair SEM images
• Deep-learning SEM denoising (for low-dose or fast-scan images)
• Automated particle analysis and morphometry

Principle

Scanning TEM focuses the electron beam to a fine probe (0.05-1 nm) and scans it across the specimen. Multiple detectors collect signals simultaneously: bright-field (BF), annular dark-field (ADF), and high-angle annular dark-field (HAADF). HAADF-STEM provides Z-contrast imaging where intensity scales approximately as Z^1.7, enabling direct interpretation of atomic columns by atomic number.

How to Build the System

Use an aberration-corrected STEM (probe-corrected, e.g., Thermo Fisher Titan Themis or JEOL ARM300F). Align the probe-corrector to minimize C₃ and C₅ aberrations, achieving sub-Ångström probe size. Adjust camera length for HAADF inner angle (typically 50-80 mrad for Z-contrast). Prepare atomically thin specimens by FIB or mechanical exfoliation. Use drift-corrected frame integration for high-quality atomic-resolution images.

Common Reconstruction Algorithms

• Atom column detection and quantification (peak finding, Gaussian fitting)
• Strain mapping via geometric phase analysis (GPA) or peak-pair analysis
• Multi-frame averaging with rigid/non-rigid registration for noise reduction
• HAADF simulation (frozen-phonon multislice) for quantitative comparison
• Deep-learning STEM image denoising and super-resolution

Principle

Shear-wave elastography measures tissue stiffness by tracking the propagation speed of shear waves generated by an acoustic radiation force impulse (ARFI) or external vibration. Shear-wave speed is proportional to the square root of the shear modulus: cₛ = √(μ/ρ). Stiffer tissues (fibrosis, tumors) have faster shear-wave propagation. Results are displayed as quantitative elasticity maps (in kPa or m/s).

How to Build the System

Use a clinical ultrasound system with shear-wave elastography mode (Supersonic Imagine Aixplorer, Siemens ARFI/VTQ, or GE 2D-SWE). The transducer generates a focused push pulse to create shear waves, then tracks their propagation with ultrafast plane-wave imaging (up to 10,000 fps). Place the ROI in a region free of large vessels and interfaces. Patient should hold breath for liver measurements. Calibrate with an elasticity phantom.

Common Reconstruction Algorithms

• Time-to-peak shear-wave arrival estimation
• Phase-gradient shear-wave speed inversion
• 2-D shear-wave elastography mapping (real-time SWE)
• Transient elastography (FibroScan 1-D measurement)
• Deep-learning elasticity estimation from B-mode + SWE data

Principle

Single Photon Emission Computed Tomography detects single gamma-ray photons emitted by a radiotracer (⁹⁹ᵐTc, ¹²³I, ²⁰¹Tl) using a rotating gamma camera with a parallel-hole or pinhole collimator. The collimator provides directional sensitivity at the cost of low geometric efficiency (~0.01 %). Projections from multiple angles are reconstructed into 3-D activity maps.

How to Build the System

A dual-head gamma camera (e.g., Siemens Symbia, GE Discovery) with NaI(Tl) scintillator crystals (9.5 mm thick) and parallel-hole collimators rotates around the patient (typically 60-128 angular stops over 360°). For cardiac SPECT, use dedicated CZT-based cameras with pinhole or multi-pinhole collimators. Acquire in step-and-shoot or continuous rotation mode. Energy windows are set around the photopeak (e.g., 140 keV ± 10 % for ⁹⁹ᵐTc).

Common Reconstruction Algorithms

• FBP with ramp-Butterworth filter
• OSEM with attenuation and scatter correction
• Resolution recovery (collimator-detector response modeling in OSEM)
• CT-based attenuation correction (SPECT/CT)
• Deep-learning SPECT reconstruction (dose reduction, resolution enhancement)

Principle

A single-pixel camera uses a spatial light modulator (DMD) to project a sequence of binary or grayscale patterns onto the scene. Each pattern multiplies the scene, and a single bucket detector (photodiode or PMT) measures the total light for each pattern, producing one scalar measurement per pattern. Compressive sensing recovers the image from far fewer measurements than Nyquist by exploiting sparsity in a transform domain.

How to Build the System

Place a DMD (e.g., Texas Instruments DLP LightCrafter) at the image plane of a relay lens. Focus the scene onto the DMD. After the DMD, collect all reflected light onto a single photodetector (avalanche photodiode for low light, or silicon photodiode for visible). Display Hadamard, random, or optimized patterns at 10-22 kHz DMD rate. Synchronize pattern display with detector readout.

Common Reconstruction Algorithms

• Basis pursuit / L1 minimization (LASSO)
• Orthogonal matching pursuit (OMP)
• Total-variation minimization (TV-CS)
• TVAL3 (TV with augmented Lagrangian and alternating direction)
• Deep compressive sensing networks (ReconNet, CSNet)

Principle

Sonar imaging uses acoustic waves (typically 50 kHz to 1 MHz) to image underwater scenes. Active sonar transmits a sound pulse and records the echoes from the seabed, objects, or water column. The propagation speed in water (~1500 m/s, varying with temperature, salinity, and pressure) determines the time-to-distance relationship. Side-scan sonar and multibeam bathymetry produce 2-D and 3-D maps of the underwater environment.

How to Build the System

For side-scan sonar: mount a towfish with two transducer arrays (port and starboard) that ensonify a swath perpendicular to the survey track. For multibeam: mount a hull-mounted array (e.g., Kongsberg EM2040, 200-400 kHz). Sound velocity profiler (SVP) measurements are essential for ray-tracing corrections. Integrate with GNSS positioning and motion reference unit (MRU) for heave, pitch, and roll compensation.

Common Reconstruction Algorithms

• Beamforming (delay-and-sum for multibeam sonar)
• Synthetic aperture sonar (SAS) processing for enhanced azimuth resolution
• Bottom detection and bathymetric surface extraction
• Acoustic backscatter classification for seabed characterization
• Deep-learning object detection for mine countermeasures or marine archaeology

Principle

Stimulated Emission Depletion microscopy breaks the diffraction limit by using a donut-shaped depletion beam to force fluorophores at the periphery of the excitation spot back to the ground state via stimulated emission. Only fluorophores at the very center of the donut emit spontaneously, shrinking the effective PSF to 30-70 nm lateral resolution depending on depletion power.

How to Build the System

Combine an excitation laser (e.g., 640 nm pulsed) with a co-aligned depletion laser (775 nm pulsed, ~1 ns) that passes through a vortex phase plate to create the donut. Use a high-NA objective (100x 1.4 NA oil). Time-gate detection (1-6 ns after excitation pulse) to reject depletion photon leakage. Single-photon counting detectors (APDs or hybrid PMTs) are essential. Align the donut null precisely at the excitation center.

Common Reconstruction Algorithms

• Richardson-Lucy deconvolution with STED PSF
• Wiener deconvolution with known STED PSF
• Deep-learning restoration (content-aware STED denoising)
• Linear unmixing for multi-color STED
• Time-gated STED (g-STED) background subtraction

Principle

Structured Illumination Microscopy projects a known sinusoidal pattern onto the specimen, shifting high-frequency spatial information into the observable passband via Moiré interference. Multiple images (typically 9-15) are acquired at different pattern orientations and phases, then computationally recombined in Fourier space to achieve ~2× lateral resolution improvement beyond the diffraction limit.

How to Build the System

Install a SIM-capable microscope (Nikon N-SIM, Zeiss Elyra 7, or custom with SLM/DMD). Use a high-NA objective (100x 1.49 NA TIRF) for maximum frequency extension. The illumination grating (SLM or fiber interference) generates the sinusoidal pattern. Acquire 3 orientations × 3-5 phases. A fast sCMOS camera captures all raw frames in ~100-500 ms for 2D-SIM. Careful alignment of the pattern contrast is critical.

Common Reconstruction Algorithms

• Gustafsson/Heintzmann frequency-domain SIM reconstruction
• Open-source fairSIM (ImageJ plugin)
• Wiener-filtered order separation and recombination
• Deep-learning SIM (ML-SIM, reconstruction from fewer frames)
• Hessian-SIM for live-cell with reduced artifacts

Principle

Structured-light depth sensing projects a known pattern (stripes, dots, coded binary patterns) onto the scene and observes the pattern deformation with a camera from a different viewpoint. The displacement (disparity) of each pattern element between projected and observed positions encodes the surface depth via triangulation. Dense depth maps are obtained by identifying pattern correspondences across the scene.

How to Build the System

Arrange a projector (DLP or laser dot projector) and camera with a known baseline separation (5-25 cm) and convergent geometry. Calibrate the projector-camera system (intrinsics and extrinsics) using a planar calibration target. For temporal coding (Gray code), project multiple patterns sequentially. For spatial coding (single-shot, e.g., Apple FaceID dot projector), use a diffractive optical element to generate a unique dot pattern.

Common Reconstruction Algorithms

• Gray code + phase shifting (sequential multi-pattern decoding)
• Single-shot coded pattern matching (speckle or pseudo-random dot decoding)
• Phase unwrapping for sinusoidal fringe projection
• Stereo matching applied to textured scenes (active stereo)
• Deep-learning depth estimation from structured-light patterns

Principle

Synthetic Aperture Radar achieves fine azimuth resolution by coherently processing radar echoes collected as the antenna moves along its flight path, synthesizing an aperture much larger than the physical antenna. The SAR signal processor applies matched filtering (pulse compression) in both range and azimuth to form a high-resolution complex image. SAR operates through clouds, at night, and in all weather conditions.

How to Build the System

Mount a microwave transmitter/receiver (C-band 5.4 GHz, L-band 1.3 GHz, or X-band 9.6 GHz) on a satellite (Sentinel-1, RADARSAT) or aircraft. The antenna illuminates a strip on the ground as the platform moves. Record the complex (I/Q) echo data with precise pulse timing and platform position/velocity from GNSS/INS. Range resolution is set by pulse bandwidth (1-200 MHz); azimuth resolution equals L_ant/2 (half the antenna length).

Common Reconstruction Algorithms

• Range-Doppler algorithm (range compression + azimuth compression)
• Chirp scaling algorithm for wide-swath SAR
• Omega-K (wavenumber domain) algorithm for high-resolution spotlight SAR
• InSAR (Interferometric SAR) for DEM generation and deformation mapping
• PolSAR decomposition (Cloude-Pottier, Freeman-Durden) for land classification

Principle

A Time-of-Flight depth camera measures the round-trip time of modulated light (typically near-infrared LEDs at 850 nm) reflected from the scene. The sensor measures the phase shift between emitted and received modulated signals at each pixel, which is proportional to the target distance: d = c·Δφ/(4π·f_mod). Typical modulation frequencies are 20-100 MHz, providing depth ranges of 0.5-10 meters with mm-cm precision.

How to Build the System

Use an integrated ToF camera module (e.g., Microsoft Azure Kinect DK, PMD CamBoard pico, Texas Instruments OPT8241). The module contains the NIR light source, modulation driver, and ToF sensor with per-pixel demodulation circuits. Mount rigidly and calibrate intrinsic parameters (lens distortion, depth offset) and phase-to-depth nonlinearities. For multi-camera setups, synchronize or frequency-multiplex to avoid interference.

Common Reconstruction Algorithms

• Four-phase demodulation for distance extraction
• Multi-frequency unwrapping for extended unambiguous range
• Flying-pixel filtering (mixed pixels at depth discontinuities)
• Multi-path interference correction
• Deep-learning depth denoising and completion

Principle

Total Internal Reflection Fluorescence microscopy creates an evanescent wave that penetrates only ~100-200 nm into the sample when the excitation beam is totally internally reflected at the glass-sample interface. This provides excellent optical sectioning of membrane-proximal events (vesicle fusion, protein dynamics at the plasma membrane) with very low background.

How to Build the System

Use a TIRF-capable objective (60-100x, 1.49 NA oil) on an inverted microscope. Launch the laser at the critical angle through the objective periphery (objective-type TIRF) or through a prism (prism-type TIRF). Verify total internal reflection by observing the evanescent field depth with a calibration sample. Cells must be plated on clean, high-RI coverslips (#1.5H, 170 μm).

Common Reconstruction Algorithms

• Single-particle tracking (SPT) algorithms
• Multi-angle TIRF for axial sectioning (variable penetration depth)
• Denoising (Gaussian filtering, wavelet, or deep-learning)
• Photobleaching step analysis for molecular counting
• Temporal median filtering for background subtraction

Principle

Transmission Electron Microscopy transmits a high-energy electron beam (80-300 keV) through an ultra-thin specimen (<100 nm). Electrons interact with the sample via elastic scattering (diffraction contrast, phase contrast) and inelastic scattering (energy loss). The transmitted beam is magnified by electromagnetic lenses to form an image with atomic-level resolution (0.05-0.2 nm in aberration-corrected TEMs).

How to Build the System

Operate a TEM (e.g., JEOL JEM-2100, Thermo Fisher Talos/Titan) under high vacuum (< 10⁻⁵ Pa). Prepare ultra-thin specimens using ultramicrotomy (biological), focused ion beam (FIB) milling (materials), or electropolishing (metals). Load samples on 3 mm TEM grids (Cu or Mo). Align the beam, correct condenser and objective astigmatism, and set appropriate defocus for phase contrast imaging. Use direct-electron detectors for highest DQE.

Common Reconstruction Algorithms

• CTF correction (Contrast Transfer Function for phase contrast imaging)
• Single-particle analysis (cryo-EM: classification, 3-D reconstruction)
• Selected-area electron diffraction (SAED) pattern analysis
• HRTEM image simulation (multislice or Bloch wave)
• Deep-learning denoising for low-dose cryo-EM (Topaz, Warp, cryoSPARC)

Principle

Two-photon excitation uses a pulsed near-infrared laser so that two photons are absorbed simultaneously by a fluorophore, producing fluorescence equivalent to a single photon of half the wavelength. Because absorption depends on the square of intensity, fluorescence is generated only at the tight focus, providing intrinsic optical sectioning without a pinhole. Deep tissue penetration (up to ~1 mm) is achieved due to reduced scattering at NIR wavelengths.

How to Build the System

Install a mode-locked Ti:Sapphire laser (680-1080 nm, ~100 fs pulses, 80 MHz, Coherent Chameleon or Spectra-Physics InSight) on a laser-scanning microscope. Use a high-NA water-dipping objective (25x 1.05 NA or 20x 1.0 NA) for deep imaging. Non-descanned detectors (GaAsP PMTs) collect scattered fluorescence close to the objective for maximum efficiency. Add a Pockels cell for fast power modulation.

Common Reconstruction Algorithms

• Adaptive background subtraction for in-depth imaging
• Motion correction and image registration for in-vivo data
• Suite2p / CaImAn (calcium imaging segmentation and trace extraction)
• Deep-learning denoising (DeepInterpolation, Noise2Void)
• Attenuation compensation (exponential depth correction)

Principle

Medical ultrasound imaging transmits short pulses of high-frequency sound waves (1-20 MHz) into tissue and detects the echoes reflected from acoustic impedance boundaries. The time delay of each echo determines the reflector depth, and beamforming focuses the transmitted and received beams to form a 2-D cross-sectional image. Spatial resolution improves with frequency but penetration depth decreases.

How to Build the System

A clinical ultrasound system consists of a multi-element transducer array (linear 7-15 MHz for superficial, curvilinear 2-5 MHz for abdominal, phased array 1-5 MHz for cardiac) connected to a beamformer and image processor. Modern systems use 128-192 element arrays with digital beamforming. Apply acoustic coupling gel between transducer and skin. Adjust gain, depth, focus, and frequency for the specific examination.

Common Reconstruction Algorithms

• Delay-and-sum (DAS) beamforming
• Adaptive beamforming (Capon, MVDR) for improved resolution
• Synthetic aperture focusing (SAFT)
• Plane-wave compounding for ultrafast imaging
• Deep-learning beamforming and speckle reduction