Physics World Model — Modality Catalog
4 imaging modalities with descriptions, experimental setups, and reconstruction guidance.
Fiber Bundle Endoscopy
Fiber bundle endoscopy transmits images through a coherent fiber bundle of 10,000-50,000 individual optical fibers. Each fiber core acts as a spatial sample, producing a honeycomb pattern. Image quality is limited by inter-core spacing (pixelation), inter-core coupling (crosstalk), and core-to-core transmission variation. White-light or narrow-band illumination is delivered through the bundle or alongside it. Reconstruction involves core localization, transmission calibration, interpolation to a regular grid, and denoising.
Fiber Bundle Endoscopy
Description
Fiber bundle endoscopy transmits images through a coherent fiber bundle of 10,000-50,000 individual optical fibers. Each fiber core acts as a spatial sample, producing a honeycomb pattern. Image quality is limited by inter-core spacing (pixelation), inter-core coupling (crosstalk), and core-to-core transmission variation. White-light or narrow-band illumination is delivered through the bundle or alongside it. Reconstruction involves core localization, transmission calibration, interpolation to a regular grid, and denoising.
Principle
Fiber-bundle endoscopy transmits an image through a flexible coherent fiber bundle (10,000-100,000 individual fiber cores) to visualize internal body cavities. Each fiber core acts as a single pixel, transmitting light from the distal end to the proximal end where a camera captures the image. The hexagonal fiber packing imposes a fixed pixelation pattern (comb/honeycomb structure) on the image.
How to Build the System
A medical endoscope has a flexible insertion tube containing the coherent fiber bundle (or a distal CMOS chip for video endoscopes), illumination fibers, working channels, and air/water channels. Light source: LED or Xenon lamp transmitted through illumination fibers. For fiber-bundle type: attach a high-resolution camera and relay lens at the proximal end. Calibrate fiber core positions and individual fiber transmission for computational image improvement.
Common Reconstruction Algorithms
- Fiber core mapping and interpolation (honeycomb artifact removal)
- Deep-learning super-resolution for fiber-bundle images
- Structure-from-motion for endoscopic 3-D reconstruction
- Defogging / dehazing for underwater or smoke-obscured endoscopy
- Real-time mosaicking for extended field-of-view endoscopy
Common Mistakes
- Honeycomb pattern artifact from fiber core spacing not removed
- Broken fibers (dark spots) accumulating over time and degrading image quality
- Specular reflections (glare) from wet tissue surfaces saturating the image
- Insufficient illumination causing noisy images in deep body cavities
- Image distortion from fiber bundle bending not corrected
How to Avoid Mistakes
- Apply fiber core interpolation or deep-learning super-resolution in post-processing
- Replace fiber bundles when broken fiber percentage exceeds acceptable threshold
- Use polarization filtering or computational specular removal algorithms
- Use bright LED sources and adjust exposure/gain for adequate signal
- Calibrate and correct for bending-dependent distortion using test patterns
Forward-Model Mismatch Cases
- The widefield fallback produces a (64,64) image, but fiber-bundle endoscopy transmits images through discrete fiber cores creating a hexagonal pixelation pattern — output shape (n_fibers,) is a 1D vector of per-core intensities
- The fiber bundle imposes a fixed sampling grid (honeycomb structure) with inter-core crosstalk and dead fibers — the widefield continuous Gaussian blur has no relationship to the discrete fiber sampling and transmission physics
How to Correct the Mismatch
- Use the endoscopy operator that models per-fiber-core sampling: each of the ~10,000-100,000 cores transmits a point sample from the distal end to the proximal camera, with known core positions and transmission coefficients
- Reconstruct using fiber-core interpolation, honeycomb artifact removal, or deep-learning super-resolution that account for the known fiber bundle geometry and per-core response
Experimental Setup — Signal Chain
Experimental Setup — Details
Key References
- Lee & Bhatt, 'Fiber bundle endoscopy advances', J. Biophotonics 12, e201900004 (2019)
Canonical Datasets
- Kvasir-SEG (polyp segmentation)
- CVC-ClinicDB (colonoscopy)
- HyperKvasir (multi-class GI dataset)
Fundus Camera
A fundus camera captures a 2D color photograph of the retinal surface by illuminating the fundus through the pupil with a ring-shaped flash and imaging the reflected light through the central pupillary zone. The optical system images the curved retina onto a flat detector with 30-50 degree field of view. Image quality is degraded by media opacities (cataract), small pupil, and uneven illumination. Fundus images are widely used for automated screening of diabetic retinopathy, glaucoma, and AMD via deep learning.
Fundus Camera
Description
A fundus camera captures a 2D color photograph of the retinal surface by illuminating the fundus through the pupil with a ring-shaped flash and imaging the reflected light through the central pupillary zone. The optical system images the curved retina onto a flat detector with 30-50 degree field of view. Image quality is degraded by media opacities (cataract), small pupil, and uneven illumination. Fundus images are widely used for automated screening of diabetic retinopathy, glaucoma, and AMD via deep learning.
Principle
A fundus camera images the posterior segment of the eye (retina, optic disc, macula, vasculature) by illuminating the retina through the pupil and capturing the reflected/backscattered light. The optical path is designed to separate illumination and observation through different portions of the pupil to avoid corneal reflections. Standard fundus imaging provides 30-50° field-of-view color photographs of the retina.
How to Build the System
Use a dedicated fundus camera (e.g., Topcon TRC-NW400, Canon CR-2 AF) or a scanning laser ophthalmoscope (Optos for widefield). Dilate the patient's pupil (tropicamide 1%) for standard fundus photography. Align the camera to center on the macula or optic disc. Set appropriate flash intensity and focus. Capture color and red-free (green channel) images. For fluorescein angiography, inject sodium fluorescein IV and capture timed image series with excitation/barrier filters.
Common Reconstruction Algorithms
- Image quality assessment and auto-focus/auto-exposure
- Vessel segmentation (U-Net, DeepVessel)
- Optic disc and cup segmentation for glaucoma screening
- Diabetic retinopathy grading (deep-learning classifiers)
- Multi-frame averaging and super-resolution for fundus images
Common Mistakes
- Insufficient pupil dilation causing vignetting at the field edges
- Corneal reflections (flare) obscuring the central retinal image
- Image out of focus due to refractive error not compensated
- Eyelash or eyelid obstruction in the image
- Uneven illumination across the retinal image
How to Avoid Mistakes
- Ensure adequate mydriasis (>5 mm pupil diameter) before imaging
- Align the camera carefully to separate illumination and observation through different pupil zones
- Use auto-focus and compensate for patient refractive error in the camera optics
- Ask patients to open eyes wide; use a fixation target for gaze direction
- Verify uniform illumination before capture; adjust camera alignment if uneven
Forward-Model Mismatch Cases
- The widefield fallback applies a generic Gaussian PSF, but fundus imaging has a unique optical path through the eye's optics (cornea and lens) with specific aberrations and the pupil-splitting illumination/observation geometry
- The retinal image is formed after double-pass through the ocular media, with wavelength-dependent absorption (hemoglobin, melanin, macular pigment) — the widefield achromatic Gaussian blur cannot model spectral absorption or ocular aberrations
How to Correct the Mismatch
- Use the fundus operator that models the eye's optical path: illumination through one pupil zone, retinal reflection/fluorescence, and collection through a separate pupil zone, with ocular aberration and media absorption
- Include wavelength-dependent retinal reflectance for color fundus imaging, or fluorescein excitation/emission model for fluorescein angiography
Experimental Setup — Signal Chain
Experimental Setup — Details
Key References
- Gulshan et al., 'Development and validation of a deep learning algorithm for detection of diabetic retinopathy', JAMA 316, 2402 (2016)
- Staal et al., 'Ridge-based vessel segmentation (DRIVE)', IEEE TMI 23, 501 (2004)
Canonical Datasets
- EyePACS (diabetic retinopathy screening)
- DRIVE (Digital Retinal Images for Vessel Extraction)
- MESSIDOR-2
- APTOS 2019 Blindness Detection
OCT Angiography
OCT angiography extends standard OCT by acquiring repeated B-scans at the same location and computing the decorrelation of the complex OCT signal between successive scans. Moving red blood cells cause temporal fluctuations that differ from static tissue, enabling label-free visualization of retinal vasculature. The contrast mechanism uses amplitude decorrelation (SSADA), phase variance, or complex-signal algorithms. Key limitations include motion artifacts, projection artifacts from superficial vessels, and limited field of view.
OCT Angiography
Description
OCT angiography extends standard OCT by acquiring repeated B-scans at the same location and computing the decorrelation of the complex OCT signal between successive scans. Moving red blood cells cause temporal fluctuations that differ from static tissue, enabling label-free visualization of retinal vasculature. The contrast mechanism uses amplitude decorrelation (SSADA), phase variance, or complex-signal algorithms. Key limitations include motion artifacts, projection artifacts from superficial vessels, and limited field of view.
Principle
OCT Angiography detects blood flow non-invasively by comparing repeated OCT B-scans at the same location. Moving red blood cells cause temporal fluctuations in the OCT signal (amplitude and/or phase), while static tissue remains constant. Decorrelation, variance, or differential analysis between repeated scans produces a motion-contrast image revealing the vasculature without the need for injectable contrast agents.
How to Build the System
Use a high-speed OCT system (≥70 kHz A-scan rate, swept-source preferred) capable of repeated B-scans at the same location. Acquire 2-4 repeated B-scans at each position with inter-scan time of 3-10 ms. An eye-tracking system is essential for ophthalmic OCTA to correct microsaccades. Process with split-spectrum amplitude-decorrelation (SSADA), optical microangiography (OMAG), or phase-variance algorithms.
Common Reconstruction Algorithms
- SSADA (Split-Spectrum Amplitude-Decorrelation Angiography)
- OMAG (Optical Micro-Angiography, complex signal differential)
- Phase-variance OCTA
- Deep-learning OCTA denoising and vessel segmentation
- Projection artifact removal algorithms
Common Mistakes
- Bulk tissue motion producing decorrelation artifacts (false flow signals)
- Projection artifacts where superficial vessel shadows appear in deeper layers
- Shadow artifacts beneath large vessels causing false flow voids
- Insufficient inter-scan interval for detecting slow capillary flow
- Motion artifacts from blinks or microsaccades corrupting OCTA volumes
How to Avoid Mistakes
- Apply bulk motion correction (axial and lateral registration) before decorrelation analysis
- Use projection artifact removal algorithms (slab subtraction or OMAG-based)
- Increase number of repeated B-scans to improve SNR and reduce shadow impact
- Optimize inter-scan time: shorter for fast flow, longer for slow capillary flow
- Use active eye tracking and discard frames with large motion; average multiple volumes
Forward-Model Mismatch Cases
- The widefield fallback applies static spatial blur, but OCTA detects blood flow by comparing repeated OCT B-scans — the temporal decorrelation between scans caused by moving red blood cells is not modeled
- OCTA is fundamentally a motion-contrast technique (flow signal = decorrelation or variance between repeated measurements) — the widefield static model has no temporal dimension and cannot detect or distinguish flowing from static tissue
How to Correct the Mismatch
- Use the OCTA operator that models repeated OCT measurements at the same location: static tissue produces correlated signals while flowing blood produces decorrelated signals between repeated scans
- Extract flow maps using SSADA (split-spectrum amplitude decorrelation) or OMAG (optical microangiography) that require multiple temporally separated OCT measurements as input
Experimental Setup — Signal Chain
Experimental Setup — Details
Key References
- Jia et al., 'Split-spectrum amplitude-decorrelation angiography (SSADA)', Opt. Express 20, 4710 (2012)
- Spaide et al., 'OCT Angiography', Prog. Retin. Eye Res. 64, 1 (2018)
Canonical Datasets
- OCTA-500 (Li et al., Scientific Data 2024)
- ROSE retinal OCTA vessel segmentation
Optical Coherence Tomography
OCT is a low-coherence interferometric imaging technique that measures depth-resolved backscattering profiles (A-scans) by interfering sample-arm reflections with a reference mirror. In spectral-domain OCT, the interference spectrum is recorded by a spectrometer and the axial profile is obtained via Fourier transform. Axial resolution is determined by the source bandwidth (typically 3-7 um in tissue) and imaging depth by spectrometer resolution. Dominant artifacts include speckle noise, motion artifacts, and sensitivity roll-off with depth.
Optical Coherence Tomography
Description
OCT is a low-coherence interferometric imaging technique that measures depth-resolved backscattering profiles (A-scans) by interfering sample-arm reflections with a reference mirror. In spectral-domain OCT, the interference spectrum is recorded by a spectrometer and the axial profile is obtained via Fourier transform. Axial resolution is determined by the source bandwidth (typically 3-7 um in tissue) and imaging depth by spectrometer resolution. Dominant artifacts include speckle noise, motion artifacts, and sensitivity roll-off with depth.
Principle
Optical Coherence Tomography uses low-coherence interferometry to produce cross-sectional images of tissue microstructure. A broadband light source (superluminescent diode, ~840 nm or ~1310 nm) is split between sample and reference arms. Interference occurs only when the path lengths match within the coherence length (~5-10 μm), providing axial resolution. Spectral-domain OCT records the spectral interferogram and uses FFT for fast depth-resolved imaging.
How to Build the System
Build or acquire a spectral-domain OCT system: broadband SLD source (center 840 nm, 50 nm bandwidth for retinal; 1310 nm for dermal/cardiac), fiber-based Michelson interferometer, galvo scanner for lateral scanning, and a spectrometer with line camera (2048-4096 pixels) for spectral detection. Calibrate wavelength-to-wavenumber mapping, dispersion compensation, and reference arm delay. For swept-source OCT, use a frequency-swept laser (100-400 kHz sweep rate) and balanced detector.
Common Reconstruction Algorithms
- FFT-based spectral-domain OCT reconstruction (spectral interferogram → A-scan)
- Dispersion compensation (numerical or hardware)
- Speckle reduction (spatial/angular compounding, or deep-learning)
- Segmentation of retinal layers (graph-based, U-Net, or transformer models)
- OCT Angiography (OCTA) via decorrelation or phase-variance of repeated B-scans
Common Mistakes
- Dispersion mismatch between sample and reference arms degrading axial resolution
- Mirror image artifact from complex conjugate ambiguity in SD-OCT
- Sensitivity roll-off at deeper imaging depths not compensated
- Motion artifacts in 3-D OCT volumes (eye motion for ophthalmic OCT)
- Incorrect refractive index assumption for depth scale calibration
How to Avoid Mistakes
- Match fiber lengths and add numerical dispersion compensation in reconstruction
- Place the zero-delay near the sample surface; use full-range OCT if needed
- Use swept-source OCT for reduced roll-off; optimize spectrometer for uniform sensitivity
- Apply eye-tracking or motion-correction algorithms; average repeated B-scans
- Calibrate depth scale with a known-thickness reference standard
Forward-Model Mismatch Cases
- The widefield fallback applies spatial blur, but OCT acquires spectral interferograms that encode depth via low-coherence interferometry — the interference fringe pattern bears no resemblance to a blurred image
- OCT depth resolution comes from the broadband source coherence length (~5-10 um), not from spatial PSF — the widefield operator cannot model the axial sectioning, dispersion, or spectral-to-depth FFT relationship
How to Correct the Mismatch
- Use the OCT operator that models spectral-domain interferometry: y(k) = |E_ref + E_sample(k)|^2, where depth information is encoded in the spectral fringe frequency
- Reconstruct A-scans via FFT of the spectral interferogram after dispersion compensation and k-linearization; B-scans are formed by lateral scanning
Experimental Setup — Signal Chain
Experimental Setup — Details
Key References
- Huang et al., 'Optical coherence tomography', Science 254, 1178 (1991)
- de Boer et al., 'Twenty-five years of OCT', Biomed. Opt. Express 8, 3248 (2017)
Canonical Datasets
- Duke SD-OCT DME dataset (Chiu et al.)
- RETOUCH Challenge (retinal OCT)
- OCTA-500 (Li et al., Scientific Data 2024)