3D Gaussian Splatting
3D Gaussian splatting represents scenes as a collection of learnable 3D Gaussian primitives, each parameterized by position, covariance (anisotropic 3D extent), opacity, and spherical harmonic color coefficients. Rendering rasterizes the Gaussians by projecting them to 2D screen space, sorting by depth, and alpha-compositing with a tile-based differentiable rasterizer. Training optimizes Gaussian parameters via gradient descent with adaptive density control (splitting, cloning, pruning). This achieves real-time (30+ fps) rendering at quality comparable to NeRF, from SfM point cloud initialization (COLMAP).
Gaussian Rasterization
Gaussian
gaussian splatting 3dgs
RGB_CAMERA
Forward-Model Signal Chain
Each primitive represents a physical operation in the measurement process. Arrows show signal flow left to right.
Π(splat) → Σ(alpha) → D(g, η₁)
Benchmark Variants & Leaderboards
3DGS
3D Gaussian Splatting
Π(splat) → Σ(alpha) → D(g, η₁)
Standard Leaderboard (Top 10)
| # | Method | Score | PSNR (dB) | SSIM | Trust | Source |
|---|---|---|---|---|---|---|
| 🥇 | NeRFactor2 | 0.831 | 35.85 | 0.966 | ✓ Certified | Barron et al., NeurIPS 2024 |
| 🥈 | GaussianShader | 0.816 | 35.18 | 0.960 | ✓ Certified | Wang et al., ICCV 2024 |
| 🥉 | 2DGS | 0.811 | 34.67 | 0.966 | ✓ Certified | Huang et al., CVPR 2024 |
| 4 | 3D-GS++ | 0.801 | 34.52 | 0.952 | ✓ Certified | Kerbl et al., SIGGRAPH 2024 |
| 5 | NeRF | 0.779 | 33.15 | 0.954 | ✓ Certified | Mildenhall et al., ECCV 2020 |
| 6 | 3D-GS | 0.775 | 33.3 | 0.940 | ✓ Certified | Kerbl et al., SIGGRAPH 2023 |
| 7 | Instant-NGP | 0.721 | 31.1 | 0.905 | ✓ Certified | Muller et al., SIGGRAPH 2022 |
| 8 | Mesh-GS | 0.710 | 30.07 | 0.918 | ✓ Certified | Li et al., ECCV 2024 |
| 9 | Mip-NeRF 360 | 0.662 | 29.4 | 0.844 | ✓ Certified | Barron et al., CVPR 2022 |
| 10 | Photogrammetry | 0.616 | 26.54 | 0.847 | ✓ Certified | Structure-from-Motion baseline |
Showing top 10 of 11 methods. View all →
Mismatch Parameters (3) click to expand
| Name | Symbol | Description | Nominal | Perturbed |
|---|---|---|---|---|
| camera_pose | ΔT | Camera pose error (mm / deg) | 0 | 1.0 |
| focal_length | Δf | Focal length error (pixels) | 0 | 5.0 |
| point_cloud_init | ΔP | Initial point cloud noise (mm) | 0 | 2.0 |
Reconstruction Triad Diagnostics
The three diagnostic gates (G1, G2, G3) characterize how reconstruction quality degrades under different error sources. Each bar shows the relative attribution.
Model: gaussian rasterization — Mismatch modes: pose error, sparse initialization, floater artifacts, popping artifacts
Noise: gaussian — Typical SNR: 25.0–45.0 dB
Requires: camera intrinsics, camera extrinsics, sfm point cloud, scene scale
Modality Deep Dive
Principle
3-D Gaussian Splatting represents a scene as a set of anisotropic 3-D Gaussians, each with position, covariance, opacity, and spherical harmonics color coefficients. Novel views are rendered by projecting (splatting) these Gaussians onto the image plane and alpha-compositing them in depth order. Unlike NeRF, rendering is rasterization-based and achieves real-time frame rates (≥100 fps) with high visual quality.
How to Build the System
Start with the same multi-view image dataset as NeRF (50-200 posed images via COLMAP). Initialize 3-D Gaussians from the SfM point cloud. Train by differentiable rasterization: project Gaussians to each training view, compute photometric loss (L1 + SSIM), and optimize positions, covariances, colors, and opacities via Adam. Adaptive densification (splitting/cloning Gaussians) and pruning runs periodically during training. Training takes ~15-30 minutes on a modern GPU.
Common Reconstruction Algorithms
- 3D Gaussian Splatting (original, Kerbl et al. 2023)
- Mip-Splatting (anti-aliased multi-scale Gaussian splatting)
- SuGaR (Surface-Aligned Gaussian Splatting for mesh extraction)
- Dynamic 3D Gaussians (for dynamic scenes / video)
- Compact-3DGS (compressed Gaussian representations)
Common Mistakes
- Insufficient initial SfM points causing sparse reconstruction
- Too few training views creating holes or floater artifacts in novel views
- Excessive Gaussian count (millions) consuming too much GPU memory
- Not using adaptive densification, leaving under-reconstructed regions
- Ignoring exposure variation between training images
How to Avoid Mistakes
- Use dense SfM initialization; increase COLMAP matching thoroughness if sparse
- Capture more views, especially in regions that are under-represented
- Apply periodic pruning of low-opacity Gaussians to control memory
- Enable adaptive densification and set proper gradient thresholds for splitting
- Apply per-image exposure compensation or normalize images before training
Forward-Model Mismatch Cases
- The widefield fallback processes a single 2D (64,64) image, but Gaussian splatting renders multi-view images from a set of 3D Gaussian primitives — output shape (n_views, H, W) encodes view-dependent appearance
- Gaussian splatting is a nonlinear rendering process (alpha-compositing of projected 3D Gaussians sorted by depth) — the widefield linear blur cannot model 3D-to-2D projection, depth ordering, or view-dependent effects
How to Correct the Mismatch
- Use the Gaussian splatting operator that projects 3D Gaussian primitives onto each camera plane via differentiable rasterization with alpha compositing
- Optimize Gaussian parameters (position, covariance, opacity, color SH coefficients) to minimize rendering loss across training views using the correct splatting forward model
Experimental Setup
24-300 (scene-dependent)
~1600x1200
SfM point cloud (COLMAP)
30
unbounded indoor / outdoor
30000
PSNR / SSIM / LPIPS
Mip-NeRF360, Tanks & Temples, Deep Blending
Signal Chain Diagram
Key References
- Kerbl et al., '3D Gaussian Splatting for Real-Time Radiance Field Rendering', SIGGRAPH 2023
Canonical Datasets
- Mip-NeRF 360 (9 scenes)
- Tanks & Temples (Knapitsch et al.)
- Deep Blending (Hedman et al.)