Passive Camera Radar
`passive-camera-radar` is an early-stage C++ scaffold for a multi-camera passive sensing pipeline:
- multiple asynchronous USB cameras
- per-camera motion extraction from successive frame differences
- backprojection of dense per-pixel motion into two decaying sparse voxel spaces
- one Earth-fixed grid and one inertial sky grid
- eventual GPU-heavy execution path with OpenGL compute shaders and OpenGL rendering
The current repository contains the architecture, a buildable starter implementation, and the first CPU-backed mock pipeline. It is intentionally structured so GPU decode, camera ingest, sparse voxel backends, and rendering can be replaced independently.
Recommended Stack
- `C++20`
- `CMake`
- `GStreamer` for the eventual multi-camera ingest and compressed decode path
- `OpenCV` for calibration and offline tooling
- `GLFW + GLAD + OpenGL 4.6`
- `GLM` for rendering math
- `Eigen` for pose / transform math
- `ImGui` for debug tools
Use `GLAD` or `GLEW`, not both. This scaffold assumes `GLAD`.
Architecture
The system should be asynchronous end-to-end. Camera frames do not need to be synchronized. Each frame is processed with its own capture timestamp and integrated into global state independently.
Dataflow
`camera thread -> timestamped frame -> per-camera motion extraction -> dense motion field -> timestamped backprojection -> Earth grid + sky grid accumulation -> time decay -> renderer snapshot`
Main subsystems
- `capture/`
- camera interfaces
- mock camera implementation
- future GStreamer ingest
- `motion/`
- frame differencing
- dense motion magnitude extraction
- thresholded debug overlay extraction
- `pose/`
- intrinsics
- timestamped pose lookup
- transform helpers
- `fusion/`
- pixel backprojection to rays
- `grid/`
- Earth-fixed sparse voxel grid
- sky angular sparse grid
- LOD quantization policy
- `gpu/`
- future OpenGL compute path
- placeholder shader entry points
- `render/`
- future OpenGL renderer
- current no-op snapshot sink
Coordinate Strategy
Two spaces are treated differently:
- `Earth grid`
- sparse Cartesian voxel grid
- suited for nearer content and camera-centric / Earth-fixed accumulation
- should become a clipmap or brick-hash structure
- `Sky grid`
- sparse angular grid
- suited for deep-space / distant-star observations where radial depth is weak or unobservable
- should become a cubemap or HEALPix-like hierarchy later
LOD Strategy
LOD should be selected so one voxel projects to roughly one pixel at the relevant distance:
`target_cell_size ~= range / fx`
Quantize that target cell size to powers of two relative to the base voxel size. The current starter code uses that principle for Earth-grid point samples derived from backprojected rays.
Current Implementation
The current codebase implements:
- a buildable C++ executable
- a native Windows Media Foundation single-camera test path
- a mock asynchronous camera source fallback
- per-camera grayscale frame differencing
- dense per-pixel motion magnitudes for voxel fusion
- pose buffering by timestamp
- calibration-file loading for intrinsics, distortion, extrinsics, and timestamped pose samples
- star-trail solved site/time/orientation metadata for inertial sky alignment
- distortion-aware pixel backprojection with precomputed undistortion ray maps
- sparse decaying Earth and sky accumulators
- a native Windows OpenGL preview window
- 3D Earth voxel view
- 3D sky voxel view on a distant sphere
- live camera frame inset with motion overlay
- placeholder GLSL compute/graphics shader files
What it does not implement yet:
- real USB camera ingest
- hardware decode
- OpenGL rendering
- compute-shader processing
- sparse brick pool / clipmap paging
- automatic in-app star detection / solve fitting
Build
Configure and build:
cmake -S . -B build
cmake --build build
Run:
.\build\passive_camera_radar.exe
To use a calibration bundle, point `PCR_CALIBRATION_FILE` at an `.ini` file before launch:
$env:PCR_CALIBRATION_FILE = ".\calibration\example-star-trail-calibration.ini"
.\build\Debug\passive_camera_radar.exe
The default build path does not require external libraries. Optional dependency discovery for the recommended stack is wired into CMake and can be expanded as the runtime path is implemented.
On Windows, the current scaffold can try a real USB/UVC camera through Media Foundation without requiring `GStreamer` or `OpenCV` to be installed system-wide.
When preview is enabled, the app runs until the preview window is closed.
Calibration File
The runtime loader accepts repeated `[camera]` and `[pose]` sections.
See [example-star-trail-calibration.ini](calibration/example-star-trail-calibration.ini) for a complete example.
`[camera]` supports:
- `id`, `width`, `height`
- either `fx` / `fy` / `cx` / `cy` or `horizontal_fov_degrees`
- `distortion_model` plus `k1..k6` and `p1`, `p2`
- `calibration_source`
- `reference_unix_time_seconds` and `camera_time_offset_seconds`
- `observer_latitude_degrees`, `observer_longitude_degrees`, `observer_altitude_m`
- either `earth_rotation_matrix` or star-solved `forward_azimuth_degrees`, `forward_altitude_degrees`, `roll_degrees`
- optional local `earth_translation_m`
`[pose]` supports timestamped overrides for either `earth_fixed` or `sky_inertial`:
- `camera_id`, `frame`, `timestamp_seconds`
- `translation_m`
- `rotation_matrix`
Preview controls
- `Mouse`: look around with captured cursor
- `W / A / S / D`: horizontal movement from current view
- `Space / Left Shift`: vertical movement
- `Left Ctrl`: speed boost
- `F11`: toggle fullscreen
- `Home`: reset the camera
- `Esc`: close the preview
Near-Term Implementation Order
1. replace the mock camera with `GStreamer` camera ingest
2. expand the native ingest path to true multi-camera asynchronous capture
3. move frame differencing, dense motion extraction, and optional debug compaction onto GPU
4. replace the simple Earth grid with sparse bricks / clipmaps
5. replace the legacy fixed-function preview with the intended `GLFW + GLAD + modern OpenGL` renderer path
6. expand calibration fitting, validation, and automatic star-solve tooling