mirror of
https://github.com/storytold/lightcraft.git
synced 2026-10-14 19:27:01 -04:00
Claude
4c1dfc5b0f
Merged HDR DNGs lost the camera's embedded lens correction: load_frame strips the lens opcodes from each source (frames are merged uncorrected) and write_linear_dng wrote no OpcodeList3, so Optics ▸ Enable Lens Corrections did nothing on the result (Sony ILCE-7M3/7RM4A, Panasonic/Leica, DNGs with their own opcodes). Each raw frame now keeps the lens corrections it applies (the DNG 1.6 skip rule already applied), re-expressed for the default crop develop keeps by the new Opcode::cropped: the centre moves into the crop and coefficients are rescaled for the crop's farthest corner. HdrResult carries the reference frame's and the DNG writes them as its OpcodeList3. Frames turned before merging (mixed orientations) and panoramas carry none. Tests: cropped corrections give the crop of the corrected whole (WarpRectilinear, WarpRectilinear2 with and without reciprocal, FixVignetteRadial); an HDR merge of cropped, corrected brackets, upright and vertical, gives the same profile correction as its reference photo (fails without the fix). Real brackets not measured yet. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_015gRtdWF1E9358FfzhzYqCS
72 lines
5.1 KiB
Markdown
72 lines
5.1 KiB
Markdown
# Photo Merge (HDR, Panorama, HDR Panorama)
|
||
|
||
`crates/merge` (layer L3, no UI dependencies) implements the algorithms; `crates/engine/src/merge.rs`
|
||
plans, runs and finishes merges in the library; the desktop UI shows a dialog with a live preview.
|
||
All algorithms are our own implementations from published papers (cited in the module docs).
|
||
|
||
## Using it
|
||
|
||
- **App:** select the photos, then *Photo Merge ▸ HDR…* (`Ctrl+H`), *Panorama…* (`Ctrl+M`) or
|
||
*HDR Panorama…* from the photo context menu (or the `dialog.merge*` commands). The dialog previews
|
||
the merge at ≤ 1024 px and re-runs the preview when an option changes; *Merge* runs the full merge
|
||
in the background (a row in the activity stack shows its progress, with ✕ to stop it). The result is written next to the first photo as
|
||
`<name>-HDR.dng`, `-Pano.dng` or `-HDR-Pano.dng`, imported and selected.
|
||
- **Commands** (CLI, MCP, control channel):
|
||
- `merge.hdr {ids?, align=true, deghost=none|low|medium|high, autoSettings=true, stack=false, preview=false, showOverlay=false, previewPath?}`
|
||
- `merge.panorama {ids?, projection=auto|spherical|cylindrical|perspective, boundaryWarp=0..100, autoCrop=false, fillEdges=false, autoSettings=true, maxMegapixels=40, preview, previewPath?}`
|
||
- `merge.hdrPanorama {ids?, bracket=0 (from EXIF), …both sets of options}`
|
||
- **CLI:** `lightcraft-cli merge hdr|panorama|hdr-panorama [options] FILES…`;
|
||
`lightcraft-cli synth-merge hdr|panorama -o DIR` writes synthetic inputs (procedural scene).
|
||
|
||
## What it does
|
||
|
||
**HDR** (`hdr.rs`): frames ordered by EXIF exposure (or median luminance); the middle one is the
|
||
reference. Auto Align: features on exposure-normalised luminance, RANSAC translation / similarity /
|
||
homography (the simplest model that explains ≥ 97 % of the matches). Exposure ratios are measured
|
||
(median log ratio of well-exposed pixels between brightness neighbours), EXIF is the fallback.
|
||
Radiance is a weighted mean in linear light, weight `k · (1 − smoothstep(0.8·clip, 0.96·clip, max
|
||
channel))`. Deghost: a per-pixel consistency test against the reference radiance (with a noise
|
||
floor), dilated and feathered masks; the union is the deghost overlay. Output: 16-bit float DNG.
|
||
Lens corrections: frames are merged uncorrected, and the DNG's `OpcodeList3` carries the reference frame's
|
||
(`WarpRectilinear`, `WarpRectilinear2`, `FixVignetteRadial`: a DNG's own, or a camera's correction a raw reader
|
||
converts, e.g. Sony, Panasonic), re-expressed for its default crop (`Opcode::cropped`). So Optics ▸ Enable Lens
|
||
Corrections works on the result as on the sources; a vertical bracket keeps its pixels unturned and its
|
||
corrections.
|
||
|
||
**Panorama** (`pano/`): MOPS-style features, pairwise RANSAC homographies with Brown & Lowe's
|
||
verification, focal from homographies (or EXIF 35 mm), rotations chained along a maximum spanning
|
||
tree, Levenberg–Marquardt bundle adjustment of rotations + focal lengths, straightening.
|
||
Projections: spherical, cylindrical, perspective; *Auto* picks perspective for compact fields of
|
||
view < 110°, cylindrical when the vertical field is < 90°, else spherical. Exposure: EXIF, then
|
||
gain compensation from the overlaps. Seams: each pixel goes to the image with the highest
|
||
centre-weighted feather weight; multi-band blending (Laplacian pyramids) in log space. Boundary
|
||
Warp: a separable stretch of the smoothed valid-area edges to the canvas edges. Auto Crop: the
|
||
largest rectangle of image data (set as the develop crop, so it stays adjustable). Fill Edges:
|
||
push–pull diffusion. Output: 16-bit integer DNG (16-bit float for HDR panoramas).
|
||
|
||
**DNG output** (`output.rs`): LinearRaw, 3 samples, Deflate tiles; values scaled below 1.0 with the
|
||
scale in `BaselineExposure`; raw sources keep their DNG colour tags (camera RGB), other sources get
|
||
`ColorMatrix1`/`ForwardMatrix1` of their RGB space.
|
||
|
||
## Measured (synthetic tests, `cargo test -p lightcraft-merge`)
|
||
|
||
- HDR, 3 brackets ±2 EV with 14-bit noise and handheld shifts: alignment error < 0.3 px, exposure
|
||
error < 0.02 EV, radiance median log error < 0.03 (noise-limited, bias < 0.01), recovered
|
||
highlights < 0.05; moving object: ghost error 0.2–2.0 without deghosting, < 0.06 with High.
|
||
- Panorama, 5 views over 182° with a 1.3× exposure drift: relative rotations < 0.1°, focal < 1 %,
|
||
BA RMS ≈ 0.35 px; median log error vs ground truth < 0.02; no seam steps.
|
||
|
||
## Limits (known gaps)
|
||
|
||
- No lens-distortion model in the panorama bundle adjustment (rectilinear lenses assumed); no
|
||
360° wrap-around; flat (translating-camera) scans are fitted with the rotation model. Panoramas and HDR
|
||
panoramas carry no lens corrections, nor does an HDR whose frames don't share an orientation (they are turned
|
||
before merging).
|
||
- No camera response calibration for JPEG brackets (sRGB decoding is taken as linear light).
|
||
- Boundary Warp is a separable stretch, not a content-preserving mesh warp; Fill Edges is diffusion,
|
||
not patch synthesis.
|
||
- *Create Stack*: the result goes on top of a collapsed stack of its sources (`Catalog::stack_with_ops`); its
|
||
caption also lists them.
|
||
- Panorama output is limited to `maxMegapixels` (default 40) — blending memory is ~25 bytes/pixel.
|
||
- The browser build has no file access for merges yet.
|