lightcraft/docs/merge.md

Photo Merge (HDR, Panorama, HDR Panorama)

crates/merge (layer L3, no UI dependencies) implements the algorithms; crates/engine/src/merge.rs plans, runs and finishes merges in the library; the desktop UI shows a dialog with a live preview. All algorithms are our own implementations from published papers (cited in the module docs).

Using it

• App: select the photos, then Photo Merge ▸ HDR… (Ctrl+H), Panorama… (Ctrl+M) or HDR Panorama… from the photo context menu (or the dialog.merge* commands). The dialog previews the merge at ≤ 1024 px and re-runs the preview when an option changes; Merge runs the full merge in the background (a row in the activity stack shows its progress, with ✕ to stop it). The result is written next to the first photo as <name>-HDR.dng, -Pano.dng or -HDR-Pano.dng, imported and selected.
• Commands (CLI, MCP, control channel):
  • merge.hdr {ids?, align=true, deghost=none|low|medium|high, autoSettings=true, stack=false, preview=false, showOverlay=false, previewPath?}
  • merge.panorama {ids?, projection=auto|spherical|cylindrical|perspective, boundaryWarp=0..100, autoCrop=false, fillEdges=false, autoSettings=true, maxMegapixels=40, preview, previewPath?}
  • merge.hdrPanorama {ids?, bracket=0 (from EXIF), …both sets of options}
• CLI: lightcraft-cli merge hdr|panorama|hdr-panorama [options] FILES…; lightcraft-cli synth-merge hdr|panorama -o DIR writes synthetic inputs (procedural scene).

What it does

HDR (hdr.rs): frames ordered by EXIF exposure (or median luminance); the middle one is the reference. Auto Align: features on exposure-normalised luminance, RANSAC translation / similarity / homography (the simplest model that explains ≥ 97 % of the matches). Exposure ratios are measured (median log ratio of well-exposed pixels between brightness neighbours), EXIF is the fallback. Radiance is a weighted mean in linear light, weight k · (1 − smoothstep(0.8·clip, 0.96·clip, max channel)). Deghost: a per-pixel consistency test against the reference radiance (with a noise floor), dilated and feathered masks; the union is the deghost overlay. Output: 16-bit float DNG. Lens corrections: frames are merged uncorrected, and the DNG's OpcodeList3 carries the reference frame's (WarpRectilinear, WarpRectilinear2, FixVignetteRadial: a DNG's own, or a camera's correction a raw reader converts, e.g. Sony, Panasonic), re-expressed for its default crop (Opcode::cropped). So Optics ▸ Enable Lens Corrections works on the result as on the sources; a vertical bracket keeps its pixels unturned and its corrections.

Panorama (pano/): MOPS-style features, pairwise RANSAC homographies with Brown & Lowe's verification, focal from homographies (or EXIF 35 mm), rotations chained along a maximum spanning tree, Levenberg–Marquardt bundle adjustment of rotations + focal lengths, straightening. Projections: spherical, cylindrical, perspective; Auto picks perspective for compact fields of view < 110°, cylindrical when the vertical field is < 90°, else spherical. Exposure: EXIF, then gain compensation from the overlaps. Seams: each pixel goes to the image with the highest centre-weighted feather weight; multi-band blending (Laplacian pyramids) in log space. Boundary Warp: a separable stretch of the smoothed valid-area edges to the canvas edges. Auto Crop: the largest rectangle of image data (set as the develop crop, so it stays adjustable). Fill Edges: push–pull diffusion. Output: 16-bit integer DNG (16-bit float for HDR panoramas).

DNG output (output.rs): LinearRaw, 3 samples, Deflate tiles; values scaled below 1.0 with the scale in BaselineExposure; raw sources keep their DNG colour tags (camera RGB), other sources get ColorMatrix1/ForwardMatrix1 of their RGB space.

Measured (synthetic tests, cargo test -p lightcraft-merge)

• HDR, 3 brackets ±2 EV with 14-bit noise and handheld shifts: alignment error < 0.3 px, exposure error < 0.02 EV, radiance median log error < 0.03 (noise-limited, bias < 0.01), recovered highlights < 0.05; moving object: ghost error 0.2–2.0 without deghosting, < 0.06 with High.
• Panorama, 5 views over 182° with a 1.3× exposure drift: relative rotations < 0.1°, focal < 1 %, BA RMS ≈ 0.35 px; median log error vs ground truth < 0.02; no seam steps.

Limits (known gaps)

• No lens-distortion model in the panorama bundle adjustment (rectilinear lenses assumed); no 360° wrap-around; flat (translating-camera) scans are fitted with the rotation model. Panoramas and HDR panoramas carry no lens corrections, nor does an HDR whose frames don't share an orientation (they are turned before merging).
• No camera response calibration for JPEG brackets (sRGB decoding is taken as linear light).
• Boundary Warp is a separable stretch, not a content-preserving mesh warp; Fill Edges is diffusion, not patch synthesis.
• Create Stack: the result goes on top of a collapsed stack of its sources (Catalog::stack_with_ops); its caption also lists them.
• Panorama output is limited to maxMegapixels (default 40) — blending memory is ~25 bytes/pixel.
• The browser build has no file access for merges yet.