From 5a61fc5f67eb6b6a8b27529b9ef10c5e7571be95 Mon Sep 17 00:00:00 2001 From: Alex Chambers Date: Fri, 9 Oct 2026 20:04:16 +0200 Subject: [PATCH] LR-PROF-CAMERACOLOR: Sony ARWs shot with DRO start without the camera JPEG's lifted shadows (#244) Sony's Dynamic Range Optimizer (factory default Auto) brightens the darker regions of the camera JPEG, not the raw. The tone curve fitted to that JPEG carried the brightening into the raw's starting look: on 8 ILCE-7CR photos with DRO on, LightCraft rendered +6.4 L* from Lightroom's default (median dE76 7.5), against -0.2 L* (3.6) for the 16 with DRO off. - lightcraft_raw::embedded_preview_dynamic_range_optimized reads Sony maker-note tag 0xb025 (every documented value: 0 off; 1-3, 8-12 and 16-23, i.e. up to Lv8, on). It was present on all 264 Sony raws checked (98 bodies); 0xb04f only says on/off and is missing on older bodies. - With DRO on, the look keeps the colour fitted to the JPEG and its tone curve is lowered, knot by knot, to Sony's curve without DRO wherever that is darker (assets/camera-tone/sony-dro-off.json, pooled from 12 CC0 raw.pixls.us ARWs of 10 interchangeable-lens bodies; sources and derivation in the file and assets/ATTRIBUTION.md). The second-last knot is lowered only as far as keeps the shoulder past the last knot at or below the fitted curve, so the result is never brighter anywhere. Compacts that map raw values darker keep their own curve. - DRO off, unknown and other makers: unchanged (byte-identical renders on 71 + 35 photos). Render cache version 23. DRO-on photos now land at +0.8 L* from Lightroom (median dE76 3.3), band by band like the DRO-off ones. Docs: camera-preview-colour.md, parity row. Co-Authored-By: Claude Opus 5.5 --- assets/ATTRIBUTION.md | 1 + assets/camera-tone/sony-dro-off.json | 52 ++++++ crates/engine/src/camera_preview.rs | 238 ++++++++++++++++++++++++++- crates/engine/src/media.rs | 2 +- crates/pipeline/src/tone.rs | 5 + crates/raw/src/lib.rs | 2 +- crates/raw/src/preview.rs | 15 ++ crates/raw/src/vendor/arw.rs | 101 ++++++++++++ docs/camera-preview-colour.md | 12 +- docs/parity.md | 2 +- 10 files changed, 424 insertions(+), 6 deletions(-) create mode 100644 assets/camera-tone/sony-dro-off.json diff --git a/assets/ATTRIBUTION.md b/assets/ATTRIBUTION.md index 52074466d..9d7afce6e 100644 --- a/assets/ATTRIBUTION.md +++ b/assets/ATTRIBUTION.md @@ -13,6 +13,7 @@ covers a whole directory), or if a licence file referenced in the Licence column | `assets/camera-profiles/*.json` | Camera colour profiles (matrix + hue/saturation/value table), built into LightCraft | ILCE-7M4: Harald Hoyer | original work: fitted by `lightcraft-cli calibrate` to the contributor's own raw photos and their in-camera JPEGs (597 ILCE-7M4 photos, 2026); aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-07 | none | | `packaging/macos/dmg/` (`background.svg`, the `background.tiff` rendered from it by `generate.py`, and the `dmg-layout.DS_Store` layout file) | LightCraft macOS DMG window background | @XusBadia | original work for this repository, derived from the LightCraft app icon (`assets/app-icon/lightcraft.svg`, referenced via ``, not copied); text is outlined Inter / JetBrains Mono glyph paths | MIT OR Apache-2.0 | 2026-10-08 | none (original); used for the Finder window of the macOS DMG, `packaging/macos/dmg/generate.py` renders it to `background.tiff` | | `assets/camera-profiles/X-H2S.json`, `assets/camera-profiles/X-T4.json` | Fujifilm camera colour profiles (matrix + hue/saturation/value table), built into LightCraft | LightCraft contributors; source photographs supplied by Sebastian Pfluegelmeier | original work: fitted by `lightcraft-cli calibrate` to 201 X-H2S and 545 X-T4 RAFs and their in-camera JPEGs; aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-08 | none | +| `assets/camera-tone/sony-dro-off.json` | Sony global tone curve with the Dynamic Range Optimizer off (32 knots), built into LightCraft for ARWs shot with DRO on | LightCraft contributors; source photographs by raw.pixls.us contributors | original work: tone curve quantile-matched over the pooled pixel pairs of 12 CC0 raw.pixls.us ARWs (DRO off, Standard creative style; ILCE-3500, -5000, -6600, -7CR, -7M3 ×3, -7S, -9M2, ILCA-77M2, SLT-A57, NEX-F3) and their embedded camera JPEGs; the file lists each source URL and SHA-256 and how the curve was derived (see also `docs/camera-preview-colour.md`); aggregate tone statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 (sources CC0) | 2026-10-09 | none | | `crates/scenes/` (generated images) | Procedural demo photographs | LightCraft contributors | original work (generated at runtime, no source imagery) | MIT OR Apache-2.0 | 2026-09-30 | n/a | | `crates/ui-egui/src/icons.rs` | UI icons drawn as vector paths in code | LightCraft contributors | original work | MIT OR Apache-2.0 | 2026-09-30 | n/a | | `docs/images/*-tetons.jpg`, `docs/images/grid-pd.jpg` (screenshots containing it) | "The Tetons and the Snake River" | Ansel Adams (1942), U.S. National Archives | https://commons.wikimedia.org/wiki/File:Adams_The_Tetons_and_the_Snake_River.jpg | Public domain (U.S. federal government work) | 2026-09-30 | developed in LightCraft; shown inside app screenshots | diff --git a/assets/camera-tone/sony-dro-off.json b/assets/camera-tone/sony-dro-off.json new file mode 100644 index 000000000..46397044f --- /dev/null +++ b/assets/camera-tone/sony-dro-off.json @@ -0,0 +1,52 @@ +{ + "description": "Sony's global tone curve with the Dynamic Range Optimizer (DRO) off, Standard creative style: knots of scene luminance (white-balanced camera RGB with baseline exposure, through the colour model) to display-linear luminance. The camera look of an ARW shot with DRO on is lowered to it wherever it is darker (crates/engine/src/camera_preview.rs, docs/camera-preview-colour.md, issue #244).", + "derivation": "Each source raw was paired with its own embedded camera JPEG at the 96-pixel proxy of the per-photo fit (camera_preview::collect_pairs). The pairs of all 12 files were pooled as one model, as lightcraft-cli calibrate pools a camera profile: a colour matrix and hue/saturation table were fitted to the pool, and the tone curve was quantile-matched (camera_preview::fit_tone) between the pooled scene luminances through that colour model and the JPEG luminances. Knots rounded to 5 decimals. Aggregate statistics only, no image content.", + "sources": [ + { "body": "ILCA-77M2", "url": "https://raw.pixls.us/data/Sony/ILCA-77M2/_DSC0244.ARW", "sha256": "f41ceff680b675460cc1fe258a24d8c8c01c52a300933f3ab6401d9a20b60dcc", "licence": "CC0" }, + { "body": "ILCE-3500", "url": "https://raw.pixls.us/data/Sony/ILCE-3500/DSC06923.ARW", "sha256": "04c4fe04425c3e6fff7c8af0f811923c35e750300373357bd54342150c5505d8", "licence": "CC0" }, + { "body": "ILCE-5000", "url": "https://raw.pixls.us/data/Sony/ILCE-5000/_DSC0793.ARW", "sha256": "a919c8535eed68577f1745fb9674ccf333962628ccdec6d48a6f635208467f6c", "licence": "CC0" }, + { "body": "ILCE-6600", "url": "https://raw.pixls.us/data/Sony/ILCE-6600/_DSC0268.ARW", "sha256": "041d71d70148ca77164f19d061e3582adb34399ea5a4baba32960f2956fb0362", "licence": "CC0" }, + { "body": "ILCE-7CR", "url": "https://raw.pixls.us/data/Sony/ILCE-7CR/DSC00798.ARW", "sha256": "f5096e8fbccf0842c8a57763cabd2836608f019eb7fd54f7b22260c2f5bc4da5", "licence": "CC0" }, + { "body": "ILCE-7M3", "url": "https://raw.pixls.us/data/Sony/ILCE-7M3/UNCOMPRESSED_14bit.ARW", "sha256": "ece80551abf64949dbe826985a80f1d9b265401ee4ea6b989e1829549616fdcd", "licence": "CC0" }, + { "body": "ILCE-7M3", "url": "https://raw.pixls.us/data/Sony/ILCE-7M3/_DSC0009.ARW", "sha256": "250784580ea527442c09004417bb0eead484f2bf3ee8f9121a776ac65bb50d0f", "licence": "CC0" }, + { "body": "ILCE-7M3", "url": "https://raw.pixls.us/data/Sony/ILCE-7M3/_DSC0010.ARW", "sha256": "3ae2ff75cfa7240d0b4458f662afd28665c5fd1e83d230ac73f5ad69d3df8592", "licence": "CC0" }, + { "body": "ILCE-7S", "url": "https://raw.pixls.us/data/Sony/ILCE-7S/DSC04125.ARW", "sha256": "7cc338a0abc8fdad32d61006f1f8f412297b93e040c7efba572eb2bd8f8e8be2", "licence": "CC0" }, + { "body": "ILCE-9M2", "url": "https://raw.pixls.us/data/Sony/ILCE-9M2/SONY_A9II_(ILCE-9M2)_-_compressed_(3:2)_14bit.ARW", "sha256": "161c2a9da2b5f1e50be6117a0b4da0ce249660b5d7de13568d301c4034716b97", "licence": "CC0" }, + { "body": "NEX-F3", "url": "https://raw.pixls.us/data/Sony/NEX-F3/_DSC4017.ARW", "sha256": "cfdc60d118b5ce2dce10d459dcb98ad63c6db3e609999932ca8964d86daf836b", "licence": "CC0" }, + { "body": "SLT-A57", "url": "https://raw.pixls.us/data/Sony/SLT-A57/a57-06125.arw", "sha256": "0e9064f369a3947f892c1d7b1971f75d3879f5673134932d5cc7a67bf4263ba4", "licence": "CC0" } + ], + "knots": [ + [0.01655, 0.01799], + [0.01970, 0.02376], + [0.02212, 0.03008], + [0.02469, 0.03634], + [0.02737, 0.04320], + [0.03049, 0.05044], + [0.03425, 0.05812], + [0.03824, 0.06667], + [0.04232, 0.07817], + [0.04658, 0.09059], + [0.05095, 0.10268], + [0.05550, 0.11731], + [0.05962, 0.13173], + [0.06409, 0.14529], + [0.06843, 0.15939], + [0.07314, 0.17363], + [0.07800, 0.18897], + [0.08298, 0.20502], + [0.08909, 0.22159], + [0.09693, 0.23964], + [0.10540, 0.26082], + [0.11579, 0.28882], + [0.12763, 0.31986], + [0.14079, 0.35198], + [0.15786, 0.39293], + [0.17949, 0.44101], + [0.20330, 0.49407], + [0.23466, 0.54244], + [0.27207, 0.60078], + [0.31673, 0.66533], + [0.38682, 0.74143], + [0.52622, 0.80307] + ] +} diff --git a/crates/engine/src/camera_preview.rs b/crates/engine/src/camera_preview.rs index dd3d7361f..0557ced5c 100644 --- a/crates/engine/src/camera_preview.rs +++ b/crates/engine/src/camera_preview.rs @@ -51,13 +51,17 @@ pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransf } let (sensor, reference) = proxies(raw, bytes, transform, PROXY)?; // A camera profile pooled from many photos knows colours this photo shows too little of; - // try its colour first and fit tone/chroma per photo (DRO and picture styles vary). + // try its colour first and fit tone/chroma per photo (picture styles vary). let profile = raw.metadata.model.as_deref().and_then(crate::camera_profiles::get); let colour = profile.as_ref().and_then(|p| Some((p.matrix().mul(&transform.matrix.inverse()?), p.hue_sat.clone()))); let look = fit_pairs_with(&sensor, &reference, colour)?; + // Sony's Dynamic Range Optimizer (on by default) brightens the camera JPEG's darker regions, + // not the raw: keep the colour fitted to the JPEG, take the tone curve without DRO. + let dro = lightcraft_raw::embedded_preview_dynamic_range_optimized(bytes) == Some(true); + let look = if dro { with_dro_off_tone(look) } else { look }; if lightcraft_pipeline::profiling() { eprintln!( - "[profile] {:?} camera look: {:?}, {:?}, hue/sat table {}, camera profile available {}", + "[profile] {:?} camera look: {:?}, {:?}, hue/sat table {}, camera profile available {}, DRO tone replaced {dro}", raw.format, look.matrix.0, look.tone, @@ -68,6 +72,57 @@ pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransf Some(look) } +/// Sony's global tone curve with the Dynamic Range Optimizer off (Standard creative style), pooled +/// from 12 CC0 raw.pixls.us ARWs of 10 interchangeable-lens bodies and their camera JPEGs; the file +/// lists them and how the curve was derived (`assets/ATTRIBUTION.md`). Pooled the same way from 55 +/// private ILCE-7CR photos with DRO off, the curve differs by at most 1.2 L* between scene luminance +/// 0.01 and 0.5. Issue #244, `docs/camera-preview-colour.md`. +const SONY_DRO_OFF_TONE: &str = include_str!("../../../assets/camera-tone/sony-dro-off.json"); + +/// [`SONY_DRO_OFF_TONE`]'s curve (`None` only if the built-in file were invalid; a test checks it). +fn sony_dro_off_tone() -> Option { + #[derive(serde::Deserialize)] + struct File { + knots: [[f32; 2]; 32], + } + static TONE: std::sync::OnceLock> = std::sync::OnceLock::new(); + *TONE.get_or_init(|| serde_json::from_str::(SONY_DRO_OFF_TONE).ok().and_then(|f| CameraTone::new(f.knots))) +} + +/// `look`, fitted to a camera JPEG brightened by Sony's Dynamic Range Optimizer, with its tone +/// curve lowered to Sony's without DRO ([`SONY_DRO_OFF_TONE`]) wherever that is darker. DRO only +/// brightens, so the fitted curve bounds the curve without it from above, everywhere: where Sony's +/// curve is brighter than the photo's own (the 1″ compacts map raw values darker), the photo keeps +/// its own. Its colour (matrix, hue/saturation table and chroma curve) stays as fitted. +fn with_dro_off_tone(look: CameraLook) -> CameraLook { + let Some(dro_off) = sony_dro_off_tone() else { return look }; + let own = look.tone.knots(); + let mut knots = own.map(|[x, y]| [x, y.min(dro_off.apply(x))]); + // Past its last knot a curve goes on as a shoulder whose rate comes from its last two knots + // (`CameraTone::apply`): lowering the second-last knot more than the last steepens it, and the + // extended highlights would end up above the fitted curve. Raise the second-last knot (never + // above the fitted curve) until the rate is at most the fitted curve's. + let ([a, b], last) = ([own[30], own[31]], knots[31]); + let rate = ((b[1] - a[1]) / (b[0] - a[0])).clamp(0.1, 16.0) / (1.0 - b[1]).max(0.01); + let floor = last[1] - rate * (1.0 - last[1]).max(0.01) * (b[0] - a[0]); + knots[30][1] = knots[30][1].max(floor).min(last[1]); + let Some(tone) = CameraTone::new(knots).and_then(|tone| tone.with_chroma(*look.tone.chroma())) else { return look }; + if !at_most(&tone, &look.tone) { + return look; + } + CameraLook { tone, ..look } +} + +/// Whether curve `a` is nowhere brighter than `b` over the scene luminances the finish stage's tone +/// table covers ([`ToneMap::camera`]). +fn at_most(a: &CameraTone, b: &CameraTone) -> bool { + use lightcraft_pipeline::tone::{GREY, LUT_MAX_EV, LUT_MIN_EV, LUT_N}; + (0..LUT_N).all(|i| { + let x = GREY * 2f32.powf(LUT_MIN_EV + (LUT_MAX_EV - LUT_MIN_EV) * i as f32 / (LUT_N - 1) as f32); + a.apply(x) <= b.apply(x) + 1e-5 + }) +} + /// Same-size proxies of the sensor (white-balanced, baseline exposure, through `transform`'s /// matrix: the generic camera ≈ sRGB model) and of the file's embedded camera JPEG (linear Rec.2020). fn proxies(raw: &RawImage, bytes: &[u8], transform: &CameraTransform, size: usize) -> Option<(Rgb32f, Rgb32f)> { @@ -1234,6 +1289,185 @@ mod tests { assert!(info.camera_tone.is_some(), "no camera look fitted"); } + /// A look with tone curve `f` at knots `xs` and a colour model and chroma curve to keep. + fn dro_test_look(xs: &[f32; 32], f: &dyn Fn(f32) -> f32) -> CameraLook { + CameraLook { + matrix: Mat3([[1.6, -0.4, -0.2], [-0.2, 1.4, -0.2], [0.0, -0.3, 1.3]]), + tone: CameraTone::new(xs.map(|x| [x, f(x)])).unwrap().with_chroma([1.3, 1.2, 1.1, 1.0, 1.0, 0.9, 0.6, 0.3]).unwrap(), + hue_sat: Some(HsvTable { hue_divisions: 4, sat_divisions: 2, val_divisions: 1, data: vec![[5.0, 1.1, 1.0]; 8], srgb_value: false }), + } + } + + /// `fixed` (from `fitted`) is `min(fitted, Sony's DRO-off curve)` at every knot but the + /// second-last, which may stay higher (at most the fitted curve) to keep the shoulder past the + /// last knot below the fitted one; nowhere brighter than `fitted`, colour unchanged. + fn assert_lowered(fitted: &CameraLook, fixed: &CameraLook, what: &str) { + let dro_off = sony_dro_off_tone().unwrap(); + assert_eq!((fixed.matrix, &fixed.hue_sat, fixed.tone.chroma()), (fitted.matrix, &fitted.hue_sat, fitted.tone.chroma()), "{what}"); + for (i, (k, own)) in fixed.tone.knots().iter().zip(fitted.tone.knots()).enumerate() { + let lowered = own[1].min(dro_off.apply(own[0])); + assert_eq!(k[0], own[0], "{what}: knot {i}"); + if i == 30 { + assert!(k[1] >= lowered && k[1] <= own[1], "{what}: knot {i} {} not in {lowered}..={}", k[1], own[1]); + } else { + assert!((k[1] - lowered).abs() < 1e-6, "{what}: knot {i} {} vs {lowered}", k[1]); + } + } + assert!(at_most(&fixed.tone, &fitted.tone), "{what}: brighter than the fitted curve somewhere"); + // the same check spelled out, from deep shadows to far past the last knot (the shoulder) + for i in 0..=2400 { + let x = 0.001 * 1.005f32.powi(i); + assert!(fixed.tone.apply(x) <= fitted.tone.apply(x) + 1e-5, "{what} at {x}: {} > {}", fixed.tone.apply(x), fitted.tone.apply(x)); + } + } + + /// Issue #244: Sony's Dynamic Range Optimizer lifts the camera JPEG's shadows, not the raw's. A + /// look fitted to such a JPEG keeps its colour (matrix, hue/saturation table, chroma curve) and + /// its curve is lowered to Sony's without DRO wherever that is darker, never raised. + #[test] + fn a_dro_look_keeps_its_colour_and_is_lowered_to_the_dro_off_tone() { + let dro_off = sony_dro_off_tone().expect("the built-in curve is valid"); + // the curve itself: black stays black, increasing, mid-grey where cameras put it (L* 50–75) + assert_eq!(dro_off.apply(0.0), 0.0); + let mut previous = 0.0; + for i in 1..200 { + let y = dro_off.apply(i as f32 * 0.005); + assert!(y > previous && y < 1.0, "{i}: {y}"); + previous = y; + } + assert!((0.18..0.48).contains(&dro_off.apply(0.18)), "{}", dro_off.apply(0.18)); + let xs: [f32; 32] = std::array::from_fn(|i| 0.006 * 1.13f32.powi(i as i32)); + // a JPEG with lifted shadows: the curve comes down to Sony's, colour and chroma curve stay + let lifted = dro_test_look(&xs, &|x| dro_off.apply(x).powf(0.8)); + let fixed = with_dro_off_tone(lifted.clone()); + assert_lowered(&lifted, &fixed, "lifted"); + assert!(fixed.tone.apply(0.02) < 0.85 * lifted.tone.apply(0.02)); + // a camera that maps raw values darker than Sony's curve keeps its own curve exactly + let darker = dro_test_look(&xs, &|x| 0.7 * dro_off.apply(x)); + assert_eq!(with_dro_off_tone(darker.clone()).tone, darker.tone); + // crossing curves: the lower of the two at the knots, still a valid increasing curve + let crossing = dro_test_look(&xs, &|x| (dro_off.apply(x) * (0.6 + x)).min(0.95)); + assert_lowered(&crossing, &with_dro_off_tone(crossing.clone()), "crossing"); + } + + /// Past its last knot a camera curve continues as a shoulder whose rate comes from its last two + /// knots. Lowering the second-last knot more than the last steepened that shoulder: the curve + /// fitted to the CC0 NEX-5T sample (raw.pixls.us, DRO Auto) went from 0.740 to 0.843 at scene + /// luminance 0.63, brighter than before. The lowered curve must stay at or below the fitted one + /// through the shoulder, on that curve and on many others. + #[test] + fn the_lowered_curve_stays_below_the_fitted_one_past_its_last_knot() { + let nex5t: [[f32; 2]; 32] = [ + [0.014948029, 0.017424058], + [0.018395446, 0.023981506], + [0.021539066, 0.030581286], + [0.024555191, 0.0374646], + [0.027602654, 0.044408206], + [0.030193308, 0.049894042], + [0.03261828, 0.055907387], + [0.034544908, 0.06156683], + [0.03669959, 0.06814718], + [0.039161213, 0.075263545], + [0.04173663, 0.0830011], + [0.044166446, 0.088166036], + [0.04682434, 0.095398724], + [0.049544845, 0.1027153], + [0.05235826, 0.11173598], + [0.05497513, 0.119017586], + [0.057503678, 0.12616614], + [0.059836045, 0.13521175], + [0.062677935, 0.14601946], + [0.06599234, 0.15573128], + [0.069809504, 0.16883339], + [0.0755404, 0.1854325], + [0.08250754, 0.20684941], + [0.089204684, 0.23466235], + [0.09650368, 0.25805998], + [0.102161564, 0.27849936], + [0.107812546, 0.29706743], + [0.11410169, 0.31379074], + [0.12000221, 0.33262977], + [0.12776275, 0.35407156], + [0.1387929, 0.38362882], + [0.19492775, 0.43935108], + ]; + let fitted = CameraLook { tone: CameraTone::new(nex5t).unwrap(), ..dro_test_look(&std::array::from_fn(|i| 0.01 + i as f32 * 0.01), &|x| x) }; + let fixed = with_dro_off_tone(fitted.clone()); + assert_lowered(&fitted, &fixed, "NEX-5T"); + assert!((fitted.tone.apply(0.63) - 0.7405).abs() < 1e-3 && fixed.tone.apply(0.63) <= fitted.tone.apply(0.63)); + assert_ne!(fixed.tone, fitted.tone, "the lifted mid-tones still come down"); + // curves that end at all kinds of scene luminances, slopes and heights relative to Sony's + let dro_off = sony_dro_off_tone().unwrap(); + let mut seed = 0x9e37_79b9_u32; + let mut next = move || { + seed = seed.wrapping_mul(1_664_525).wrapping_add(1_013_904_223); + f32::from((seed >> 16) as u16) / 65536.0 + }; + for case in 0..300 { + let (start, span) = (0.002 + 0.03 * next(), 1.05 + 0.15 * next()); + let xs: [f32; 32] = std::array::from_fn(|i| start * span.powi(i as i32)); + let (lift, gain, top) = (0.5 + 0.6 * next(), 0.6 + 0.8 * next(), next()); + // brighter or darker than Sony's, with a last knot sometimes lowered less than the one before + let f = |x: f32| (gain * dro_off.apply(x).powf(lift) + top * 0.2 * (x / xs[31]).powi(8)).min(0.995); + let mut ys = xs.map(f); + for i in 1..32 { + ys[i] = ys[i].max(ys[i - 1]); + } + let Some(tone) = CameraTone::new(std::array::from_fn(|i| [xs[i], ys[i]])) else { continue }; + let fitted = CameraLook { tone, ..dro_test_look(&std::array::from_fn(|i| 0.01 + i as f32 * 0.01), &|x| x) }; + assert_lowered(&fitted, &with_dro_off_tone(fitted.clone()), &format!("case {case}")); + } + } + + /// Issue #244 (skipped without the corpus): the public ILCE-7RM2 sample (Standard creative + /// style) was shot with DRO Auto, so its curve comes down to Sony's without DRO in the shadows the + /// camera lifted; the ILCE-7M4 sample (DRO Auto, Vivid) has deeper shadows than that curve and + /// keeps them, only its brighter highlights come down. Both keep the colour fitted to their + /// JPEGs. The ILCE-7M3 sample (DRO off) keeps its own fit. + #[test] + fn corpus_arw_with_dro_is_lowered_to_the_dro_off_tone() { + let dir = std::env::var_os("LIGHTCRAFT_CORPUS") + .map(std::path::PathBuf::from) + .unwrap_or_else(|| std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus")) + .join("raw"); + let samples = [ + ("arw-sony-a7rm2-12bit-uncompressed.arw", true, [0.01, 0.05, 0.1]), + ("arw-sony-a7m4-14bit.arw", true, [0.3, 0.4, 0.5]), + ("arw-sony-a7m3-compressed.arw", false, [0.0; 3]), + ]; + for (name, dro, lowered) in samples { + let path = dir.join(name); + let bytes = match std::fs::read(&path) { + Ok(bytes) => bytes, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => { + eprintln!("skip: {} absent", path.display()); + continue; + } + Err(e) => panic!("{}: {e}", path.display()), + }; + assert_eq!(lightcraft_raw::embedded_preview_dynamic_range_optimized(&bytes), Some(dro), "{name}"); + let mut raw = lightcraft_raw::decode(&bytes).unwrap(); + raw.opcodes.list3.retain(|op| !op.is_lens_correction()); + let transform = lightcraft_raw::color::camera_transform(&raw, lightcraft_raw::color::as_shot_white_xy(&raw)); + let look = fit_preview(&raw, &bytes, &transform).expect("a camera look"); + // the look fitted to the JPEG, as before this change + let (sensor, reference) = proxies(&raw, &bytes, &transform, PROXY).unwrap(); + let profile = raw.metadata.model.as_deref().and_then(crate::camera_profiles::get); + let colour = profile.as_ref().and_then(|p| Some((p.matrix().mul(&transform.matrix.inverse()?), p.hue_sat.clone()))); + let fitted = fit_pairs_with(&sensor, &reference, colour).unwrap(); + if dro { + assert_lowered(&fitted, &look, name); + for x in lowered { + assert!(look.tone.apply(x) < 0.95 * fitted.tone.apply(x), "{name} at {x}: {} vs {}", look.tone.apply(x), fitted.tone.apply(x)); + } + } else { + assert_eq!((look.matrix, &look.hue_sat, look.tone), (fitted.matrix, &fitted.hue_sat, fitted.tone), "{name}"); + } + let (_, info) = crate::files::load_bytes(&bytes, 400).unwrap(); + assert_eq!(info.camera_tone, Some(look.tone), "{name}"); + } + } + /// Issue #232: an ILCE-7RM2 camera JPEG is lens-corrected ("Distortion Comp.: Auto"), so on a /// façade of window frames the raw and its JPEG are a pixel or two apart. The colours agree /// everywhere away from the edges, and the look must still be found there. diff --git a/crates/engine/src/media.rs b/crates/engine/src/media.rs index 1231221b9..fcfbbe208 100644 --- a/crates/engine/src/media.rs +++ b/crates/engine/src/media.rs @@ -62,7 +62,7 @@ impl SettingsHashes { } /// Bump when the pipeline's output changes, to invalidate cached thumbnails. -pub const RENDER_CACHE_VERSION: u64 = 22; +pub const RENDER_CACHE_VERSION: u64 = 23; /// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache). pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512]; diff --git a/crates/pipeline/src/tone.rs b/crates/pipeline/src/tone.rs index 1a1fcc201..cc076669e 100644 --- a/crates/pipeline/src/tone.rs +++ b/crates/pipeline/src/tone.rs @@ -67,6 +67,11 @@ impl CameraTone { &self.chroma } + /// The (scene, display) luminance knots, in increasing order. + pub fn knots(&self) -> &[[f32; 2]; 32] { + &self.knots + } + pub fn apply(&self, y: f32) -> f32 { if !y.is_finite() || y <= 0.0 { return 0.0; diff --git a/crates/raw/src/lib.rs b/crates/raw/src/lib.rs index 963a4be89..63f26b060 100644 --- a/crates/raw/src/lib.rs +++ b/crates/raw/src/lib.rs @@ -49,7 +49,7 @@ pub use lightcraft_geom::Orientation; pub use lightcraft_meta::Metadata; pub use lightcraft_raster::Rgb32f; pub use opcodes::{Opcode, OpcodeLists}; -pub use preview::{PreviewColorSpace, embedded_preview, embedded_preview_color_space}; +pub use preview::{PreviewColorSpace, embedded_preview, embedded_preview_color_space, embedded_preview_dynamic_range_optimized}; pub use semantic::{SemanticMask, semantic_masks}; use lightcraft_color::Xy; diff --git a/crates/raw/src/preview.rs b/crates/raw/src/preview.rs index 371b5e4ab..d34fb4c6f 100644 --- a/crates/raw/src/preview.rs +++ b/crates/raw/src/preview.rs @@ -31,6 +31,21 @@ pub fn embedded_preview_color_space(bytes: &[u8]) -> Option { } } +/// Whether the camera optimised its embedded JPEG's dynamic range with local tone mapping that the raw data doesn't +/// carry: Sony's Dynamic Range Optimizer (maker note `0xb025`, see `vendor::arw`). Such a JPEG is brighter in its +/// darker regions than the raw developed with the camera's global tone curve. `Some(false)` when the file says it +/// was off, `None` when it doesn't say (other makers, notes without the tag, undocumented values). +pub fn embedded_preview_dynamic_range_optimized(bytes: &[u8]) -> Option { + let tiff = Tiff::parse(bytes).ok()?; + let make = tiff.ifds.first()?.string(t::MAKE)?; + if !make.trim().to_ascii_uppercase().starts_with("SONY") { + return None; + } + let e = tiff.exif()?.get(t::MAKER_NOTE)?; + let mn = makernote::parse_makernote(bytes, e.offset, e.count() as u64, tiff.order, &make)?; + crate::vendor::arw::dynamic_range_optimizer(&mn) +} + /// Whether `b` looks like a displayable (DCT) JPEG: SOI, and the first SOF marker is not lossless. fn is_dct_jpeg(b: &[u8]) -> bool { if b.len() < 4 || b[0] != 0xff || b[1] != 0xd8 { diff --git a/crates/raw/src/vendor/arw.rs b/crates/raw/src/vendor/arw.rs index 7c2990466..2e3eec22f 100644 --- a/crates/raw/src/vendor/arw.rs +++ b/crates/raw/src/vendor/arw.rs @@ -42,6 +42,23 @@ const REFERENCE_BLACK_WHITE: u16 = 532; /// Maker-note tags (ExifTool Sony tag names): the enciphered `Tag2010` block and `FullImageSize` (height, width). const MN_TAG2010: u16 = 0x2010; const MN_FULL_IMAGE_SIZE: u16 = 0xb02b; +/// Maker-note `DynamicRangeOptimizer` (ExifTool Sony tag documentation, int32u): 0 Off, 1 Standard, 2 Advanced +/// Auto, 3 Auto, 8–12 Advanced Lv1–Lv5, 16–23 Lv1–Lv8. Sony writes a second tag of that name, `0xb04f` (0 Off, +/// 1 Standard, 2 Plus), but it adds nothing: on the 241 Sony raws (75 bodies) of 264 checked that carry both, it +/// reads 1 whenever `0xb025` is on, whatever the level, and 0 when off; older bodies (DSLR-A100 to A900, +/// NEX-3/5/C3, SLT-A33/A35/A55) leave it out. `0xb025` was present on all 264 files (98 bodies). +const MN_DYNAMIC_RANGE_OPTIMIZER: u16 = 0xb025; + +/// Whether the camera's Dynamic Range Optimizer (DRO) was on for this shot. DRO is a local tone operator that +/// brightens darker regions of the camera's JPEG (and the preview embedded in the ARW) but leaves the raw data +/// alone; the factory default is Auto. `None` when the maker note doesn't record it or holds an undocumented value. +pub(crate) fn dynamic_range_optimizer(note: &makernote::MakerNote) -> Option { + match note.ifd.u64(MN_DYNAMIC_RANGE_OPTIMIZER)? { + 0 => Some(false), + 1..=3 | 8..=12 | 16..=23 => Some(true), + _ => None, + } +} /// Inverse of Sony's maker-note byte substitution. ExifTool's Sony tag documentation states that the data of /// tags `0x2010`, `0x9050` and `0x94xx` "is encrypted by a simple substitution cipher" (no decoder source was @@ -732,6 +749,90 @@ mod tests { assert!(tag2010_wb("DSC-RX100M3", &encipher(&odd), Little).is_none()); } + /// A Sony-style file: IFD0 (`make`) → Exif → maker note `SONY DSC \0\0\0` + an IFD of `entries` (tag, type, + /// count, inline value bytes), the layout of the camera's own notes; `count` overrides the entry count. + fn sony_file(order: lightcraft_tiff::ByteOrder, make: &str, entries: &[(u16, u16, u32, [u8; 4])], count: Option) -> Vec { + use lightcraft_tiff::{ByteOrder, IfdBuilder, TiffWriter, Value}; + let (u16b, u32b) = match order { + ByteOrder::Little => (u16::to_le_bytes as fn(u16) -> [u8; 2], u32::to_le_bytes as fn(u32) -> [u8; 4]), + ByteOrder::Big => (u16::to_be_bytes as fn(u16) -> [u8; 2], u32::to_be_bytes as fn(u32) -> [u8; 4]), + }; + let mut note = b"SONY DSC \0\0\0".to_vec(); + note.extend(u16b(count.unwrap_or(entries.len() as u16))); + for &(tag, kind, n, value) in entries { + note.extend(u16b(tag)); + note.extend(u16b(kind)); + note.extend(u32b(n)); + note.extend(value); + } + note.extend(u32b(0)); + let mut exif = IfdBuilder::new(); + exif.set(t::MAKER_NOTE, Value::Undefined(note)); + let mut ifd0 = IfdBuilder::new(); + ifd0.set(t::MAKE, Value::Ascii(make.into())); + ifd0.set(t::MODEL, Value::Ascii("ILCE-7CR".into())); + ifd0.set_child(t::EXIF_IFD, exif); + TiffWriter::new(order, false).write(&[ifd0]).unwrap() + } + + #[test] + fn dynamic_range_optimizer_is_read_from_the_sony_note() { + use lightcraft_tiff::ByteOrder::{Big, Little}; + let long = |order, v: u32| match order { + Little => v.to_le_bytes(), + Big => v.to_be_bytes(), + }; + let short = |order, v: u16| match order { + Little => [v.to_le_bytes()[0], v.to_le_bytes()[1], 0, 0], + Big => [v.to_be_bytes()[0], v.to_be_bytes()[1], 0, 0], + }; + let dro = |bytes: &[u8]| crate::embedded_preview_dynamic_range_optimized(bytes); + for order in [Little, Big] { + // every documented value: Off; Standard, Advanced Auto, Auto (the factory default); Advanced Lv1–5; Lv1–8 + let documented: Vec = [0, 1, 2, 3].into_iter().chain(8..=12).chain(16..=23).collect(); + for &value in &documented { + let file = sony_file(order, "SONY", &[(MN_DYNAMIC_RANGE_OPTIMIZER, 4, 1, long(order, value))], None); + assert_eq!(dro(&file), Some(value != 0), "{order:?} {value}"); + } + // undocumented values say nothing + for value in (0..=64).filter(|v| !documented.contains(v)).chain([255, 65535, u32::MAX]) { + let file = sony_file(order, "SONY", &[(MN_DYNAMIC_RANGE_OPTIMIZER, 4, 1, long(order, value))], None); + assert_eq!(dro(&file), None, "{order:?} {value}"); + } + // stored as a SHORT: same reading + assert_eq!(dro(&sony_file(order, "SONY", &[(MN_DYNAMIC_RANGE_OPTIMIZER, 3, 1, short(order, 18))], None)), Some(true)); + // the coarse 0xb04f alone is not used, nor is an ASCII or empty value + assert_eq!(dro(&sony_file(order, "SONY", &[(0xb04f, 3, 1, short(order, 1))], None)), None); + assert_eq!(dro(&sony_file(order, "SONY", &[(MN_DYNAMIC_RANGE_OPTIMIZER, 2, 2, *b"3\0\0\0")], None)), None); + assert_eq!(dro(&sony_file(order, "SONY", &[(MN_DYNAMIC_RANGE_OPTIMIZER, 4, 0, [0; 4])], None)), None); + // other makers' notes are not read as Sony's + assert_eq!(dro(&sony_file(order, "NIKON CORPORATION", &[(MN_DYNAMIC_RANGE_OPTIMIZER, 4, 1, long(order, 3))], None)), None); + } + } + + #[test] + fn hostile_sony_notes_are_read_safely() { + use lightcraft_tiff::ByteOrder::Little; + let entry = [(MN_DYNAMIC_RANGE_OPTIMIZER, 4, 1, 3u32.to_le_bytes())]; + let good = sony_file(Little, "SONY", &entry, None); + assert_eq!(crate::embedded_preview_dynamic_range_optimized(&good), Some(true)); + // an entry count far beyond the note: at most the entries actually there are read + assert_ne!(crate::embedded_preview_dynamic_range_optimized(&sony_file(Little, "SONY", &entry, Some(u16::MAX))), Some(false)); + // an entry pointing outside the file + let outside = [(MN_DYNAMIC_RANGE_OPTIMIZER, 4, 2, 0xffff_fff0u32.to_le_bytes())]; + assert_eq!(crate::embedded_preview_dynamic_range_optimized(&sony_file(Little, "SONY", &outside, None)), None); + // every truncation of a good file, and no file at all + for len in 0..good.len() { + let _ = crate::embedded_preview_dynamic_range_optimized(&good[..len]); + } + assert_eq!(crate::embedded_preview_dynamic_range_optimized(b"not a TIFF"), None); + assert_eq!(crate::embedded_preview_dynamic_range_optimized(&[]), None); + // a Sony file without a maker note + let mut ifd0 = lightcraft_tiff::IfdBuilder::new(); + ifd0.set(t::MAKE, lightcraft_tiff::Value::Ascii("SONY".into())); + assert_eq!(crate::embedded_preview_dynamic_range_optimized(&lightcraft_tiff::TiffWriter::new(Little, false).write(&[ifd0]).unwrap()), None); + } + #[test] fn sr2_black_level_decodes_and_rejects_garbage() { use lightcraft_tiff::ByteOrder::Little; diff --git a/docs/camera-preview-colour.md b/docs/camera-preview-colour.md index dfad77c22..feab1040a 100644 --- a/docs/camera-preview-colour.md +++ b/docs/camera-preview-colour.md @@ -74,9 +74,19 @@ This is a per-file camera-look estimate, **not measured spectral calibration or One photo can show too little of a colour for its own fit to learn it: in a second shot of the lime shirt only a few dozen proxy pixels show it, next to a hillside of foliage at the same hue that the camera renders differently, and the shirt stayed yellow (hue 68° against the JPEG's 82°; a 192-pixel proxy only reached 74°). `lightcraft-cli calibrate [--max N] [--out DIR] FOLDERS…` therefore pools the colour pairs of many ARWs, NEFs or RAFs per camera model — each against its own embedded JPEG, at a 192-pixel proxy, at most 4000 pairs per photo, files spread evenly over the folders — and fits one matrix (from white-balanced camera RGB, so no single photo's white point is baked in) and one hue/saturation/value table. Models need at least 5 usable photos. -Profiles are JSON files `.json` in `$LIGHTCRAFT_CAMERA_PROFILES`, else `/camera-profiles` (macOS `~/Library/Application Support/LightCraft/camera-profiles`); a local profile replaces a built-in one. Built-in profiles live in `assets/camera-profiles/` (listed in `assets/ATTRIBUTION.md`) and are compiled in, so the app, CLI and web build share them: ILCE-7M4, fitted to 597 photos (2.2 million colour pairs) shot in 2026, mostly with the Standard creative style and DRO Auto; X-H2S, 201 photos; X-T4, 545 photos. They hold aggregate colour statistics only. A photo of a profiled model takes its colour from the profile and fits only its own tone and chroma curves (DRO and picture styles vary per shot); the acceptance gates still apply. A rejected profile fit retries the photo's own colour and tone fit before falling back to neutral rendering. Files are read once per process and validated (version, bounded invertible matrix, table shape and finite data); a damaged file is ignored with a warning. The profiles folder's contents are part of the render cache keys, so thumbnails rendered before a profile existed are redone; smart previews built before keep their colour until rebuilt. +Profiles are JSON files `.json` in `$LIGHTCRAFT_CAMERA_PROFILES`, else `/camera-profiles` (macOS `~/Library/Application Support/LightCraft/camera-profiles`); a local profile replaces a built-in one. Built-in profiles live in `assets/camera-profiles/` (listed in `assets/ATTRIBUTION.md`) and are compiled in, so the app, CLI and web build share them: ILCE-7M4, fitted to 597 photos (2.2 million colour pairs) shot in 2026, mostly with the Standard creative style and DRO Auto; X-H2S, 201 photos; X-T4, 545 photos. They hold aggregate colour statistics only. A photo of a profiled model takes its colour from the profile and fits only its own tone and chroma curves (picture styles vary per shot; with Sony's DRO on, that tone curve is then lowered as described below); the acceptance gates still apply. A rejected profile fit retries the photo's own colour and tone fit before falling back to neutral rendering. Files are read once per process and validated (version, bounded invertible matrix, table shape and finite data); a damaged file is ignored with a warning. The profiles folder's contents are part of the render cache keys, so thumbnails rendered before a profile existed are redone; smart previews built before keep their colour until rebuilt. +## Sony Dynamic Range Optimizer (DRO) + +Sony bodies apply DRO, a local tone operator that brightens darker regions, to their JPEG (and the preview embedded in the ARW) only; the raw data are unchanged, and the factory default is DRO Auto. The tone curve fitted to that JPEG carried the brightening into the raw's starting look, so DRO photos opened lighter than in Lightroom, darktable, Photomator or Affinity, which develop the raw without it (issue #244). On 24 ILCE-7CR photos (Standard creative style) with Lightroom 9.5.1 default exports, mean L* at 200 px: with DRO off, Lightroom matches the camera JPEG (+0.15 L*, all within ±1.4) and LightCraft matches Lightroom (−0.19 L*, median ΔE76 3.6); with DRO Lv3 or Auto (8 photos), Lightroom is 2.4–10.7 L* darker than the camera JPEG and LightCraft followed the JPEG, +6.4 L* from Lightroom (median ΔE76 7.5). The tone curves fitted to 131 ILCE-7CR photos show where: at scene luminance 0.01, 0.02, 0.05 and 0.1 the median DRO Lv3/Auto curve is 8, 10, 9 and 6 L* above the DRO-off one, and from 0.3 up they agree. + +The ARW decoder reads the setting from maker-note tag `0xb025` (ExifTool's Sony tag documentation: 0 Off; 1 Standard, 2 Advanced Auto, 3 Auto, 8–12 Advanced Lv1–5, 16–23 Lv1–8; other values are treated as unknown). It was present on all 264 Sony raws checked (98 bodies, DSLR-A100 to ILCE-7CR); the second tag of that name, `0xb04f`, only says off or on and is missing on the older bodies. With DRO on, the look keeps the colour fitted to the JPEG (matrix, hue/saturation table, chroma curve) and its tone curve is lowered, knot by knot, to Sony's curve without DRO wherever that is darker (`assets/camera-tone/sony-dro-off.json`, listed in `assets/ATTRIBUTION.md`). Past its last knot a curve continues as a shoulder whose rate comes from its last two knots, so the second-last knot is lowered only as far as keeps that shoulder at or below the fitted one; the result is never brighter than the fitted curve at any scene luminance. Sony's curve is quantile-matched, as one photo's tone is, over the pooled pairs of 12 CC0 raw.pixls.us ARWs shot with DRO off and the Standard creative style on 10 interchangeable-lens bodies (ILCE-3500, -5000, -6600, -7M3, -7S, -7CR, -9M2, ILCA-77M2, SLT-A57, NEX-F3; the file lists their URLs and SHA-256); pooled the same way from 55 private ILCE-7CR photos with DRO off, the curve differs by at most 1.2 L* between scene luminance 0.01 and 0.5. DRO only brightens, so the photo's own curve bounds the curve without it from above: the 1″ compacts map raw values darker (with DRO off, the DSC-RX100M2, -RX100M5 and -RX10M4 curves lie 8–16 L* below Sony's at scene luminance 0.18) and keep their own curve where it is the darker one. Photos with DRO off or unknown, and other makers, are unchanged. + +Measured before → after (mean L* and CIE76 ΔE at 200 px): on the 24 ILCE-7CR photos with Lightroom references, the 8 with DRO on go from +6.4 L* (+4.4 to +10.5) to +0.8 L* (−0.4 to +1.9) against Lightroom's default, median ΔE 7.5 → 3.3; by Lightroom lightness band (L* 0–25, 25–50, 50–75, 75–100) they now differ from it by +2.3, +1.4, −0.8 and −3.0 L* (before +7.8, +9.4, +3.8, −0.8), like the 16 with DRO off (+2.5, −0.1, −2.0, −3.2), which render byte-identically. Of all 131 ILCE-7CR photos, the 56 fitted with DRO on now render 6.1 L* darker than their camera JPEGs on average (0.3 lighter before), as Lightroom renders the measured ones 5.7 L* darker; the 71 with DRO off are byte-identical. Of 116 CC0 Sony files from 100 bodies, 79 were shot with DRO on (71 of them Auto); the 57 changed of the 69 fitted ones render 2.1 L* darker on average (at most 10.8), the 12 others (six compacts, four ILCE-7M4 shots in Vivid, an ILCE-1 and an ILCE-7RM4, whose own curves are already darker) and all 35 DRO-off files are unchanged. The issue's ILCE-7M3 file goes from +0.8 to −6.5 L* against its camera JPEG. Applied to photos shot with DRO off, whose JPEG shows the camera's own curve, the rule would move the 68 fitted ILCE-7CR photos by −0.4 L* (median; at most −4.1) and 25 of 28 fitted CC0 ones by at most −1.8 L*; the exceptions are an ILCE-3500 (−4.7) and the cine bodies FX3 and FX30 (−7.2 and −5.1), whose picture profiles are not the Standard style. + +Picture styles other than Standard are lowered to the Standard curve too where it is darker; Lightroom ignores them as well. Nikon Active D-Lighting, Canon Auto Lighting Optimizer and Fujifilm D-Range settings are likely to have the same effect and are not handled yet. Render cache version 23 redraws thumbnails; smart previews built before keep the brightened tone curve in their header until rebuilt. + ## Nikon crop and preview colour metadata Nikon maker-note `CropArea` (0x0045) supplies the default `[left, top, width, height]` crop. The decoder validates the rectangle against the active sensor area and falls back to that area for missing or invalid values. The CFA origin is unchanged; cropping follows demosaicing. diff --git a/docs/parity.md b/docs/parity.md index 4c27755b5..905eef910 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -296,7 +296,7 @@ under the UI command that wraps it (a no-op error); the UI command now wins. | LR-PROF-ADOBE | Standard raw looks (own equivalents) | P0 | ✅ | `crates/engine/src/presets.rs` (`PROFILES`), `crates/pipeline/src/profiles.rs` | six own looks: Color, Neutral, Vivid, Landscape, Portrait, Monochrome | | LR-PROF-ADAPTIVE | Adaptive profiles | P2 | ⬜ | | | | LR-PROF-CAMERA | Camera-matching looks | P2 | ⬜ | | | -| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/raw/src/profile.rs`, `crates/raw/src/gaintable.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices and the camera-profile look they carry (`ProfileHueSatMap`, `ProfileLookTable`, `ProfileToneCurve`, read from the file at run time per DNG spec ch. 6 — Lightroom-converted DNGs rendered flat and muted without them, issue #138). A DNG's `ProfileGainTableMap` / `ProfileGainTableMap2` (Apple ProRAW's local tone map; both versions, DNG 1.7 precedence, the file's byte order with a big-endian fallback) is read and written back by DNG export and not rendered by default, as Lightroom Classic renders ProRAW without it (measured on one CC0 iPhone 12 Pro ProRAW); the Profile option "Camera local tone mapping" (`profile.camera_local_tone`, per photo, off by default, `crates/pipeline/src/local_tone.rs`) renders it as the camera does, per output pixel in the scene-linear stage (CPU, also under the GPU path); Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3 and CR2, Pentax PEF and Samsung SRW get a guarded file-local fit to their own embedded JPEG — chromaticity matrix + hue/saturation/value table + tone and chroma curves, relative WB — or, with a bundled or local per-model profile pooled from many photos (`lightcraft-cli calibrate`, ARW, NEF and RAF so far, `crates/engine/src/camera_profiles.rs`), the profile's colour and only their own tone/chroma curves (on 29 held-out ILCE-7M4 photos mean ΔE vs the camera JPEG 3.54 → 3.10), kept with smart previews (see docs/camera-preview-colour.md); ARW: on 10 public raw.pixls.us samples from 8 bodies 7 fits were accepted, mean ΔE vs the camera JPEG 17–26 → 3–10; NEF: 13 of 13 decodable samples (D750, D780, D850, D7500, Z 50 and local D7500 shots) accepted, ΔE 13–46 → 3–8 (issue #150); RW2 / RWL: 140 of 174 public samples accepted, from 93 of the 114 bodies with a preview, median ΔE 5.0 with matching lightness (13 of the 14 corpus files with a preview: ΔE 9.6–27 → 2.8–8.9; most rejections are compacts and kit zooms whose camera JPEG is distortion-corrected); a fit that fails the gates on all pixels is retried away from edges, where a lens-corrected camera JPEG still lines up (issue #232: the six distortion-corrected public ILCE-7RM2 samples ΔE 16.6–16.9 → 3.7–3.8, the corpus GX80 19.1 → 7.9); RAF: bundled X-H2S/X-T4 profiles from 201/545 photos; on 150 withheld files mean ΔE improves slightly over per-file fitting (X-H2S 1.70 → 1.62, X-T4 2.43 → 2.34); 23 of 24 references from 13 bodies accepted, including the dark X-H2S samples (8.74 → 2.25) (`docs/camera-preview-colour.md`); this is not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today | +| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/raw/src/profile.rs`, `crates/raw/src/gaintable.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices and the camera-profile look they carry (`ProfileHueSatMap`, `ProfileLookTable`, `ProfileToneCurve`, read from the file at run time per DNG spec ch. 6 — Lightroom-converted DNGs rendered flat and muted without them, issue #138). A DNG's `ProfileGainTableMap` / `ProfileGainTableMap2` (Apple ProRAW's local tone map; both versions, DNG 1.7 precedence, the file's byte order with a big-endian fallback) is read and written back by DNG export and not rendered by default, as Lightroom Classic renders ProRAW without it (measured on one CC0 iPhone 12 Pro ProRAW); the Profile option "Camera local tone mapping" (`profile.camera_local_tone`, per photo, off by default, `crates/pipeline/src/local_tone.rs`) renders it as the camera does, per output pixel in the scene-linear stage (CPU, also under the GPU path); Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3 and CR2, Pentax PEF and Samsung SRW get a guarded file-local fit to their own embedded JPEG — chromaticity matrix + hue/saturation/value table + tone and chroma curves, relative WB — or, with a bundled or local per-model profile pooled from many photos (`lightcraft-cli calibrate`, ARW, NEF and RAF so far, `crates/engine/src/camera_profiles.rs`), the profile's colour and only their own tone/chroma curves (on 29 held-out ILCE-7M4 photos mean ΔE vs the camera JPEG 3.54 → 3.10), kept with smart previews (see docs/camera-preview-colour.md); Sony ARWs shot with DRO on (maker note `0xb025`, on by default) keep that colour but their tone curve is lowered to Sony's curve without DRO, whose lifted shadows are in the camera JPEG only (issue #244: 8 ILCE-7CR photos +6.4 → +0.8 L* from Lightroom's default, median ΔE 7.5 → 3.3); ARW: on 10 public raw.pixls.us samples from 8 bodies 7 fits were accepted, mean ΔE vs the camera JPEG 17–26 → 3–10; NEF: 13 of 13 decodable samples (D750, D780, D850, D7500, Z 50 and local D7500 shots) accepted, ΔE 13–46 → 3–8 (issue #150); RW2 / RWL: 140 of 174 public samples accepted, from 93 of the 114 bodies with a preview, median ΔE 5.0 with matching lightness (13 of the 14 corpus files with a preview: ΔE 9.6–27 → 2.8–8.9; most rejections are compacts and kit zooms whose camera JPEG is distortion-corrected); a fit that fails the gates on all pixels is retried away from edges, where a lens-corrected camera JPEG still lines up (issue #232: the six distortion-corrected public ILCE-7RM2 samples ΔE 16.6–16.9 → 3.7–3.8, the corpus GX80 19.1 → 7.9); RAF: bundled X-H2S/X-T4 profiles from 201/545 photos; on 150 withheld files mean ΔE improves slightly over per-file fitting (X-H2S 1.70 → 1.62, X-T4 2.43 → 2.34); 23 of 24 references from 13 bodies accepted, including the dark X-H2S samples (8.74 → 2.25) (`docs/camera-preview-colour.md`); this is not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today | | LR-PROF-CREATIVE | Creative profiles (own) | P2 | ✅ | `cmd:develop.profile`, `crates/pipeline/src/profiles.rs` | 16 own looks in Film / Cinematic / Muted / B&W (tone + point-curve fades, colour grading, mixer / B&W mix); scale with `ctl:profile.amount`; sliders untouched | | LR-PROF-LEGACY | Legacy profiles | P2 | ⬜ | | | | LR-PROF-NONRAW | Profiles for non-raw files | P0 | ✅ | `cmd:develop.profile` | same looks apply to JPEG/TIFF |