LR-IMP-FORMATS: CR3 C-RAW samples that overshoot the bit depth are clamped

C-RAW is quantised wavelet coding: next to clipped highlights its
reconstruction overshoots the sensor range by a few quantisation steps.
On 17 CC0 CR3 files (SX70 HS, G5 X Mark II, M6 Mark II, EOS R, RP, 90D,
M200, R5 Mark II, R6 Mark II, R6 Mark III, R8) samples reached 16384 to
16419 at 14 bits, never below 0, and decoding failed with "decoded
sample outside its bit depth". Lossless files of the same bodies clip at
exactly 16383. Wavelet samples are now clamped to the CMP1 bit depth;
lossless samples outside it are still corrupt data.

The other 93 CR3 files have no sample outside the range and render
pixel-identically. Render cache version 13 -> 14.
This commit is contained in:
radialmonster committed 2026-10-08 18:04:01 -04:00
1 parent 67b7bc8bc6
commit 7094ba1b6e
3 files changed
+65 -6

No files matched your search

+1 -1
View File
@@ -62,7 +62,7 @@ impl SettingsHashes {
}
/// Bump when the pipeline's output changes, to invalidate cached thumbnails.
pub const RENDER_CACHE_VERSION: u64 = 13;
pub const RENDER_CACHE_VERSION: u64 = 14;
/// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache).
pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512];
+59 -3
View File
@@ -103,21 +103,33 @@ pub(crate) fn decode(sample: &[u8], config: &Cr3Compression) -> Result<Vec<u16>>
output.resize(count, 0);
}
for (i, &value) in decoded.iter().enumerate() {
let sensor_value =
value.checked_add(midpoint).filter(|&v| v >= 0 && v <= maximum).ok_or_else(|| corrupt("decoded sample outside its bit depth"))?;
let sensor_value = sensor_sample(value, midpoint, maximum, config.levels != 0)?;
let (x, y) = (tile.x + 2 * (i % (tile.width / 2)) + p % 2, tile.y + 2 * (i / (tile.width / 2)) + p / 2);
let slot = y
.checked_mul(width)
.and_then(|n| n.checked_add(x))
.and_then(|n| output.get_mut(n))
.ok_or_else(|| corrupt("tile writes outside image"))?;
*slot = sensor_value as u16;
*slot = sensor_value;
}
}
}
Ok(output)
}
/// A decoded plane value as a sensor sample of the CMP1 bit depth.
///
/// Lossless planes reproduce the sensor exactly, so a value outside the bit depth is corrupt data.
/// Wavelet (C-RAW) planes are quantized: next to clipped highlights the reconstruction overshoots
/// the sensor's range by up to a few quantization steps (measured on 17 CC0 files: at most 36 codes
/// above 16383 at 14 bits, never below 0), while lossless files of the same bodies clip at exactly
/// 16383. Those samples are clamped to the range the sensor can record.
fn sensor_sample(value: i32, midpoint: i32, maximum: i32, quantized: bool) -> Result<u16> {
let sample = value.checked_add(midpoint).ok_or_else(|| corrupt("decoded sample outside its bit depth"))?;
let sample = if quantized { sample.clamp(0, maximum) } else { sample };
u16::try_from(sample).ok().filter(|_| sample <= maximum).ok_or_else(|| corrupt("decoded sample outside its bit depth"))
}
/// Header-only probing must reject the same unsupported coding variants as full decoding.
pub(crate) fn validate(sample: &[u8], config: &Cr3Compression) -> Result<()> {
supports(config)?;
@@ -913,6 +925,50 @@ mod tests {
}
}
#[test]
fn craw_overshoot_at_clipped_highlights_is_clamped_to_the_bit_depth() {
// As `three_level_craw_reconstructs_sensor_samples`, but each LL3 coefficient is +8200
// (Rice escape: 41 zeros, 1, 21-bit literal 16400 = signed +8200). Flat synthesis gives
// 8192 + 8200 = 16392, 9 codes above the 14-bit maximum, as in clipped C-RAW highlights.
let mut config = coding();
config.width = 16;
config.height = 16;
config.tile_width = 16;
config.tile_height = 16;
config.header_size = 540;
config.levels = 3;
let mut sample = Vec::new();
marker(&mut sample, 0xff01, 80, 0);
for plane in 0..4 {
marker(&mut sample, 0xff02, 20, (plane << 28) | 0x0800_0000);
for band in 0..10 {
marker(&mut sample, 0xff03, [8, 1, 1, 1, 1, 1, 1, 2, 2, 2][band as usize], (band << 28) | 0x0020_0000);
}
}
for _ in 0..4 {
sample.extend([0, 0, 0, 0, 0, 0x40, 0x80, 0x20]); // LL3 = +8200.
sample.extend([0x80; 3]); // The three 1×1 HF3 bands.
sample.extend([0xf0; 3]); // Three 2×2 all-zero HF2 bands.
for _ in 0..3 {
sample.extend([0xff, 0xf8]); // Three 4×4 all-zero HF1 bands.
}
}
validate(&sample, &config).unwrap();
assert_eq!(decode(&sample, &config).unwrap(), vec![16383; 256]);
}
#[test]
fn only_quantized_planes_clamp_out_of_range_samples() {
assert_eq!(sensor_sample(8200, 8192, 16383, true).unwrap(), 16383);
assert_eq!(sensor_sample(-8300, 8192, 16383, true).unwrap(), 0);
assert_eq!(sensor_sample(8191, 8192, 16383, false).unwrap(), 16383);
assert_eq!(sensor_sample(-8192, 8192, 16383, false).unwrap(), 0);
// Lossless planes reproduce the sensor exactly: outside the bit depth is corrupt data.
assert!(matches!(sensor_sample(8192, 8192, 16383, false), Err(RawError::Corrupt(_))));
assert!(matches!(sensor_sample(-8193, 8192, 16383, false), Err(RawError::Corrupt(_))));
assert!(matches!(sensor_sample(i32::MAX, 8192, 16383, true), Err(RawError::Corrupt(_))));
}
#[test]
fn measured_quantization_and_allocation_limits() {
for (quant, expected) in [(4, 1), (9, 1), (10, 2), (16, 4), (22, 8), (26, 12), (32, 25), (36, 40), (41, 72), (42, 80), (43, 90)] {
+5 -2
View File
@@ -22,8 +22,11 @@ CRX version `0x100`, four Bayer planes and `ff01`/`ff02`/`ff03` marker headers
support lossless RAW and three-level C-RAW with horizontal tiles. Uniform
quantization values 4–43 are independently verified. Version `0x200` C-RAW
supports the `ff11`/`ff12`/`ff13` marker family, a single 14-bit Bayer tile and
its adaptive quantization map. Other unverified coding variants return explicit
unsupported errors. Header probing validates the same coding variants and
its adaptive quantization map. C-RAW is quantized, so next to clipped highlights
its reconstruction can overshoot the sensor range by a few quantization steps
(up to 36 codes above 16383 in 17 CC0 files); those samples are clamped to the
CMP1 bit depth. A lossless sample outside the bit depth is still corrupt data.
Other unverified coding variants return explicit unsupported errors. Header probing validates the same coding variants and
tile/plane layout as full decoding.
`crates/raw/tests/cr3_corpus.rs` checks every full-sensor sample against a