From dbae11b843c65229e48d8e0564c028cc4603fda9 Mon Sep 17 00:00:00 2001 From: Davi Andrei <67552001+SaymonMaster14@users.noreply.github.com> Date: Thu, 8 Oct 2026 22:18:03 -0300 Subject: [PATCH] Windows: NVENC H.265 Main 10 HDR (PQ / HLG) export HDR sequences export as real 10-bit HEVC Main 10 through NVENC where the GPU reports 10-bit HEVC: float BT.2020 R'G'B' -> rgbf_to_yuv420_10 -> P010 input; VUI BT.2020 + PQ/HLG; HDR10 mastering-display / content-light as SEI on every IDR (PQ); colr/mdcv/clli in the sample entry. New register_hdr_probe / hdr_available in the export crate; macOS and GPUs without 10-bit keep tone-mapping HDR sequences to 8-bit SDR HEVC. ABI asserts extended and re-verified against nvEncodeAPI.h 12.1 (0 drift). 8-bit SDR HEVC unchanged; the two #328 tests that asserted HDR is declined now expect Main 10. Co-Authored-By: Claude Sonnet 5.5 Co-Authored-By: Claude Opus 5.5 --- ATTRIBUTION.md | 2 +- README.md | 4 +- ROADMAP.md | 7 +- crates/export/src/hdr_tests.rs | 201 ++++++++++ crates/export/src/job.rs | 17 +- crates/export/src/lib.rs | 103 ++++- crates/export/src/settings.rs | 4 +- crates/platform/README.md | 51 ++- crates/platform/src/lib.rs | 6 +- crates/platform/src/nvenc/abi_tests.rs | 42 ++ crates/platform/src/nvenc/export.rs | 134 +++++-- crates/platform/src/nvenc/ffi.rs | 64 +++- crates/platform/src/nvenc/hevc.rs | 86 ++++- crates/platform/src/nvenc/mod.rs | 213 ++++++++++- crates/platform/src/nvenc/session.rs | 124 +++++- crates/platform/tests/nvenc_hevc_export.rs | 19 +- .../platform/tests/nvenc_hevc_hdr_export.rs | 362 ++++++++++++++++++ crates/platform/tests/nvenc_hevc_main10.rs | 324 ++++++++++++++++ crates/platform/tests/nvenc_hevc_warm.rs | 12 +- crates/ui-egui/src/panels/export_mode.rs | 15 +- docs/adr/0001-platform-ffi.md | 8 + docs/architecture.md | 8 +- docs/testing.md | 9 +- 23 files changed, 1705 insertions(+), 110 deletions(-) create mode 100644 crates/export/src/hdr_tests.rs create mode 100644 crates/platform/tests/nvenc_hevc_hdr_export.rs create mode 100644 crates/platform/tests/nvenc_hevc_main10.rs diff --git a/ATTRIBUTION.md b/ATTRIBUTION.md index 1ba879b6..6f0877b2 100644 --- a/ATTRIBUTION.md +++ b/ATTRIBUTION.md @@ -146,7 +146,7 @@ the original copyright and permission notice in its header. | File | Author | Source | Licence | |---|---|---|---| -| `crates/platform/src/nvenc/ffi.rs` | NVIDIA Corporation (`nvEncodeAPI.h`); FilmCraft contributors (Rust transcription) | Structures (including `NV_ENC_CONFIG_HEVC` and its bit-field masks), function table, codec / profile GUIDs (H.264, HEVC Main) and constants (`NV_ENC_LEVEL_HEVC_*`, tier, CU size) transcribed from `nvEncodeAPI.h`, NVIDIA Video Codec SDK, API 12.1 (https://developer.nvidia.com/video-codec-sdk) | MIT (Copyright (c) 2010-2023 NVIDIA Corporation; notice kept in the file header) | +| `crates/platform/src/nvenc/ffi.rs` | NVIDIA Corporation (`nvEncodeAPI.h`); FilmCraft contributors (Rust transcription) | Structures (including `NV_ENC_CONFIG_HEVC` and its bit-field masks, `NV_ENC_PIC_PARAMS_HEVC`, `NV_ENC_SEI_PAYLOAD`, `NV_ENC_TIME_CODE`), function table, codec / profile GUIDs (H.264, HEVC Main, HEVC Main 10) and constants (`NV_ENC_LEVEL_HEVC_*`, tier, CU size, `NV_ENC_BUFFER_FORMAT_YUV420_10BIT`, `NV_ENC_CAPS_SUPPORT_10BIT_ENCODE`, BT.2020 / PQ / HLG VUI codes) transcribed from `nvEncodeAPI.h`, NVIDIA Video Codec SDK, API 12.1 (https://developer.nvidia.com/video-codec-sdk) | MIT (Copyright (c) 2010-2023 NVIDIA Corporation; notice kept in the file header) | ## Downloaded at runtime diff --git a/README.md b/README.md index b78ff6ca..3b0bca8e 100644 --- a/README.md +++ b/README.md @@ -287,9 +287,9 @@ We track two numbers ([ROADMAP.md](ROADMAP.md#honest-assessment-2026-10-05)): The biggest gaps today: -- **Speed on big footage.** Hardware decoding works on macOS and Windows; Linux has no hardware path yet. Blend modes and the most common effects run on the GPU, but Lumetri, keys and export rendering still run on the CPU; H.264 encoding can use the hardware encoder on macOS and on Windows with an NVIDIA GPU (opt-in, Export ▸ Hardware encoding), and H.265 export is hardware-only on both (NVENC; Main 10 and HDR not yet) ([#30](https://github.com/storytold/filmcraft/issues/30)). +- **Speed on big footage.** Hardware decoding works on macOS and Windows; Linux has no hardware path yet. Blend modes and the most common effects run on the GPU, but Lumetri, keys and export rendering still run on the CPU; H.264 encoding can use the hardware encoder on macOS and on Windows with an NVIDIA GPU (opt-in, Export ▸ Hardware encoding), and H.265 export is hardware-only on both (HDR sequences export Main 10 PQ / HLG with NVENC; VideoToolbox writes 8-bit SDR) ([#30](https://github.com/storytold/filmcraft/issues/30)). - **No plugins.** No VST3 / Audio Units or OpenFX hosting. -- **Delivery codecs.** H.264 is our only software delivery-codec export; H.265 (8-bit Main, SDR) exports only through a hardware encoder (NVIDIA on Windows, VideoToolbox on macOS); no AV1 export yet. +- **Delivery codecs.** H.264 is our only software delivery-codec export; H.265 exports only through a hardware encoder (NVIDIA on Windows: Main 8-bit SDR and Main 10 HDR PQ / HLG; VideoToolbox on macOS: 8-bit Main, SDR); no AV1 export yet. - **Real-world media and platforms.** Our decoders are bit-exact on conformance streams, but camera and phone files in the wild are less tested. Windows and Linux get far less testing than macOS. - **AI features.** Few so far; speech to text is optional and off by default. diff --git a/ROADMAP.md b/ROADMAP.md index fb9b1c02..d735f1ad 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -35,7 +35,7 @@ before choosing work. | **Feature breadth** (menus, commands, panels) | ~86–95% | **~85%** | Menu diff against Premiere 26.5; effect, transition and audio-effect catalogues complete by test. The long tail is mostly there. | | **Correctness in depth** | not tracked | **~60–70%** | A first-time contributor found four real bugs in core paths within hours (#5–#8: AIFF import failed, Slide left linked audio behind, transitions didn't follow ripple trims, export ignored start/end times). User reports: Color Matte always grey with no picker (#29); ALSA audio error on Linux (#23). Settings are "most wired", not all. | | **Codecs and media** | ~90% | **~75%** | Our decoders are bit-exact on conformance streams. Real camera and phone media (variable frame rate, damaged files, unusual containers) is far less tested. No camera RAW (RED, BRAW, ARRIRAW), no E-AC-3. | -| **Export** | ~92% | **~70%** | Delivery: H.264, plus H.265 (8-bit Main, SDR) through a hardware encoder only (NVENC on Windows, VideoToolbox on macOS); ProRes, DNxHR and image sequences cover mastering. No AV1 export, no HDR or 10-bit HEVC. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. | +| **Export** | ~92% | **~70%** | Delivery: H.264, plus H.265 through a hardware encoder only (NVENC on Windows: Main 8-bit SDR and Main 10 HDR PQ / HLG; VideoToolbox on macOS: Main 8-bit SDR); ProRes, DNxHR and image sequences cover mastering. No AV1 export, no 10-bit / HDR HEVC on macOS. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. | | **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS (VideoToolbox, #33; active in the desktop app only since #41, so releases up to 0.2.1 decoded in software) and Windows (Media Foundation / Direct3D 11 DXVA, H.264, HEVC Main / Main 10, VP9 and AV1 4:2:0 8/10-bit, #30). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects and export rendering are CPU; H.264 encoding can use VideoToolbox on macOS (opt-in per export, no B-frames) and NVENC on Windows with an NVIDIA GPU (opt-in per export), and H.265 (HEVC) export exists there only, through the same hardware encoders (macOS and Windows). Linux has no hardware path. 8K isn't real-time (AV1 is on Windows with a hardware decoder). The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. | | **Stability** | not tracked | **improving, unproven** | The never-crash pass (no panics in product code, last-resort guards) landed on 2026-10-04. There's no field record from real users' projects yet. | | **Plugins and pro ecosystem** | not tracked | **~0–10%** | No audio plugin hosting (VST3 / Audio Units) and no third-party video effects (OpenFX). For many professional editors this alone rules FilmCraft out. Team Projects and Productions are out of scope by design. | @@ -51,7 +51,7 @@ In priority order. Agents should prefer this work over adding more checklist ite - hardware decode on Linux (VA-API); - zero-copy decoded frames into wgpu; - the remaining effects on the GPU: Lumetri, keys, Vignette, Video Limiter, masks (31 common effects are done); - - GPU export, then hardware encode: H.264 and H.265 on macOS are in (H.264 opt-in; H.265 has no other encoder), H.264 and H.265 (Main, 8-bit) on NVENC (Windows); Main 10 / HDR HEVC, Linux and other Windows vendors remain. + - GPU export, then hardware encode: H.264 and H.265 on macOS are in (H.264 opt-in; H.265 has no other encoder), H.264 and H.265 (Main 8-bit, and Main 10 HDR PQ / HLG) on NVENC (Windows); Main 10 / HDR HEVC on VideoToolbox, Linux and other Windows vendors remain. 2. **A measured parity number.** Add `cargo xtask parity`, run in CI. It should cover menus, commands, effects, transitions, panels, preferences and formats against the Premiere snapshot in `plan/premiere/`. It should report presence and fidelity separately (exact, approximate, stub). @@ -63,7 +63,7 @@ In priority order. Agents should prefer this work over adding more checklist ite through XML / AAF. 5. **Windows and Linux at runtime.** CI that runs the test suite and a headless playback/export smoke test on both, not just release builds. Fix the Linux audio path (#23). -6. **Delivery codecs.** HEVC export is in on macOS (VideoToolbox, 8-bit, no software encoder); still to do: HEVC elsewhere, 10-bit / HDR HEVC, and AV1 (`rav1e`, BSD-2). +6. **Delivery codecs.** HEVC export is in on macOS (VideoToolbox, 8-bit, no software encoder) and on Windows (NVENC, 8-bit and Main 10 HDR); still to do: HEVC elsewhere, 10-bit / HDR HEVC on macOS, and AV1 (`rav1e`, BSD-2). 7. **Plugin hosting** (VST3 / Audio Units, then OpenFX). Needs FFI, so it falls under the isolated `unsafe` crate rule (AGENTS.md §0.3). **Owner decision needed** before starting. 8. **AI features** with openly licensed local models: transcription on by default, text-based @@ -159,6 +159,7 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining ## Log +- **2026-10-08 (night):** HDR H.265 (HEVC Main 10, PQ and HLG, BT.2020, limited range) export on Windows through NVENC (#30): an HDR sequence now exports as real 10-bit HDR (`rgbf_to_yuv420_10` into P010 input buffers; VUI BT.2020 + PQ / HLG; the HDR10 mastering-display and content-light messages as SEI on every IDR for PQ, through the generic SEI payload array because NVENC 12.1 has no mastering-display field; `colr` / `mdcv` / `clli` in the sample entry), where the GPU reports 10-bit HEVC (new `register_hdr_probe` / `hdr_available` in the export crate; elsewhere, and on macOS, HDR sequences still tone-map to 8-bit SDR HEVC, and `settings.sdr` forces that). ffprobe reads `hevc` / `Main 10` / `hvc1` / `yuv420p10le` / `bt2020` / `smpte2084` or `arib-std-b67` / `bt2020nc` / `tv` with the mastering-display and content-light side data for PQ; ffmpeg's decode equals ours code for code; the in-process round trip is at 71.1 dB luma PSNR on the 10-bit scale with 803 distinct levels on a ramp and the peak code 940 intact (RTX 5060). 8-bit SDR H.265 is unchanged. - **2026-10-08:** H.265 (HEVC Main, 8-bit 4:2:0, SDR) export on Windows through NVENC (#30): the NVENC session, ring and NV12 path of the H.264 encoder now serve both codecs (the H.264 export is byte-for-byte unchanged); the format is available where the GPU has an HEVC encoder, choosing it is the opt-in, and `hvcC` is built from the SPS the encoder wrote. ffprobe reads `hevc` / `Main` / `hvc1` / `yuv420p` / BT.709; our own HEVC decoder decodes it at 48.9 dB luma PSNR (RTX 5060). Main 10 and HDR remain. - **2026-10-06:** 21 community PRs landed (#19, #28, #38, #39, #41–#60). Highlights: hardware decoding is now actually on in the desktop app (#41: `register()` sat inside a `log::info!` that never ran, so #33's VideoToolbox path only worked in the CLI and benchmarks); decode memory on long timelines cut sharply (frame pool, shared GOP-cache budget, positional file reads); exports are byte-identical across machines (#19); crash fix for odd track names over MCP (#44); ripple trim, ripple delete and speed changes move split-edit clips once (#56–#58); Japanese interface using an installed system font (#38; no fonts bundled); MCP annotations, resources, export progress and cancellation (#28). diff --git a/crates/export/src/hdr_tests.rs b/crates/export/src/hdr_tests.rs new file mode 100644 index 00000000..f5f04808 --- /dev/null +++ b/crates/export/src/hdr_tests.rs @@ -0,0 +1,201 @@ +//! HDR plumbing shared by hardware encoders: the 10-bit 4:2:0 converter, the HDR probe registry, the +//! decision whether an export is HDR, and the one definition of the HDR10 static metadata. +use super::*; +use std::sync::atomic::{AtomicBool, Ordering}; + +const BT2020: (f32, f32) = (0.2627, 0.0593); + +fn convert(rgb: &[f32], w: usize, h: usize) -> (Vec, Vec, Vec) { + let (mut y, mut u, mut v) = (Vec::new(), Vec::new(), Vec::new()); + rgbf_to_yuv420_10(rgb, w, h, BT2020.0, BT2020.1, &mut y, &mut u, &mut v).unwrap(); + (y, u, v) +} + +fn flat(w: usize, h: usize, px: [f32; 3]) -> Vec { + px.iter().copied().cycle().take(w * h * 3).collect() +} + +#[test] +fn black_white_and_grey_have_the_limited_range_codes() { + for (px, code) in [([0.0; 3], 64u16), ([1.0; 3], 940), ([0.5; 3], 502)] { + let (y, u, v) = convert(&flat(4, 4, px), 4, 4); + assert!(y.iter().all(|c| *c == code), "{px:?}: {y:?}"); + assert!(u.iter().chain(&v).all(|c| *c == 512), "{px:?}: neutral chroma"); + assert_eq!((y.len(), u.len(), v.len()), (16, 4, 4)); + } +} + +#[test] +fn bt2020_primaries_have_exact_codes() { + // (R', G', B') -> (Y, Cb, Cr) with the BT.2020 non-constant-luminance matrix, 10-bit limited range + for (px, want) in [ + ([1.0, 0.0, 0.0], (294u16, 387u16, 960u16)), + ([0.0, 1.0, 0.0], (658, 189, 100)), + ([0.0, 0.0, 1.0], (116, 960, 476)), + ([1.0, 0.5, 0.0], (591, 225, 754)), + ] { + let (y, u, v) = convert(&flat(2, 2, px), 2, 2); + assert_eq!((y[0], u[0], v[0]), want, "{px:?}"); + } +} + +#[test] +fn out_of_range_and_non_finite_values_clamp_instead_of_wrapping() { + // above 1, below 0 and infinities clamp to the legal range; NaN counts as 0 (black, not code 0) + let mut px = vec![0.0f32; 4 * 3]; + px[..3].copy_from_slice(&[2.0, 2.0, 2.0]); + px[3..6].copy_from_slice(&[-1.0, -5.0, -0.5]); + px[6..9].copy_from_slice(&[f32::INFINITY, f32::INFINITY, f32::INFINITY]); + px[9..].copy_from_slice(&[f32::NAN, f32::NEG_INFINITY, f32::NAN]); + let (y, u, v) = convert(&px, 4, 1); + assert_eq!(y, vec![940, 64, 940, 64]); + assert!(u.iter().chain(&v).all(|c| (64..=960).contains(c)), "{u:?} {v:?}"); + // all NaN: black + let (y, u, v) = convert(&[f32::NAN; 2 * 2 * 3], 2, 2); + assert_eq!((y, u, v), (vec![64; 4], vec![512], vec![512])); +} + +#[test] +fn odd_sizes_average_the_pixels_they_have() { + // 3x3: the chroma planes are 2x2; the right column and bottom row average one or two pixels + let (y, u, v) = convert(&flat(3, 3, [1.0, 0.0, 0.0]), 3, 3); + assert_eq!((y.len(), u.len(), v.len()), (9, 4, 4)); + assert!(y.iter().all(|c| *c == 294) && u.iter().all(|c| *c == 387) && v.iter().all(|c| *c == 960)); + // 1x1 + let (y, u, v) = convert(&[0.0, 0.0, 1.0], 1, 1); + assert_eq!((y, u, v), (vec![116], vec![960], vec![476])); + // chroma is the mean over the 2x2 block: red and blue pixels give the mean of their Cb and Cr + let rgb = [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0]; + let (y, u, v) = convert(&rgb, 2, 2); + assert_eq!(y, vec![294, 116, 294, 116]); + assert_eq!((u[0], v[0]), ((387u16 + 960) / 2, (960u16 + 476) / 2)); +} + +#[test] +fn a_ramp_has_every_code_a_10_bit_picture_can_have() { + // a smooth horizontal ramp: 877 distinct luma codes in 64..=940, far more than 8 bits hold + let w = 1024usize; + let rgb: Vec = (0..w).flat_map(|x| [x as f32 / (w - 1) as f32; 3]).collect(); + let (y, _, _) = convert(&rgb, w, 1); + let mut codes: Vec = y.clone(); + codes.dedup(); + assert_eq!((y[0], y[w - 1]), (64, 940)); + assert_eq!(codes.len(), 877, "{} distinct codes", codes.len()); + assert!(y.windows(2).all(|p| p[0] <= p[1])); +} + +#[test] +fn hostile_input_is_an_error_not_a_panic() { + let (mut y, mut u, mut v) = (Vec::new(), Vec::new(), Vec::new()); + let mut run = |rgb: &[f32], w: usize, h: usize| rgbf_to_yuv420_10(rgb, w, h, 0.2627, 0.0593, &mut y, &mut u, &mut v); + assert!(run(&[], 0, 0).is_err()); + assert!(run(&[0.0; 3], 0, 1).is_err()); + assert!(run(&[0.0; 3], 1, 0).is_err()); + assert!(run(&[0.0; 11], 2, 2).is_err(), "short"); + assert!(run(&[0.0; 13], 2, 2).is_err(), "long"); + assert!(run(&[0.0; 6], 2, 2).is_err()); + assert!(run(&[0.0; 3], usize::MAX, 2).is_err(), "w*h overflows"); + assert!(run(&[0.0; 3], usize::MAX / 2, 3).is_err(), "w*h*3 overflows"); + assert!(run(&[0.0; 3], 1 << 40, 1 << 40).is_err()); +} + +#[test] +fn the_hdr_probe_registry_follows_its_probes() { + static HERE: AtomicBool = AtomicBool::new(false); + fn probe() -> bool { + HERE.load(Ordering::SeqCst) + } + // a format nothing registered a probe for is never HDR; registering twice is harmless + assert!(!hdr_available(Format::ProRes)); + register_hdr_probe(Format::ProRes, probe); + register_hdr_probe(Format::ProRes, probe); + assert!(!hdr_available(Format::ProRes)); + HERE.store(true, Ordering::SeqCst); + assert!(hdr_available(Format::ProRes)); + assert!(!hdr_available(Format::DnxHr), "probes are per format"); + HERE.store(false, Ordering::SeqCst); + assert!(!hdr_available(Format::ProRes)); +} + +#[test] +fn which_exports_are_hdr() { + use crate::job::hdr_output; + let never = |_: Format| false; + let hevc_only = |f: Format| f == Format::Hevc; + // H.264, ProRes, DNxHR and APV follow the working space, whatever the probes say + for f in [Format::H264, Format::ProRes, Format::DnxHr, Format::Apv] { + assert!(hdr_output(true, false, f, never), "{f:?}"); + assert!(!hdr_output(false, false, f, hevc_only), "{f:?}: SDR working space"); + assert!(!hdr_output(true, true, f, hevc_only), "{f:?}: SDR asked for"); + } + // H.265 only where a registered encoder writes HDR + assert!(!hdr_output(true, false, Format::Hevc, never), "no probe: tone-mapped SDR, as with VideoToolbox"); + assert!(hdr_output(true, false, Format::Hevc, hevc_only)); + assert!(!hdr_output(true, true, Format::Hevc, hevc_only), "settings.sdr forces SDR"); + assert!(!hdr_output(false, false, Format::Hevc, hevc_only)); + // formats without HDR output + assert!(!hdr_output(true, false, Format::Mjpeg, |_| true)); +} + +#[test] +fn only_pq_has_static_metadata_and_it_is_what_h264_writes() { + assert!(ColorSignal::HLG.static_metadata().is_none()); + assert!(ColorSignal::default().static_metadata().is_none()); + let (md, cll) = ColorSignal::PQ.static_metadata().unwrap(); + assert_eq!(cll, (0, 0)); + assert_eq!(md, filmcraft_isobmff::MasteringDisplay::bt2020(1000.0, 0.0001)); + assert_eq!(md.to_bytes().len(), 24); + // the box on the sample entry is the same value + let mut e = filmcraft_isobmff::SampleEntry::hevc(Default::default(), 64, 64); + ColorSignal::PQ.apply_to(&mut e, false); + let v = e.video.as_ref().unwrap(); + assert_eq!((v.mastering_display, v.content_light), (Some(md), Some((0, 0)))); + let mut e = filmcraft_isobmff::SampleEntry::hevc(Default::default(), 64, 64); + ColorSignal::HLG.apply_to(&mut e, false); + let v = e.video.as_ref().unwrap(); + assert!(v.color.is_some() && v.mastering_display.is_none() && v.content_light.is_none()); +} + +/// Remove emulation-prevention bytes (`00 00 03` -> `00 00`). +fn unescape(b: &[u8]) -> Vec { + let mut out = Vec::new(); + let mut zeros = 0; + for &x in b { + if zeros >= 2 && x == 3 { + zeros = 0; + continue; + } + zeros = if x == 0 { zeros + 1 } else { 0 }; + out.push(x); + } + out +} + +#[test] +fn the_software_h264_sei_is_the_static_metadata_laid_out_as_messages() { + let s = ExportSettings { format: Format::H264, signal: ColorSignal::PQ, ..Default::default() }; + let mut enc = h264_factory(Format::H264, 64, 64, FrameRate::FPS_24, &s).unwrap().unwrap(); + let rgb = vec![0.5f32; 64 * 64 * 3]; + let mut packets = enc.encode(&EncoderFrame { width: 64, height: 64, rgba: &[], hdr: Some(&rgb), index: 0 }).unwrap(); + packets.extend(enc.flush().unwrap()); + // find the SEI NAL unit (type 6) of the length-prefixed first access unit + let data = &packets[0].data; + let (mut pos, mut sei) = (0usize, None); + while let Some(len) = data.get(pos..pos + 4) { + let n = u32::from_be_bytes([len[0], len[1], len[2], len[3]]) as usize; + let nal = &data[pos + 4..pos + 4 + n]; + if nal[0] & 0x1f == 6 { + sei = Some(unescape(&nal[1..])); + } + pos += 4 + n; + } + let sei = sei.expect("an SEI NAL unit in the first access unit"); + let (md, (cll, fall)) = ColorSignal::PQ.static_metadata().unwrap(); + let mut want = vec![137u8, 24]; + want.extend(md.to_bytes()); + want.extend([144u8, 4]); + want.extend(cll.to_be_bytes()); + want.extend(fall.to_be_bytes()); + want.push(0x80); + assert_eq!(sei, want); +} diff --git a/crates/export/src/job.rs b/crates/export/src/job.rs index c9ce7b8f..c07228f1 100644 --- a/crates/export/src/job.rs +++ b/crates/export/src/job.rs @@ -85,6 +85,18 @@ pub fn stepped(format: Format) -> bool { matches!(format, Format::H264 | Format::Hevc | Format::ProRes | Format::DnxHr | Format::Apv | Format::Mjpeg | Format::MxfOp1a | Format::MxfOpAtom) } +/// Whether an export is HDR: an HDR working space, SDR not asked for, and a format whose encoder +/// writes HDR (`hdr_available` says so for H.265, which has no built-in encoder). +pub(crate) fn hdr_output(working_hdr: bool, sdr: bool, format: Format, hdr_available: impl Fn(Format) -> bool) -> bool { + working_hdr + && !sdr + && match format { + Format::H264 | Format::ProRes | Format::DnxHr | Format::Apv => true, + Format::Hevc => hdr_available(Format::Hevc), + _ => false, + } +} + fn make_venc(settings: &ExportSettings, w: u32, h: u32, rate: FrameRate) -> Result> { video_factories() .read() @@ -102,9 +114,10 @@ impl Exporter { } settings.validate()?; let q = project.sequence(seq).ok_or(ExportError::NoSequence)?; - // HDR sequences export HDR (H.264 / ProRes / DNxHR / APV) unless SDR is asked for (H.265 export is 8-bit SDR) + // HDR sequences export HDR (H.264 / ProRes / DNxHR / APV, and H.265 where a registered encoder + // writes Main 10) unless SDR is asked for; elsewhere H.265 is tone-mapped 8-bit SDR let pipe = q.settings.color; - let hdr_out = pipe.working.is_hdr() && !settings.sdr && matches!(settings.video_format(), Format::H264 | Format::ProRes | Format::DnxHr | Format::Apv); + let hdr_out = hdr_output(pipe.working.is_hdr(), settings.sdr, settings.video_format(), crate::hdr_available); let mut settings = settings.clone(); settings.signal = match (hdr_out, pipe.working) { (true, filmcraft_color::WorkingSpace::Rec2100Pq) => ColorSignal::PQ, diff --git a/crates/export/src/lib.rs b/crates/export/src/lib.rs index 419a7368..544aaf94 100644 --- a/crates/export/src/lib.rs +++ b/crates/export/src/lib.rs @@ -64,7 +64,9 @@ pub enum Format { H264, /// MPEG-4 (or QuickTime, see [`Multiplexer`]), H.265 / HEVC Main (8-bit) video + AAC audio. There is no /// built-in encoder: a platform hardware encoder registers one ([`register_encoder`]) and says so - /// with [`register_format_probe`]; [`available`] is false without it. + /// with [`register_format_probe`]; [`available`] is false without it. HDR sequences export as HDR + /// (Main 10, PQ / HLG) only where an encoder also says so with [`register_hdr_probe`] + /// ([`hdr_available`]); elsewhere they are tone-mapped to SDR. #[serde(rename = "hevc", alias = "Hevc")] Hevc, /// QuickTime, Apple ProRes 422 (HQ unless the settings pick another flavour) + PCM. @@ -420,12 +422,18 @@ impl ColorSignal { } else { filmcraft_isobmff::ColorInfo::Nclx { primaries: p, transfer: t, matrix: m, full_range: false } }); - if self.transfer == 16 { - v.mastering_display = Some(filmcraft_isobmff::MasteringDisplay::bt2020(1000.0, 0.0001)); - v.content_light = Some((0, 0)); + if let Some((md, cll)) = self.static_metadata() { + v.mastering_display = Some(md); + v.content_light = Some(cll); } } } + /// HDR10 static metadata of a PQ signal: the mastering display (BT.2020/D65, 1000 / 0.0001 cd/m²) + /// and (MaxCLL, MaxFALL) = (0, 0), unknown. `None` for HLG and SDR. The one definition of these + /// values: the `mdcv` / `clli` boxes and the SEI messages of every encoder come from here. + pub fn static_metadata(&self) -> Option<(filmcraft_isobmff::MasteringDisplay, (u16, u16))> { + (self.transfer == 16).then(|| (filmcraft_isobmff::MasteringDisplay::bt2020(1000.0, 0.0001), (0, 0))) + } } impl Default for ExportSettings { @@ -692,6 +700,29 @@ pub fn register_format_probe(format: Format, probe: fn() -> bool) { } } +fn hdr_probes() -> &'static RwLock> { + static P: OnceLock>> = OnceLock::new(); + P.get_or_init(|| RwLock::new(Vec::new())) +} + +/// Say how to find out whether the registered encoder of `format` can write HDR (PQ / HLG) on this +/// machine: `probe` runs when [`hdr_available`] asks (it should cache its answer). Only H.265 needs +/// one: its encoders are hardware-only and not all of them take 10-bit pictures, so an HDR sequence +/// exports as tone-mapped SDR unless a probe says otherwise. Registering the same probe twice is harmless. +pub fn register_hdr_probe(format: Format, probe: fn() -> bool) { + let mut g = hdr_probes().write().unwrap_or_else(|e| e.into_inner()); + if !g.iter().any(|(f, p)| *f == format && std::ptr::fn_addr_eq(*p, probe)) { + g.push((format, probe)); + } +} + +/// Whether a registered probe says the encoder of `format` writes HDR here (false without one). +pub fn hdr_available(format: Format) -> bool { + // the probes run outside the lock: one may take a while + let probes: Vec bool> = hdr_probes().read().unwrap_or_else(|e| e.into_inner()).iter().filter(|(f, _)| *f == format).map(|(_, p)| *p).collect(); + probes.into_iter().any(|p| p()) +} + /// Whether a format can currently be exported: it has a built-in encoder, or a registered probe /// says its encoder works here. pub fn available(format: Format) -> bool { @@ -1160,6 +1191,60 @@ pub fn rgbf_to_yuv420_8(rgb: &[f32], w: usize, h: usize, kr: f32, kb: f32, y: &m }); } +/// Limited-range 10-bit 4:2:0 (code values 0..1023, not MSB-aligned) from encoded R'G'B' floats with +/// matrix (Kr, Kb): Y = 64 + 876·Y', C = 512 + 896·C, chroma averaged over each 2x2 block (one pixel +/// or pair at an odd edge). `rgb` is 3 floats per pixel; NaN counts as 0, infinities and +/// out-of-range values clamp to 0..1 first, and codes clamp to 4..1019 like [`rgbf_to_yuv422_10`] +/// (so black is 64, peak white 940 and the chroma of any colour stays within 64..960). +/// The planes are resized to `w * h` and `ceil(w / 2) * ceil(h / 2)`. A `rgb` slice of the wrong +/// length, a zero size or a size whose arithmetic overflows is an error. +#[allow(clippy::too_many_arguments)] +pub fn rgbf_to_yuv420_10(rgb: &[f32], w: usize, h: usize, kr: f32, kb: f32, y: &mut Vec, u: &mut Vec, v: &mut Vec) -> Result<()> { + let bad = || ExportError::Encode(format!("a {w}x{h} HDR picture needs w*h*3 floats, got {}", rgb.len())); + let px = w.checked_mul(h).ok_or_else(bad)?; + if w == 0 || h == 0 || px.checked_mul(3) != Some(rgb.len()) { + return Err(bad()); + } + let (cw, ch) = (w.div_ceil(2), h.div_ceil(2)); + let kg = 1.0 - kr - kb; + let (sb, sr) = (2.0 * (1.0 - kb), 2.0 * (1.0 - kr)); + y.clear(); + y.resize(px, 0); + u.clear(); + u.resize(cw * ch, 0); + v.clear(); + v.resize(cw * ch, 0); + let unit = |x: f32| if x.is_nan() { 0.0 } else { x.clamp(0.0, 1.0) }; + let code = |x: f32| x.round().clamp(4.0, 1019.0) as u16; + y.par_chunks_mut(w * 2).zip(u.par_chunks_mut(cw).zip(v.par_chunks_mut(cw))).enumerate().for_each(|(cy, (yr, (ur, vr)))| { + let rows = yr.len() / w; + let mut us = vec![0f32; cw]; + let mut vs = vec![0f32; cw]; + let mut cnt = vec![0f32; cw]; + for (dy, yrow) in yr.chunks_exact_mut(w).enumerate().take(rows) { + let row = cy * 2 + dy; + let Some(src) = rgb.get(row * w * 3..(row + 1) * w * 3) else { continue }; + for (x, (p, yo)) in src.chunks_exact(3).zip(yrow.iter_mut()).enumerate() { + let &[r, g, b] = p else { continue }; + let (r, g, b) = (unit(r), unit(g), unit(b)); + let yy = kr * r + kg * g + kb * b; + *yo = code(64.0 + 876.0 * yy); + if let (Some(a), Some(c), Some(n)) = (us.get_mut(x / 2), vs.get_mut(x / 2), cnt.get_mut(x / 2)) { + *a += (b - yy) / sb; + *c += (r - yy) / sr; + *n += 1.0; + } + } + } + for (cx, (uo, vo)) in ur.iter_mut().zip(vr.iter_mut()).enumerate() { + let n = cnt.get(cx).copied().unwrap_or(1.0).max(1.0); + *uo = code(512.0 + 896.0 * us.get(cx).copied().unwrap_or(0.0) / n); + *vo = code(512.0 + 896.0 * vs.get(cx).copied().unwrap_or(0.0) / n); + } + }); + Ok(()) +} + /// Slices per H.264 picture: one per four macroblock rows. The encoder's default follows the core /// count, which would make the stream (and every decoded picture) depend on the machine; this is /// what it picks on a machine with enough cores. @@ -1199,12 +1284,11 @@ fn h264_factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSetti } if s.signal.is_hdr() { cfg.color = filmcraft_h264enc::ColorConfig { primaries: s.signal.primaries, transfer: s.signal.transfer, matrix: s.signal.matrix, full_range: false }; - if s.signal.transfer == 16 { - let md = filmcraft_isobmff::MasteringDisplay::bt2020(1000.0, 0.0001).to_bytes(); + if let Some((md, cll)) = s.signal.static_metadata() { let mut b = [0u8; 24]; - b.copy_from_slice(&md); + b.copy_from_slice(&md.to_bytes()); cfg.mastering_display = Some(b); - cfg.content_light = Some((0, 0)); + cfg.content_light = Some(cll); } } Some( @@ -1404,3 +1488,6 @@ mod mxf_tests; #[cfg(test)] mod determinism_tests; + +#[cfg(test)] +mod hdr_tests; diff --git a/crates/export/src/settings.rs b/crates/export/src/settings.rs index 91d89a16..dad3b0e4 100644 --- a/crates/export/src/settings.rs +++ b/crates/export/src/settings.rs @@ -509,7 +509,9 @@ impl ExportSettings { v += &format!(", keyframe every {} frames", r.keyint); } Format::Hevc => { - v += &format!(", HEVC Main (hardware encoder), {}", self.bitrate_mode.label()); + // Main 10 only where a registered encoder writes HDR (NVENC); VideoToolbox is 8-bit + let profile = if crate::hdr_available(Format::Hevc) { "HEVC Main / Main 10 for HDR" } else { "HEVC Main" }; + v += &format!(", {profile} (hardware encoder), {}", self.bitrate_mode.label()); v += &match self.bitrate_mode { BitrateMode::Cbr => format!(", {}", mbps(r.target_kbps)), _ => format!(", Target {}, Max {}", mbps(r.target_kbps), mbps(r.max_kbps)), diff --git a/crates/platform/README.md b/crates/platform/README.md index 4aecde21..5733009e 100644 --- a/crates/platform/README.md +++ b/crates/platform/README.md @@ -156,7 +156,7 @@ level 4 High tier, which many hardware decoders refuse; it is now level 4.1 Main caller that asks while it runs waits for the same answer. The Hardware encoding toggle governs H.264 only: with H.265 selected NVENC is used whether it is Auto or Off, because nothing else can encode it. -- **What it writes:** HEVC Main, 8-bit 4:2:0, SDR BT.709 limited range (the VUI says so, with the +- **What it writes:** HEVC Main, 8-bit 4:2:0, SDR BT.709 limited range (Main 10 HDR: see below) (the VUI says so, with the frame rate as `time_scale / num_units_in_tick`), progressive, preset P5 with high-quality tuning, constant or one-pass variable bitrate, an IDR at every keyframe distance, one B-frame when the GPU reports support and the GOP is longer than two pictures (shorter ones are written without), in @@ -179,12 +179,43 @@ level 4 High tier, which many hardware decoders refuse; it is now level 4.1 Main offsets are `pts - dts` in frame durations, and the media starts `delay` frames early (edit list). - **There is no fallback encoder.** A request NVENC cannot take is counted in `export.hardware.declined` and ends the export with "H.265 export with NVENC: …" and the reason: - HDR, interlaced output, two-pass (`ExportSettings::validate` refuses it first), non-square pixels, + 10-bit on a GPU without it (HDR), interlaced output, two-pass (`ExportSettings::validate` refuses it first), non-square pixels, MXF, sizes above 65535, odd sizes, sizes outside the GPU's limits (129×33 to 8192×8192 on the RTX 5060). On a machine without an HEVC encoder the format is not available and an export fails with "encoder not available yet". Frames, sessions and declines are the same `export.hardware` counters as for H.264. -- Main 10 / HDR is not done (see Not yet). +- **Main 10 HDR (PQ / HLG).** An HDR sequence (Rec. 2100 PQ or HLG working space) exports as HEVC + Main 10 when `settings.sdr` is off and this GPU has a 10-bit HEVC encoder. `register()` hands + `nvenc::hevc_hdr_available` (a Main 10 session, cached, asked on the same background thread as the + 8-bit probe and short-circuiting when HEVC itself is missing) to `filmcraft_export::register_hdr_probe`, + and the export job makes the export HDR for `Format::Hevc` only when + `filmcraft_export::hdr_available(Format::Hevc)` says so: on macOS (VideoToolbox is 8-bit) and on GPUs + without 10-bit an HDR sequence is tone-mapped to 8-bit SDR HEVC exactly as before, and `settings.sdr` + still forces that. The pixels are real: `EncoderFrame::hdr` (encoded BT.2020 R'G'B', 3 floats per + pixel) goes through `filmcraft_export::rgbf_to_yuv420_10` (BT.2020 non-constant-luminance matrix, + Y = 64 + 876·Y', C = 512 + 896·C, 2x2 chroma average, NaN counts as 0, values clamp to 0..1 and codes + to 4..1019, wrong-length input is an error) and into a **P010** input buffer (`code << 6`, + little-endian, chroma interleaved after the luma rows, pitch in bytes; a driver pitch under 2 bytes + per pixel is refused). One buffer format (`NV_ENC_BUFFER_FORMAT_YUV420_10BIT`) is used for the + initialisation, the input buffers and every picture; `pixelBitDepthMinus8` is 2, the profile GUID + is Main 10, `NV_ENC_CAPS_SUPPORT_10BIT_ENCODE` is queried first and a GPU without it declines + with "no 10-bit (HEVC Main 10) support". The VUI says BT.2020 primaries, transfer 16 (PQ) or 18 + (HLG), matrix 9, limited range. The `hvcC` is checked against what was asked (`hevc_config(…, 10)`: + profile 2, 10-bit 4:2:0). NVENC chose the High tier for level 4 at 12 Mb/s; the record just says so. + Pictures of the wrong depth for the encoder (an `hdr` picture to a Main encoder, an RGBA one to + Main 10, or a float slice of the wrong length) are errors, never panics. +- **HDR10 static metadata is SEI, not an API.** NVENC 12.1 has no mastering-display field, so PQ + streams get the mastering display colour volume (payload 137) and content light level (144) + messages through `NV_ENC_PIC_PARAMS_HEVC::seiPayloadArray` on every IDR picture, with the values + of `ColorSignal::static_metadata` (BT.2020/D65, 1000 / 0.0001 cd/m², MaxCLL/MaxFALL 0), the same + that the software H.264 encoder's SEI and the `mdcv` / `clli` boxes carry. NVENC writes them as + two prefix SEI NAL units (type 39) in front of the IDR slice, adding the emulation-prevention + byte its `00 00 00 01` needs; nothing is attached to other pictures; HLG has no static metadata and + gets none. The payload bytes and the descriptor array are heap blocks owned by the `Session` for + its whole life (it is moved by value, and the driver may queue a picture), and IDR pictures are + known from a submission counter (`n % gop == 0`). The sample entry carries `colr` (nclx BT.2020, + PQ / HLG, limited), and for PQ `mdcv` and `clli`; 8-bit SDR HEVC is unchanged and has no `colr` + (the VUI carries its description, as in the software H.264 path). ## Guarantees @@ -215,9 +246,12 @@ level 4 High tier, which many hardware decoders refuse; it is now level 4.1 Main | `tests/nvenc_hevc.rs` (Windows, NVIDIA with HEVC) | HEVC from NVENC (1280×720, 6 Mbps, 72 frames, keyframe every 24) decodes with our HEVC decoder at worst 48.9 dB luma / 51.0 dB chroma PSNR; IDR at 0, 24, 48; dts strictly increasing, pts a permutation, `sps_max_num_reorder_pics` within the dts shift; no parameter sets in the samples; the `hvcC`'s profile / tier / level bytes are the SPS's own; VUI BT.709 limited and timing `(1, 24)`; hostile configurations, sizes, planes and encoders dropped mid-stream give errors, never panics; keyframes every 1–2 pictures without B-frames | | `tests/nvenc_hevc_export.rs` (Windows, NVIDIA with HEVC) | H.265 export through the real pipeline, MP4 and QuickTime, toggle Auto and Off: counters (72 frames, 1 session, 0 declined), our decoder against the software H.264 export at worst 54.8 dB luma PSNR, ffprobe `codec_name=hevc`, `profile=Main`, `codec_tag_string=hvc1`, `pix_fmt=yuv420p`, BT.709 limited, 72 frames, 24/1, keyframes at 0 and 48, start 0, `ffmpeg -xerror` clean; with AAC audio; 1920×1080 and 642×362 cropped back from the coded size; every decline (HDR, analysis pass, two-pass, interlaced, non-square pixels, MXF, sizes over 65535, odd sizes, sizes outside the GPU's limits) an error naming NVENC, counted once; H.264 with hardware encoding Off never touches NVENC | | `tests/nvenc_hevc_probe.rs` (Windows) | the HEVC probe in a fresh process: its cost, the cached answer, `available(Hevc)` following it after `register()` (twice) | -| `tests/nvenc_hevc_warm.rs` (Windows) | `register()` answers the HEVC question on a thread of its own, before anyone asks; the answer is kept | -| `src/nvenc/abi_tests.rs` (Windows) | FFI structs' sizes, alignments, field offsets, constants, GUIDs and the bit-field masks of the `flags` words (H.264 and HEVC) against a C compiler's view of NVIDIA's `nvEncodeAPI.h` (12.1) | -| `src/nvenc/{mod,hevc}.rs` unit tests (Windows) | HEVC NAL types, stripping and parameter-set splitting (including `IDR_N_LP`, whose header byte reads as a PPS in H.264, and one-byte or truncated NAL units); the `hvcC` built from real VPS / SPS / PPS, every truncation and bit flip of the SPS, wrong profile, size and NAL types; HEVC levels | +| `tests/nvenc_hevc_warm.rs` (Windows) | `register()` answers the HEVC and the Main 10 questions on a thread of its own, before anyone asks; the answers are kept and `hdr_available(Hevc)` follows the Main 10 probe | +| `tests/nvenc_hevc_main10.rs` (Windows, NVIDIA with 10-bit HEVC) | HEVC Main 10 in process, PQ and HLG: a float picture (grey ramp to peak white, a moving box, saturated BT.2020 bars) is converted with `rgbf_to_yuv420_10`, encoded and decoded with our HEVC decoder (`Yuv16`, 10 bits, codes below 1024): worst 71.1 dB luma / 70.6 dB chroma PSNR on the 10-bit scale, 803 distinct luma levels on a ramp row, peak code 940 survives; IDR at 0 and 24; the VUI is BT.2020 / 16 or 18 / BT.2020 NCL / limited; the IDR samples carry two prefix SEI NAL units whose messages are byte-for-byte the expected 137 / 144 payloads (HLG: none; other pictures: none); Main and Main 10 refuse each other's pictures; hostile sizes, bitrates, signals, huge or empty SEI payloads, encoders dropped with SEI in flight | +| `tests/nvenc_hevc_hdr_export.rs` (Windows, NVIDIA with 10-bit HEVC) | HDR sequences through the real export job (PQ MP4, HLG MP4, PQ QuickTime): 24 frames, 1 session, 0 declined; ffprobe `hevc` / `Main 10` / `hvc1` / `yuv420p10le`, `bt2020` / `smpte2084` or `arib-std-b67` / `bt2020nc` / `tv`; PQ first-frame side data Mastering display (max 10000000/10000, min 1/10000) and Content light level, HLG none; `ffmpeg -xerror` clean; ffmpeg's yuv420p10le picture equals our decoder's, code for code; our importer reads PQ / HLG, BT.2020, 10-bit, and the `colr` / `mdcv` / `clli` boxes; luma PSNR against a ProRes HDR export of the same sequence 62.6 dB (PQ) / 61.8 dB (HLG); `settings.sdr` and SDR sequences stay Main 8-bit BT.709 with no HDR boxes; the factory rejects mismatched or wrong-length pictures and clamps NaN / infinity | +| `src/nvenc/abi_tests.rs` (Windows) | FFI structs' sizes, alignments, field offsets, constants, GUIDs and the bit-field masks of the `flags` words (H.264, HEVC, and the HEVC picture parameters with the SEI payload array) against a C compiler's view of NVIDIA's `nvEncodeAPI.h` (12.1) | +| `src/nvenc/{mod,hevc,export}.rs` unit tests (Windows) | HEVC NAL types, stripping and parameter-set splitting (including `IDR_N_LP`, whose header byte reads as a PPS in H.264, and one-byte or truncated NAL units); the `hvcC` built from real VPS / SPS / PPS (Main and Main 10: accepted as what was asked for, refused as the other, other depths refused), every truncation and bit flip of the SPS, wrong profile, size and NAL types; HEVC levels; P010 layout (shift, little-endian, interleaving, pitch in bytes, narrow pitches and short buffers); the 10-bit capability decline; the SEI messages | +| `crates/export/src/hdr_tests.rs` | `rgbf_to_yuv420_10`: exact codes for black, white, grey and the BT.2020 primaries and secondaries, NaN / infinity / out-of-range, odd sizes, wrong lengths and overflowing sizes; the HDR probe registry; which exports are HDR (HEVC SDR without a probe, HDR with one, `settings.sdr` forcing SDR); the SEI of the software H.264 encoder equals `ColorSignal::static_metadata` | Fixtures are made with ffmpeg into `target/fixtures/platform/` (generator only, never linked); tests skip without ffmpeg or without a hardware decoder. @@ -245,6 +279,7 @@ playback with no dropped frames at Full, 1/2 and 1/4. Details in Zero-copy upload of decoded pictures into wgpu textures (`CVPixelBuffer`s on macOS, Direct3D 11 textures on Windows); B-frames in the VideoToolbox encoders and 10-bit / HDR HEVC (Main 10) in -hardware encoding; VA-API (Linux) decoders; field-coded H.264; 10-bit and HDR encoding with NVENC -(Main 10 HEVC, 4:4:4), AV1 with NVENC, H.265 with the other Windows vendors and on Linux, and encoders +VideoToolbox; VA-API (Linux) decoders; field-coded H.264; 4:4:4 and 4:2:2 encoding with NVENC, HDR +H.264 with NVENC (HDR H.264 stays with the software encoder), HDR10 MaxCLL / MaxFALL measured from +the pictures (they are written as 0, unknown), AV1 with NVENC, H.265 with the other Windows vendors and on Linux, and encoders from other vendors on Windows (through Media Foundation); VP9 / AV1 4:4:4 and 12-bit on Windows. diff --git a/crates/platform/src/lib.rs b/crates/platform/src/lib.rs index 45422e31..2f595304 100644 --- a/crates/platform/src/lib.rs +++ b/crates/platform/src/lib.rs @@ -7,7 +7,8 @@ //! registration does nothing and reports [`Availability::Unavailable`]. It also registers a //! hardware H.264 encoder factory (`filmcraft_export::register_encoder`) that only acts when an //! export asks for it (`ExportSettings::hardware_encoding` = `Auto`), see [`hardware_encode`]; on -//! macOS and on Windows (NVENC, [`nvenc`]) it also makes the H.265 export format available. +//! macOS and on Windows (NVENC, [`nvenc`]) it also makes the H.265 export format available, and on Windows +//! (Main 10) lets HDR sequences export as HDR H.265. //! //! Hardware decoding never makes a file undecodable: //! @@ -74,6 +75,9 @@ pub fn register() -> Availability { #[cfg(target_os = "windows")] { filmcraft_export::register_format_probe(filmcraft_export::Format::Hevc, nvenc::hevc_available); + // HDR sequences export as HEVC Main 10 (PQ / HLG) only where NVENC has a 10-bit encoder; elsewhere + // (and on macOS, whose VideoToolbox path is 8-bit) they are tone-mapped to SDR, as before + filmcraft_export::register_hdr_probe(filmcraft_export::Format::Hevc, nvenc::hevc_hdr_available); nvenc::warm_hevc_probe(); } #[cfg(target_os = "windows")] diff --git a/crates/platform/src/nvenc/abi_tests.rs b/crates/platform/src/nvenc/abi_tests.rs index 9e399612..7ed9f0b9 100644 --- a/crates/platform/src/nvenc/abi_tests.rs +++ b/crates/platform/src/nvenc/abi_tests.rs @@ -336,6 +336,41 @@ fn layouts_match_the_header() { assert_eq!(offset_of!(NV_ENCODE_API_FUNCTION_LIST, nvEncRestoreEncoderState), 336, "NV_ENCODE_API_FUNCTION_LIST.nvEncRestoreEncoderState"); assert_eq!(offset_of!(NV_ENCODE_API_FUNCTION_LIST, nvEncLookaheadPicture), 344, "NV_ENCODE_API_FUNCTION_LIST.nvEncLookaheadPicture"); assert_eq!(offset_of!(NV_ENCODE_API_FUNCTION_LIST, reserved2), 352, "NV_ENCODE_API_FUNCTION_LIST.reserved2"); + assert_eq!((size_of::(), align_of::()), (16, 8), "NV_ENC_SEI_PAYLOAD"); + assert_eq!(offset_of!(NV_ENC_SEI_PAYLOAD, payloadSize), 0, "NV_ENC_SEI_PAYLOAD.payloadSize"); + assert_eq!(offset_of!(NV_ENC_SEI_PAYLOAD, payloadType), 4, "NV_ENC_SEI_PAYLOAD.payloadType"); + assert_eq!(offset_of!(NV_ENC_SEI_PAYLOAD, payload), 8, "NV_ENC_SEI_PAYLOAD.payload"); + assert_eq!((size_of::(), align_of::()), (8, 4), "NV_ENC_CLOCK_TIMESTAMP_SET"); + assert_eq!(offset_of!(NV_ENC_CLOCK_TIMESTAMP_SET, timeOffset), 4, "NV_ENC_CLOCK_TIMESTAMP_SET.timeOffset"); + assert_eq!((size_of::(), align_of::()), (28, 4), "NV_ENC_TIME_CODE"); + assert_eq!(offset_of!(NV_ENC_TIME_CODE, displayPicStruct), 0, "NV_ENC_TIME_CODE.displayPicStruct"); + assert_eq!(offset_of!(NV_ENC_TIME_CODE, clockTimestamp), 4, "NV_ENC_TIME_CODE.clockTimestamp"); + assert_eq!((size_of::(), align_of::()), (1536, 8), "NV_ENC_PIC_PARAMS_HEVC"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, displayPOCSyntax), 0, "NV_ENC_PIC_PARAMS_HEVC.displayPOCSyntax"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, refPicFlag), 4, "NV_ENC_PIC_PARAMS_HEVC.refPicFlag"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, temporalId), 8, "NV_ENC_PIC_PARAMS_HEVC.temporalId"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, forceIntraRefreshWithFrameCnt), 12, "NV_ENC_PIC_PARAMS_HEVC.forceIntraRefreshWithFrameCnt"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, sliceTypeData), 24, "NV_ENC_PIC_PARAMS_HEVC.sliceTypeData"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, sliceTypeArrayCnt), 32, "NV_ENC_PIC_PARAMS_HEVC.sliceTypeArrayCnt"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, sliceMode), 36, "NV_ENC_PIC_PARAMS_HEVC.sliceMode"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, sliceModeData), 40, "NV_ENC_PIC_PARAMS_HEVC.sliceModeData"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, ltrMarkFrameIdx), 44, "NV_ENC_PIC_PARAMS_HEVC.ltrMarkFrameIdx"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, ltrUseFrameBitmap), 48, "NV_ENC_PIC_PARAMS_HEVC.ltrUseFrameBitmap"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, ltrUsageMode), 52, "NV_ENC_PIC_PARAMS_HEVC.ltrUsageMode"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, seiPayloadArrayCnt), 56, "NV_ENC_PIC_PARAMS_HEVC.seiPayloadArrayCnt"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, reserved), 60, "NV_ENC_PIC_PARAMS_HEVC.reserved"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, seiPayloadArray), 64, "NV_ENC_PIC_PARAMS_HEVC.seiPayloadArray"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, timeCode), 72, "NV_ENC_PIC_PARAMS_HEVC.timeCode"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, reserved2), 100, "NV_ENC_PIC_PARAMS_HEVC.reserved2"); + assert_eq!(offset_of!(NV_ENC_PIC_PARAMS_HEVC, reserved3), 1048, "NV_ENC_PIC_PARAMS_HEVC.reserved3"); + assert_eq!(offset_of!(NV_ENC_CLOCK_TIMESTAMP_SET, bits), 0, "NV_ENC_CLOCK_TIMESTAMP_SET.bits"); + assert_eq!(offset_of!(NV_ENC_CODEC_PIC_PARAMS, hevcPicParams), 0, "NV_ENC_CODEC_PIC_PARAMS.hevcPicParams"); + assert_eq!(offset_of!(NV_ENC_CODEC_PIC_PARAMS, reserved), 0, "NV_ENC_CODEC_PIC_PARAMS.reserved"); + assert_eq!( + offset_of!(NV_ENC_PIC_PARAMS_HEVC, flags), + 16, + "NV_ENC_PIC_PARAMS_HEVC.flags (the C program sets constrainedFrame / ltrUseFrames and reads the word at 16: 0x1 / 0x8)" + ); } #[test] @@ -400,6 +435,12 @@ fn constants_match_the_header() { assert_eq!(NV_ENC_LEVEL_HEVC_61 as i64, 183, "NV_ENC_LEVEL_HEVC_61"); assert_eq!(NV_ENC_LEVEL_HEVC_62 as i64, 186, "NV_ENC_LEVEL_HEVC_62"); assert_eq!(NVENCAPI_VERSION as i64, 16777228, "NVENCAPI_VERSION"); + assert_eq!(NV_ENC_BUFFER_FORMAT_YUV420_10BIT as i64, 65536, "NV_ENC_BUFFER_FORMAT_YUV420_10BIT"); + assert_eq!(NV_ENC_CAPS_SUPPORT_10BIT_ENCODE as i64, 39, "NV_ENC_CAPS_SUPPORT_10BIT_ENCODE"); + assert_eq!(NV_ENC_VUI_COLOR_PRIMARIES_BT2020 as i64, 9, "NV_ENC_VUI_COLOR_PRIMARIES_BT2020"); + assert_eq!(NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SMPTE2084 as i64, 16, "NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SMPTE2084"); + assert_eq!(NV_ENC_VUI_TRANSFER_CHARACTERISTIC_ARIB_STD_B67 as i64, 18, "NV_ENC_VUI_TRANSFER_CHARACTERISTIC_ARIB_STD_B67"); + assert_eq!(NV_ENC_VUI_MATRIX_COEFFS_BT2020_NCL as i64, 9, "NV_ENC_VUI_MATRIX_COEFFS_BT2020_NCL"); } #[test] @@ -412,6 +453,7 @@ fn guids_match_the_header() { assert_eq!(NV_ENC_PRESET_P5_GUID.to_u128(), 0x21c6e6b4297a4cba998fb6cbde72ade3u128, "NV_ENC_PRESET_P5_GUID"); assert_eq!(NV_ENC_CODEC_HEVC_GUID.to_u128(), 0x790cdc8845224d7b9425bda9975f7603u128, "NV_ENC_CODEC_HEVC_GUID"); assert_eq!(NV_ENC_HEVC_PROFILE_MAIN_GUID.to_u128(), 0xb514c39ab55b40fa878ff1253b4dfdecu128, "NV_ENC_HEVC_PROFILE_MAIN_GUID"); + assert_eq!(NV_ENC_HEVC_PROFILE_MAIN10_GUID.to_u128(), 0xfa4d2b6c3a5b411a80180a3f5e3c9be5u128, "NV_ENC_HEVC_PROFILE_MAIN10_GUID"); } /// The bit-field masks of the `flags` words (the C compiler's layout of each bit-field set alone). diff --git a/crates/platform/src/nvenc/export.rs b/crates/platform/src/nvenc/export.rs index a7733207..b43a1c86 100644 --- a/crates/platform/src/nvenc/export.rs +++ b/crates/platform/src/nvenc/export.rs @@ -3,9 +3,12 @@ //! //! - H.264: it takes an export only when Export ▸ Hardware encoding is Auto and NVENC can do what //! the settings ask; otherwise it returns `None` and the software encoder runs, exactly as before. -//! - H.265 (HEVC Main, 8-bit 4:2:0, SDR BT.709, progressive): there is no software encoder, so the -//! Hardware encoding toggle does not apply (as with VideoToolbox) and choosing the format is the -//! opt-in. What NVENC cannot do (HDR, interlaced, two-pass, non-square pixels, sizes outside its +//! - H.265 (HEVC Main 8-bit 4:2:0 SDR BT.709, or Main 10 HDR: 10-bit 4:2:0, BT.2020, PQ or HLG, limited +//! range; progressive): there is no software encoder, so the Hardware encoding toggle does not apply +//! (as with VideoToolbox) and choosing the format is the opt-in. The export crate hands HDR pictures +//! over (`EncoderFrame::hdr`) only where [`super::hevc_hdr_available`] said this GPU has a 10-bit +//! encoder, otherwise an HDR sequence is tone-mapped to SDR as before. What NVENC cannot do +//! (interlaced, two-pass, non-square pixels, 10-bit on a GPU without it, sizes outside its //! limits...) is counted as declined and ends the export with an error that says why; when this //! machine has no HEVC encoder at all, [`filmcraft_export::available`] says so and the factory //! returns `None` (the export's own "encoder not available" error). @@ -15,12 +18,13 @@ //! two write different streams. use filmcraft_export::{ - BitrateMode, EncodedPacket, EncoderFrame, ExportError, ExportSettings, FieldOrder, Format, H264Pass, H264Profile, HardwareEncoding, Result, VideoEncoder, + BitrateMode, ColorSignal, EncodedPacket, EncoderFrame, ExportError, ExportSettings, FieldOrder, Format, H264Pass, H264Profile, HardwareEncoding, Result, + VideoEncoder, }; use filmcraft_isobmff::{AvcConfig, SampleEntry}; use filmcraft_time::FrameRate; -use super::{Codec, Config, Nvenc, Profile}; +use super::{Codec, Config, Nvenc, Profile, Sei, Signal}; /// The NVENC export encoder. struct NvencEncoder { @@ -28,9 +32,38 @@ struct NvencEncoder { w: u32, h: u32, rate: FrameRate, + /// The stream's colour signal (`colr` / `mdcv` / `clli` on the sample entry for HDR). + signal: ColorSignal, y: Vec, u: Vec, v: Vec, + /// Main 10: the 10-bit planes of the current picture. + y16: Vec, + u16: Vec, + v16: Vec, +} + +/// SEI payload types of the HDR10 static metadata (H.265 Annex D): mastering display colour volume +/// and content light level information. +const SEI_MASTERING_DISPLAY: u32 = 137; +const SEI_CONTENT_LIGHT_LEVEL: u32 = 144; + +/// The encoder's signal for an export's colour signal: the VUI code points and, for PQ, the HDR10 +/// static metadata as SEI messages (the same values and bytes the `mdcv` / `clli` boxes and the software +/// H.264 encoder's SEI carry: [`ColorSignal::static_metadata`]). HLG has no static metadata. +pub fn nvenc_signal(signal: &ColorSignal) -> Signal { + let mut sei = Vec::new(); + if let Some((md, (max_cll, max_fall))) = signal.static_metadata() { + sei.push(Sei { payload_type: SEI_MASTERING_DISPLAY, payload: md.to_bytes() }); + let mut cll = max_cll.to_be_bytes().to_vec(); + cll.extend_from_slice(&max_fall.to_be_bytes()); + sei.push(Sei { payload_type: SEI_CONTENT_LIGHT_LEVEL, payload: cll }); + } + Signal { primaries: signal.primaries, transfer: signal.transfer, matrix: signal.matrix, sei } +} + +fn new_encoder(enc: Nvenc, w: u32, h: u32, rate: FrameRate, signal: ColorSignal) -> NvencEncoder { + NvencEncoder { enc, w, h, rate, signal, y: Vec::new(), u: Vec::new(), v: Vec::new(), y16: Vec::new(), u16: Vec::new(), v16: Vec::new() } } impl NvencEncoder { @@ -55,8 +88,14 @@ impl VideoEncoder for NvencEncoder { let (w, h) = (u16::try_from(self.w).unwrap_or(u16::MAX), u16::try_from(self.h).unwrap_or(u16::MAX)); match self.enc.codec() { Codec::H264 => SampleEntry::avc(AvcConfig::new(vec![sps.to_vec()], vec![pps.to_vec()], 4), w, h), - // `Nvenc::new` built and validated the record, so there is always one for HEVC - Codec::Hevc => SampleEntry::hevc(self.enc.hevc_config().cloned().unwrap_or_default(), w, h), + Codec::Hevc => { + // `Nvenc::new` built and validated the record, so there is always one for HEVC + let mut e = SampleEntry::hevc(self.enc.hevc_config().cloned().unwrap_or_default(), w, h); + // HDR: `colr` (BT.2020 + PQ / HLG) and, for PQ, `mdcv` / `clli`, as the software H.264 encoder does + // (nclx, whatever the container: readers take it from either). SDR stays as it was, with the VUI only. + self.signal.apply_to(&mut e, false); + e + } } } @@ -65,17 +104,25 @@ impl VideoEncoder for NvencEncoder { } fn encode(&mut self, f: &EncoderFrame) -> Result> { - if f.hdr.is_some() { - return Err(ExportError::Unsupported("NVENC does not take HDR pictures here".into())); - } if f.width != self.w || f.height != self.h { return Err(ExportError::Encode(format!("NVENC: a {}x{} picture for a {}x{} encoder", f.width, f.height, self.w, self.h))); } - filmcraft_export::rgba_to_yuv420_8(f.rgba, self.w as usize, self.h as usize, &mut self.y, &mut self.u, &mut self.v); - let ps = self - .enc - .encode(&self.y, &self.u, &self.v, f.index) - .map_err(|e| ExportError::Encode(format!("NVENC: {e} (turn Export ▸ Hardware encoding off to use the software encoder)")))?; + let (w, h) = (self.w as usize, self.h as usize); + let hint = if self.enc.codec() == Codec::H264 { " (turn Export ▸ Hardware encoding off to use the software encoder)" } else { "" }; + let ps = match (self.enc.is_ten_bit(), f.hdr) { + // Main 10 takes the encoded R'G'B' floats: 10-bit limited-range BT.2020 NCL (or the signal's matrix) + (true, Some(rgb)) => { + let (kr, kb) = self.signal.kr_kb(); + filmcraft_export::rgbf_to_yuv420_10(rgb, w, h, kr, kb, &mut self.y16, &mut self.u16, &mut self.v16)?; + self.enc.encode_10(&self.y16, &self.u16, &self.v16, f.index).map_err(|e| ExportError::Encode(format!("NVENC: {e}{hint}")))? + } + (true, None) => return Err(ExportError::Unsupported("NVENC Main 10 needs the HDR picture (EncoderFrame::hdr)".into())), + (false, Some(_)) => return Err(ExportError::Unsupported("NVENC Main (8-bit) does not take HDR pictures".into())), + (false, None) => { + filmcraft_export::rgba_to_yuv420_8(f.rgba, w, h, &mut self.y, &mut self.u, &mut self.v); + self.enc.encode(&self.y, &self.u, &self.v, f.index).map_err(|e| ExportError::Encode(format!("NVENC: {e}{hint}")))? + } + }; filmcraft_export::note_hw_encode_frame(); Ok(self.packets(ps)) } @@ -98,12 +145,13 @@ fn config(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) - if !(format == Format::H264 || hevc) || s.format.is_mxf() { return Err("only MP4 / MOV H.264 and H.265 exports".into()); } - if s.signal.is_hdr() { - return Err(if hevc { - "HDR (hardware H.265 here is 8-bit SDR Main)".into() - } else { - "HDR (8-bit H.264 here carries it with software signalling)".into() - }); + let hdr = s.signal.is_hdr(); + if hdr && !hevc { + return Err("HDR (8-bit H.264 here carries it with software signalling)".into()); + } + // HEVC: BT.709 SDR (Main) or BT.2020 NCL with PQ / HLG (Main 10); any other description is not written + if hevc && (hdr && (s.signal.primaries != 9 || s.signal.matrix != 9) || !hdr && s.signal != ColorSignal::default()) { + return Err(format!("the colour signal {:?} (H.265 here is BT.709, or BT.2020 with PQ / HLG)", s.signal)); } if !matches!(s.h264_pass, H264Pass::Single) || (hevc && s.bitrate_mode == BitrateMode::Vbr2Pass) { return Err("two-pass VBR".into()); @@ -135,6 +183,7 @@ fn config(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) - cbr: s.bitrate_mode == BitrateMode::Cbr, keyint: s.keyframe_distance.filter(|k| *k > 0).unwrap_or_else(|| (f64::from(fps.0) / f64::from(fps.1) * 2.0).round().max(1.0) as u32), profile: match (hevc, s.h264_profile) { + (true, _) if hdr => Profile::HevcMain10, (true, _) => Profile::HevcMain, (false, H264Profile::Baseline) => Profile::Baseline, (false, H264Profile::Main) => Profile::Main, @@ -167,7 +216,7 @@ pub fn factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettin match Nvenc::new(&cfg) { Ok(enc) => { filmcraft_export::note_hw_encode_session(); - Some(Ok(Box::new(NvencEncoder { enc, w, h, rate, y: Vec::new(), u: Vec::new(), v: Vec::new() }))) + Some(Ok(Box::new(new_encoder(enc, w, h, rate, s.signal)))) } Err(why) => declined(&why), } @@ -185,10 +234,10 @@ fn hevc_factory(w: u32, h: u32, rate: FrameRate, s: &ExportSettings) -> Option c, Err(why) => return declined(&why), }; - match Nvenc::new(&cfg) { + match Nvenc::with_signal(&cfg, &nvenc_signal(&s.signal)) { Ok(enc) => { filmcraft_export::note_hw_encode_session(); - Some(Ok(Box::new(NvencEncoder { enc, w, h, rate, y: Vec::new(), u: Vec::new(), v: Vec::new() }))) + Some(Ok(Box::new(new_encoder(enc, w, h, rate, s.signal)))) } // no HEVC encoder here at all: not a hardware attempt, the export's own "encoder not available" error Err(why) if !super::hevc_available() => { @@ -220,11 +269,18 @@ mod tests { // the H.264 level and profile settings mean nothing to HEVC let h = ExportSettings { h264_level: Some(41), h264_profile: H264Profile::Baseline, ..ok.clone() }; assert_eq!(config(Format::Hevc, 1280, 720, FrameRate::FPS_24, &h).unwrap().level, None); + // HDR (PQ / HLG, BT.2020) is Main 10, SDR stays Main; H.264 still declines HDR + for signal in [ColorSignal::PQ, ColorSignal::HLG] { + let hdr = ExportSettings { signal, ..ok.clone() }; + assert_eq!(config(Format::Hevc, 1280, 720, FrameRate::FPS_24, &hdr).unwrap().profile, Profile::HevcMain10); + assert!(config(Format::H264, 1280, 720, FrameRate::FPS_24, &ExportSettings { format: Format::H264, ..hdr }).is_err()); + } // the Hardware encoding toggle does not matter for HEVC (it has no software encoder) let off = ExportSettings { hardware_encoding: HardwareEncoding::Off, ..ok.clone() }; assert!(config(Format::Hevc, 1280, 720, FrameRate::FPS_24, &off).is_ok()); for (what, s) in [ - ("HDR", ExportSettings { signal: filmcraft_export::ColorSignal::PQ, ..ok.clone() }), + ("an unwritable colour signal", ExportSettings { signal: ColorSignal { primaries: 9, transfer: 1, matrix: 1 }, ..ok.clone() }), + ("HDR with BT.709 primaries", ExportSettings { signal: ColorSignal { primaries: 1, transfer: 16, matrix: 9 }, ..ok.clone() }), ("two-pass", ExportSettings { bitrate_mode: BitrateMode::Vbr2Pass, ..ok.clone() }), ("analysis pass", ExportSettings { h264_pass: H264Pass::First, ..ok.clone() }), ("interlaced", ExportSettings { field_order: FieldOrder::UpperFirst, ..ok.clone() }), @@ -243,6 +299,34 @@ mod tests { assert!(config(Format::ProRes, 1280, 720, FrameRate::FPS_24, &ok).is_err()); } + #[test] + fn the_sei_messages_are_the_hdr10_static_metadata_the_software_h264_path_writes() { + let pq = nvenc_signal(&ColorSignal::PQ); + assert_eq!((pq.primaries, pq.transfer, pq.matrix), (9, 16, 9)); + assert_eq!(pq.sei.len(), 2); + // mastering display colour volume (137): the 24-byte payload of ST 2086, the `mdcv` box's bytes + let (md, _) = ColorSignal::PQ.static_metadata().unwrap(); + assert_eq!((pq.sei[0].payload_type, pq.sei[0].payload.as_slice()), (137, md.to_bytes().as_slice())); + assert_eq!(pq.sei[0].payload.len(), 24); + // BT.2020 primaries (G, B, R order), D65, 1000 / 0.0001 cd/m2 in 0.0001 units + assert_eq!( + pq.sei[0].payload, + [0x21, 0x34, 0x9B, 0xAA, 0x19, 0x96, 0x08, 0xFC, 0x8A, 0x48, 0x39, 0x08, 0x3D, 0x13, 0x40, 0x42, 0x00, 0x98, 0x96, 0x80, 0x00, 0x00, 0x00, 0x01] + ); + // content light level (144): MaxCLL 0, MaxFALL 0 (unknown) + assert_eq!((pq.sei[1].payload_type, pq.sei[1].payload.as_slice()), (144, &[0u8, 0, 0, 0][..])); + // laid out as one SEI RBSP (type, size, payload, ...) these are the bytes the software H.264 encoder + // writes for the same metadata (checked against its real output in the export crate's hdr_tests) + let rbsp: Vec = + pq.sei.iter().flat_map(|m| [m.payload_type as u8, m.payload.len() as u8].into_iter().chain(m.payload.iter().copied())).chain([0x80]).collect(); + assert_eq!(&rbsp[..2], &[137, 24]); + assert_eq!(&rbsp[26..], &[144, 4, 0, 0, 0, 0, 0x80]); + // HLG: the colour description only; SDR: BT.709, no messages + let hlg = nvenc_signal(&ColorSignal::HLG); + assert_eq!((hlg.primaries, hlg.transfer, hlg.matrix, hlg.sei.len()), (9, 18, 9, 0)); + assert_eq!(nvenc_signal(&ColorSignal::default()), Signal::default()); + } + #[test] fn oversized_pictures_are_declined() { let s = ExportSettings { format: Format::H264, hardware_encoding: HardwareEncoding::Auto, ..Default::default() }; diff --git a/crates/platform/src/nvenc/ffi.rs b/crates/platform/src/nvenc/ffi.rs index f0bcfee9..c53042fd 100644 --- a/crates/platform/src/nvenc/ffi.rs +++ b/crates/platform/src/nvenc/ffi.rs @@ -72,6 +72,8 @@ pub const NV_ENC_ERR_LOCK_BUSY: NvStatus = 13; pub const NV_ENC_DEVICE_TYPE_DIRECTX: u32 = 0; pub const NV_ENC_MEMORY_HEAP_SYSMEM_CACHED: u32 = 2; pub const NV_ENC_BUFFER_FORMAT_NV12: u32 = 0x1; +/// P010: 10-bit 4:2:0, 16-bit little-endian samples with the value in the high 10 bits. +pub const NV_ENC_BUFFER_FORMAT_YUV420_10BIT: u32 = 0x10000; pub const NV_ENC_PARAMS_RC_VBR: u32 = 1; pub const NV_ENC_PARAMS_RC_CBR: u32 = 2; pub const NV_ENC_TUNING_INFO_HIGH_QUALITY: u32 = 1; @@ -103,10 +105,15 @@ pub const NV_ENC_VUI_VIDEO_FORMAT_UNSPECIFIED: u32 = 5; pub const NV_ENC_VUI_COLOR_PRIMARIES_BT709: u32 = 1; pub const NV_ENC_VUI_TRANSFER_CHARACTERISTIC_BT709: u32 = 1; pub const NV_ENC_VUI_MATRIX_COEFFS_BT709: u32 = 1; +pub const NV_ENC_VUI_COLOR_PRIMARIES_BT2020: u32 = 9; +pub const NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SMPTE2084: u32 = 16; +pub const NV_ENC_VUI_TRANSFER_CHARACTERISTIC_ARIB_STD_B67: u32 = 18; +pub const NV_ENC_VUI_MATRIX_COEFFS_BT2020_NCL: u32 = 9; /// `NV_ENC_CAPS` values (the enum counts from zero in the header's order). pub const NV_ENC_CAPS_NUM_MAX_BFRAMES: u32 = 0; pub const NV_ENC_CAPS_SUPPORTED_RATECONTROL_MODES: u32 = 1; +pub const NV_ENC_CAPS_SUPPORT_10BIT_ENCODE: u32 = 39; pub const NV_ENC_CAPS_WIDTH_MAX: u32 = 16; pub const NV_ENC_CAPS_HEIGHT_MAX: u32 = 17; pub const NV_ENC_CAPS_WIDTH_MIN: u32 = 45; @@ -118,6 +125,7 @@ pub const NV_ENC_H264_PROFILE_BASELINE_GUID: GUID = GUID::from_values(0x0727bcaa pub const NV_ENC_H264_PROFILE_MAIN_GUID: GUID = GUID::from_values(0x60b5c1d4, 0x67fe, 0x4790, [0x94, 0xd5, 0xc4, 0x72, 0x6d, 0x7b, 0x6e, 0x6d]); pub const NV_ENC_H264_PROFILE_HIGH_GUID: GUID = GUID::from_values(0xe7cbc309, 0x4f7a, 0x4b89, [0xaf, 0x2a, 0xd5, 0x37, 0xc9, 0x2b, 0xe3, 0x10]); pub const NV_ENC_HEVC_PROFILE_MAIN_GUID: GUID = GUID::from_values(0xb514c39a, 0xb55b, 0x40fa, [0x87, 0x8f, 0xf1, 0x25, 0x3b, 0x4d, 0xfd, 0xec]); +pub const NV_ENC_HEVC_PROFILE_MAIN10_GUID: GUID = GUID::from_values(0xfa4d2b6c, 0x3a5b, 0x411a, [0x80, 0x18, 0x0a, 0x3f, 0x5e, 0x3c, 0x9b, 0xe5]); pub const NV_ENC_PRESET_P4_GUID: GUID = GUID::from_values(0x90a7b826, 0xdf06, 0x4862, [0xb9, 0xd2, 0xcd, 0x6d, 0x73, 0xa0, 0x86, 0x81]); pub const NV_ENC_PRESET_P5_GUID: GUID = GUID::from_values(0x21c6e6b4, 0x297a, 0x4cba, [0x99, 0x8f, 0xb6, 0xcb, 0xde, 0x72, 0xad, 0xe3]); @@ -401,10 +409,62 @@ pub struct NV_ENC_CREATE_BITSTREAM_BUFFER { pub reserved2: [*mut c_void; 64], } -/// `NV_ENC_CODEC_PIC_PARAMS`: a union of codec structures, all zero here (defaults). +/// `NV_ENC_SEI_PAYLOAD`: a user SEI message (payload type and bytes) for a picture. #[repr(C)] #[derive(Clone, Copy)] -pub struct NV_ENC_CODEC_PIC_PARAMS { +pub struct NV_ENC_SEI_PAYLOAD { + pub payloadSize: u32, + pub payloadType: u32, + pub payload: *mut u8, +} + +/// `NV_ENC_CLOCK_TIMESTAMP_SET`: countingType:1, discontinuityFlag:1, cntDroppedFrames:1, nFrames:8, +/// secondsValue:6, minutesValue:6, hoursValue:5, reserved2:4 in `bits`, then the time offset. +#[repr(C)] +#[derive(Clone, Copy)] +pub struct NV_ENC_CLOCK_TIMESTAMP_SET { + pub bits: u32, + pub timeOffset: u32, +} + +/// `NV_ENC_TIME_CODE` (`MAX_NUM_CLOCK_TS` is 3). +#[repr(C)] +#[derive(Clone, Copy)] +pub struct NV_ENC_TIME_CODE { + pub displayPicStruct: u32, + pub clockTimestamp: [NV_ENC_CLOCK_TIMESTAMP_SET; 3], +} + +/// `NV_ENC_PIC_PARAMS_HEVC`: the per-picture HEVC parameters; all zero but the SEI payload array here. +#[repr(C)] +#[derive(Clone, Copy)] +pub struct NV_ENC_PIC_PARAMS_HEVC { + pub displayPOCSyntax: u32, + pub refPicFlag: u32, + pub temporalId: u32, + pub forceIntraRefreshWithFrameCnt: u32, + /// constrainedFrame:1, sliceModeDataUpdate:1, ltrMarkFrame:1, ltrUseFrames:1, reservedBitFields:28. + pub flags: u32, + pub sliceTypeData: *mut u8, + pub sliceTypeArrayCnt: u32, + pub sliceMode: u32, + pub sliceModeData: u32, + pub ltrMarkFrameIdx: u32, + pub ltrUseFrameBitmap: u32, + pub ltrUsageMode: u32, + pub seiPayloadArrayCnt: u32, + pub reserved: u32, + pub seiPayloadArray: *mut NV_ENC_SEI_PAYLOAD, + pub timeCode: NV_ENC_TIME_CODE, + pub reserved2: [u32; 237], + pub reserved3: [*mut c_void; 61], +} + +/// `NV_ENC_CODEC_PIC_PARAMS`: a union of codec structures (H.264 and AV1 stay opaque inside `reserved`). +#[repr(C)] +#[derive(Clone, Copy)] +pub union NV_ENC_CODEC_PIC_PARAMS { + pub hevcPicParams: NV_ENC_PIC_PARAMS_HEVC, pub reserved: [u32; 388], align: [*mut c_void; 0], } diff --git a/crates/platform/src/nvenc/hevc.rs b/crates/platform/src/nvenc/hevc.rs index e0bda40f..e962e724 100644 --- a/crates/platform/src/nvenc/hevc.rs +++ b/crates/platform/src/nvenc/hevc.rs @@ -1,6 +1,9 @@ //! What is specific to HEVC in the NVENC backend (safe code): the level codes, and the `hvcC` //! record built from the parameter sets the encoder wrote. //! +//! Main (8-bit) and Main 10 (10-bit) are both written; the caller says which it asked the encoder for +//! and the record is refused if the SPS says otherwise. +//! //! Nothing in the record is invented: the profile, tier, compatibility flags, level, chroma format, //! bit depths and temporal layers come from the SPS (parsed with `filmcraft_hevc`), and the 48 //! general constraint flags and `sps_temporal_id_nesting_flag` are read from the SPS bits that carry @@ -65,8 +68,14 @@ const PTL_LEN: usize = 12; /// The `hvcC` record of a stream of `size` pictures (width, height) whose parameter sets are `vps`, /// `sps` and `pps` (NAL units with their headers, without start codes or length prefixes). An error -/// says why the stream is not one this backend writes: HEVC Main, 8-bit 4:2:0. -pub fn hevc_config(vps: &[u8], sps: &[u8], pps: &[u8], size: (u32, u32)) -> Result { +/// says why the stream is not the one that was asked for: `bit_depth` 8 is HEVC Main (profile 1), +/// 10 is Main 10 (profile 2), both 4:2:0 with equal luma and chroma depth; any other depth is refused. +pub fn hevc_config(vps: &[u8], sps: &[u8], pps: &[u8], size: (u32, u32), bit_depth: u8) -> Result { + let (profile_idc, profile_name) = match bit_depth { + 8 => (1u8, "Main"), + 10 => (2, "Main 10"), + other => return Err(format!("{other}-bit HEVC is not written by this backend")), + }; for (nal, kind, name) in [(vps, 32u8, "VPS"), (sps, 33, "SPS"), (pps, 34, "PPS")] { if nal_type(Codec::Hevc, nal) != Some(kind) || nal.len() <= 2 { return Err(format!("the encoder's {name} is not a {name} NAL unit")); @@ -85,12 +94,13 @@ pub fn hevc_config(vps: &[u8], sps: &[u8], pps: &[u8], size: (u32, u32)) -> Resu let temporal_id_nested = rbsp.first().is_some_and(|b| b & 1 != 0); let ptl = &parsed.ptl; - if ptl.profile_space != 0 || ptl.profile_idc != 1 { - return Err(format!("the encoder wrote HEVC profile {} (space {}), not Main", ptl.profile_idc, ptl.profile_space)); + if ptl.profile_space != 0 || ptl.profile_idc != profile_idc { + return Err(format!("the encoder wrote HEVC profile {} (space {}), not {profile_name}", ptl.profile_idc, ptl.profile_space)); } - if parsed.chroma_format_idc != 1 || parsed.bit_depth_luma != 8 || parsed.bit_depth_chroma != 8 { + let depth = u32::from(bit_depth); + if parsed.chroma_format_idc != 1 || parsed.bit_depth_luma != depth || parsed.bit_depth_chroma != depth { return Err(format!( - "the encoder wrote chroma format {} at {}/{} bits, not 8-bit 4:2:0", + "the encoder wrote chroma format {} at {}/{} bits, not {bit_depth}-bit 4:2:0", parsed.chroma_format_idc, parsed.bit_depth_luma, parsed.bit_depth_chroma )); } @@ -140,7 +150,7 @@ mod tests { } fn record() -> HevcConfig { - hevc_config(&unhex(VPS), &unhex(SPS), &unhex(PPS), (1280, 720)).unwrap() + hevc_config(&unhex(VPS), &unhex(SPS), &unhex(PPS), (1280, 720), 8).unwrap() } #[test] @@ -202,7 +212,7 @@ mod tests { fn the_picture_size_has_to_match_after_cropping() { let (v, s, p) = (unhex(VPS), unhex(SPS), unhex(PPS)); for wrong in [(1280, 736), (1920, 1080), (1280, 719), (0, 0)] { - let e = hevc_config(&v, &s, &p, wrong).unwrap_err(); + let e = hevc_config(&v, &s, &p, wrong, 8).unwrap_err(); assert!(e.contains("1280x720"), "{e}"); } } @@ -215,10 +225,10 @@ mod tests { let (v, s, p) = (unhex(VPS), unhex(SPS), unhex(PPS)); let mut other_profile = s.clone(); other_profile[3] = (other_profile[3] & !0x1f) | 2; // Main 10 - assert!(hevc_config(&v, &other_profile, &p, (1280, 720)).unwrap_err().contains("not Main")); + assert!(hevc_config(&v, &other_profile, &p, (1280, 720), 8).unwrap_err().contains("not Main")); let mut other_space = s.clone(); other_space[3] |= 0x40; - assert!(hevc_config(&v, &other_space, &p, (1280, 720)).is_err()); + assert!(hevc_config(&v, &other_space, &p, (1280, 720), 8).is_err()); } #[test] @@ -226,14 +236,14 @@ mod tests { let (v, s, p) = (unhex(VPS), unhex(SPS), unhex(PPS)); // every truncation of the SPS for n in 0..s.len() { - assert!(hevc_config(&v, &s[..n], &p, (1280, 720)).is_err(), "SPS cut at {n}"); + assert!(hevc_config(&v, &s[..n], &p, (1280, 720), 8).is_err(), "SPS cut at {n}"); } // a bit flip anywhere in the SPS: an error or a record, but never a panic for byte in 0..s.len() { for bit in 0..8 { let mut m = s.clone(); m[byte] ^= 1 << bit; - let r = std::panic::catch_unwind(|| hevc_config(&v, &m, &p, (1280, 720))); + let r = std::panic::catch_unwind(|| hevc_config(&v, &m, &p, (1280, 720), 8)); assert!(r.is_ok(), "panic with bit {bit} of byte {byte} flipped"); } } @@ -246,11 +256,59 @@ mod tests { (&v, &p, &s), ]; for (v, s, p) in cases { - assert!(hevc_config(v, s, p, (64, 64)).is_err(), "{v:?} {s:?} {p:?}"); + assert!(hevc_config(v, s, p, (64, 64), 8).is_err(), "{v:?} {s:?} {p:?}"); } // a parameter set too long for the record's 16-bit lengths let mut huge = s.clone(); huge.resize(70_000, 0x55); - assert!(hevc_config(&v, &huge, &p, (1280, 720)).is_err()); + assert!(hevc_config(&v, &huge, &p, (1280, 720), 8).is_err()); + } + + // The parameter sets NVENC wrote for a 1280x720, 24 fps HEVC Main 10 stream with the PQ signal (VUI: BT.2020 + // primaries, SMPTE ST 2084, BT.2020 NCL matrix, limited range) on the same GPU. + const VPS10: &str = "40010c01ffff2220000003009000000300000300789940c0000003004000000614"; + const SPS10: &str = "420101222000000300900000030000030078a00280802e1f12d96654a421191bff0c05a8488048a000000300200000030301"; + const PPS10: &str = "4401c1933c0cc9"; + + #[test] + fn a_main_10_stream_is_accepted_as_main_10_and_refused_as_main() { + let (v, s, p) = (unhex(VPS10), unhex(SPS10), unhex(PPS10)); + let c = hevc_config(&v, &s, &p, (1280, 720), 10).unwrap(); + assert_eq!((c.general_profile_space, c.general_tier_flag, c.general_profile_idc), (0, true, 2), "NVENC picked the High tier for level 4 at 12 Mbps"); + assert_eq!(c.general_profile_compatibility_flags, 0x2000_0000, "Main 10 only"); + assert_eq!(c.general_constraint_indicator_flags, 0x9000_0000_0000); + assert_eq!(c.general_level_idc, 120, "level 4"); + assert_eq!((c.chroma_format_idc, c.bit_depth_luma, c.bit_depth_chroma), (1, 10, 10)); + assert_eq!(HevcConfig::parse(&c.to_bytes()).unwrap(), c); + // the same stream where Main (8-bit) was asked for, and the 8-bit stream where Main 10 was asked for + let e = hevc_config(&v, &s, &p, (1280, 720), 8).unwrap_err(); + assert!(e.contains("not Main") && !e.contains("Main 10"), "{e}"); + let e = hevc_config(&unhex(VPS), &unhex(SPS), &unhex(PPS), (1280, 720), 10).unwrap_err(); + assert!(e.contains("not Main 10"), "{e}"); + // no other depth + for depth in [0, 7, 9, 12, 16, 255] { + assert!(hevc_config(&v, &s, &p, (1280, 720), depth).is_err(), "{depth}"); + } + // a Main 10 profile byte with 8-bit depth (or the reverse) is refused whichever is asked for + let mut forged = s.clone(); + forged[3] = (forged[3] & !0x1f) | 1; // profile_idc 1 (Main) but a 10-bit SPS + for depth in [8, 10] { + assert!(hevc_config(&v, &forged, &p, (1280, 720), depth).is_err(), "{depth}"); + } + } + + #[test] + fn truncated_and_flipped_main_10_parameter_sets_never_panic() { + let (v, s, p) = (unhex(VPS10), unhex(SPS10), unhex(PPS10)); + for n in 0..s.len() { + assert!(hevc_config(&v, &s[..n], &p, (1280, 720), 10).is_err(), "SPS cut at {n}"); + } + for byte in 0..s.len() { + for bit in 0..8 { + let mut m = s.clone(); + m[byte] ^= 1 << bit; + assert!(std::panic::catch_unwind(|| hevc_config(&v, &m, &p, (1280, 720), 10)).is_ok(), "panic with bit {bit} of byte {byte} flipped"); + } + } } } diff --git a/crates/platform/src/nvenc/mod.rs b/crates/platform/src/nvenc/mod.rs index feb7ca4e..7448045c 100644 --- a/crates/platform/src/nvenc/mod.rs +++ b/crates/platform/src/nvenc/mod.rs @@ -1,4 +1,4 @@ -//! NVIDIA NVENC hardware encoding, H.264 and H.265 (HEVC) Main (Windows). +//! NVIDIA NVENC hardware encoding, H.264 and H.265 (HEVC) Main / Main 10 (Windows). //! //! The encoder runs on the GPU's NVENC engine through the driver's `nvEncodeAPI64.dll` (API 12.1, //! [`ffi`]; no CUDA, no SDK to install): pictures go in as NV12 input buffers, the Annex B output @@ -11,6 +11,10 @@ //! ```text //! RGBA (the export pipeline) ──► BT.709 limited 4:2:0 (the software encoder's conversion) //! ──► NV12 input buffer ──NVENC──► Annex B ──► length-prefixed samples + avcC / hvcC +//! +//! HDR (PQ / HLG) HEVC Main 10: +//! encoded BT.2020 R'G'B' floats ──► 10-bit limited 4:2:0 (`filmcraft_export::rgbf_to_yuv420_10`) +//! ──► P010 input buffer ──NVENC──► same path; VUI BT.2020 + transfer, HDR10 SEI on every IDR (PQ) //! ``` //! //! `unsafe` is confined to `ffi` (data) and `session` (every driver call); this module is safe @@ -32,10 +36,10 @@ use std::collections::VecDeque; use filmcraft_isobmff::HevcConfig; -pub use self::session::{Caps, Codec, Params}; +pub use self::session::{Caps, Codec, Params, Sei, Signal}; use self::session::{Locked, Session, Submitted}; -/// H.264 profile (`profile_idc` 66 / 77 / 100), or HEVC Main (8-bit 4:2:0). +/// H.264 profile (`profile_idc` 66 / 77 / 100), or HEVC Main (8-bit 4:2:0) / Main 10 (10-bit 4:2:0). #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum Profile { Baseline, @@ -43,13 +47,15 @@ pub enum Profile { High, /// H.265 Main: chooses the HEVC codec (like `VtProfile::HevcMain` for VideoToolbox). HevcMain, + /// H.265 Main 10: 10-bit 4:2:0 (P010 input), for HDR (PQ / HLG) and 10-bit SDR. + HevcMain10, } impl Profile { /// The codec this profile belongs to. pub fn codec(self) -> Codec { match self { - Profile::HevcMain => Codec::Hevc, + Profile::HevcMain | Profile::HevcMain10 => Codec::Hevc, _ => Codec::H264, } } @@ -89,7 +95,7 @@ pub struct Packet { /// Pictures in flight: input / output buffer pairs. const RING: usize = 8; -/// An NVENC encoder (H.264, or HEVC when the profile is [`Profile::HevcMain`]). +/// An NVENC encoder (H.264, or HEVC when the profile is [`Profile::HevcMain`] or [`Profile::HevcMain10`]). pub struct Nvenc { session: Session, codec: Codec, @@ -107,6 +113,8 @@ pub struct Nvenc { ready: usize, emitted: i64, size: (u32, u32), + /// Main 10: pictures go in as 10-bit P010 ([`Nvenc::encode_10`]), otherwise as 8-bit NV12 ([`Nvenc::encode`]). + ten_bit: bool, } /// The H.264 encoder (the name it had before HEVC joined it). @@ -156,6 +164,9 @@ pub fn warm_hevc_probe() { if std::panic::catch_unwind(hevc_available).is_err() { log::warn!("the hardware HEVC probe panicked; the format list will ask again"); } + if std::panic::catch_unwind(hevc_hdr_available).is_err() { + log::warn!("the hardware HEVC Main 10 probe panicked; the export will ask again"); + } }); if let Err(e) = spawned { log::info!("hardware HEVC probe thread not started: {e}"); @@ -163,14 +174,58 @@ pub fn warm_hevc_probe() { }); } +/// Whether this system's NVENC can encode HEVC Main 10 (10-bit): HEVC works and a Main 10 session +/// opened for a small picture, once (the answer is kept). What makes HDR H.265 exports possible: +/// `filmcraft_export::hdr_available(Format::Hevc)`. +pub fn hevc_hdr_available() -> bool { + *HEVC_HDR_AVAILABLE.get_or_init(|| { + if !hevc_available() { + return false; + } + let probe = Config { + width: 640, + height: 360, + fps: (30, 1), + bitrate_kbps: 2000, + max_bitrate_kbps: 3000, + cbr: false, + keyint: 30, + profile: Profile::HevcMain10, + level: None, + sar: None, + bframes: false, + }; + let result = Nvenc::with_signal(&probe, &Signal::default()); + if let Err(why) = &result { + log::info!("no hardware HEVC Main 10 encoder: {why}"); + } + result.is_ok() + }) +} + +/// The answer of [`hevc_hdr_available`], once asked. +static HEVC_HDR_AVAILABLE: std::sync::OnceLock = std::sync::OnceLock::new(); + +/// Whether [`hevc_hdr_available`] has its answer yet (diagnostics and tests). +pub fn hevc_hdr_probed() -> bool { + HEVC_HDR_AVAILABLE.get().is_some() +} + /// Whether [`hevc_available`] has its answer yet (diagnostics and tests). pub fn hevc_probed() -> bool { HEVC_AVAILABLE.get().is_some() } impl Nvenc { - /// Open an encoder, or say why NVENC does not take this configuration. + /// Open an encoder with the default BT.709 signal, or say why NVENC does not take this configuration. pub fn new(cfg: &Config) -> Result { + Self::with_signal(cfg, &Signal::default()) + } + + /// Open an encoder whose HEVC stream carries `signal` (colour description in the VUI, SEI messages + /// on every IDR picture), or say why NVENC does not take this configuration. H.264 takes the + /// default signal only. + pub fn with_signal(cfg: &Config, signal: &Signal) -> Result { let (w, h) = (cfg.width, cfg.height); if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 { return Err(format!("{w}x{h}: NVENC needs even dimensions")); @@ -179,6 +234,9 @@ impl Nvenc { return Err("frame rate".into()); } let codec = cfg.profile.codec(); + if codec == Codec::H264 && *signal != Signal::default() { + return Err("H.264 here is BT.709 without SEI messages".into()); + } let level = match (codec, cfg.level) { (Codec::Hevc, Some(l)) => Some(hevc::level_code(l).ok_or_else(|| format!("{}.{} is not an HEVC level", l / 10, l % 10))?), // left to itself NVENC answers a bitrate above its chosen level's Main tier limit with the @@ -191,6 +249,7 @@ impl Nvenc { }; let mut session = Session::open()?; let caps = session.caps(codec)?; + check_caps(&caps, cfg)?; if w < caps.min_size.0 || h < caps.min_size.1 || w > caps.max_size.0 || h > caps.max_size.1 { return Err(format!("{w}x{h} is outside NVENC's {}x{} - {}x{}", caps.min_size.0, caps.min_size.1, caps.max_size.0, caps.max_size.1)); } @@ -198,6 +257,7 @@ impl Nvenc { // pictures is simply written without B-frames (H.264 keeps declining those, to the software encoder) let room_for_bframes = codec == Codec::H264 || cfg.keyint > 2; let bframes = u32::from(cfg.bframes && cfg.profile != Profile::Baseline && caps.max_bframes >= 1 && room_for_bframes); + let ten_bit = cfg.profile == Profile::HevcMain10; let params = Params { codec, width: w, @@ -209,9 +269,11 @@ impl Nvenc { gop: cfg.keyint.max(1), profile: match cfg.profile { Profile::Baseline => 0, - Profile::Main | Profile::HevcMain => 1, + Profile::Main | Profile::HevcMain | Profile::HevcMain10 => 1, Profile::High => 2, }, + ten_bit, + signal: signal.clone(), level, sar: cfg.sar, bframes, @@ -226,7 +288,7 @@ impl Nvenc { Codec::Hevc => { let (vps, sps, pps) = split_hevc_parameter_sets(¶ms)?; // validated now: a stream whose parameter sets we cannot describe is a declined export - let record = hevc::hevc_config(&vps, &sps, &pps, (w, h))?; + let record = hevc::hevc_config(&vps, &sps, &pps, (w, h), if ten_bit { 10 } else { 8 })?; (vps, sps, pps, Some(record)) } }; @@ -244,6 +306,7 @@ impl Nvenc { ready: 0, emitted: 0, size: (w, h), + ten_bit, }) } @@ -272,13 +335,39 @@ impl Nvenc { self.delay } - /// Encode picture `index` from planar 4:2:0 (`u`, `v` at half size, rows `w` and `w / 2` bytes). - /// Returns the pictures that came out (usually the oldest; none while the ring fills). + /// Whether this encoder takes 10-bit pictures ([`Nvenc::encode_10`]) instead of 8-bit ones ([`Nvenc::encode`]). + pub fn is_ten_bit(&self) -> bool { + self.ten_bit + } + + /// Encode picture `index` from planar 8-bit 4:2:0 (`u`, `v` at half size, rows `w` and `w / 2` bytes). + /// Returns the pictures that came out (usually the oldest; none while the ring fills). An error + /// for a Main 10 encoder. pub fn encode(&mut self, y: &[u8], u: &[u8], v: &[u8], index: u64) -> Result, String> { + if self.ten_bit { + return Err("an 8-bit picture for a Main 10 encoder".into()); + } let (w, h) = (self.size.0 as usize, self.size.1 as usize); if y.len() < w * h || u.len() < w / 2 * (h / 2) || v.len() < w / 2 * (h / 2) { return Err("the picture is smaller than the encoder's size".into()); } + self.submit_picture(index, |l| fill_nv12(l, y, u, v, w, h)) + } + + /// Encode picture `index` from planar 10-bit 4:2:0 (code values 0..=1023 in `u16`, `u`, `v` at + /// half size). The pictures that came out, as for [`Nvenc::encode`]. An error for a Main encoder. + pub fn encode_10(&mut self, y: &[u16], u: &[u16], v: &[u16], index: u64) -> Result, String> { + if !self.ten_bit { + return Err("a 10-bit picture for a Main (8-bit) encoder".into()); + } + let (w, h) = (self.size.0 as usize, self.size.1 as usize); + if y.len() < w * h || u.len() < w / 2 * (h / 2) || v.len() < w / 2 * (h / 2) { + return Err("the picture is smaller than the encoder's size".into()); + } + self.submit_picture(index, |l| fill_p010(l, y, u, v, w, h)) + } + + fn submit_picture(&mut self, index: u64, fill: impl FnOnce(Locked<'_>)) -> Result, String> { let mut out = Vec::new(); let slot = match self.free.pop() { Some(s) => s, @@ -291,7 +380,7 @@ impl Nvenc { self.free.pop().ok_or("no free encoder buffer")? } }; - let st = self.session.submit(slot, index, |l| fill_nv12(l, y, u, v, w, h)); + let st = self.session.submit(slot, index, fill); let st = match st { Ok(s) => s, Err(e) => { @@ -359,6 +448,44 @@ fn fill_nv12(l: Locked<'_>, y: &[u8], u: &[u8], v: &[u8], w: usize, h: usize) { } } +/// Copy planar 10-bit 4:2:0 into a P010 input buffer: each sample is a little-endian `u16` with the +/// 10-bit code in its high bits (`code << 6`), the chroma plane interleaved (Cb, Cr) after the luma rows, +/// `pitch` in bytes. Codes above 10 bits are masked; a row the pitch cannot hold is skipped, never overrun. +fn fill_p010(l: Locked<'_>, y: &[u16], u: &[u16], v: &[u16], w: usize, h: usize) { + let pitch = l.pitch; + let (luma, chroma) = l.data.split_at_mut(pitch.saturating_mul(h).min(l.data.len())); + if w == 0 || pitch == 0 { + return; + } + let put = |dst: &mut [u8], code: u16| dst.copy_from_slice(&((code & 0x3ff) << 6).to_le_bytes()); + for (row, dst) in y.chunks_exact(w).zip(luma.chunks_exact_mut(pitch)).take(h) { + let Some(dst) = dst.get_mut(..w.saturating_mul(2)) else { continue }; + for (d, c) in dst.chunks_exact_mut(2).zip(row) { + put(d, *c); + } + } + let cw = w / 2; + if cw == 0 { + return; + } + for ((ur, vr), dst) in u.chunks_exact(cw).zip(v.chunks_exact(cw)).zip(chroma.chunks_exact_mut(pitch)).take(h / 2) { + let Some(dst) = dst.get_mut(..w.saturating_mul(2)) else { continue }; + for ((d, a), b) in dst.chunks_exact_mut(4).zip(ur).zip(vr) { + let (du, dv) = d.split_at_mut(2); + put(du, *a); + put(dv, *b); + } + } +} + +/// Why this GPU's encoder cannot take `cfg` (beyond its size limits), or `Ok`: Main 10 needs 10-bit support. +fn check_caps(caps: &Caps, cfg: &Config) -> Result<(), String> { + if cfg.profile == Profile::HevcMain10 && !caps.ten_bit { + return Err("this GPU's encoder has no 10-bit (HEVC Main 10) support".into()); + } + Ok(()) +} + /// The NAL units of an Annex B byte stream (start codes `00 00 01` / `00 00 00 01`). pub fn annex_b_nals(data: &[u8]) -> Vec<&[u8]> { let mut starts = Vec::new(); @@ -497,6 +624,69 @@ mod tests { fill_nv12(Locked { data: &mut buf, pitch }, &[1], &[], &[], 1, 1); } + #[test] + fn p010_is_shifted_little_endian_with_interleaved_chroma_in_bytes() { + let (w, h, pitch) = (4usize, 2usize, 12usize); // 2 bytes per sample + 4 bytes of padding + let y: Vec = vec![0, 1, 64, 1023, 940, 512, 4, 1019]; + let (u, v) = (vec![512, 960], vec![64, 1023]); + let mut buf = vec![0xAAu8; pitch * h * 3 / 2]; + fill_p010(Locked { data: &mut buf, pitch }, &y, &u, &v, w, h); + let sample = |at: usize| u16::from_le_bytes([buf[at], buf[at + 1]]); + assert_eq!((0..4).map(|x| sample(x * 2)).collect::>(), vec![0, 1 << 6, 64 << 6, 1023 << 6]); + assert_eq!((0..4).map(|x| sample(pitch + x * 2)).collect::>(), vec![940 << 6, 512 << 6, 4 << 6, 1019 << 6]); + // chroma after pitch * h bytes: Cb Cr Cb Cr + let c = pitch * h; + assert_eq!((0..4).map(|x| sample(c + x * 2)).collect::>(), vec![512 << 6, 64 << 6, 960 << 6, 1023 << 6]); + // the low 6 bits of every sample are zero and the padding is untouched + assert!((0..4).all(|x| sample(x * 2) & 0x3f == 0)); + assert_eq!(&buf[8..12], &[0xAA; 4]); + // a code beyond 10 bits is masked, not shifted into the next field + let mut b2 = vec![0u8; pitch * h * 3 / 2]; + fill_p010(Locked { data: &mut b2, pitch }, &[0xFFFF; 8], &[0xFFFF; 2], &[0xFFFF; 2], w, h); + assert_eq!(u16::from_le_bytes([b2[0], b2[1]]), 1023 << 6); + } + + #[test] + fn p010_never_writes_past_a_narrow_pitch_or_short_buffer() { + // a pitch narrower than 2 bytes per pixel (rejected by `submit`) must not panic here either + let (w, h) = (8usize, 4usize); + for pitch in [0usize, 1, 7, 8, 15, 16] { + let mut buf = vec![0u8; pitch * h * 3 / 2]; + let y = vec![1000u16; w * h]; + let (u, v) = (vec![500u16; w * h / 4], vec![600u16; w * h / 4]); + fill_p010(Locked { data: &mut buf, pitch }, &y, &u, &v, w, h); + } + // empty and short inputs, a buffer shorter than the pitch says + fill_p010(Locked { data: &mut [], pitch: 0 }, &[], &[], &[], 0, 0); + fill_p010(Locked { data: &mut [0u8; 5], pitch: 64 }, &[1; 64], &[1; 16], &[1; 16], 8, 8); + let mut buf = vec![0u8; 64]; + fill_p010(Locked { data: &mut buf, pitch: 16 }, &[1], &[], &[], 1, 1); + } + + #[test] + fn main_10_needs_a_gpu_with_10_bit_support() { + let cfg = |profile| Config { + width: 640, + height: 360, + fps: (24, 1), + bitrate_kbps: 1000, + max_bitrate_kbps: 1500, + cbr: false, + keyint: 24, + profile, + level: None, + sar: None, + bframes: false, + }; + let caps = |ten_bit| Caps { ten_bit, max_bframes: 2, min_size: (130, 128), max_size: (8192, 8192) }; + assert!(check_caps(&caps(true), &cfg(Profile::HevcMain10)).is_ok()); + let e = check_caps(&caps(false), &cfg(Profile::HevcMain10)).unwrap_err(); + assert!(e.contains("10-bit"), "{e}"); + for p in [Profile::HevcMain, Profile::Main, Profile::High, Profile::Baseline] { + assert!(check_caps(&caps(false), &cfg(p)).is_ok(), "{p:?} does not need 10-bit"); + } + } + #[test] fn empty_nal_units_are_skipped() { assert!(annex_b_nals(&[0, 0, 1, 0, 0, 1]).is_empty()); @@ -584,6 +774,7 @@ mod tests { #[test] fn the_hevc_profile_picks_the_hevc_codec() { assert_eq!(Profile::HevcMain.codec(), Codec::Hevc); + assert_eq!(Profile::HevcMain10.codec(), Codec::Hevc); for p in [Profile::Baseline, Profile::Main, Profile::High] { assert_eq!(p.codec(), Codec::H264); } diff --git a/crates/platform/src/nvenc/session.rs b/crates/platform/src/nvenc/session.rs index 249dd4fb..50ae1656 100644 --- a/crates/platform/src/nvenc/session.rs +++ b/crates/platform/src/nvenc/session.rs @@ -115,9 +115,34 @@ impl Codec { } } +/// One SEI message the encoder writes in front of every IDR picture (HEVC): its payload type and bytes. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Sei { + pub payload_type: u32, + pub payload: Vec, +} + +/// The colour description of a stream (ITU-T H.273 code points, limited range) and the SEI messages +/// that go with it. The default is BT.709 without SEI. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Signal { + pub primaries: u8, + pub transfer: u8, + pub matrix: u8, + pub sei: Vec, +} + +impl Default for Signal { + fn default() -> Self { + Signal { primaries: 1, transfer: 1, matrix: 1, sei: Vec::new() } + } +} + /// What the encoder says it can do for a codec. #[derive(Clone, Copy, Debug)] pub struct Caps { + /// 10-bit encoding (Main 10) is supported. + pub ten_bit: bool, pub max_bframes: u32, pub min_size: (u32, u32), pub max_size: (u32, u32), @@ -134,8 +159,12 @@ pub struct Params { pub max_bitrate: u32, pub cbr: bool, pub gop: u32, - /// H.264 only: 0 baseline, 1 main, 2 high (HEVC is always Main). + /// H.264 only: 0 baseline, 1 main, 2 high (HEVC is Main, or Main 10 with `ten_bit`). pub profile: u8, + /// HEVC Main 10: 10-bit 4:2:0 pictures (P010), otherwise 8-bit (NV12). + pub ten_bit: bool, + /// HEVC: the VUI colour description and the SEI messages of every IDR picture (H.264 keeps BT.709). + pub signal: Signal, /// The codec's own level code: `level × 10` for H.264, `level × 30` for HEVC. pub level: Option, pub sar: Option<(u32, u32)>, @@ -143,7 +172,7 @@ pub struct Params { pub bframes: u32, } -/// An input buffer locked for writing: NV12 rows go to `data` at `pitch` bytes per row, the +/// An input buffer locked for writing: NV12 (or P010) rows go to `data` at `pitch` bytes per row, the /// chroma plane after `pitch * height` bytes. pub struct Locked<'a> { pub data: &'a mut [u8], @@ -182,6 +211,17 @@ pub struct Session { slots: Vec, size: (u32, u32), out_size: u32, + /// `NV_ENC_BUFFER_FORMAT_*` of the input buffers and pictures, and its bytes per sample. + fmt: u32, + bytes_per_sample: usize, + /// The SEI messages of every IDR picture. The payload bytes and the array of descriptors live on + /// the heap, owned by the session for its whole life, so the pointers handed to the driver stay + /// valid however the `Session` value is moved and however long the driver keeps a picture queued. + sei_payloads: Vec>, + sei_array: Box<[NV_ENC_SEI_PAYLOAD]>, + /// Pictures submitted, and the distance between IDR pictures (to know which pictures get the SEI). + submitted: u64, + gop: u64, } // SAFETY: an encoder session and its buffers are driver objects used from one thread at a time (the @@ -237,7 +277,20 @@ impl Session { } enc }; - Ok(Session { api, enc, _device: device, slots: Vec::new(), size: (0, 0), out_size: 0 }) + Ok(Session { + api, + enc, + _device: device, + slots: Vec::new(), + size: (0, 0), + out_size: 0, + fmt: NV_ENC_BUFFER_FORMAT_NV12, + bytes_per_sample: 1, + sei_payloads: Vec::new(), + sei_array: Box::new([]), + submitted: 0, + gop: 1, + }) } /// The encoder's limits for `codec` (an error when the GPU cannot encode it). @@ -258,6 +311,8 @@ impl Session { } }; Ok(Caps { + // a failing query means no 10-bit encoder + ten_bit: get(NV_ENC_CAPS_SUPPORT_10BIT_ENCODE).unwrap_or(0) != 0, max_bframes: get(NV_ENC_CAPS_NUM_MAX_BFRAMES)?, min_size: (get(NV_ENC_CAPS_WIDTH_MIN)?, get(NV_ENC_CAPS_HEIGHT_MIN)?), max_size: (get(NV_ENC_CAPS_WIDTH_MAX)?, get(NV_ENC_CAPS_HEIGHT_MAX)?), @@ -267,6 +322,24 @@ impl Session { /// Configure the encoder (preset P5, high quality tuning, the settings' rate control) and /// create `ring` pairs of input / output buffers. pub fn initialize(&mut self, p: &Params, ring: usize) -> Result<(), String> { + if p.ten_bit && p.codec != Codec::Hevc { + return Err("10-bit encoding is HEVC Main 10 here".into()); + } + if !p.signal.sei.is_empty() && p.codec != Codec::Hevc { + return Err("SEI messages are written for HEVC only".into()); + } + self.fmt = if p.ten_bit { NV_ENC_BUFFER_FORMAT_YUV420_10BIT } else { NV_ENC_BUFFER_FORMAT_NV12 }; + self.bytes_per_sample = if p.ten_bit { 2 } else { 1 }; + self.gop = u64::from(p.gop.max(1)); + self.submitted = 0; + // the SEI payloads: heap blocks that outlive every picture (see the field documentation) + self.sei_payloads = p.signal.sei.iter().map(|m| m.payload.clone().into_boxed_slice()).collect(); + let mut array = Vec::with_capacity(self.sei_payloads.len()); + for (m, bytes) in p.signal.sei.iter().zip(self.sei_payloads.iter_mut()) { + let size = u32::try_from(bytes.len()).map_err(|_| "an SEI payload longer than 4 GiB".to_string())?; + array.push(NV_ENC_SEI_PAYLOAD { payloadSize: size, payloadType: m.payload_type, payload: bytes.as_mut_ptr() }); + } + self.sei_array = array.into_boxed_slice(); let preset_fn = self.api.list.nvEncGetEncodePresetConfigEx.ok_or("no nvEncGetEncodePresetConfigEx")?; let init_fn = self.api.list.nvEncInitializeEncoder.ok_or("no nvEncInitializeEncoder")?; // SAFETY: zeroed parameter blocks with their versions; the preset call fills `preset`; the @@ -282,6 +355,7 @@ impl Session { let mut cfg: Box = Box::new(preset.presetCfg); cfg.version = NV_ENC_CONFIG_VER; cfg.profileGUID = match (p.codec, p.profile) { + (Codec::Hevc, _) if p.ten_bit => NV_ENC_HEVC_PROFILE_MAIN10_GUID, (Codec::Hevc, _) => NV_ENC_HEVC_PROFILE_MAIN_GUID, (Codec::H264, 0) => NV_ENC_H264_PROFILE_BASELINE_GUID, (Codec::H264, 1) => NV_ENC_H264_PROFILE_MAIN_GUID, @@ -325,7 +399,7 @@ impl Session { h.minCUSize = NV_ENC_HEVC_CUSIZE_AUTOSELECT; h.maxCUSize = NV_ENC_HEVC_CUSIZE_AUTOSELECT; h.idrPeriod = p.gop.max(1); - // parameter sets only in the first IDR (and the hvcC), no access unit delimiters, 8-bit 4:2:0 + // parameter sets only in the first IDR (and the hvcC), no access unit delimiters, 4:2:0 at 8 or 10 bits h.flags &= !(HEVC_OUTPUT_AUD | HEVC_DISABLE_SPSPPS | HEVC_REPEAT_SPSPPS @@ -333,17 +407,20 @@ impl Session { | HEVC_CHROMA_FORMAT_MASK | HEVC_PIXEL_BIT_DEPTH_MASK); h.flags |= 1 << HEVC_CHROMA_FORMAT_SHIFT; + if p.ten_bit { + h.flags |= 2 << HEVC_PIXEL_BIT_DEPTH_SHIFT; + } h.maxNumRefFramesInDPB = 0; - // BT.709 limited range, like the H.264 path; an HEVC VUI counts frames, not fields: - // frame rate = time_scale / num_units_in_tick + // the stream's colour description (BT.709 unless the signal says otherwise), limited + // range; an HEVC VUI counts frames, not fields: frame rate = time_scale / num_units_in_tick let vui = &mut h.hevcVUIParameters; vui.videoSignalTypePresentFlag = 1; vui.videoFormat = NV_ENC_VUI_VIDEO_FORMAT_UNSPECIFIED; vui.videoFullRangeFlag = 0; vui.colourDescriptionPresentFlag = 1; - vui.colourPrimaries = NV_ENC_VUI_COLOR_PRIMARIES_BT709; - vui.transferCharacteristics = NV_ENC_VUI_TRANSFER_CHARACTERISTIC_BT709; - vui.colourMatrix = NV_ENC_VUI_MATRIX_COEFFS_BT709; + vui.colourPrimaries = u32::from(p.signal.primaries); + vui.transferCharacteristics = u32::from(p.signal.transfer); + vui.colourMatrix = u32::from(p.signal.matrix); vui.timingInfoPresentFlag = 1; vui.numUnitInTicks = p.fps.1; vui.timeScale = p.fps.0; @@ -364,7 +441,7 @@ impl Session { init.enablePTD = 1; init.encodeConfig = &mut *cfg; init.tuningInfo = NV_ENC_TUNING_INFO_HIGH_QUALITY; - init.bufferFormat = NV_ENC_BUFFER_FORMAT_NV12; + init.bufferFormat = self.fmt; let s = init_fn(self.enc, &mut *init); if s != NV_ENC_SUCCESS { return Err(status("initialising the encoder", s, self.enc, self.api)); @@ -391,7 +468,7 @@ impl Session { i.width = self.size.0; i.height = self.size.1; i.memoryHeap = NV_ENC_MEMORY_HEAP_SYSMEM_CACHED; - i.bufferFmt = NV_ENC_BUFFER_FORMAT_NV12; + i.bufferFmt = self.fmt; let s = create_in(self.enc, &mut i); if s != NV_ENC_SUCCESS { return Err(status("creating an input buffer", s, self.enc, self.api)); @@ -451,8 +528,8 @@ impl Session { ); let (w, h) = self.size; // SAFETY: the buffer handle is one this session created. `LockInputBuffer` returns a - // writable block of `pitch * h * 3 / 2` bytes (NV12: luma rows then chroma rows) that stays - // valid until `UnlockInputBuffer`; the slice borrows it only inside `fill`. + // writable block of `pitch * h * 3 / 2` bytes (NV12 or P010: luma rows then chroma rows, the pitch + // in bytes) that stays valid until `UnlockInputBuffer`; the slice borrows it only inside `fill`. let pitch = unsafe { let mut l: NV_ENC_LOCK_INPUT_BUFFER = std::mem::zeroed(); l.version = NV_ENC_LOCK_INPUT_BUFFER_VER; @@ -461,11 +538,11 @@ impl Session { if st != NV_ENC_SUCCESS || l.bufferDataPtr.is_null() || l.pitch == 0 { return Err(status("locking an input buffer", st, self.enc, self.api)); } - // a driver pitch narrower than a row would make `fill` write past each row - if (l.pitch as usize) < w as usize { + // a driver pitch narrower than a row (in bytes: P010 samples are two) would make `fill` write past each row + if (l.pitch as usize) < (w as usize).saturating_mul(self.bytes_per_sample) { // the lock succeeded, so release it before giving up (its status no longer matters) let _ = unlock(self.enc, input); - return Err(format!("the driver returned an input pitch of {} bytes for {w}-pixel rows", l.pitch)); + return Err(format!("the driver returned an input pitch of {} bytes for {w}-pixel rows of {}-byte samples", l.pitch, self.bytes_per_sample)); } let len = (l.pitch as usize).saturating_mul(h as usize).saturating_mul(3) / 2; fill(Locked { data: std::slice::from_raw_parts_mut(l.bufferDataPtr.cast::(), len), pitch: l.pitch as usize }); @@ -475,16 +552,27 @@ impl Session { } l.pitch }; - // SAFETY: a zeroed parameter block with its version; the buffers are this session's. + // IDR pictures (every `gop` pictures from the first, in submission = display order) carry the SEI messages + let idr = !self.sei_array.is_empty() && self.submitted.is_multiple_of(self.gop); + self.submitted = self.submitted.saturating_add(1); + // SAFETY: a zeroed parameter block with its version; the buffers are this session's. The SEI + // descriptor array and the payload bytes it points to are heap blocks owned by `self`, which + // stay allocated (and unmoved) until the session is dropped, so they outlive this call and + // whatever time the driver keeps the picture queued. let st = unsafe { let mut e: NV_ENC_PIC_PARAMS = std::mem::zeroed(); + if idr { + let h = &mut e.codecPicParams.hevcPicParams; + h.seiPayloadArrayCnt = u32::try_from(self.sei_array.len()).unwrap_or(0); + h.seiPayloadArray = self.sei_array.as_mut_ptr(); + } e.version = NV_ENC_PIC_PARAMS_VER; e.inputWidth = w; e.inputHeight = h; e.inputPitch = pitch; e.inputBuffer = input; e.outputBitstream = output; - e.bufferFmt = NV_ENC_BUFFER_FORMAT_NV12; + e.bufferFmt = self.fmt; e.pictureStruct = NV_ENC_PIC_STRUCT_FRAME; e.inputTimeStamp = pts; e.inputDuration = 1; diff --git a/crates/platform/tests/nvenc_hevc_export.rs b/crates/platform/tests/nvenc_hevc_export.rs index 1f0922ee..95f074bb 100644 --- a/crates/platform/tests/nvenc_hevc_export.rs +++ b/crates/platform/tests/nvenc_hevc_export.rs @@ -237,8 +237,10 @@ fn what_nvenc_cannot_do_is_an_error_that_says_why() { // (what, settings, size): each declined by the configuration alone, so even a machine without // NVENC counts and explains it let cases: Vec<(&str, ExportSettings, (u32, u32))> = vec![ - ("HDR (PQ)", ExportSettings { signal: ColorSignal::PQ, ..base.clone() }, (1280, 720)), - ("HDR (HLG)", ExportSettings { signal: ColorSignal::HLG, ..base.clone() }, (1280, 720)), + // HDR is Main 10 since the HDR round (see nvenc_hevc_hdr_export.rs); what stays declined is a colour description + // this backend does not write + ("an unwritable colour signal", ExportSettings { signal: ColorSignal { primaries: 9, transfer: 1, matrix: 1 }, ..base.clone() }, (1280, 720)), + ("HDR with BT.709 primaries", ExportSettings { signal: ColorSignal { primaries: 1, transfer: 16, matrix: 9 }, ..base.clone() }, (1280, 720)), ("an analysis pass", ExportSettings { h264_pass: H264Pass::First, ..base.clone() }, (1280, 720)), ("two-pass VBR", ExportSettings { bitrate_mode: BitrateMode::Vbr2Pass, ..base.clone() }, (1280, 720)), ("non-square pixels", ExportSettings { pixel_aspect: Some((4, 3)), ..base.clone() }, (1280, 720)), @@ -254,6 +256,19 @@ fn what_nvenc_cannot_do_is_an_error_that_says_why() { assert!(e.to_string().contains("NVENC"), "{what}: {e}"); assert_eq!(delta(before), (0, 0, 1), "{what}: counted as declined, no session"); } + // HDR (PQ / HLG) is taken as Main 10 where the GPU has 10-bit HEVC, declined (counted, with the reason) where it has not + for signal in [ColorSignal::PQ, ColorSignal::HLG] { + let before = hw_encode_stats(); + let r = factory(Format::Hevc, 640, 360, rate, &ExportSettings { signal, ..base.clone() }); + if filmcraft_platform::nvenc::hevc_hdr_available() { + assert!(matches!(r, Some(Ok(_))), "{signal:?}"); + assert_eq!(delta(before), (0, 1, 0), "{signal:?}"); + } else if hevc_available() { + let Some(Err(e)) = r else { panic!("{signal:?}: expected a declined request") }; + assert!(e.to_string().contains("10-bit"), "{signal:?}: {e}"); + assert_eq!(delta(before), (0, 0, 1), "{signal:?}"); + } + } // interlaced output is refused by the export's own validation, before any encoder is asked let (p, seq, m) = project(640, 360, 6); let before = hw_encode_stats(); diff --git a/crates/platform/tests/nvenc_hevc_hdr_export.rs b/crates/platform/tests/nvenc_hevc_hdr_export.rs new file mode 100644 index 00000000..f30b57fc --- /dev/null +++ b/crates/platform/tests/nvenc_hevc_hdr_export.rs @@ -0,0 +1,362 @@ +//! HDR export as H.265 Main 10 (NVENC) through the real export pipeline: an HDR (Rec. 2100 PQ / HLG) +//! sequence gives MP4 / QuickTime files that ffprobe (an external oracle only) describes as `hvc1` +//! Main 10, yuv420p10le, BT.2020 with the right transfer, limited range, with HDR10 static metadata for +//! PQ; that ffmpeg decodes without errors to the same 10-bit pictures our own decoder makes; that +//! our importer reads back as PQ / HLG, 10-bit; whose pixels follow a ProRes HDR export of the same +//! sequence. SDR stays Main 8-bit BT.709, `settings.sdr` forces it. Its own test binary: the encoder +//! registry and the counters are process-wide. Skips (SKIPPED) only without a 10-bit HEVC encoder. +#![cfg(target_os = "windows")] + +use std::path::Path; +use std::process::Command; +use std::sync::Arc; + +use filmcraft_color::{ColorPipeline, WorkingSpace}; +use filmcraft_export::{BitrateMode, ColorSignal, EncoderFrame, ExportSettings, Format, Progress, export, hw_encode_stats}; +use filmcraft_media::generators::GeneratorSource; +use filmcraft_media::{DemoScene, FrameRequest, Generator, MediaSource}; +use filmcraft_platform::nvenc::export::factory; +use filmcraft_platform::nvenc::{hevc_available, hevc_hdr_available}; +use filmcraft_project::{ItemKind, Label, MediaClip, MediaRef, Project, SequenceSettings, TrackKind}; +use filmcraft_render::SourceMap; +use filmcraft_time::{FrameRate, Tick, TimeRange}; + +/// The encoder counters are process-wide: one test at a time. +static SERIAL: std::sync::Mutex<()> = std::sync::Mutex::new(()); + +const W: u32 = 1280; +const H: u32 = 720; +const FRAMES: i64 = 24; + +fn project(working: WorkingSpace) -> (Arc, filmcraft_project::ItemId, SourceMap) { + let rate = FrameRate::FPS_24; + let mut p = Project::new("x"); + let g = GeneratorSource::new(Generator::Demo(DemoScene::OceanSunset), W, H, rate, rate.tick_of(FRAMES)); + let info = g.info().clone(); + let clip = p.add_item( + &info.name.clone(), + Label::Iris, + ItemKind::Media(MediaClip { + media: MediaRef::Generator(g.generator.clone()), + info, + interpret: Default::default(), + mark_in: None, + mark_out: None, + markers: vec![], + offline: false, + proxy: None, + identity: None, + }), + None, + ); + let color = ColorPipeline { working, ..ColorPipeline::REC709 }; + let seq = p.new_sequence("s", SequenceSettings { width: W, height: H, frame_rate: rate, color, ..Default::default() }, 1, 1, None); + let v = p.make_track_item(clip, TrackKind::Video, Tick::ZERO, TimeRange::new(Tick::ZERO, rate.tick_of(FRAMES)), rate).unwrap(); + p.sequence_mut(seq).unwrap().video_tracks[0].items.push(v); + let mut m = SourceMap::default(); + m.0.insert(clip, Arc::new(g)); + (Arc::new(p), seq, m) +} + +fn tmp(name: &str) -> String { + let d = std::env::temp_dir().join(format!("fc-nvenc-hdr-{}", std::process::id())); + std::fs::create_dir_all(&d).unwrap(); + d.join(name).to_string_lossy().to_string() +} + +fn decode(path: &str) -> Vec> { + let bytes: Arc<[u8]> = std::fs::read(path).unwrap().into(); + let src = filmcraft_codecs::open_bytes(path, bytes).unwrap(); + let rate = src.info().frame_rate(); + let n = rate.frame_at(src.info().duration); + (0..n).map(|i| src.video_frame(FrameRequest::full(rate.tick_of(i))).unwrap()).collect() +} + +fn hevc(path: String) -> ExportSettings { + ExportSettings { format: Format::Hevc, path, include_audio: false, bitrate_kbps: 10_000, bitrate_mode: BitrateMode::Vbr1Pass, ..Default::default() } +} + +fn probe(ffprobe: &Path, path: &str) -> std::collections::HashMap { + let entries = "stream=codec_name,profile,codec_tag_string,pix_fmt,color_space,color_transfer,color_primaries,color_range,width,height,nb_frames,level"; + let out = Command::new(ffprobe).args(["-v", "error", "-select_streams", "v:0", "-show_entries", entries, "-of", "default=nw=1", path]).output().unwrap(); + assert!(out.status.success(), "ffprobe: {}", String::from_utf8_lossy(&out.stderr)); + String::from_utf8_lossy(&out.stdout).lines().filter_map(|l| l.split_once('=')).map(|(k, v)| (k.to_string(), v.to_string())).collect() +} + +/// The side data ffprobe reports for the first frame. +fn first_frame_side_data(ffprobe: &Path, path: &str) -> String { + let out = Command::new(ffprobe) + .args([ + "-v", + "error", + "-select_streams", + "v:0", + "-read_intervals", + "%+#1", + "-show_frames", + "-show_entries", + "frame=side_data_list", + "-of", + "default=nw=1", + path, + ]) + .output() + .unwrap(); + String::from_utf8_lossy(&out.stdout).to_string() +} + +fn assert_ffmpeg_decodes(ffmpeg: &Path, path: &str) { + let out = Command::new(ffmpeg).args(["-v", "error", "-xerror", "-i", path, "-f", "null", "-"]).output().unwrap(); + assert!(out.status.success() && out.stderr.is_empty(), "ffmpeg: {}", String::from_utf8_lossy(&out.stderr)); +} + +/// The first picture as ffmpeg decodes it to yuv420p10le: (Y, Cb, Cr) code values. +fn ffmpeg_first_picture(ffmpeg: &Path, path: &str, w: usize, h: usize) -> (Vec, Vec, Vec) { + let out = Command::new(ffmpeg).args(["-v", "error", "-i", path, "-frames:v", "1", "-f", "rawvideo", "-pix_fmt", "yuv420p10le", "-"]).output().unwrap(); + assert!(out.status.success(), "ffmpeg: {}", String::from_utf8_lossy(&out.stderr)); + let samples: Vec = out.stdout.chunks_exact(2).map(|b| u16::from_le_bytes([b[0], b[1]])).collect(); + assert_eq!(samples.len(), w * h * 3 / 2, "one yuv420p10le picture"); + let (y, c) = samples.split_at(w * h); + let (u, v) = c.split_at(c.len() / 2); + (y.to_vec(), u.to_vec(), v.to_vec()) +} + +fn frame<'a>(rgba: &'a [u8], hdr: Option<&'a [f32]>, index: u64) -> EncoderFrame<'a> { + EncoderFrame { width: 640, height: 360, rgba, hdr, index } +} + +fn psnr10(a: &[u16], b: &[u16]) -> f64 { + let mse = a.iter().zip(b).map(|(x, y)| (f64::from(*x) - f64::from(*y)).powi(2)).sum::() / a.len() as f64; + if mse == 0.0 { 99.0 } else { 10.0 * (1023.0f64 * 1023.0 / mse).log10() } +} + +fn delta(before: filmcraft_export::HwEncodeStats) -> (u64, u64, u64) { + let after = hw_encode_stats(); + (after.frames - before.frames, after.sessions - before.sessions, after.declined - before.declined) +} + +fn yuv16(f: &filmcraft_frame::VideoFrame) -> ([Arc>; 3], u32) { + match &f.data { + filmcraft_frame::PixelData::Yuv16 { planes, bits, .. } => (planes.clone(), *bits), + _ => panic!("16-bit planes expected, got {}", f.format_label()), + } +} + +#[test] +fn hdr_sequences_export_as_main_10() { + let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner); + filmcraft_platform::register(); + assert_eq!(filmcraft_export::hdr_available(Format::Hevc), hevc_hdr_available(), "HDR H.265 is available iff NVENC can encode Main 10"); + if !hevc_hdr_available() { + eprintln!("SKIPPED: no NVENC HEVC Main 10 here (hevc: {})", hevc_available()); + return; + } + let ffprobe = filmcraft_testkit::ffprobe_or_skip("nvenc hdr export"); + let ffmpeg = filmcraft_testkit::ffmpeg_or_skip("nvenc hdr export"); + for (ws, ext, ff_transfer, pq) in [ + (WorkingSpace::Rec2100Pq, "mp4", "smpte2084", true), + (WorkingSpace::Rec2100Hlg, "mp4", "arib-std-b67", false), + (WorkingSpace::Rec2100Pq, "mov", "smpte2084", true), + ] { + let name = format!("{}.{ext}", ws.id()); + let (p, seq, m) = project(ws); + + // the ProRes HDR export of the same sequence is the reference for the pixels (10-bit BT.2020, same matrix and range) + let ref_path = tmp(&format!("ref-{}.mov", ws.id())); + let before = hw_encode_stats(); + export( + &p, + seq, + &ExportSettings { format: Format::ProRes, path: ref_path.clone(), include_audio: false, ..Default::default() }, + &m, + &Progress::default(), + ) + .unwrap(); + assert_eq!(delta(before), (0, 0, 0), "ProRes never touches NVENC"); + let reference = decode(&ref_path); + + let path = tmp(&name); + let s = hevc(path.clone()); + let before = hw_encode_stats(); + let report = export(&p, seq, &s, &m, &Progress::default()).unwrap_or_else(|e| panic!("{name}: {e}")); + assert_eq!(report.frames, FRAMES as u64, "{name}"); + assert_eq!(delta(before), (FRAMES as u64, 1, 0), "{name}: every picture through one NVENC session, nothing declined"); + + // our importer: PQ / HLG, BT.2020, 10-bit planes + let bytes: Arc<[u8]> = std::fs::read(&path).unwrap().into(); + let src = filmcraft_codecs::open_bytes(&path, bytes.clone()).unwrap(); + let v = src.info().video.clone().unwrap(); + let (want_transfer, want_hdr) = if pq { (filmcraft_color::Transfer::Pq, true) } else { (filmcraft_color::Transfer::Hlg, false) }; + assert_eq!( + (v.color.transfer, v.color.primaries, v.color.matrix), + (want_transfer, filmcraft_color::Primaries::Bt2020, filmcraft_color::Matrix::Bt2020Ncl), + "{name}" + ); + if want_hdr { + let hdr = v.hdr.unwrap_or_else(|| panic!("{name}: no HDR metadata")); + assert_eq!((hdr.mastering_max_nits, hdr.max_cll, hdr.peak_nits()), (Some(1000.0), None, Some(1000.0)), "{name}"); + } + // and the sample entry boxes + let file = filmcraft_isobmff::open(&bytes[..]).unwrap(); + let vt = file.track_of_kind(filmcraft_isobmff::TrackKind::Video).expect("a video track"); + let ve = file.tracks[vt].entries[0].video.clone().expect("a video sample entry"); + assert!(matches!(ve.color, Some(filmcraft_isobmff::ColorInfo::Nclx { primaries: 9, matrix: 9, full_range: false, .. })), "{name}: {:?}", ve.color); + assert_eq!(ve.mastering_display.is_some() && ve.content_light == Some((0, 0)), pq, "{name}: mdcv / clli"); + + let pictures = decode(&path); + assert_eq!(pictures.len(), FRAMES as usize, "{name}"); + let (mut worst, mut best_levels) = (99.0f64, usize::MAX); + let (mut peak, mut ref_peak) = (0u16, 0u16); + for (a, b) in pictures.iter().zip(&reference) { + let (pa, bits) = yuv16(a); + let (pb, _) = yuv16(b); + assert_eq!(bits, 10, "{name}: decoded bit depth"); + assert!(pa[0].iter().all(|c| *c < 1024)); + worst = worst.min(psnr10(&pa[0], &pb[0])); + peak = peak.max(pa[0].iter().copied().max().unwrap_or(0)); + ref_peak = ref_peak.max(pb[0].iter().copied().max().unwrap_or(0)); + let mut levels: Vec = pa[0].to_vec(); + levels.sort_unstable(); + levels.dedup(); + best_levels = best_levels.min(levels.len()); + } + eprintln!( + "{name}: {} bytes, worst luma PSNR against the ProRes HDR export {worst:.1} dB, {best_levels} distinct luma levels per picture (min), peak luma code {peak} (ProRes {ref_peak})", + report.bytes + ); + assert!(worst > 35.0, "{name}: {worst:.1} dB against the ProRes HDR export"); + assert!(best_levels > 256, "{name}: {best_levels} distinct luma levels: not 10-bit"); + assert!(peak.abs_diff(ref_peak) < 24, "{name}: peak {peak} against {ref_peak}"); + + let (Some(ffprobe), Some(ffmpeg)) = (&ffprobe, &ffmpeg) else { continue }; + let f = probe(ffprobe, &path); + let get = |k: &str| f.get(k).map(String::as_str).unwrap_or("?"); + assert_eq!((get("codec_name"), get("profile"), get("codec_tag_string"), get("pix_fmt")), ("hevc", "Main 10", "hvc1", "yuv420p10le"), "{name}: {f:?}"); + assert_eq!( + (get("color_primaries"), get("color_transfer"), get("color_space"), get("color_range")), + ("bt2020", ff_transfer, "bt2020nc", "tv"), + "{name}: {f:?}" + ); + assert_eq!((get("width"), get("height"), get("nb_frames")), ("1280", "720", "24"), "{name}: {f:?}"); + let side = first_frame_side_data(ffprobe, &path); + if pq { + assert!( + side.contains("Mastering display metadata") && side.contains("max_luminance=10000000/10000") && side.contains("min_luminance=1/10000"), + "{name}: {side}" + ); + assert!(side.contains("Content light level metadata") && side.contains("max_content=0") && side.contains("max_average=0"), "{name}: {side}"); + } else { + assert!(!side.contains("Mastering display") && !side.contains("Content light level"), "{name}: {side}"); + } + assert_ffmpeg_decodes(ffmpeg, &path); + // ffmpeg's yuv420p10le picture is our decoder's picture + let (fy, fu, fv) = ffmpeg_first_picture(ffmpeg, &path, W as usize, H as usize); + let (ours, _) = yuv16(&pictures[0]); + let diff = |a: &[u16], b: &[u16]| a.iter().zip(b).map(|(x, y)| x.abs_diff(*y)).max().unwrap_or(0); + eprintln!( + "{name}: ffprobe {f:?}\n{name}: first-frame side data {:?}\n{name}: ffmpeg vs our decoder, max code difference Y {} Cb {} Cr {}", + side.lines().collect::>(), + diff(&fy, &ours[0]), + diff(&fu, &ours[1]), + diff(&fv, &ours[2]) + ); + assert_eq!( + (diff(&fy, &ours[0]), diff(&fu, &ours[1]), diff(&fv, &ours[2])), + (0, 0, 0), + "{name}: our HEVC decoder and ffmpeg disagree on the 10-bit pictures" + ); + } +} + +#[test] +fn sdr_stays_main_8_bit_and_the_sdr_setting_tone_maps_hdr_sequences() { + let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner); + filmcraft_platform::register(); + if !hevc_hdr_available() { + eprintln!("SKIPPED: no NVENC HEVC Main 10 here"); + return; + } + let ffprobe = filmcraft_testkit::ffprobe_or_skip("nvenc hdr export"); + // an HDR sequence with SDR asked for, and a plain SDR sequence: Main, 8-bit, BT.709 + for (what, ws, sdr) in [ + ("HDR sequence, settings.sdr", WorkingSpace::Rec2100Pq, true), + ("HLG sequence, settings.sdr", WorkingSpace::Rec2100Hlg, true), + ("SDR sequence", ColorPipeline::REC709.working, false), + ] { + let (p, seq, m) = project(ws); + let path = tmp(&format!("sdr-{}.mp4", what.replace([' ', ','], "-"))); + let s = ExportSettings { sdr, ..hevc(path.clone()) }; + let before = hw_encode_stats(); + export(&p, seq, &s, &m, &Progress::default()).unwrap_or_else(|e| panic!("{what}: {e}")); + assert_eq!(delta(before), (FRAMES as u64, 1, 0), "{what}"); + let pictures = decode(&path); + assert!(matches!(&pictures[0].data, filmcraft_frame::PixelData::Yuv8 { .. }), "{what}: 8-bit planes"); + assert_eq!(pictures[0].color.transfer, filmcraft_color::Transfer::Bt709, "{what}"); + let bytes: Arc<[u8]> = std::fs::read(&path).unwrap().into(); + let file = filmcraft_isobmff::open(&bytes[..]).unwrap(); + let vt = file.track_of_kind(filmcraft_isobmff::TrackKind::Video).unwrap(); + let ve = file.tracks[vt].entries[0].video.clone().unwrap(); + assert!(ve.mastering_display.is_none() && ve.content_light.is_none(), "{what}: no HDR boxes"); + if let Some(ffprobe) = &ffprobe { + let f = probe(ffprobe, &path); + let get = |k: &str| f.get(k).map(String::as_str).unwrap_or("?"); + assert_eq!( + (get("profile"), get("pix_fmt"), get("color_primaries"), get("color_transfer"), get("color_space"), get("color_range")), + ("Main", "yuv420p", "bt709", "bt709", "bt709", "tv"), + "{what}: {f:?}" + ); + assert!(!first_frame_side_data(ffprobe, &path).contains("Mastering"), "{what}"); + } + } +} + +#[test] +fn the_factory_rejects_mismatched_pictures_and_clamps_hostile_floats() { + let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner); + filmcraft_platform::register(); + if !hevc_hdr_available() { + eprintln!("SKIPPED: no NVENC HEVC Main 10 here"); + return; + } + let rate = FrameRate::FPS_24; + let (w, h) = (640u32, 360u32); + let rgba = vec![128u8; (w * h * 4) as usize]; + let hdr_settings = ExportSettings { signal: ColorSignal::PQ, ..hevc(tmp("x.mp4")) }; + let sdr_settings = hevc(tmp("y.mp4")); + let Some(Ok(mut ten)) = factory(Format::Hevc, w, h, rate, &hdr_settings) else { panic!("a Main 10 encoder") }; + let Some(Ok(mut eight)) = factory(Format::Hevc, w, h, rate, &sdr_settings) else { panic!("a Main encoder") }; + let good = vec![0.5f32; (w * h * 3) as usize]; + // the HDR encoder wants the HDR picture, the SDR encoder the RGBA one + assert!(ten.encode(&frame(&rgba, None, 0)).is_err()); + assert!(eight.encode(&frame(&rgba, Some(&good), 0)).is_err()); + // wrong-length floats (short, long, empty) and a wrong picture size: errors, nothing submitted + for bad in [&good[..good.len() - 1], &[][..], &vec![0.5f32; good.len() + 3][..]] { + assert!(ten.encode(&frame(&rgba, Some(bad), 0)).is_err(), "{} floats", bad.len()); + } + assert!(ten.encode(&EncoderFrame { width: w + 2, height: h, rgba: &rgba, hdr: Some(&good), index: 0 }).is_err()); + // NaN, infinities and out-of-range values clamp: the pictures encode + let mut hostile = vec![0.25f32; good.len()]; + for (i, x) in hostile.iter_mut().enumerate() { + *x = [f32::NAN, f32::INFINITY, f32::NEG_INFINITY, -3.0, 7.0, 0.5][i % 6]; + } + let mut packets = 0; + for i in 0..4 { + packets += ten.encode(&frame(&rgba, Some(&hostile), i)).unwrap().len(); + } + packets += ten.flush().unwrap().len(); + assert_eq!(packets, 4); + // the sample entry of the HDR encoder carries colr + mdcv + clli, the SDR one nothing HDR + let e = ten.sample_entry(); + let v = e.video.as_ref().unwrap(); + assert!(v.color.is_some() && v.mastering_display.is_some() && v.content_light == Some((0, 0))); + assert!(eight.sample_entry().video.as_ref().unwrap().color.is_none()); + // HLG: colr only + let Some(Ok(hlg)) = factory(Format::Hevc, w, h, rate, &ExportSettings { signal: ColorSignal::HLG, ..sdr_settings.clone() }) else { + panic!("an HLG encoder") + }; + let v = hlg.sample_entry().video.unwrap(); + assert!(v.color.is_some() && v.mastering_display.is_none() && v.content_light.is_none()); + // H.264 still declines HDR (software signalling), and says why + let h264 = ExportSettings { format: Format::H264, signal: ColorSignal::PQ, ..sdr_settings }; + assert!(factory(Format::H264, w, h, rate, &h264).is_none()); +} diff --git a/crates/platform/tests/nvenc_hevc_main10.rs b/crates/platform/tests/nvenc_hevc_main10.rs new file mode 100644 index 00000000..76d0c9cc --- /dev/null +++ b/crates/platform/tests/nvenc_hevc_main10.rs @@ -0,0 +1,324 @@ +//! NVENC H.265 Main 10 (HDR PQ / HLG) encoding (Windows, NVIDIA GPU with 10-bit HEVC encoding): the +//! pictures that go in are the 10-bit codes `rgbf_to_yuv420_10` makes of float PQ / HLG R'G'B', the +//! pictures that come out of our own HEVC decoder are 10-bit and close to them (smooth ramps keep far more +//! than 256 levels, bright code values survive), the VUI and the HDR10 SEI messages are right, and +//! hostile inputs are errors. Skips (with SKIPPED) only on a machine without a 10-bit HEVC encoder. +#![cfg(target_os = "windows")] + +use filmcraft_bitstream::unescape_rbsp; +use filmcraft_export::{ColorSignal, rgbf_to_yuv420_10}; +use filmcraft_hevc::params::Sps; +use filmcraft_isobmff::SampleEntry; +use filmcraft_platform::nvenc::export::nvenc_signal; +use filmcraft_platform::nvenc::{Codec, Config, Nvenc, Packet, Profile, Signal, hevc_available, hevc_hdr_available, nal_type}; + +const W: usize = 1280; +const H: usize = 720; +const FRAMES: usize = 48; + +/// NVENC sessions are a limited resource on consumer GPUs: one test at a time. +static SERIAL: std::sync::Mutex<()> = std::sync::Mutex::new(()); + +fn config(w: u32, h: u32) -> Config { + Config { + width: w, + height: h, + fps: (24, 1), + bitrate_kbps: 12000, + max_bitrate_kbps: 18000, + cbr: false, + keyint: 24, + profile: Profile::HevcMain10, + level: None, + sar: None, + bframes: true, + } +} + +/// A float HDR test picture (encoded R'G'B', 3 per pixel): the top half is a smooth grey ramp from black to +/// peak (all of it, so PQ passes 1000 nits near 75 % of the way), a box moves across it; the bottom half +/// is bars of saturated BT.2020 colours (the primaries and secondaries at 0.75, beyond BT.709) and dark steps. +fn picture(i: usize) -> Vec { + let mut v = vec![0f32; W * H * 3]; + let bars: [[f32; 3]; 8] = + [[0.75, 0.0, 0.0], [0.0, 0.75, 0.0], [0.0, 0.0, 0.75], [0.0, 0.75, 0.75], [0.75, 0.0, 0.75], [0.75, 0.75, 0.0], [1.0, 1.0, 1.0], [0.05, 0.05, 0.05]]; + let (bx, by) = ((i * 11) % (W - 80), (i * 5) % (H / 2 - 80)); + for y in 0..H { + for x in 0..W { + let o = (y * W + x) * 3; + let px = if y < H / 2 { + if x >= bx && x < bx + 80 && y >= by && y < by + 80 { [0.9, 0.3, 0.1] } else { [x as f32 / (W - 1) as f32; 3] } + } else { + bars[x * 8 / W] + }; + v[o..o + 3].copy_from_slice(&px); + } + } + v +} + +/// The converter's 10-bit planes of a picture. +fn planes(rgb: &[f32], signal: &ColorSignal) -> (Vec, Vec, Vec) { + let (kr, kb) = signal.kr_kb(); + let (mut y, mut u, mut v) = (Vec::new(), Vec::new(), Vec::new()); + rgbf_to_yuv420_10(rgb, W, H, kr, kb, &mut y, &mut u, &mut v).expect("convert"); + (y, u, v) +} + +type Sources = Vec<(Vec, Vec, Vec)>; + +fn encode_all(cfg: &Config, signal: &ColorSignal, frames: usize) -> Option<(Nvenc, Vec, Sources)> { + if !hevc_hdr_available() { + eprintln!("SKIPPED: no NVENC HEVC Main 10 encoder"); + return None; + } + let mut enc = Nvenc::with_signal(cfg, &nvenc_signal(signal)).unwrap_or_else(|why| panic!("NVENC Main 10 is available but refused {cfg:?}: {why}")); + let (mut packets, mut sources) = (Vec::new(), Vec::new()); + for i in 0..frames { + let src = planes(&picture(i), signal); + packets.extend(enc.encode_10(&src.0, &src.1, &src.2, i as u64).unwrap()); + sources.push(src); + } + packets.extend(enc.flush().unwrap()); + Some((enc, packets, sources)) +} + +fn psnr10(a: &[u16], b: &[u16]) -> f64 { + let mse = a.iter().zip(b).map(|(x, y)| (f64::from(*x) - f64::from(*y)).powi(2)).sum::() / a.len() as f64; + if mse == 0.0 { 99.0 } else { 10.0 * (1023.0f64 * 1023.0 / mse).log10() } +} + +/// The (NAL type, bytes after the 2-byte header) of every NAL unit of a length-prefixed sample. +fn sample_nals(sample: &[u8]) -> Vec<(u8, Vec)> { + let mut out = Vec::new(); + let mut pos = 0; + while let Some(len) = sample.get(pos..pos + 4) { + let n = u32::from_be_bytes([len[0], len[1], len[2], len[3]]) as usize; + pos += 4; + let nal = sample.get(pos..pos + n).expect("a NAL unit inside the sample"); + out.push((nal_type(Codec::Hevc, nal).expect("a NAL header"), nal[2..].to_vec())); + pos += n; + } + assert_eq!(pos, sample.len(), "samples are made of whole NAL units"); + out +} + +/// The SEI messages (type, payload) of a prefix SEI NAL unit's RBSP. +fn sei_messages(nal_body: &[u8]) -> Vec<(u32, Vec)> { + let rbsp = unescape_rbsp(nal_body); + let (mut out, mut i) = (Vec::new(), 0usize); + // the RBSP ends with the stop bit byte 0x80 (rbsp_trailing_bits) + while i < rbsp.len() && rbsp[i] != 0x80 { + let mut t = 0u32; + while rbsp[i] == 0xff { + t += 255; + i += 1; + } + t += u32::from(rbsp[i]); + i += 1; + let mut n = 0usize; + while rbsp[i] == 0xff { + n += 255; + i += 1; + } + n += usize::from(rbsp[i]); + i += 1; + out.push((t, rbsp[i..i + n].to_vec())); + i += n; + } + out +} + +fn hex(b: &[u8]) -> String { + b.iter().map(|x| format!("{x:02x}")).collect() +} + +fn round_trip(name: &str, signal: ColorSignal, want_sei: bool) { + let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner); + let Some((enc, packets, sources)) = encode_all(&config(W as u32, H as u32), &signal, FRAMES) else { return }; + assert!(enc.is_ten_bit()); + assert_eq!(packets.len(), FRAMES, "{name}: one packet per picture"); + let keys: Vec = packets.iter().filter(|p| p.key).map(|p| p.pts).collect(); + assert_eq!(keys, vec![0, 24], "{name}: IDR pictures"); + + // where NVENC put the SEI: every IDR sample has prefix SEI NAL units (type 39) before its slices; nothing else does + let expected = nvenc_signal(&signal).sei; + for (k, p) in packets.iter().enumerate() { + let nals = sample_nals(&p.data); + let types: Vec = nals.iter().map(|n| n.0).collect(); + assert!(types.iter().all(|t| !(32..=37).contains(t)), "{name}: packet {k} carries parameter sets {types:?}"); + let seis: Vec<(u32, Vec)> = nals.iter().filter(|n| n.0 == 39).flat_map(|n| sei_messages(&n.1)).collect(); + if p.key { + if k < 2 { + eprintln!("{name}: IDR sample {k} NAL types in order {types:?}"); + } + let got: Vec<(u32, Vec)> = seis.clone(); + let want: Vec<(u32, Vec)> = expected.iter().map(|m| (m.payload_type, m.payload.clone())).collect(); + assert_eq!(got, want, "{name}: SEI messages of the IDR sample {k}"); + let slice_at = types.iter().position(|t| matches!(t, 19 | 20)).expect("an IDR slice"); + assert!(types.iter().take(slice_at).filter(|t| **t == 39).count() >= usize::from(want_sei), "{name}: prefix SEI before the slice"); + assert!(types.iter().all(|t| *t != 40), "{name}: no suffix SEI"); + } else { + assert!(seis.is_empty(), "{name}: packet {k} (not an IDR) carries SEI {seis:?}"); + } + } + assert_eq!(expected.is_empty(), !want_sei); + + // the record and the SPS: Main 10, 10-bit 4:2:0, BT.2020 + the transfer, limited range + let hvcc = enc.hevc_config().expect("an hvcC record").clone(); + let sps_nal = enc.parameter_sets().0.to_vec(); + eprintln!("{name}: VPS {}\n{name}: SPS {}\n{name}: PPS {}", hex(enc.vps()), hex(&sps_nal), hex(enc.parameter_sets().1)); + assert_eq!((hvcc.general_profile_idc, hvcc.chroma_format_idc, hvcc.bit_depth_luma, hvcc.bit_depth_chroma), (2, 1, 10, 10), "{name}"); + eprintln!( + "{name}: hvcC level {} compat {:#010x} constraints {:#014x}", + hvcc.general_level_idc, hvcc.general_profile_compatibility_flags, hvcc.general_constraint_indicator_flags + ); + let rbsp = unescape_rbsp(&sps_nal[2..]); + let sps = Sps::parse(&rbsp).unwrap(); + let vui = sps.vui.clone().expect("a VUI"); + assert_eq!((vui.colour_primaries, vui.transfer_characteristics, vui.matrix_coefficients, vui.full_range), (9, signal.transfer, 9, false), "{name}"); + + // our decoder: 10-bit planes + let entry = SampleEntry::hevc(hvcc, W as u16, H as u16); + let mut dec = filmcraft_codecs::software_video_decoder(&entry).unwrap(); + let mut out = Vec::new(); + for p in &packets { + out.extend(dec.decode(&p.data, p.pts).unwrap()); + } + out.extend(dec.flush()); + assert_eq!(out.len(), FRAMES, "{name}"); + let (mut worst_y, mut worst_c) = (99.0f64, 99.0f64); + let (mut max_code, mut max_ramp_code) = (0u16, 0u16); + let mut distinct_min = usize::MAX; + for f in &out { + let filmcraft_frame::PixelData::Yuv16 { planes: p, bits, .. } = &f.frame.data else { panic!("{name}: 16-bit planes, got {}", f.frame.format_label()) }; + assert_eq!(*bits, 10, "{name}: decoded bit depth"); + assert_eq!((f.frame.width as usize, f.frame.height as usize), (W, H)); + assert!(p.iter().all(|pl| pl.iter().all(|c| *c < 1024)), "{name}: codes are 10-bit values, not MSB-aligned"); + let src = &sources[f.pts as usize]; + worst_y = worst_y.min(psnr10(&p[0], &src.0)); + worst_c = worst_c.min(psnr10(&p[1], &src.1)).min(psnr10(&p[2], &src.2)); + max_code = max_code.max(p[0].iter().copied().max().unwrap_or(0)); + // the ramp: one row of the top half, away from the moving box (row 340 is below it) + let row = &p[0][340 * W..341 * W]; + max_ramp_code = max_ramp_code.max(row.iter().copied().max().unwrap_or(0)); + let mut levels: Vec = row.to_vec(); + levels.sort_unstable(); + levels.dedup(); + distinct_min = distinct_min.min(levels.len()); + } + let src_max = sources.iter().map(|s| s.0[340 * W..341 * W].iter().copied().max().unwrap_or(0)).max().unwrap_or(0); + eprintln!( + "{name}: worst luma PSNR {worst_y:.1} dB, worst chroma {worst_c:.1} dB (10-bit scale), ramp row: {distinct_min} distinct decoded levels (min over frames), \ + max decoded ramp code {max_ramp_code} (source {src_max}), max luma code {max_code}, {} bytes", + packets.iter().map(|p| p.data.len()).sum::() + ); + assert!(worst_y > 60.0, "{name}: luma PSNR {worst_y:.1} dB"); + assert!(worst_c > 60.0, "{name}: chroma PSNR {worst_c:.1} dB"); + assert!(distinct_min > 700, "{name}: only {distinct_min} distinct levels on a smooth ramp: not 10-bit"); + assert!(max_ramp_code >= src_max.saturating_sub(4) && max_ramp_code > 900, "{name}: bright codes lost: {max_ramp_code} (source {src_max})"); +} + +#[test] +fn pq_decodes_to_the_10_bit_source() { + round_trip("PQ", ColorSignal::PQ, true); +} + +#[test] +fn hlg_decodes_to_the_10_bit_source_without_static_metadata() { + round_trip("HLG", ColorSignal::HLG, false); +} + +/// Run `f`, failing the test (not the process) if it panics. +fn calm(what: &str, f: impl FnOnce() -> T) -> T { + match std::panic::catch_unwind(std::panic::AssertUnwindSafe(f)) { + Ok(v) => v, + Err(_) => panic!("{what}: panicked"), + } +} + +#[test] +fn main_10_and_main_encoders_refuse_each_others_pictures() { + let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner); + if !hevc_hdr_available() { + eprintln!("SKIPPED: no NVENC HEVC Main 10 encoder"); + return; + } + let (w, h) = (640usize, 360usize); + let mut ten = Nvenc::with_signal(&config(w as u32, h as u32), &nvenc_signal(&ColorSignal::PQ)).unwrap(); + let mut eight = Nvenc::new(&Config { profile: Profile::HevcMain, ..config(w as u32, h as u32) }).unwrap(); + assert!(ten.is_ten_bit() && !eight.is_ten_bit()); + let (y8, c8) = (vec![16u8; w * h], vec![128u8; w * h / 4]); + let (y10, c10) = (vec![64u16; w * h], vec![512u16; w * h / 4]); + assert!(calm("8-bit into Main 10", || ten.encode(&y8, &c8, &c8, 0)).err().unwrap().contains("Main 10")); + assert!(calm("10-bit into Main", || eight.encode_10(&y10, &c10, &c10, 0)).err().unwrap().contains("8-bit")); + // wrong-length planes + assert!(calm("short luma", || ten.encode_10(&y10[..w * h - 1], &c10, &c10, 0)).is_err()); + assert!(calm("short chroma", || ten.encode_10(&y10, &c10[..c10.len() - 1], &c10, 0)).is_err()); + assert!(calm("empty", || ten.encode_10(&[], &[], &[], 0)).is_err()); + // nothing was submitted by the refusals: the encoders still work + ten.encode_10(&y10, &c10, &c10, 0).unwrap(); + eight.encode(&y8, &c8, &c8, 0).unwrap(); + assert!(!ten.flush().unwrap().is_empty() && !eight.flush().unwrap().is_empty()); + // H.264 takes no colour signal, no 10-bit profile + let h264 = Config { profile: Profile::Main, ..config(w as u32, h as u32) }; + assert!(Nvenc::with_signal(&h264, &nvenc_signal(&ColorSignal::PQ)).is_err()); + assert!(Nvenc::with_signal(&h264, &Signal::default()).is_ok() || !filmcraft_platform::nvenc::available()); +} + +#[test] +fn hostile_main_10_configurations_give_errors_not_crashes() { + let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner); + if !hevc_hdr_available() { + eprintln!("SKIPPED: no NVENC HEVC Main 10 encoder"); + return; + } + let base = config(640, 360); + let sig = nvenc_signal(&ColorSignal::PQ); + for (w, h) in [(0, 0), (641, 360), (640, 361), (16, 16), (u32::MAX, u32::MAX), (100_000, 64)] { + assert!(calm("size", || Nvenc::with_signal(&config(w, h), &sig)).is_err(), "{w}x{h}"); + } + let variants: Vec<(&str, Config, Signal)> = vec![ + ("zero bitrate", Config { bitrate_kbps: 0, max_bitrate_kbps: 0, ..base.clone() }, sig.clone()), + ("huge bitrate", Config { bitrate_kbps: u32::MAX, max_bitrate_kbps: u32::MAX, ..base.clone() }, sig.clone()), + ("keyint 1", Config { keyint: 1, ..base.clone() }, sig.clone()), + ("zero keyint", Config { keyint: 0, ..base.clone() }, sig.clone()), + ("odd signal codes", base.clone(), Signal { primaries: 255, transfer: 255, matrix: 255, sei: vec![] }), + ( + "a huge SEI payload", + base.clone(), + Signal { sei: vec![filmcraft_platform::nvenc::Sei { payload_type: 137, payload: vec![0xAB; 100_000] }], ..sig.clone() }, + ), + ("an empty SEI payload", base.clone(), Signal { sei: vec![filmcraft_platform::nvenc::Sei { payload_type: 144, payload: vec![] }], ..sig.clone() }), + ]; + for (name, c, s) in variants { + match calm(name, || Nvenc::with_signal(&c, &s)) { + Ok(mut enc) => { + let (y, u) = (vec![300u16; 640 * 360], vec![512u16; 640 * 360 / 4]); + for i in 0..3u64 { + let _ = calm(name, || enc.encode_10(&y, &u, &u, i)); + } + let _ = calm(name, || enc.flush()); + } + Err(why) => eprintln!("{name}: declined ({why})"), + } + } + // dropped mid-stream with SEI buffers in flight, repeatedly + for round in 0..8 { + let mut enc = Nvenc::with_signal(&base, &sig).unwrap_or_else(|e| panic!("round {round}: {e}")); + let (y, u) = (vec![300u16; 640 * 360], vec![512u16; 640 * 360 / 4]); + for i in 0..5 { + enc.encode_10(&y, &u, &u, i).unwrap(); + } + } +} + +#[test] +fn the_probe_follows_the_encoder() { + let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner); + let hdr = hevc_hdr_available(); + assert!(!hdr || hevc_available(), "Main 10 implies HEVC"); + assert_eq!(hevc_hdr_available(), hdr, "the answer is kept"); + let made = Nvenc::with_signal(&config(640, 360), &Signal::default()); + assert_eq!(made.is_ok(), hdr, "{:?}", made.err()); +} diff --git a/crates/platform/tests/nvenc_hevc_warm.rs b/crates/platform/tests/nvenc_hevc_warm.rs index 33944520..adc0ebb2 100644 --- a/crates/platform/tests/nvenc_hevc_warm.rs +++ b/crates/platform/tests/nvenc_hevc_warm.rs @@ -5,18 +5,26 @@ use std::time::{Duration, Instant}; -use filmcraft_platform::nvenc::{hevc_available, hevc_probed}; +use filmcraft_platform::nvenc::{hevc_available, hevc_hdr_available, hevc_hdr_probed, hevc_probed}; #[test] fn register_answers_the_hevc_question_in_the_background() { - assert!(!hevc_probed(), "nothing has asked yet"); + assert!(!hevc_probed() && !hevc_hdr_probed(), "nothing has asked yet"); filmcraft_platform::register(); let t0 = Instant::now(); while !hevc_probed() { assert!(t0.elapsed() < Duration::from_secs(60), "register() did not start the probe"); std::thread::sleep(Duration::from_millis(10)); } + // the Main 10 probe runs on the same thread, right after + while !hevc_hdr_probed() { + assert!(t0.elapsed() < Duration::from_secs(60), "register() did not start the Main 10 probe"); + std::thread::sleep(Duration::from_millis(10)); + } eprintln!("answered in the background after {:?}", t0.elapsed()); + let hdr = hevc_hdr_available(); + assert!(!hdr || hevc_available(), "Main 10 implies HEVC"); + assert_eq!(filmcraft_export::hdr_available(filmcraft_export::Format::Hevc), hdr); // the answer is kept: asking and registering again change nothing let answer = hevc_available(); filmcraft_platform::register(); diff --git a/crates/ui-egui/src/panels/export_mode.rs b/crates/ui-egui/src/panels/export_mode.rs index 23d3d7e9..cf682a4a 100644 --- a/crates/ui-egui/src/panels/export_mode.rs +++ b/crates/ui-egui/src/panels/export_mode.rs @@ -389,6 +389,8 @@ fn settings_column(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect) { .map(|q| (q.settings.width, q.settings.height, q.settings.frame_rate, q.settings.sample_rate)) .unwrap_or((1920, 1080, FrameRate::FPS_24, 48_000)); let has_captions = app.session.active_sequence().is_some_and(|q| !q.caption_tracks.is_empty()); + // an HDR sequence exports HDR (H.265 Main 10 where the hardware encoder has it) unless SDR is asked for + let seq_hdr = app.session.active_sequence().is_some_and(|q| q.settings.color.working.is_hdr()); let all_presets = app.session.export_presets.all(); let favs: Vec = all_presets.iter().filter(|p| app.session.export_presets.is_favorite(&p.name)).map(|p| p.name.clone()).collect(); let mut pick_folder = false; @@ -463,7 +465,7 @@ fn settings_column(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect) { ui.add_space(8.0); let s = &mut ex.settings; if s.has_video() && section(ui, &mut reg, &mut ex.open_sections, "video", "Video", &t) { - video_section(ui, &mut reg, s, &t, seq_w, seq_h); + video_section(ui, &mut reg, s, &t, seq_w, seq_h, seq_hdr); } if (s.has_audio() || !s.has_video() @@ -545,7 +547,7 @@ fn settings_column(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect) { reg.flush(app); } -fn video_section(ui: &mut egui::Ui, reg: &mut Reg, s: &mut ExportSettings, t: &Tokens, seq_w: u32, seq_h: u32) { +fn video_section(ui: &mut egui::Ui, reg: &mut Reg, s: &mut ExportSettings, t: &Tokens, seq_w: u32, seq_h: u32, seq_hdr: bool) { // frame size let mut match_size = s.frame_size.is_none(); row(ui, t, "Frame Size", |ui| { @@ -602,8 +604,13 @@ fn video_section(ui: &mut egui::Ui, reg: &mut Reg, s: &mut ExportSettings, t: &T f @ (Format::H264 | Format::Hevc) => { let hevc = f == Format::Hevc; if hevc { - // the hardware encoder writes one profile: Main, 8-bit 4:2:0, level chosen by the encoder - row(ui, t, "Profile", |ui| ui.label("Main (8-bit)")); + // the hardware encoder writes Main (8-bit 4:2:0), or Main 10 for HDR sequences where it can; level chosen by the encoder + let label = match (filmcraft_engine::export::hdr_available(Format::Hevc), seq_hdr && !s.sdr) { + (true, true) => "Main 10 (10-bit, HDR)", + (true, false) => "Main (8-bit; Main 10 for HDR sequences)", + (false, _) => "Main (8-bit)", + }; + row(ui, t, "Profile", |ui| ui.label(label)); row(ui, t, "Encoder", |ui| ui.label("Hardware (H.265 has no software encoder)")); } else { row(ui, t, "Profile", |ui| { diff --git a/docs/adr/0001-platform-ffi.md b/docs/adr/0001-platform-ffi.md index 4186db60..622be6b6 100644 --- a/docs/adr/0001-platform-ffi.md +++ b/docs/adr/0001-platform-ffi.md @@ -95,6 +95,14 @@ Everything else in this record applies unchanged: `unsafe` stays in `crates/plat has a `// SAFETY:` comment, no panic crosses the FFI boundary, the public API is safe, and the crate compiles everywhere (`register()` is a no-op off macOS). +## Addendum (2026-10-08): NVENC HEVC Main 10 (HDR) + +HDR H.265 added no `unsafe` module either. `nvenc::ffi` gained data declarations (the HEVC picture +parameters, `NV_ENC_SEI_PAYLOAD`, the 10-bit buffer format) checked by the generated layout tests, and +`nvenc::session` gained the P010 pitch check and the SEI pointers it hands to the driver, which are +heap blocks owned by the session for its whole life (`// SAFETY:` comments say so). The conversion from +float pictures to 10-bit planes is safe code in the export crate. + ## Addendum (2026-10-07): hardware H.265 (HEVC) encoding The same VideoToolbox session wrapper also creates HEVC sessions (`VtProfile::HevcMain`). The rules diff --git a/docs/architecture.md b/docs/architecture.md index a72643c9..cc69a953 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -273,8 +273,12 @@ file ──► codecs (MP4/MOV, MKV, audio) demux + decode, GOP-aware see (the software encoder runs, counted in `export.hardware.declined`) two-pass VBR, HDR, MXF, interlaced output, sizes outside NVENC's limits, and systems without an NVIDIA GPU or driver. A failure during an export ends it with an error: the software encoder cannot take over a hardware - stream. The same NVENC session also encodes H.265 (HEVC Main, 8-bit 4:2:0, SDR; the `hvcC` is built - from the SPS the encoder wrote): `register()` hands `platform::nvenc::hevc_available` to + stream. The same NVENC session also encodes H.265 (HEVC Main 8-bit 4:2:0 SDR, or Main 10 HDR: PQ / HLG, + BT.2020, limited range, from the float R'G'B' pictures through `rgbf_to_yuv420_10` into P010 input + buffers, with the HDR10 static metadata as SEI on every IDR for PQ; the `hvcC` is built from the SPS + the encoder wrote). An HDR sequence exports as HDR H.265 only where a registered probe says so + (`filmcraft_export::register_hdr_probe`, `hdr_available`; NVENC with 10-bit support), otherwise it is + tone-mapped to SDR as on macOS: `register()` hands `platform::nvenc::hevc_available` to `filmcraft_export::register_format_probe`, so the H.265 format exists where an HEVC encoder does, and choosing it is the opt-in (the Hardware encoding toggle governs H.264 only). With no software H.265 encoder, what NVENC declines is counted and ends the export with an error that says why. diff --git a/docs/testing.md b/docs/testing.md index f6745fd3..cbb2c9ee 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -72,14 +72,15 @@ are never linked or shipped ([AGENTS.md](../AGENTS.md) §2). AV1 fixtures (libvpx-vp9, libaom-av1; 360p, 1080p, 2160p, hidden alt-ref frames, two GOPs) need an ffmpeg with those encoders. The NVENC tests (`crates/platform/tests/nvenc.rs` and `nvenc_export.rs` for H.264, `nvenc_hevc.rs`, -`nvenc_hevc_export.rs`, `nvenc_hevc_probe.rs` and `nvenc_hevc_warm.rs` for H.265; Windows only) skip, -printing `SKIPPED`, without an NVIDIA GPU with NVENC (H.265: when the HEVC probe says there is no HEVC -encoder; on a machine that has one, a configuration NVENC refuses fails the test). The FFI layout tests in +`nvenc_hevc_export.rs`, `nvenc_hevc_probe.rs` and `nvenc_hevc_warm.rs` for H.265, `nvenc_hevc_main10.rs` and +`nvenc_hevc_hdr_export.rs` for Main 10 HDR; Windows only) skip, printing `SKIPPED`, without an NVIDIA GPU +with NVENC (H.265: when the HEVC probe says there is no HEVC encoder; Main 10: when the Main 10 probe says +there is no 10-bit encoder; on a machine that has one, a configuration NVENC refuses fails the test). The FFI layout tests in `crates/platform/src/nvenc/abi_tests.rs` were generated from a C program built with MSVC (`cl` after `vcvars64.bat`) against NVIDIA's MIT-licensed `nvEncodeAPI.h` (12.1); the header and the program are not in the repository. To regenerate them, put the header in a scratch directory and write a program that prints, for every type, field, constant and GUID of `src/nvenc/ffi.rs` that the -asserts name (H.264 and HEVC), `sizeof` / `alignof` / `offsetof`, the enum and macro values cast to +asserts name (H.264, HEVC and the HEVC picture parameters with their SEI payload array), `sizeof` / `alignof` / `offsetof`, the enum and macro values cast to 64-bit integers and the GUIDs as 128-bit hex; for the bit-fields of the `flags` words, which `offsetof` cannot take, zero a structure, set that one field to its maximum and print the 32-bit word. Compare the numbers with the asserts (a difference is drift, which must be understood before