Docs: Windows hardware decoding (platform README, ADR 0001, architecture, performance, roadmap)

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01MMorBHwct72udS2LiC3ZHp

AGENTS.md

@@ -21,7 +21,7 @@ panic path, and fix a crash before building on top of it. The cross-app standard
    code alone (a literal that always parses): a single-item `#[allow(clippy::expect_used)]` with the reason.
 3. **No `unsafe`** (`unsafe_code = "forbid"` for the workspace). The single exception is
    `crates/platform`, which needs `unsafe` to call the operating system's media APIs (hardware
-   video decoding through VideoToolbox; later encoding and other OSes) and does nothing else. It
+   video decoding through VideoToolbox and Media Foundation / Direct3D 11; later encoding and Linux) and does nothing else. It
    uses `unsafe_code = "deny"` with `#[allow(unsafe_code)]` only on its FFI modules, a
    `// SAFETY:` comment on every `unsafe` block, a safe `Result`-returning public API, and the
    pure-Rust decoder as the tested fallback ([ADR 0001](docs/adr/0001-platform-ffi.md)).

README.md

@@ -286,7 +286,7 @@ We track two numbers ([ROADMAP.md](ROADMAP.md#honest-assessment-2026-10-05)):
 
 The biggest gaps today:
 
-- **Speed on big footage.** Hardware decoding works on macOS; Windows and Linux have no hardware path yet. Blend modes and the most common effects run on the GPU, but Lumetri, keys, export and encoding still run on the CPU ([#30](https://github.com/storytold/filmcraft/issues/30)).
+- **Speed on big footage.** Hardware decoding works on macOS and Windows; Linux has no hardware path yet. Blend modes and the most common effects run on the GPU, but Lumetri, keys, export and encoding still run on the CPU ([#30](https://github.com/storytold/filmcraft/issues/30)).
 - **No plugins.** No VST3 / Audio Units or OpenFX hosting.
 - **Delivery codecs.** H.264 is our only delivery-codec export; no HEVC or AV1 export yet.
 - **Real-world media and platforms.** Our decoders are bit-exact on conformance streams, but camera and phone files in the wild are less tested. Windows and Linux get far less testing than macOS.

ROADMAP.md

@@ -36,7 +36,7 @@ before choosing work.
 | **Correctness in depth** | not tracked | **~60–70%** | A first-time contributor found four real bugs in core paths within hours (#5–#8: AIFF import failed, Slide left linked audio behind, transitions didn't follow ripple trims, export ignored start/end times). User reports: Color Matte always grey with no picker (#29); ALSA audio error on Linux (#23). Settings are "most wired", not all. |
 | **Codecs and media** | ~90% | **~75%** | Our decoders are bit-exact on conformance streams. Real camera and phone media (variable frame rate, damaged files, unusual containers) is far less tested. No camera RAW (RED, BRAW, ARRIRAW), no E-AC-3. |
 | **Export** | ~92% | **~70%** | Delivery: H.264 only (ProRes, DNxHR and image sequences cover mastering). No HEVC or AV1 export. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. |
-| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS only (VideoToolbox, #33; active in the desktop app only since #41, so releases up to 0.2.1 decoded in software). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects, export and encoding are CPU. Linux and Windows have no hardware path. 8K and AV1 aren't real-time. The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. |
+| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS (VideoToolbox, #33; active in the desktop app only since #41, so releases up to 0.2.1 decoded in software) and Windows (Media Foundation / Direct3D 11 DXVA, H.264 and HEVC Main / Main 10, #30). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects, export and encoding are CPU. Linux has no hardware path. 8K and AV1 aren't real-time. The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. |
 | **Stability** | not tracked | **improving, unproven** | The never-crash pass (no panics in product code, last-resort guards) landed on 2026-10-04. There's no field record from real users' projects yet. |
 | **Plugins and pro ecosystem** | not tracked | **~0–10%** | No audio plugin hosting (VST3 / Audio Units) and no third-party video effects (OpenFX). For many professional editors this alone rules FilmCraft out. Team Projects and Productions are out of scope by design. |
 | **AI features** | partial | **~25–35%** | Speech to text exists but is off in default builds (the `whisper` feature). Enhance Speech is a DSP chain, not a model. Auto Reframe is approximate. No Generative Extend, and no media-intelligence search or auto colour. |
@@ -48,7 +48,7 @@ before choosing work.
 In priority order. Agents should prefer this work over adding more checklist items.
 
 1. **Hardware acceleration** (#30), the most visible gap to users:
-   - hardware decode on Linux (VA-API) and Windows (Media Foundation);
+   - hardware decode on Linux (VA-API); VP9 / AV1 on Windows;
    - zero-copy decoded frames into wgpu;
    - the remaining effects on the GPU: Lumetri, keys, Vignette, Video Limiter, masks (31 common effects are done);
    - GPU export, then hardware encode (H.264 / HEVC).
@@ -154,7 +154,7 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
 
 ## Running now
 
-- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 31 common effects on the GPU (GPU2). Next: VA-API (Linux) and Media Foundation (Windows) decode, zero-copy upload, Lumetri on the GPU.
+- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 31 common effects on the GPU (GPU2). Windows: Media Foundation / Direct3D 11 H.264 + HEVC decode (HW2). Next: VA-API (Linux) decode, zero-copy upload, Lumetri on the GPU.
 - **Next:** [Where we are lacking](#where-we-are-lacking) items 1–5. GPU export waits for the export-crate work (#19 and the never-crash follow-up) to land.
 
 ## Log

docs/adr/0001-platform-ffi.md

@@ -25,15 +25,21 @@ Containment rules:
 
 1. The crate does not use `lints.workspace = true`. Its own `[lints]` table copies the workspace
    lints except `unsafe_code = "deny"` (not `forbid`), and adds
-   `clippy::undocumented_unsafe_blocks = "deny"`. Only the FFI modules (`videotoolbox`) carry
-   `#[allow(unsafe_code)]`; the rest of the crate (the fallback logic in `hybrid`) has no `unsafe`.
+   `clippy::undocumented_unsafe_blocks = "deny"`. Only the FFI modules (`videotoolbox`, and
+   `media_foundation::gpu` / `media_foundation::mft` on Windows) carry `#[allow(unsafe_code)]`; the
+   rest of the crate (the fallback logic in `hybrid`, the decoder logic in `media_foundation`,
+   `annexb`, `biplanar`) has no `unsafe`.
 2. Every `unsafe` block has a `// SAFETY:` comment saying why it is sound.
 3. The public API is safe: no `pub unsafe fn`, no raw pointers or FFI types in public signatures;
    failures are `Result`s. The crate keeps the never-crash `deny(clippy::unwrap_used, …)`
    attribute of every clean crate.
 4. No panic crosses the FFI boundary: the VideoToolbox output callback runs under `catch_unwind`
-   and reports a panic as a failed frame.
-5. The crate compiles on every target. OS bindings are target-specific dependencies; elsewhere
+   and reports a panic as a failed frame. The Windows backend has no callbacks into Rust (it drives
+   a synchronous decoder MFT with `ProcessInput` / `ProcessOutput`), so nothing can unwind into
+   COM.
+5. The crate compiles on every target. OS bindings are target-specific dependencies (the
+   `objc2-*` crates on macOS, the official `windows` crate on Windows: MIT OR Apache-2.0, already
+   in the lockfile through wgpu / winit, with only the Win32 features the backend uses); elsewhere
    `register()` reports `Unavailable` and does nothing. It is not part of the wasm check (L5).
 
 ## The fallback guarantee
@@ -44,7 +50,10 @@ Registering a hardware decoder never makes a file undecodable, and never changes
   decoding is Off, for formats the hardware path does not take (field-coded H.264, bit depths
   other than 8 / 10, 4:4:4, mismatched luma / chroma depth), and when the OS cannot create a
   hardware session for the stream (hardware decoding is *required*, so the OS's own software
-  decoder is never used instead of ours).
+  decoder is never used instead of ours). On Windows that means the decoder MFT must be
+  Direct3D-aware, the GPU's DXVA decoder must list the stream, and every picture must come back as
+  a Direct3D 11 texture: Microsoft's decoder MFTs quietly decode in software when DXVA is not
+  available, and a system-memory picture is treated as a failure.
 - A hardware decoder that fails mid-stream (decode error, session invalidated by a GPU change or
   sleep, in-band parameter sets that differ from the sample entry) becomes the software decoder
   for that stream without the caller noticing: `HybridDecoder` replays the samples since the last

docs/architecture.md

@@ -72,7 +72,7 @@ and `filmcraft-cli`.
 | `engine` | L4 | `Session`, command registry, undo history, media pool, jobs, interchange glue |
 | `ui-egui` | L5 | the egui frontend: docking, panels, timeline, monitors, playback, control-channel handlers |
 | `automation` | L5 | MCP server (`rmcp`, stdio), headless or bridged to the running app |
-| `platform` | L5 | OS media FFI only: hardware video decoding (VideoToolbox H.264 / HEVC on macOS; a no-op elsewhere) behind `codecs::VideoDecoder`, with transparent fallback to our decoders. The one crate allowed `unsafe` ([ADR 0001](adr/0001-platform-ffi.md), [README](../crates/platform/README.md)) |
+| `platform` | L5 | OS media FFI only: hardware video decoding (VideoToolbox H.264 / HEVC on macOS; Media Foundation / Direct3D 11 H.264 / HEVC on Windows; a no-op elsewhere) behind `codecs::VideoDecoder`, with transparent fallback to our decoders. The one crate allowed `unsafe` ([ADR 0001](adr/0001-platform-ffi.md), [README](../crates/platform/README.md)) |
 | `filmcraft` | L6 | desktop binary: eframe/wgpu window, cpal audio output, file dialogs, native macOS menu, TCP control server |
 | `filmcraft-cli` | L6 | headless CLI: `exec`, `run`, `inspect`, `describe`, `commands`, `import`, `export`, `render`, `probe`, `mcp`; `--bridge` targets the running app |
 | `filmcraft-web` | L6 | the browser app (wasm32): eframe web runner on WebGPU/WebGL2, Blob-backed services, OPFS recovery, WebAudio, WebCodecs, `window.filmcraft` API ([web.md](web.md)) |
@@ -200,7 +200,7 @@ pub struct CommandSpec {
 ```text
 file ──► codecs (MP4/MOV, MKV, audio)        demux + decode, GOP-aware seek
           │   decoder registry: h264, hevc, vp9, av1, prores, dnx, mjpeg (+ any registered first:
-          │   platform's VideoToolbox H.264 / HEVC on macOS, falling back to ours)
+          │   platform's VideoToolbox (macOS) or Media Foundation (Windows) H.264 / HEVC, falling back to ours)
           ▼
         media::MediaSource ──► frame cache (byte-budgeted LRU, shared)
           ▼
@@ -231,7 +231,8 @@ file ──► codecs (MP4/MOV, MKV, audio)        demux + decode, GOP-aware see
   hardware decoder can take precedence. The GOP cache never holds its lock while decoding.
 - **Hardware decoding.** The apps (desktop, CLI / headless MCP, bench) call
   `filmcraft_platform::register()` at startup, which on macOS registers a VideoToolbox factory for
-  `avcC` / `hvcC` streams (8 / 10-bit, 4:2:0 and 4:2:2; elsewhere it does nothing). The factory
+  `avcC` / `hvcC` streams (8 / 10-bit, 4:2:0 and 4:2:2; elsewhere it does nothing), and on Windows a
+  Media Foundation one (below). The factory
   declines (our decoder is used) when Settings ▸ Playback ▸ Hardware decoding is Off
   (`codecs::hw::set_hardware_decoding`, applied by the engine whenever preferences change), for
   formats it does not take, and when the OS cannot create a *hardware* session for the stream.
@@ -246,7 +247,18 @@ file ──► codecs (MP4/MOV, MKV, audio)        demux + decode, GOP-aware see
   `codecs::software_video_decoder` (built-in factories only), replays the samples since the last
   restart point and continues in software for that instance; it is logged and counted. A source
   already open keeps its decoder when the setting changes, until it is reopened. `perf.stats`
-  `decode.hardware` reports hardware vs software frames, sessions, declines and fallbacks.
+  `decode.hardware` reports hardware vs software frames, sessions, declines and fallbacks, and the
+  registered `backend`.
+  On Windows the decoder (`platform::media_foundation`) is a Direct3D-aware decoder MFT driven at
+  the level of single access units (no Source Reader: the demuxer above already delivers the
+  samples): the `avcC` / `hvcC` sample becomes Annex B, the MFT is given the process's Direct3D 11
+  video device through an `IMFDXGIDeviceManager` so it decodes with DXVA, and its NV12 (8-bit) /
+  P010 (10-bit) texture is read back through a staging texture (the one GPU to CPU copy) into
+  planar `Yuv8` / `Yuv16`. H.264 Baseline / Main / High and HEVC Main / Main 10, 4:2:0, progressive;
+  everything else, and any stream the GPU's DXVA decoder does not list, is declined. A decoder that
+  would hand back system-memory pictures (Microsoft's decoder MFTs do that when DXVA is not
+  available) fails the stream, so the hybrid continues with our decoder and Windows' software
+  decoding is never used in its place. Zero-copy (the texture straight to wgpu) is not done yet.
   Decoders run slices on rayon, so an export worker waiting inside a decode can pick up another
   frame of the same source. A request that finds the shared decoder busy decodes with a private
   decoder.

docs/performance.md

@@ -125,6 +125,53 @@ the draft plan's box decimation of the planes, cheap next to software decoding,
 CPU cost. Zero-copy upload of the decoded `CVPixelBuffer` into wgpu (no copy-out at all) is the
 next step (issue #30).
 
+## Results (HW2: Windows Media Foundation / Direct3D 11 hardware decoding, Off → Auto)
+
+Same binary (release build of the HW2 commit), Settings ▸ Playback ▸ Hardware decoding switched
+with `--hw off` / `--hw auto`, 2026-10-07: Intel Xeon E5-2680 v4 (14 cores / 28 threads, 2.4 GHz),
+NVIDIA GeForce RTX 5060 8 GB (driver 617.14), Windows 11 Pro 26200, idle machine. Two alternating
+rounds; the table gives the range of the two (decode: best of 3 within each). Reproduce with
+`cargo xtask bench --sections decode --only dec_h264 --repeat 3 --hw off|auto` (likewise `dec_hevc`)
+and `--sections playback --only "h264-2160 full" --hw off|auto`; set `FILMCRAFT_FFMPEG` if ffmpeg is not
+on `PATH` for the fixture generator. Pictures are identical either way (bit-exact parity tests,
+`crates/platform/tests/media_foundation.rs`). The decoder MFTs were Microsoft's H.264 decoder
+(`Microsoft H264 Video Decoder MFT`) and the `HEVCVideoExtension` decoder, both decoding with DXVA
+on the RTX 5060's NVDEC: `nvidia-smi dmon -s u` showed the decoder engine at 18–27 % during an
+Auto run and 0 % otherwise, and every picture came back as a Direct3D 11 texture.
+
+### Decode (every frame through the media stack)
+
+| codec | size | CPU ms/frame Off → **Auto** | fps Off → **Auto** |
+|---|---|---|---|
+| H.264 | 1080p | 77–80 → **3.3** | 162–166 → **347–353** |
+| H.264 | 2160p | 310–315 → **11.9** | 39–43 → **93** |
+| HEVC Main | 1080p | 69–73 → **3.5** | 89–93 → **307–308** |
+| HEVC Main | 2160p | 315–348 → **12–13** | 26–28 → **87–90** |
+| HEVC Main 10 | 2160p | 314–336 → **17.6** | 32 → **57** |
+
+Counters: every Auto row had `hw frames` = frames × 3 repeats (360 at 1080p, 216 at 2160p), 3
+sessions, 0 fallbacks, 0 declined; every Off row had 0 hardware frames and 0 sessions.
+
+What is left on the CPU is the readback: `examples/mfprobe.rs --time` splits a 2160p H.264 picture
+into 1.1 ms DXVA decode (feeding the MFT), 2.8 ms GPU to CPU copy (staging texture + map) and 7 ms
+CPU conversion (Annex B, biplanar to planar Y'CbCr); Main 10 is 1.4 / 4.5 / 10 ms (twice the bytes).
+A zero-copy path removes the last two.
+
+### Program-monitor playback (8 s, GPU path)
+
+| case | shown/dropped Off → **Auto** | CPU ms/frame Off → **Auto** |
+|---|---|---|
+| H.264 1080p Full | 192/0 → **192/0** | 86–89 → **8.5–8.8** |
+| H.264 2160p Full | 192/0 → **189–190/2–3** | 338–353 → **17–20** |
+| H.264 2160p 1/2 | 192/0 → **190/2** | 355–361 → **22** |
+| HEVC 2160p Full | 112–152/40–80 → **191–192/0–1** | 355–364 → **18** |
+
+This machine has 28 software-decoding threads, so Off keeps up with 4K H.264 (at 340 ms of CPU per
+frame, about 8 cores busy); Auto uses a twentieth of that CPU but drops 2–3 of 192 frames
+(readback latency at the start of the run). 4K HEVC is where Off drops frames and Auto does not.
+Draft playback (1/2 draft: 410–416 → 93–105 CPU ms/frame) is dominated by the draft plan's decimation,
+as on macOS, because the hardware ignores draft mode.
+
 ## Results (GPU1: blend modes on the GPU compositor, #30, before → after)
 
 Before = this change with the old whole-frame CPU fallback for non-Normal blend modes put back

docs/testing.md

@@ -66,7 +66,9 @@ are never linked or shipped ([AGENTS.md](../AGENTS.md) §2).
   the matrix.
 - Fixture sources are synthetic: `testsrc2`, `mandelbrot`, SMPTE bars, noise and fades, sine tones.
   H.264 and HEVC fixtures need ffmpeg built with libx264 and libx265. VideoToolbox fixtures are
-  generated only on macOS.
+  generated only on macOS. The Media Foundation parity tests (`crates/platform/tests/media_foundation.rs`,
+  Windows only) make their 640x360, 1080p and 2160p H.264 / HEVC / Main 10 fixtures with ffmpeg and skip
+  without a Direct3D 11 video device or the HEVC Video Extensions.
 
 ### Pass criteria per codec