Merge main (gap assessment) into M13.35: keep parity numbers, record G7's decision and evidence

Co-Authored-By: Claude Opus 5.5 <[email protected]>

docs/gaps.md

@@ -0,0 +1,134 @@
+# Where EffectCraft falls short
+
+An honest assessment of how far EffectCraft is from being a real replacement for After Effects,
+and the work that closes the gap. This document is meant for contributors and agents choosing
+what to work on. The [ROADMAP](../ROADMAP.md) summarises it; [parity.md](parity.md) is the
+feature-by-feature checklist it builds on.
+
+*Assessed 5 October 2026. Estimates marked "≈" are judgements from the evidence listed, not
+measurements. Update this file when the evidence changes.*
+
+## Two different questions
+
+[parity.md](parity.md) reports **≈ 99%**. That number answers one question: *does each After
+Effects feature exist?* It scores a 92-item catalogue we wrote ourselves, graded by the same agents
+that built the features, with partial features counting half. As a measure of breadth it is fair,
+and the breadth is real: every one of After Effects' effects exists by name, along with the
+menus, panels, 3D, tracking, expressions, scripting and export.
+
+It does not answer the question users care about: *can someone who uses After Effects for a living
+do real work in EffectCraft?* Nobody has measured that yet. Our best estimate is **≈ 30–50%**,
+limited mainly by four things: projects we can't open, behaviour nobody has checked against After
+Effects, reliability on platforms other than macOS, and the third-party plug-ins professional
+projects depend on.
+
+The checklist is also internally inconsistent: parity.md's per-area table still shows Interface
+75%, Project 68%, Animation 70% and Paint 0% next to the 99% headline. Some rows are stale; none
+of them have been reconciled.
+
+## By dimension
+
+| Dimension | Estimate | Evidence |
+|---|---|---|
+| Breadth of features | ≈ 95%+ | All 306 effects, the After Effects menus and panels, Classic and Advanced 3D, tracking, expressions, scripting, render queue (parity.md) |
+| Behaves like After Effects | unmeasured, ≈ 60–80% | One outside contributor found four bugs in features marked done within a day (PRs #6–#10): Hold keyframes eased the motion into them, `keyInSpatialTangent` / `keyOutSpatialTangent` had the wrong names, a zero frame rate crashed, Find and Enter Full Screen shared Ctrl+F off macOS. There are about 10 After Effects reference captures in total. No test compares our renders or property values with After Effects itself; the tests check what the agents believed After Effects does |
+| Opening existing After Effects work | ≈ 0% | `.aep` / `.aepx` projects cannot be opened. Third-party After Effects plug-ins cannot run. Expressions and the scripting object model are strong, so scripts and expressions carry over |
+| Stability | improving | 23 never-crash PRs landed on 4 October; [AGENTS.md](../AGENTS.md) "Never crash" now binds every crate. On 5 October `cargo xtask ci` failed on main under Rust 1.99's clippy (a fix is in progress) |
+| Real-user experience | ≈ 70%, uneven by platform | Issues #41–#47 (all from the Linux AppImage, 5 October): panning the viewer snaps back, panels can't be resized or rearranged, drag-and-drop and double-click import don't work, layer rename gets stuck, the Layer Settings arrow does nothing, the Project panel clips, the Wayland window icon is generic. parity.md counts docking as done. Most checking happens on macOS. No localisation, no accessibility work |
+| Performance | unknown against After Effects | Internal numbers only (e.g. Advanced 3D 290 ms/frame at 1080p on the GPU, an M4 Pro under load). Nothing benchmarked against After Effects; no large real projects (4K footage, hundreds of layers) tested |
+| Media formats | ≈ 80% | H.264, ProRes, HEVC, AV1, image sequences and audio exist. The new HEVC / AV1 encoders have no B-frames, multi-reference or SAO / CDEF, so files are larger than from mature encoders. Camera formats (BRAW, R3D, ProRes RAW, variable-frame-rate phone video) are unverified |
+| AI-assisted tools | ≈ 50% | Face tracking uses a classical shape-model fitter. Roto Brush 2.0 / 3.0 can use MobileSAM since M13.35 (pure-Rust inference, an optional download): on our synthetic moving-disc test it scores IoU 0.989 on the base frame and ≥ 0.980 over 20 propagated frames (classic: 0.973), but nothing compares it with After Effects yet |
+| Maturity | early | First commit 1 October 2026. ≈ 285,000 lines of almost entirely agent-written Rust, ≈ 2,050 tests, 8 external issue reports so far. After Effects has around 30 years of edge cases behind it |
+
+## Where we're going: workstreams in priority order
+
+Each workstream lists what "done" means, so progress can be measured rather than self-graded.
+
+### G1. Measure fidelity against After Effects
+
+The most important missing piece: it turns every other estimate here into a measurement.
+
+- Build a corpus of original test projects (our own content, no Adobe assets), one or more per
+  feature id in `plan/aftereffects/feature-catalog.md`: keyframe interpolation of every kind,
+  blend modes, track mattes, each effect at default and at non-default settings, text animators,
+  expressions, 3D, motion blur, time remapping.
+- Drive After Effects through ExtendScript (see CLAUDE.md) to record property values at sampled
+  times and render reference frames; render the same projects in EffectCraft headless.
+- Compare values exactly and frames with a perceptual metric; publish a per-feature fidelity
+  score and a fidelity column in parity.md.
+- Clean room: After Effects output stays local in `plan/aftereffects/ref/` (gitignored). Commit
+  only our projects, the harness and the scores, never After Effects frames.
+- Done when: every P0 and P1 feature has at least one corpus project, scores are reproducible from
+  one command, and parity.md reports breadth and measured fidelity side by side.
+
+### G2. Real-user reliability on every platform
+
+- Fix the open user issues (#41–#47), each with a regression test.
+- Run the headless snapshot and control-channel checks on Linux (X11 and Wayland) and Windows, not
+  only macOS. Interactions that only fail with real input (docking drags, viewer pan, drag-and-drop
+  import, inline rename) need scripted input tests.
+- Done when: no open bug blocks a basic workflow (import, arrange, animate, preview, render) on
+  any of the three desktop platforms, and every bug users report gets triaged within a day.
+
+### G3. Stability and a green main
+
+- Keep `cargo xtask ci` green on the current stable toolchain. A toolchain release that breaks the
+  gate is a P0.
+- Fuzz the inputs we don't control: project files, imported media and documents, expressions,
+  scripts, control-channel and MCP requests. Every crash found becomes a regression test
+  (AGENTS.md "Never crash").
+- Done when: the gate is green on stable and the fuzzers run regularly without new crashes.
+
+### G4. Open After Effects projects
+
+- `.aep` / `.aepx` import is the largest single barrier to switching. It needs an **owner
+  decision** on clean-room scope (whether and how the format may be studied), like the open
+  `.prproj` question in `plan/STATUS.md`.
+- Also in this area: relative footage paths and relinking when a project moves (parity.md, Project).
+- Done when: the decision is recorded and, if approved, a corpus of real-world-shaped projects
+  opens with its comps, layers, keyframes, effects and expressions intact.
+
+### G5. Performance at real-world scale
+
+- Benchmark projects at 1080p and 4K with many layers, heavy effects, long footage and nested
+  comps; record preview frame times, RAM preview fill and render times on reference machines.
+- Compare with After Effects on the same machine where possible.
+- Done when: benchmark numbers are tracked over time and regressions fail a check.
+
+### G6. Media depth
+
+- Encoder efficiency: B-frames and multi-reference for HEVC / AV1, SAO / CDEF / restoration, Opus
+  FEC / DTX.
+- Verify decode of the camera and phone formats people actually bring, including variable frame
+  rate; document what is not supported.
+
+### G7. Learned models for Roto Brush and face tracking
+
+- Decided on 5 October 2026 for Roto Brush: licensed weights with pure-Rust inference, if the
+  impact on the system is low, in a composable, swappable module. Done in M13.35: the
+  `effectcraft-segment` crate (a `MaskModel` interface, a registry that only accepts
+  open-source licences, MobileSAM under Apache-2.0), weights downloaded on demand and verified,
+  Settings ▸ Roto Brush to choose. Next: measure it against After Effects (G1); faster encoders
+  (GPU) for long shots.
+- Face tracking: still an owner decision (a learned detector, or the classical fitter as final).
+
+### G8. Plug-in ecosystem
+
+- After Effects SDK plug-ins cannot run in EffectCraft. Our own WebAssembly plug-in API exists
+  ([plugins.md](plugins.md)). Grow it: documentation, examples, and original effects that cover
+  what the most common third-party plug-ins are used for (particles, glows, 3D objects, sabers).
+
+### G9. Reach
+
+- Localisation of the interface, accessibility (keyboard navigation, screen readers, contrast),
+  user documentation and tutorials.
+
+## For agents choosing work
+
+1. Read `plan/STATUS.md` for owner blockers and running work, then this file.
+2. Prefer G1–G3 over new features. A feature that exists but behaves differently from After
+   Effects is not done.
+3. When you fix a behaviour bug in a feature parity.md marks as done, add it to the evidence
+   above. That is how this assessment stays honest.
+4. Don't raise parity.md's numbers without evidence: a G1 score, a user-facing check on more than
+   one platform, or both.

docs/parity.md

@@ -2,17 +2,25 @@
 
 How close EffectCraft is to After Effects 2026, feature by feature, and how much work is left.
 
-## Current status (audit at commit `d39c0e8`, 4 October 2026; updated 5 October for M3.9–M4.12 and M13.35)
+> **Read this as a measure of breadth, not of real-world readiness.** It checks whether each
+> feature exists, using our own catalogue, graded by the agents that built it. It does not measure
+> whether the feature behaves like After Effects (nothing compares the two yet), whether After
+> Effects projects open (they don't), or how the app holds up for real users on every platform.
+> Those gaps, and the work that closes them, are in [gaps.md](gaps.md). Don't raise the numbers
+> here without the evidence gaps.md asks for. The per-area table further down has stale rows that
+> have not been reconciled with the headline.
+
+## Current status (audit at commit `d39c0e8`, 4 October 2026; updated 5 October for M3.9–M4.12)
 
 | Measure | Value |
 |---|---|
-| **Feature parity, weighted by tier** (P0 ×3, P1 ×2, P2 ×1) | **≈ 99.4%** counting every partial feature as half done; ≈ 99.9% using per-feature fractions |
-| Unweighted | ≈ 98.9% (half-credit) / 99.9% (fractions) |
-| P0 / P1 / P2 | 100% / 98% / 95% (half-credit); 100% / 99.8% / 99% (fractions) |
-| Features done / partial / missing | 90 / 2 / 0 of 92 (MSK-4 Roto Brush done in M13.35) |
+| **Feature parity, weighted by tier** (P0 ×3, P1 ×2, P2 ×1) | **≈ 99%** counting every partial feature as half done; ≈ 99.8% using per-feature fractions |
+| Unweighted | ≈ 98.4% (half-credit) / 99.6% (fractions) |
+| P0 / P1 / P2 | 100% / 98% / 90% (half-credit); 100% / 99.7% / 97.5% (fractions) |
+| Features done / partial / missing | 89 / 3 / 0 of 92 |
 | **Effects** | **306** effects, every one implemented in full ([effects.md](effects.md)); 280 run on the GPU |
 | Disabled menu entries left | 1, on purpose: Import ▸ Vanishing Point (.vpe), whose format has no public specification |
-| Remaining work | ≈ 8.5 agent-hours at the audit (≈ 26 on the conservative audit scale), before M13.35 landed Roto Brush's trained model; the learned face tracker is the one learned-model item left |
+| Remaining work | ≈ 8.5 agent-hours at the pace measured so far (≈ 26 on the conservative audit scale); ≈ 3.5 without the learned-model items |
 | **Wall-clock estimate** | **≈ 2–3 hours** with five agents in parallel; ≈ 1 hour without the learned-model items |
 
 Partial features:
@@ -20,6 +28,7 @@ Partial features:
 | id | Tier | Done | What is missing |
 |---|---|---|---|
 | EFF-5 GPU effects | P1 | 0.98 | Fractal (its escape iteration needs f64: WGSL has none and Metal's fast math defeats double-f32 emulation) and the analysis / tool effects (Camera-Shake Deblur, Detail-preserving Upscale, Puppet, Rolling Shutter Repair, Warp Stabilizer, Mocha Shape, Roto Brush, Match Grain, Paint, Camera Tracker, Face Tracker) render on the CPU. M13.28 moved the remaining pixel effects (CC blur / glass family, Color Link, Cineon, HDR, Inner/Outer Key, Basic 3D…) and the geometry and audio generators (Lightning, Advanced Lightning, Beam, Lens Flare, Radio Waves, Vegas, Stroke, Scribble, Write-on, Paint Bucket, Eyedropper Fill, CC Glue Gun, CC Threads, Audio Spectrum / Waveform, Basic / Path Text) onto the GPU; M13.29 the particle effects (CC Particle World / Systems II, Particle Playground, CC Ball Action, CC Pixel Polly, CC Scatterize) and Curl Noise, and removed the Shatter wireframe, Foam extras, Timewarp matte, strong Warp Fisheye / Twist and custom `.ocio` fallbacks. The 13 audio filters stay on the CPU by design |
+| MSK-4 Roto Brush | P2 | 0.85 | Roto Brush 2.0 / 3.0 can use a trained model since M13.35 (MobileSAM, optional download); its quality has not been compared with After Effects (G1), so the score stands |
 | TRK-3 Face tracking | P2 | 0.9 | Classical fitter: weak on profile and occluded faces; Rolling Shutter Ripple is approximated |
 
 Since this audit: the project operations (M3.9–M3.14) and preview (M4.5–M4.12) passes fixed
@@ -433,7 +442,8 @@ stretch) and shown within its In–Out range. Hovering names them ("beat (marker
 double-click opens the nested comp at the marker. Agents read the same list with
 `markers.nested {layer}`.
 
-Since (M13.35): Roto Brush can use a trained segmentation model; MSK-4 is done.
+Since (M13.35): Roto Brush can use a trained segmentation model. MSK-4 keeps its score until G1
+measures it against After Effects ([gaps.md](gaps.md)).
 - The model sits behind a swappable interface: `effectcraft_segment::MaskModel` plus a registry
   where every entry must have an open-source licence compatible with ours, a source URL, a size
   and a SHA-256.