Merge pull request #1110 from eliasx45/upstream/render-bench

docs/gaps.md

@@ -249,6 +249,10 @@ Issues #41–#47 (all from the Linux AppImage 0.1.1, 5 October): panning the vie
 
 - Benchmark projects at 1080p and 4K with many layers, heavy effects, long footage and nested
   comps; record preview frame times, RAM preview fill and render times on reference machines.
+  Started 10 October 2026: `cargo xtask bench` renders generated 1080p / 4K scenarios (many
+  layers with common effects, nested precomps, footage) at Full and Half and records peak memory,
+  CPU and wall time per run, with `--compare` against an earlier run (docs/testing.md, "Render
+  benchmark").
 - Compare with After Effects on the same machine where possible.
 - Done when: benchmark numbers are tracked over time and regressions fail a check.
 

docs/testing.md

@@ -46,3 +46,27 @@ cargo run --release -p effectcraft-cli -- bench --n 10 --play 30
 
 prints per-layer and per-effect timings for one frame, and playback timings with and without the
 layer cache.
+
+### Render benchmark (`cargo xtask bench`)
+
+```sh
+nice -n 10 cargo xtask bench --bin target/release/effectcraft-cli   # or no --bin: one `--release` build into target/bench
+nice -n 10 cargo xtask bench --quick --compare before.jsonl        # 8 frames instead of 60, % change against an earlier run
+cargo xtask bench --project F.ecproj --comp "Main"                 # any project's comp
+```
+
+measures memory, CPU and wall time of whole `effectcraft-cli` runs, so every optimisation has a
+before and after number. It generates four deterministic scenarios into `target/bench/projects`
+with engine commands (no committed media): `demo` (the built-in demo comp), `layers` (1080p,
+100 solid, text and shape layers with Gaussian Blur, Glow, Levels, Curves, Drop Shadow and
+keyframes), `precomps` (static and animated precomps nested three deep, each used several times)
+and `footage` (a rendered PNG sequence and H.264 clip on several 1080p layers, plus a 4K still).
+Each is opened (`open`), rendered at Full and Half (PNG sequence, deleted afterwards) and as one
+frame. `/usr/bin/time -l` on macOS gives peak memory footprint, max RSS, user + sys CPU and
+instructions retired; `/usr/bin/time -v` on Linux gives max RSS and CPU; elsewhere wall time only.
+
+It prints a table and appends one JSON line per run to `target/bench/results.jsonl` (git rev,
+date, load average, scenario, settings, metrics); `--compare OLD.jsonl` prints the % change against
+the latest matching runs there, `--compare OLD --with NEW` compares two files without rendering.
+The table notes the 1-minute load average: on a busy machine wall time is noise, so compare
+peak memory and instructions retired, and rerun when the load differs a lot.

xtask/src/main.rs

@@ -10,7 +10,10 @@
 //!   `<file>.attribution` sidecar and an entry in `ATTRIBUTION.md` (AGENTS.md §1).
 //! - `ico <out.ico> <in.png>…`: pack PNGs into a Windows `.ico` (used by `packaging/icons.sh`).
 //! - `ci`: fmt check, clippy -D warnings, tests, layers, assets, wasm.
+//! - `bench [--bin CLI] [--quick] [--compare OLD.jsonl] …`: render benchmark of generated scenarios
+//!   (memory, CPU, wall time) into `target/bench/results.jsonl` (`bench.rs`, docs/testing.md).
 
+mod bench;
 mod ico;
 mod version;
 
@@ -420,6 +423,7 @@ fn main() -> ExitCode {
         "assets" => assets(),
         "ci" => ci(),
         "web" => web(&std::env::args().skip(2).collect::<Vec<_>>()),
+        "bench" => bench::run(&root(), &std::env::args().skip(2).collect::<Vec<_>>()),
         "ico" => {
             let rest: Vec<String> = std::env::args().skip(2).collect();
             ico::run(&rest.iter().map(String::as_str).collect::<Vec<_>>())
@@ -428,7 +432,7 @@ fn main() -> ExitCode {
             let rest: Vec<String> = std::env::args().skip(2).collect();
             version::run(&root(), &rest.iter().map(String::as_str).collect::<Vec<_>>())
         }
-        _ => Err("usage: cargo xtask <layers|assets|wasm|web [--dev] [--serve PORT]|ico OUT.ico IN.png…|version [set X.Y.Z]|ci>".into()),
+        _ => Err("usage: cargo xtask <layers|assets|wasm|web [--dev] [--serve PORT]|ico OUT.ico IN.png…|version [set X.Y.Z]|ci|bench [--bin CLI] [--quick] [--project F --comp C] [--compare OLD.jsonl] (--help)>".into()),
     };
     match r {
         Ok(()) => ExitCode::SUCCESS,