G5: cargo xtask bench: render benchmark of generated scenarios (peak memory, CPU, wall) with results.jsonl and --compare

(cherry picked from commit 40b232d02069925e9c72677f52ee6f356495a736)

(cherry picked from commit a2d89bda11b7217c83c98ab681b8f9b4a334786d)
(cherry picked from commit 63951595222207b6bfd0ecde72f991628c6f12d9)

(cherry picked from commit 20c2bb0dcb)
This commit is contained in:
Elias authored and Elias committed 2026-10-12 19:18:32 +01:00
1 parent 470ea7a3af
commit e23b926efc
4 files changed
+994 -1

No files matched your search

+4
View File
@@ -249,6 +249,10 @@ Issues #41–#47 (all from the Linux AppImage 0.1.1, 5 October): panning the vie
- Benchmark projects at 1080p and 4K with many layers, heavy effects, long footage and nested
comps; record preview frame times, RAM preview fill and render times on reference machines.
Started 10 October 2026: `cargo xtask bench` renders generated 1080p / 4K scenarios (many
layers with common effects, nested precomps, footage) at Full and Half and records peak memory,
CPU and wall time per run, with `--compare` against an earlier run (docs/testing.md, "Render
benchmark").
- Compare with After Effects on the same machine where possible.
- Done when: benchmark numbers are tracked over time and regressions fail a check.
+24
View File
@@ -46,3 +46,27 @@ cargo run --release -p effectcraft-cli -- bench --n 10 --play 30
prints per-layer and per-effect timings for one frame, and playback timings with and without the
layer cache.
### Render benchmark (`cargo xtask bench`)
```sh
nice -n 10 cargo xtask bench --bin target/release/effectcraft-cli # or no --bin: one `--release` build into target/bench
nice -n 10 cargo xtask bench --quick --compare before.jsonl # 8 frames instead of 60, % change against an earlier run
cargo xtask bench --project F.ecproj --comp "Main" # any project's comp
```
measures memory, CPU and wall time of whole `effectcraft-cli` runs, so every optimisation has a
before and after number. It generates four deterministic scenarios into `target/bench/projects`
with engine commands (no committed media): `demo` (the built-in demo comp), `layers` (1080p,
100 solid, text and shape layers with Gaussian Blur, Glow, Levels, Curves, Drop Shadow and
keyframes), `precomps` (static and animated precomps nested three deep, each used several times)
and `footage` (a rendered PNG sequence and H.264 clip on several 1080p layers, plus a 4K still).
Each is opened (`open`), rendered at Full and Half (PNG sequence, deleted afterwards) and as one
frame. `/usr/bin/time -l` on macOS gives peak memory footprint, max RSS, user + sys CPU and
instructions retired; `/usr/bin/time -v` on Linux gives max RSS and CPU; elsewhere wall time only.
It prints a table and appends one JSON line per run to `target/bench/results.jsonl` (git rev,
date, load average, scenario, settings, metrics); `--compare OLD.jsonl` prints the % change against
the latest matching runs there, `--compare OLD --with NEW` compares two files without rendering.
The table notes the 1-minute load average: on a busy machine wall time is noise, so compare
peak memory and instructions retired, and rerun when the load differs a lot.