mirror of
https://github.com/storytold/effectcraft.git
synced 2026-10-14 19:27:05 -04:00
Elias
e23b926efc
(cherry picked from commit 40b232d02069925e9c72677f52ee6f356495a736)
(cherry picked from commit a2d89bda11b7217c83c98ab681b8f9b4a334786d)
(cherry picked from commit 63951595222207b6bfd0ecde72f991628c6f12d9)
(cherry picked from commit 20c2bb0dcb)
73 lines
4.0 KiB
Markdown
73 lines
4.0 KiB
Markdown
# Testing
|
|
|
|
```sh
|
|
cargo test --workspace # everything
|
|
cargo xtask ci # fmt, clippy -D warnings, tests, layers, assets, wasm
|
|
```
|
|
|
|
## What the tests cover
|
|
|
|
- **Engine commands** (`crates/engine/src/tests*.rs`): every command family, including undo and
|
|
redo, and that every menu entry resolves to a registered command.
|
|
- **Rendering** (`crates/render`): pixel checks on small compositions, for blend modes, masks,
|
|
mattes, time remapping, 3D projection, lighting and shadows. Fast paths (the layer cache, the
|
|
affine compositor, blurs, motion blur accumulation) are checked against their reference
|
|
implementations.
|
|
- **Effects** (`crates/effects`): each effect has a determinism test and at least one behaviour
|
|
test (identity at neutral settings, known pixel results, or simulations giving the same frame
|
|
whether you seek straight to it or play up to it). A registry-wide test checks that effects
|
|
which read the clock are declared time-dependent. `crates/effects/tests/sim_golden.rs` pins
|
|
exact pixel hashes of the simulation effects the GPU shares plans with (CC Rainfall … Card
|
|
Wipe) at several settings, times, bit depths and resolutions, so any change to their CPU output
|
|
fails (re-pin with `SIM_GOLDEN_PRINT=1` after an intended change; pinned for aarch64 macOS,
|
|
with the hashes another macOS release's libm gives for unchanged code in `LIBM_VARIANTS`).
|
|
- **Export** (`crates/export/tests`): every format is encoded and decoded back, checking frame
|
|
count, size and pixels. When `ffmpeg`/`ffprobe` are installed they are used as an outside
|
|
check; they are never linked or shipped.
|
|
- **Automation** (`crates/automation`, `apps/effectcraft-cli/tests`): MCP protocol round trips and
|
|
the command-line tool's JSON output.
|
|
- **Interface** (`crates/ui-egui`): headless egui_kittest tests for panels and dialogs.
|
|
|
|
## Looking at the interface without a window
|
|
|
|
```sh
|
|
cargo run -p effectcraft-ui-egui --example snapshot -- --out ui.png \
|
|
--step '{"method":"engine.execute","params":{"command":"layer.select","params":{"layers":["#2"]}}}'
|
|
```
|
|
|
|
Each `--step` is a control-channel request; `{"method":"snap","params":{"path":"x.png"}}` writes an
|
|
intermediate image.
|
|
|
|
## Benchmarks
|
|
|
|
```sh
|
|
cargo run --release -p effectcraft-cli -- bench --n 10 --play 30
|
|
```
|
|
|
|
prints per-layer and per-effect timings for one frame, and playback timings with and without the
|
|
layer cache.
|
|
|
|
### Render benchmark (`cargo xtask bench`)
|
|
|
|
```sh
|
|
nice -n 10 cargo xtask bench --bin target/release/effectcraft-cli # or no --bin: one `--release` build into target/bench
|
|
nice -n 10 cargo xtask bench --quick --compare before.jsonl # 8 frames instead of 60, % change against an earlier run
|
|
cargo xtask bench --project F.ecproj --comp "Main" # any project's comp
|
|
```
|
|
|
|
measures memory, CPU and wall time of whole `effectcraft-cli` runs, so every optimisation has a
|
|
before and after number. It generates four deterministic scenarios into `target/bench/projects`
|
|
with engine commands (no committed media): `demo` (the built-in demo comp), `layers` (1080p,
|
|
100 solid, text and shape layers with Gaussian Blur, Glow, Levels, Curves, Drop Shadow and
|
|
keyframes), `precomps` (static and animated precomps nested three deep, each used several times)
|
|
and `footage` (a rendered PNG sequence and H.264 clip on several 1080p layers, plus a 4K still).
|
|
Each is opened (`open`), rendered at Full and Half (PNG sequence, deleted afterwards) and as one
|
|
frame. `/usr/bin/time -l` on macOS gives peak memory footprint, max RSS, user + sys CPU and
|
|
instructions retired; `/usr/bin/time -v` on Linux gives max RSS and CPU; elsewhere wall time only.
|
|
|
|
It prints a table and appends one JSON line per run to `target/bench/results.jsonl` (git rev,
|
|
date, load average, scenario, settings, metrics); `--compare OLD.jsonl` prints the % change against
|
|
the latest matching runs there, `--compare OLD --with NEW` compares two files without rendering.
|
|
The table notes the 1-minute load average: on a busy machine wall time is noise, so compare
|
|
peak memory and instructions retired, and rerun when the load differs a lot.
|