effectcraft/docs/web.md

EffectCraft on the web

apps/effectcraft-web runs the same engine and egui UI as the desktop app in the browser: the workspace compiled to wasm32-unknown-unknown, started by eframe's web runner on WebGPU (WebGL2 fallback). It is a static site: one .wasm, its wasm-bindgen JavaScript glue, index.html, a few small scripts (worker, audio worklet, service worker), a web manifest and icons. Nothing is uploaded anywhere: projects, media, settings and renders stay in the browser (its private storage) and on the user's machine.

Build and run

Tool	Version	Install
Rust target	wasm32-unknown-unknown	rustup target add wasm32-unknown-unknown
wasm-bindgen CLI	exactly the wasm-bindgen crate version (0.2.129)	cargo install wasm-bindgen-cli --version 0.2.129 --locked
wasm-opt (optional)	any recent binaryen	brew install binaryen; used by cargo xtask web when found

cargo xtask web                 # release build → <target>/web/dist
cargo xtask web --dev           # unoptimised build (faster to compile, slow to run)
cargo xtask web --serve 8765    # build, then serve dist on http://127.0.0.1:8765/

<target> is $CARGO_TARGET_DIR or target. dist holds index.html, effectcraft_web.js + effectcraft_web_bg.wasm (+ snippets/, the browser glue js/host.js), worker.js, audio-worklet.js, sw.js (its precache list and version filled in by the build), manifest.webmanifest, favicon.svg and icon-256.png / icon-512.png (copied from assets/app-icon). --serve is a tiny localhost-only static server that sends Cross-Origin-Opener-Policy / Cross-Origin-Embedder-Policy, so the page is cross-origin isolated like a production deployment should be. Any static server works (python3 -m http.server -d target/web/dist 8765); it must serve .wasm as application/wasm. Service workers need a secure context: https://, or http://localhost / 127.0.0.1.

The release .wasm is about 38 MB before wasm-opt (code: the effects, codecs, the expression engine and the UI; data, mostly the bundled fonts; function names for readable panics). Served compressed it is a fraction of that, and the service worker caches it after the first visit.

On other targets the web crate holds only its portable storage model (store.rs, unit-tested by cargo test --workspace); cargo xtask wasm (part of cargo xtask ci) checks the whole crate, with every L0–L4 crate and effectcraft-ui-egui, for wasm32-unknown-unknown.

URL flags: ?empty (a blank project; the session is not restored or recorded), ?demo (the demo project instead of the last session), ?home (show the start screen), ?storage=indexeddb / ?storage=memory (force a storage backend), ?noworkers (render and analyse on the page's thread), ?nosw (don't register the service worker).

How the desktop pieces map to the browser

Desktop	Web
frame render thread pool (ui-egui/src/frames.rs)	Frames::pump: queued frames render on the UI thread after each egui frame, most important first (the viewer's frame, then prefetch), within 40 ms (24 ms while playing)
Render Queue on a background thread	a Web Worker running a second engine instance (below); progress streams back, the page never waits
tracker / mask tracker / Warp Stabilizer / Roto Brush Freeze threads	the same workers; the analysed property group comes back as one undo step
rayon parallel loops (raster, effects, export batches)	rayon's global pool falls back to the calling thread when threads are unavailable: same code, serial
std::time::Instant / SystemTime (panic on wasm32)	web-time (re-exports std::time on native)
std::fs project reads/writes (FsServices)	files::WebServices: the virtual file table, persisted (below); saving also downloads the .ecproj
config directory: settings, shortcut presets, recent projects, the recovery sentinel	store::WebConfig in browser storage
auto-save folder	/EffectCraft Auto-Save/ in browser storage (ConfigStore::files)
media reads (MediaPool, probe)	files::WebImporter: bytes from the file table, handed to MediaPool::add_bytes / probe_bytes
export writes (effectcraft-export)	the job's sink (effectcraft_host::FileExporter { sink }): files land in the table and download after the render; several files (an image sequence) download as one stored .zip
rfd file dialogs	<input type=file> for File ▸ Open / Import; drop files anywhere on the page (.ecproj opens, everything else imports); "Save As" / "Output To" pick a download name
system fonts	not scanned; the bundled fonts (Inter, Noto Serif, JetBrains Mono) are always there
TCP control channel / MCP	window.effectcraft (below)
cpal audio output	Web Audio (below)
GPU compositor on the desktop's wgpu device	the same compositor on eframe's WebGPU device (below)

Persistence

Everything that lives in files on the desktop lives in the browser's Origin Private File System (OPFS); where OPFS can't be written from the page (older Safari), IndexedDB; with neither (some private windows), memory only. effectcraft.info().storage.backend says which. The page asks for persistent storage (navigator.storage.persist()), which browsers grant to installed apps, so the store isn't evicted under storage pressure.

The engine's storage traits are synchronous and the browser's are not, so the app works on an in-memory mirror (store.rs): every stored entry is read before the app starts, reads and writes hit the mirror, and changes flush in the background (one write per changed key, in order; OPFS writes are atomic: a swap file replaced on close). Keys:

Key	What
config/prefs.json, config/shortcuts.json	Settings (with File ▸ Open Recent's list) and keyboard shortcut presets
config/session.lock	the crash-recovery sentinel
config/session.ecproj, config/session.json	the session snapshot: the open project and editor state (active comp, time, selection), written at most once a second while they change
files/<path>	the file table: imported media, projects saved with File ▸ Save / Save As or saveToBrowser, auto-saves

On the next visit the app reopens the snapshot (unsaved changes stay unsaved: the project is dirty), registers every stored media file with the media pool so its footage decodes, and File ▸ Open Recent reopens saved projects from storage. Without a snapshot (first visit) it opens the demo project. Render outputs are not stored: they download.

Background jobs: Web Workers

wasm threads need shared memory, which needs a nightly build-std toolchain, so this build is single-threaded per instance. Long jobs run in Web Workers instead, each with its own instance of the same module (the page compiles it once and posts the WebAssembly.Module, so nothing is compiled twice) and its own memory (crates/engine/src/offload.rs, src/worker.rs):

1. The session's Offload serializes a WorkerRequest: the project JSON, the footage paths it reads, and the job (render with the resolved queue items and output paths; warp, track, maskTrack, rotoFreeze with the target and analysis parameters).
2. js/host.js takes an idle worker from its pool (or starts one), sends it the footage bytes it doesn't have yet, then the request.
3. The worker runs the job on a plain engine session, blocking, and posts WorkerReplys: render progress (at most every 100 ms), item started/finished, rendered files (transferred), analysis progress, the analysed property group, failed, done.
4. The page applies them every frame: Session::poll_render updates the queue items and the progress bar, Session::poll_offload writes an analysis result as one undo step (the effect / tracker / mask group is replaced; other edits made meanwhile are kept) and the downloads start.

Stopping a job terminates its worker (a worker can't be interrupted mid-frame): the item being rendered becomes "User Stopped", an analysis writes nothing. renderQueue.render {wait: true} (the default for agents) and analysis commands with wait: true still run on the page's thread and return when done. Roto Brush propagation fills the session's segmentation cache rather than the project, so it runs on the page's thread too, as do viewer frames (Frames::pump).

Audio

Audio preview goes through Web Audio: an AudioWorklet (audio-worklet.js; a ScriptProcessorNode where worklets are unavailable) plays interleaved stereo blocks. The mixdown is the desktop's (AudioPlayback, mix_comp): without a feeder thread, AudioPlayback::pump mixes ahead on every UI frame (≈ 350 ms queued) and the output device takes what keeps it ≈ 150 ms ahead of the playhead. A/V sync follows AudioContext.currentTime: the playback clock is the frames played by the context clock minus its output latency, and the viewer shows the frame under the audible sample, as on the desktop. The Audio panel's meters read the same feed.

Browsers start an AudioContext suspended until the user interacts with the page; the app resumes it on the first pointer or key event, so audio plays from the first preview after any click or key press.

GPU

eframe starts on WebGPU where the browser has it (WebGL2 otherwise), and the GPU compositor (effectcraft-gpu) runs on the same device: viewer frames are composited by compute shaders and drawn straight from their texture, with no readback. Steps that need the CPU mid-render (3D runs, adjustment layers, CPU-only effects) fall back to the CPU renderer for that frame, because the page's thread can't wait for a GPU readback. The Info panel's pixel readout and the eyedroppers read GPU frames back asynchronously (Gpu::read_display_async: the pixels arrive a frame later). On WebGL2, or without a usable adapter, everything renders on the CPU. effectcraft.info().gpu reports {compositor, viewerOnGpu}.

Offline and install (PWA)

manifest.webmanifest (name, icons, standalone display) makes the app installable; sw.js precaches every file of the build into a cache named after the build's hash, serves them cache-first (the cached responses keep the server's COOP/COEP headers, so the page stays cross-origin isolated offline), and drops older caches when a new build activates. After the first visit the app loads without a network.

window.effectcraft: the agent / test API

The control channel of the desktop app (docs/control-protocol.md) as promises. Results resolve with the method's result, errors reject with the message.

Call
effectcraft.request(method, params)	any control method: ui.inspect, ui.click, ui.key, ui.playback, ui.menu.invoke, render.frame…
effectcraft.execute(command, params)	engine.execute: an engine command (same ids and params as MCP / effectcraft-cli)
effectcraft.commands() / effectcraft.inspect()	engine.commands / ui.inspect
effectcraft.renderFrame(params)	render.frame with base64: true: {comp, time, width, height, png}
effectcraft.screenshot(params)	ui.screenshot; without path the PNG comes back inline as png (base64)
effectcraft.addFile(fileOrUrl, name?)	put a File/Blob (or fetched URL) into the file table (stored) and open (.ecproj) or import it; resolves with its path
effectcraft.files() / effectcraft.readFile(path)	the file table [{path, size}] / a file's bytes (Uint8Array), e.g. a render
effectcraft.saveToBrowser(path?)	save the project to browser storage without downloading it (default: its path, or /<name>.ecproj); resolves with {path, bytes}
effectcraft.listStored()	{backend, usage, quota, persisted, pending, files: [{path, size, modified}], config: [name]}
effectcraft.removeStored(path)	delete a stored file
effectcraft.flush()	resolves once every change is written to browser storage
effectcraft.info()	graphics backend, gpu, storage, audio ({state, sampleRate, backend, posted, played, underruns}), workers, restored, webgpu, serviceWorker, version, crossOriginIsolated, load timings

window.effectcraftLoad holds {wasmMs, readyMs} (module fetch + compile, and until the app runs).

await effectcraft.execute("layer.newSolid", {color: "#ff8800"});
await effectcraft.request("ui.menu.invoke", {id: "renderQueue.add", params: {format: "gif", resolution: 0.25}});
await effectcraft.execute("renderQueue.render", {wait: false});  // renders in a worker; the GIF downloads
await effectcraft.saveToBrowser("/my-project.ecproj");            // File ▸ Open Recent has it after a reload

Browser test

apps/effectcraft-web/tests/smoke.mjs drives headless Chrome over the DevTools protocol (Node ≥ 22, no npm packages): load, the GPU path (WebGPU → GPU compositor, viewer on the GPU), the demo comp in the viewer, render.frame, Render Queue GIF and PNG sequence (.zip) downloads, a background render in a worker while measuring the page's event-loop gaps (must stay under 400 ms) and its progress, the AudioContext starting on a (simulated) key press and a tone playing through it with the meters moving, Save As and re-opening the saved project, persistence across a reload (session snapshot, a stored project, imported media, a setting, Open Recent) and an offline reload through the service worker. It writes screenshots and report.json:

cargo xtask web --serve 8765 &
node apps/effectcraft-web/tests/smoke.mjs --url http://127.0.0.1:8765/ --out target/web/smoke

Native unit tests cover the storage model (apps/effectcraft-web/src/store.rs: write coalescing, file table, ConfigStore / FileOps, auto-save and crash recovery through the browser store) and the worker protocol (crates/engine/src/offload.rs: serde round trips of every request and reply, a render through an in-process offload).

Gaps

• Viewer frames, RAM preview and Roto Brush propagation still render on the page's thread (one frame at a time between UI frames).
• GPU effects run only in the desktop GPU path: in the browser, effects run on the CPU (inside workers for renders) because the page's thread can't wait for mid-render readbacks.
• Cancelling an analysis drops its partial result (the worker is terminated).
• Stored media count against the origin's storage quota; effectcraft.listStored() shows usage, removeStored frees it (there is no storage manager in the UI yet).
• engine.execute rejects the spread Render Settings / Output Module keys of renderQueue.add (resolution, timeSpan…) through its strict parameter check; use ui.menu.invoke as above.