M0.13: web render workers — pipeline renders off the main thread

Renders run in dedicated module Web Workers, each a second instance of the app's wasm module
(index.html compiles it once and posts the WebAssembly.Module to worker.js). A job crosses as
JSON (WireJob: source, settings, request); the worker reads originals and cached thumbnails from
browser storage itself, keeps decoded sources and per-view stage caches, and transfers RGBA back.
Jobs follow the worker that decoded the photo; the loupe waits for it. No wasm threads or
SharedArrayBuffer needed; falls back to inline rendering if no worker starts (?workers=0).

Thumbnail disk cache on the web: thumbs/<key>.jpg in OPFS/IndexedDB with an index on the main
thread, pruned to the desktop budget (2 GB → 80 %, LRU).
ui-egui: RenderOffload hook in the Renderer (queue/priorities/dedup stay in the UI).
engine: RenderJob.origin + MediaCache::origin_ref; preview: encode_jpeg/decode_jpeg public.
xtask web: copies worker.js; --serve sends COOP/COEP/CORP (documented in docs/web.md).

Headless Chrome, cold load + grid scroll, 24 demo photos: main-thread busy 12.4 s → 0.7–1.1 s,
long tasks 23 (max 1.1 s) → 0; grid filled in 6.3–8.4 s vs 13–16 s; slider job 4 ms.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Brandon ThomasandClaude Opus 5.5 committed 2026-10-01 08:36:15 -04:00
1 parent 961cf4afd2
commit 8f2058a124
12 files changed
+986 -50

No files matched your search

+60 -16
View File
@@ -18,42 +18,86 @@ it's missing or a different version.
Build, then serve:
```sh
cargo xtask web # → <target>/web/{index.html, lightcraft_web.js, lightcraft_web_bg.wasm}
cargo xtask web # → <target>/web/{index.html, worker.js, lightcraft_web.js, lightcraft_web_bg.wasm}
cargo xtask web --serve # build, then serve on http://127.0.0.1:8080/ (or `--serve 9000`)
cargo xtask web --dev # unoptimized build with debug info (faster to compile, slow to run)
```
`<target>` is `target/` unless `CARGO_TARGET_DIR` is set. Any static HTTP server works, as long as
it serves `.wasm` as `application/wasm`. The bundle can't be opened from `file://`. If `wasm-opt`
it serves `.wasm` as `application/wasm`. The bundle can't be opened from `file://` (module
workers and OPFS need an HTTP(S) origin; `localhost`/`127.0.0.1` count as secure). If `wasm-opt`
(binaryen) is on `PATH`, the release build also runs it over the module.
The release build uses the `web` Cargo profile (release + thin LTO, no debug info).
## Deploying: headers
`cargo xtask web --serve` sends these on every response, and a production server should too:
```
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
Cross-Origin-Resource-Policy: same-origin
```
COOP + COEP make the page *cross-origin isolated* (`crossOriginIsolated === true`). That is
**required** for `SharedArrayBuffer`, i.e. for a future wasm-threads build (`+atomics`, nightly
`build-std`), and gives full-resolution `performance.now()`. The current render workers (below)
don't share memory, so the app also runs without them; with COEP on, every subresource must be
same-origin or send CORP/CORS headers (the bundle has no third-party resources).
## What works
- **Demo library on load.** The procedural scenes from `lightcraft-scenes` are loaded at
start, so the page always has something in it.
- **A persistent library.** The library lives in the browser's storage for the page's origin:
[OPFS](https://developer.mozilla.org/docs/Web/API/File_System_API/Origin_private_file_system)
when the main thread can write it (`FileSystemFileHandle.createWritable`: Chrome, Edge,
Firefox, recent Safari), otherwise IndexedDB. Both hold the same layout:
- `library/catalog.snap`, `library/catalog.log`: the same crash-safe journal as the desktop app
(`lightcraft-catalog`), plus `presets.json`, `view.json`, `prefs.json` and `ui.json` (panel
layout). The catalog `Store` is a memory mirror loaded at start-up; every change is flushed in
the background within a frame or two, each file replaced atomically, in modification order
(`apps/lightcraft-web/src/files.rs`). View state and UI prefs are saved every second when they
change (a tab can close without notice).
- `originals/<content hash>`: the bytes of every imported photo. The catalog refers to them as
`web/<hash>/<file name>`; the main thread keeps recently used originals in memory (≤ 768 MB)
and loads the rest on demand (the active photo is prefetched).
- `thumbs/<key>.jpg` + `thumbs/index.json`: the rendered-thumbnail cache, with the desktop's
budget (2 GB, least recently used pruned to 80 %).
A new library starts with the procedural demo photos. The app asks for persistent storage
(`navigator.storage.persist()`); without it the browser may evict the data under storage
pressure. URL options: `?store=idb` forces IndexedDB, `?store=memory` keeps nothing, `?reset`
deletes the stored library first.
- **Importing photos with no filesystem.** *File ▸ Add Photos…* (<kbd>⌘⇧I</kbd>) opens the
browser's file picker. You can also drop files anywhere on the page. The bytes are kept in memory
(`store::MemStore`) under synthetic paths (`mem/<n>/<name>`) and decoded by the same engine code
as the desktop app (`lightcraft_engine::files::{probe_bytes, load_bytes}`). That covers
JPEG/PNG/TIFF/WebP and the supported raw formats.
browser's file picker. You can also drop files anywhere on the page. The bytes are written to
storage, then imported and decoded by the same engine code as the desktop app
(`lightcraft_engine::files::{probe_bytes, load_bytes}`): JPEG/PNG/TIFF/WebP and the supported
raw formats. "Copy into library" is the same as "Add" here.
- **Rendering in Web Workers.** Renders don't run on the main thread: up to four dedicated
workers (`hardwareConcurrency − 1`, `?workers=N` to override, `?workers=0` for the old inline
path) each run a second instance of the same wasm module. `index.html` compiles the module once
and posts the compiled `WebAssembly.Module` to `worker.js`. A job crosses as JSON (the photo's
source, develop settings, request); the worker reads the original or the cached thumbnail from
storage itself, keeps decoded sources and the loupe's stage cache, renders, and transfers the
RGBA bytes back. Jobs for a photo go back to the worker that decoded it. This needs no wasm
threads or `SharedArrayBuffer`. If no worker starts, rendering falls back to the main thread.
- **Every develop control** (sliders, curves, mixer, grading, masking, crop…) works as it does
on the desktop, since it's the same crate.
- **Export downloads the file.** *Export…* (<kbd>⌘⇧E</kbd>) runs the same `app.export` path as
the desktop app (`lightcraft_engine::export`: JPEG/PNG/TIFF/WebP, sizing, naming). The host's
`write` service hands each file to the browser as a download instead of writing it to disk.
- **Automation.** `await lightcraft.command("library.info", "{}")` runs any engine or UI command
by id on the next frame and resolves to the JSON result (`web.stats` reports storage, workers
and the render queue). This is how the headless-Chrome checks drive the page.
## Not yet
- **Persistence.** Nothing is saved across reloads: the catalog, edits, imported files and UI
prefs all start fresh. The planned route is IndexedDB/OPFS for the op log, and file handles
where the browser supports them.
- **Worker threads.** Renders run on the main thread, one job per frame. On a cold load the grid
fills in over a few seconds, and the UI can stall briefly while a thumbnail is generated. The fix
is Web Workers, which need wasm threads (`+atomics`, SharedArrayBuffer, COOP/COEP headers).
- **Bundle size.** The module is about 12 MB uncompressed, because it includes every codec and
raw decoder. Serving it gzip/brotli-compressed or running `wasm-opt` shrinks it a lot.
- **Exports and auto-adjustments run on the main thread** (they need the original's pixels
there). Right after a reload, the first such command on a photo whose original isn't in memory
yet fails with "still loading" and works a moment later.
- **Durability is "a frame later", not fsync-before-return:** a crash or tab kill in the few
milliseconds between a command and its flush loses that command.
- **Preset files** (import/export `.json`) aren't wired to browser pickers yet.
- **Control channel / MCP.** These are desktop-only, because they need a TCP socket.
- **AVIF export** is untested in the browser; it's the one encoder that may not be wasm-safe.