mirror of
https://github.com/storytold/lightcraft.git
synced 2026-10-14 19:27:01 -04:00
Brandon ThomasandClaude Opus 5.5
2218ca5a4c
A GPU render no longer hands back an image it cannot vouch for: - set the wgpu device-lost callback (wgpu never reports a lost device as an error, so a driver reset went unnoticed); readback waits with a timeout and checks the poll result and map callback instead of blocking forever - each render runs in OutOfMemory/Validation/Internal error scopes; errors fail that render (OOM and limit failures for that render only, device errors for the process) and it is redone on the CPU - every buffer is checked against the device limit when created (no validation error, no process-wide disable); LIGHTCRAFT_GPU_LIMITS=webgpu|<MiB> creates the device with lower limits to reproduce small adapters - the output buffer is cleared and the per-pixel kernel writes alpha 255: any unwritten pixel (work silently dropped, the issue's all-black export) fails the render; an all-black result is re-rendered on the CPU - renders submit per stage and every 16 M invocations, the per-pixel stage runs in ~4 MP bands: no long submission for a slow iGPU's watchdog to reset - ui.inspect perf.gpuReason / perf.gpuFallback, app.gpu reason/lastFallback, System Info and Settings say why the GPU is off (e.g. llvmpipe skipped) and why the last render fell back Tests: crates/gpu/tests/fallback.rs (limit, mid-render limit, validation error, dropped work, black photo), engine export falls back to the CPU image, banded 3000x2000 GPU/CPU equivalence. Co-Authored-By: Claude Opus 5.5 <[email protected]>
76 lines
6.0 KiB
Markdown
76 lines
6.0 KiB
Markdown
# Control protocol
|
||
|
||
`lightcraft --control 7980` (or `LIGHTCRAFT_CONTROL_PORT=7980`) starts a JSON-lines server on
|
||
`127.0.0.1:7980` (loopback only). One request per line, one reply per line, in order:
|
||
|
||
```text
|
||
→ {"id": 1, "method": "engine.execute", "params": {"command": "photo.rate", "params": {"rating": 4}}}
|
||
← {"id": 1, "ok": true, "result": null}
|
||
← {"id": 2, "ok": false, "error": "unknown command `nope`"}
|
||
```
|
||
|
||
Requests are answered on the UI thread between frames (timeout 60 s). The MCP server's connect
|
||
mode ([mcp.md](mcp.md)) is a thin layer over this channel. Implementation:
|
||
`crates/ui-egui/src/control.rs` (methods) and `apps/lightcraft/src/control_server.rs` (transport).
|
||
|
||
## Methods
|
||
|
||
| Method | Params | Result |
|
||
|---|---|---|
|
||
| `engine.execute` (alias `ui.menu.invoke`) | `{command, params?}` | Run any engine or UI command (see `engine.commands`) |
|
||
| `engine.commands` | — | Engine + UI commands: id, label, menu, shortcut, params doc, enabled |
|
||
| `ui.menu.list` | — | Menu entries (flat: id, label, menu path, shortcut, enabled) |
|
||
| `ui.menu.tree` | — | The menu bar as shown (File … Help): items `{id, params?, label, shortcut?, enabled, checked?}`, separators, submenus — the model behind the native macOS menu bar and the in-window menus |
|
||
| `ui.inspect` | — | UI state, window, canvas/image rects, `scroll: {grid, filmstrip}` (scroll offsets in points, `null` until drawn), active photo, selection, perf (`frameMs` = layout, `logicMs` = per-frame logic before it, `updateMs` = both, `maxUpdateMs`, `fps`, render queue …, `gpu` = adapter in use, `gpuReason` = why renders don't use the GPU, `gpuFallback` = latest render redone on the CPU and why — see `docs/gpu-pipeline.md`), status, memory (bytes per cache, see `library.memory`; plus stage caches and textures), `export: {running: {total, done, current} \| null, last}` |
|
||
| `ui.widgets` | `{filter?}` | On-screen widgets `{id, rect: [x, y, w, h]}` (screen points) |
|
||
| `ui.clickWidget` / `ui.dragWidget` / `ui.hoverWidget` | `{id, count?, fx?, fy?}` / `{id, toX?, toY?, dx?, dy?, steps?}` / `{id, fx?, fy?}` | Real egui input on a widget (hover: the pointer rests on it, e.g. for preset/profile previews) |
|
||
| `ui.move` / `ui.click` / `ui.drag` | `{x, y, count?, button?}` / `{x, y, toX, toY, steps?}` | Raw pointer input, screen points |
|
||
| `ui.pointer` | `{events: [{kind: down\|drag\|up, x, y}], alt?, shift?, cmd?}` | Gesture in normalized image coordinates (Detail view) |
|
||
| `ui.key` | `{key, cmd?, shift?, alt?, ctrl?}` | Key press |
|
||
| `ui.text` | `{text}` | Text input |
|
||
| `ui.scroll` | `{dx, dy}` | Mouse wheel |
|
||
| `ui.set` | partial UI state, e.g. `{"view": "detail"}` | Resulting UI state |
|
||
| `ui.dialog.confirm` / `ui.dialog.cancel` | — | Close the open dialog |
|
||
| `ui.resize` | `{width, height}` | Resize the window |
|
||
| `ui.screenshot` | `{path?, headless?}` | `{path, width, height}` once the frame (with finished renders) is captured. `headless: true` draws the UI on the CPU (no compositor needed); a windowed capture that gets no frame within 2 s falls back to headless automatically |
|
||
| `engine.execute {command: "app.export", params}` | export params (see `docs/mcp.md`), plus `preset`, `dir` / `path`, `ids`, `background` | Writes the files and returns `{files}`; with `background: true` (what the Export dialog and menus use) it returns `{background: true, total}` at once and the batch runs on a worker thread — poll `ui.inspect` → `export` |
|
||
| `ui.render` | `{id?, size?, path?}` | Render a photo (PNG to `path`), `{width, height}` |
|
||
| `app.quit` | — | Close the app |
|
||
|
||
### When the library can't be saved
|
||
|
||
With a persistent library, every command that changes something is written to the catalog journal (fsynced) before
|
||
it replies. If that write fails (disk full, volume gone, permissions), the command replies `ok: false` with
|
||
`"saved in memory but not written to disk: <reason>; LightCraft will retry"`. The change itself **is** applied (and
|
||
undoable) and stays queued: the next command, and the app's frame loop every couple of seconds, retry the write, so
|
||
nothing is lost once the disk is writable again — unless the app quits first. Meanwhile `ui.inspect` → `unsaved`
|
||
is `{ops, error}` (else `null`), `library.info` reports `unsavedOps` / `unsavedError`, and the top bar's cloud icon
|
||
shows a warning (widget `indicator:unsaved`). Queries and commands that change nothing still succeed. A failed
|
||
compaction (snapshot) is not a failed command — the log is kept whole — and only shows in `library.info` →
|
||
`lastError`.
|
||
|
||
## Headless rendering (no window, no GPU)
|
||
|
||
The egui UI can be rasterized on the CPU (`crates/ui-egui/src/softpaint.rs`, driven by
|
||
`crates/ui-egui/src/headless.rs`): same tessellation, gamma-space premultiplied blending and
|
||
scissor clipping as egui's GPU backends, so the image matches the window (minus GPU dithering).
|
||
|
||
- **In the running app:** `{"method": "ui.screenshot", "params": {"path": "a.png", "headless": true}}`.
|
||
The UI is drawn into an offscreen context from the app's logic tick, which keeps running when the
|
||
window is occluded or the display sleeps/locks. Windowed screenshots fall back to this after 2 s.
|
||
- **Without any app window:** `lightcraft-cli snapshot` runs a whole app session headlessly and
|
||
answers the same control requests (same handler) from a JSON-lines script:
|
||
|
||
```text
|
||
lightcraft-cli snapshot --demo -o grid.png --size 1600x1000 [--scale 2]
|
||
lightcraft-cli snapshot --library DIR --script tour.jsonl -o shot.png
|
||
```
|
||
|
||
`tour.jsonl` holds one request per line (`#` comments allowed), e.g.
|
||
`{"method": "ui.set", "params": {"view": "detail", "right": "edit", "openSections": ["optics"]}}`
|
||
then `{"method": "ui.screenshot"}`. Replies are printed to stdout. A `ui.screenshot` without
|
||
`path` writes `-o` (then `OUT-2.png`, `OUT-3.png`, …); `ui.settle {timeoutMs?}` waits until no
|
||
renders are in flight. Each request runs frames until it is answered and its injected input
|
||
(clicks, keys, drags) has played out. Widget ids for `ui.clickWidget` come from `ui.widgets`
|
||
(e.g. `button:upright-auto`). A 1600×1000 demo snapshot takes ~1–2 s (debug build).
|