mirror of
https://github.com/storytold/lightcraft.git
synced 2026-10-14 19:27:01 -04:00
Jason Baldwin
a677478b92
egui's zoom_with_keyboard was left on, so at the end of every frame it took the same keys as View ▸ Zoom In / Zoom Out / Zoom to Fit for its browser-style interface zoom. On macOS the native menu bar got the keys first; on Windows and Linux the whole UI scaled. theme::apply turns the option off for every context (desktop, headless, tests). Headless tests that pressed Cmd+Plus / Cmd+Minus to exercise the interface zoom's viewport handling set the zoom factor directly; the CLI snapshot test that did the same is dropped (the unit tests cover the mechanism). New tests_zoom_keys.rs: the shortcuts step the photo zoom while the zoom factor stays, and Ctrl+wheel over the photo zooms it in the Edit view.
120 lines
11 KiB
Markdown
120 lines
11 KiB
Markdown
# Control protocol
|
||
|
||
`lightcraft --control 7980` (or `LIGHTCRAFT_CONTROL_PORT=7980`) starts a JSON-lines server on
|
||
`127.0.0.1:7980` (loopback only). One request per line, one reply per line, in order:
|
||
|
||
```text
|
||
→ {"id": 1, "method": "engine.execute", "params": {"command": "photo.rate", "params": {"rating": 4}}}
|
||
← {"id": 1, "ok": true, "result": null}
|
||
← {"id": 2, "ok": false, "error": "unknown command `nope`"}
|
||
```
|
||
|
||
Requests are answered on the UI thread between frames (timeout 60 s).
|
||
|
||
**Only requests are read.** Every line must be a JSON object with a string `method` (`id` and `params` are
|
||
optional; blank lines are skipped). Anything else — text that isn't JSON, a JSON array or number, an object without
|
||
`method`, invalid UTF-8, or a line longer than 4 MiB — gets one error reply
|
||
(`{"ok": false, "error": "… closing the connection"}`) and the server **closes the connection**; nothing sent after
|
||
it on that connection runs. This keeps an HTTP request (for example a web page's cross-origin `fetch` to
|
||
`127.0.0.1:<port>`) from smuggling a command in its body: its request line is rejected first. Junk never reaches the
|
||
UI thread, and at most 16 connections are served at once (further ones get an error line and are closed). Clients
|
||
that hit an error reply should reconnect. The port has no authentication, so only enable it when you need it. The MCP server's connect
|
||
mode ([mcp.md](mcp.md)) is a thin layer over this channel. Implementation:
|
||
`crates/ui-egui/src/control.rs` (methods) and `apps/lightcraft/src/control_server.rs` (transport).
|
||
|
||
## Methods
|
||
|
||
| Method | Params | Result |
|
||
|---|---|---|
|
||
| `engine.execute` (alias `ui.menu.invoke`) | `{command, params?}` | Run any engine or UI command (see `engine.commands`) |
|
||
| `engine.commands` | — | Engine + UI commands: id, label, menu, shortcut, params doc, enabled |
|
||
| `ui.menu.list` | — | Menu entries (flat: id, label, menu path, shortcut, enabled) |
|
||
| `ui.menu.tree` | — | The menu bar as shown (File … Help): items `{id, params?, label, shortcut?, enabled, checked?}`, separators, submenus — the model behind the native macOS menu bar and the in-window menus |
|
||
| `ui.inspect` | — | UI state, window, `loupe` (`source`, and `region: {full, window, pending}` when a zoomed view also renders the window on screen at no more than 100 % — `full` is the frame it is cut from, `window` its `[x, y, w, h]` — and `regionBefore` for the Before side of a Before/After view), canvas/image rects, `scroll: {grid, filmstrip}` (scroll offsets in points, `null` until drawn), active photo, selection, perf (`frameMs` = layout, `logicMs` = per-frame logic before it, `updateMs` = both, `maxUpdateMs`, `fps`, render queue …, `gpu` = adapter in use, `gpuReason` = why renders don't use the GPU, `gpuFallback` = latest render redone on the CPU and why — see `docs/gpu-pipeline.md`), status, memory (bytes per cache, see `library.memory`; plus stage caches — `budgetBytes`, `trimmed`, `sharedSourceBytes` (the one device copy of a big original) — and textures), `export: {running: {total, done, current} \| null, last}`, `notices` (warnings waiting to be shown, e.g. a damaged settings file; OK = `button:noticeOk`), `quitPrompt` (why quitting was stopped: unsaved changes; `button:quitRetry` / `button:quitAnyway` / `button:quitCancel`), `import: {done, total, imported, cancelled} \| null` (an import runs on a worker thread), `tasks` (other background work: `Find Missing Photos`, `Auto Import`), `fileDialogs` (commands waiting on a native file dialog, which runs off the UI thread; they run again with the answer when it closes) |
|
||
| `ui.widgets` | `{filter?}` | On-screen widgets `{id, rect: [x, y, w, h]}` (screen points) |
|
||
| `ui.clickWidget` / `ui.dragWidget` / `ui.hoverWidget` | `{id, count?, fx?, fy?}` / `{id, toX?, toY?, dx?, dy?, steps?}` / `{id, fx?, fy?}` | Real egui input on a widget (hover: the pointer rests on it, e.g. for preset/profile previews) |
|
||
| `ui.move` / `ui.click` / `ui.drag` | `{x, y, count?, button?}` / `{x, y, toX, toY, steps?}` | Raw pointer input, screen points |
|
||
| `ui.pointer` | `{events: [{kind: down\|drag\|up, x, y}], alt?, shift?, cmd?}` | Gesture in normalized image coordinates (Detail view) |
|
||
| `ui.key` | `{key, cmd?, shift?, alt?, ctrl?}` | Key press |
|
||
| `ui.text` | `{text}` | Text input |
|
||
| `ui.scroll` | `{dx, dy, cmd?, ctrl?, shift?, alt?}` | Wheel / two-finger scroll at the current pointer; pans over the image, modifier-scroll zooms |
|
||
| `ui.zoom` | `{factor}` | Pinch zoom at the current pointer (positive scale multiplier; 1 = unchanged). Position it first with `ui.move` or `ui.hoverWidget` |
|
||
| `ui.set` | partial UI state, e.g. `{"view": "detail"}` | Resulting UI state |
|
||
| `ui.dialog.confirm` / `ui.dialog.cancel` | — | Close the open dialog |
|
||
| `ui.resize` | `{width, height}` | Resize the window |
|
||
| `ui.screenshot` | `{path?, headless?}` | `{path, width, height}` once the frame (with finished renders) is captured. `headless: true` draws the UI on the CPU (no compositor needed); a windowed capture that gets no frame within 2 s falls back to headless automatically |
|
||
| `engine.execute {command: "app.export", params}` | export params (see `docs/mcp.md`), plus `preset`, `dir` / `path`, `ids`, `background` | Writes the files and returns `{files}`; with `background: true` (what the Export dialog and menus use) it returns `{background: true, total}` at once and the batch runs on a worker thread — poll `ui.inspect` → `export` |
|
||
| `ui.render` | `{id?, size?, path?}` | Render a photo (PNG to `path`), `{width, height}` |
|
||
| `app.quit` | — | Close the app |
|
||
|
||
Image navigation is also available directly as the UI command `view.navigate`, with
|
||
`{zoom?: "fit" | "fill" | {"percent": number}, pan?: [x, y]}`. Percentage zoom accepts fractional
|
||
values greater than 0 and at most 800; pan is the normalized image centre, with coordinates from 0 to 1.
|
||
Pinching keeps the image point under the pointer steady and zooms between Fit and 800%; two-finger
|
||
scrolling pans in both axes and respects the operating system's scrolling direction and momentum.
|
||
These gestures work in Detail (including editing tools and full-screen preview), Compare and Reference
|
||
views, and only apply over their image areas. Panning stops at the image edges.
|
||
|
||
### When the library can't be saved
|
||
|
||
With a persistent library, every command that changes something is written to the catalog journal (fsynced) before
|
||
it replies. If that write fails (disk full, volume gone, permissions), the command replies `ok: false` with
|
||
`"saved in memory but not written to disk: <reason>; LightCraft will retry"`. The change itself **is** applied (and
|
||
undoable) and stays queued: the next command, and the app's frame loop every couple of seconds, retry the write, so
|
||
nothing is lost once the disk is writable again — unless the app quits first. Meanwhile `ui.inspect` → `unsaved`
|
||
is `{ops, error}` (else `null`), `library.info` reports `unsavedOps` / `unsavedError`, and the top bar's cloud icon
|
||
shows a warning (widget `indicator:unsaved`). Queries and commands that change nothing still succeed. A failed
|
||
compaction (snapshot) is not a failed command — the log is kept whole — and only shows in `library.info` →
|
||
`lastError`.
|
||
|
||
### When the library can't be opened
|
||
|
||
If the desktop app can't open its library at launch (another program has it open, an unreadable or newer-format
|
||
catalog, a missing drive), the session starts empty and in memory — never with demo photos — and a window asks what
|
||
to do: `ui.inspect` → `libraryProblem` is `{path, error, temporarySession, pendingImport}` (else `null`); its buttons
|
||
are `button:libraryRetry`, `button:libraryChoose`, `button:libraryTemporary` (Continue Without Saving) and
|
||
`button:libraryQuit`. A temporary session shows a banner (`indicator:temporarySession`, `button:libraryReopen`) and
|
||
writes nothing. `app.openLibrary` opening a library ends it.
|
||
|
||
## Headless rendering (no window, no GPU)
|
||
|
||
The egui UI can be rasterized on the CPU (`crates/ui-egui/src/softpaint.rs`, driven by
|
||
`crates/ui-egui/src/headless.rs`): same tessellation, gamma-space premultiplied blending and
|
||
scissor clipping as egui's GPU backends, so the image matches the window (minus GPU dithering).
|
||
|
||
- **In the running app:** `{"method": "ui.screenshot", "params": {"path": "a.png", "headless": true}}`.
|
||
The UI is drawn into an offscreen context from the app's logic tick, which keeps running when the
|
||
window is occluded or the display sleeps/locks. Windowed screenshots fall back to this after 2 s.
|
||
- **Without any app window:** `lightcraft-cli snapshot` runs a whole app session headlessly.
|
||
Photo development also defaults to the CPU, so the initial frames do not discover a GPU adapter
|
||
or load a graphics driver. `LIGHTCRAFT_GPU=1` does not opt snapshots back into GPU rendering.
|
||
This applies only to the snapshot session: even `--library DIR` leaves the desktop app's saved
|
||
`ui.json` preferences untouched. It answers the same control requests (same handler) from a JSON-lines script:
|
||
|
||
```text
|
||
lightcraft-cli snapshot --demo -o grid.png --size 1600x1000 [--scale 2]
|
||
lightcraft-cli snapshot --library DIR --script tour.jsonl -o shot.png
|
||
```
|
||
|
||
Headless screenshot dimensions must be finite and at least one logical point, with a finite,
|
||
positive scale. Rounded output must be at least one pixel per edge, at most 16,384 pixels
|
||
per edge and 64 million pixels in total. `snapshot` and `ui.resize` reject requests outside
|
||
these limits before layout or rasterization; supplied resize dimensions must be numbers.
|
||
An interface zoom (egui's zoom factor; the keyboard never changes it — ⌘= / ⌘− / ⌘0 zoom the
|
||
photo, issue #566) changes text and control sizes while preserving the snapshot viewport's requested
|
||
physical pixel dimensions. Later `ui.resize` requests use current egui points; validation
|
||
and rasterization both convert them to native viewport points before rounding the output
|
||
pixels. A native edge below one point after zooming out is valid if it rounds to at least
|
||
one pixel.
|
||
|
||
`tour.jsonl` holds one request per line (`#` comments allowed), e.g.
|
||
`{"method": "ui.set", "params": {"view": "detail", "right": "edit", "openSections": ["optics"]}}`
|
||
then `{"method": "ui.screenshot"}`. Replies are printed to stdout. A failed request
|
||
(`"ok": false`) does not stop the script — the remaining lines and the final screenshot still
|
||
run — but the exit status is non-zero when any request failed, as with `run --keep-going`, so
|
||
CI and nightly runs can judge a snapshot by its exit status. A `ui.screenshot` without
|
||
`path` writes `-o` (then `OUT-2.png`, `OUT-3.png`, …); `ui.settle {timeoutMs?}` waits until no
|
||
renders are in flight. Each request runs frames until it is answered and its injected input
|
||
(clicks, keys, drags) has played out. Widget ids for `ui.clickWidget` come from `ui.widgets`
|
||
(e.g. `button:upright-auto`). A 1600×1000 demo snapshot takes ~1–2 s (debug build).
|