mirror of
https://github.com/storytold/lightcraft.git
synced 2026-10-14 19:27:01 -04:00
Brandon ThomasandClaude Opus 5.5
4788f3bfff
At the 2,000-record threshold the UI thread serialised and fsynced the whole catalog (1-2 s
at 85k photos after streaming). Now `persist()` starts `Journal::snapshot_in_background`:
- the worker writes `catalog.snap` for op N from a clone of the catalog (photos are Arc-shared,
so the clone is cheap) through `Store::background_writer` (a second FsStore handle; MemStore
opt-in; none on wasm/web, which stays synchronous);
- appends go on as before (fsynced before every command returns) and are also kept in memory
as the "tail" (records > N);
- `Journal::poll` (from `persist` and the per-frame `persist_if_dirty`) sees the worker done and
atomically replaces the log with exactly the tail.
Crash points: before the snapshot rename -> old snapshot + whole log; between the rename and the
log rewrite -> new snapshot + whole log (records <= N skipped as stale, existing rule); after ->
new snapshot + tail. A failed snapshot keeps the log whole and retries after another
max_records records (no retry storm). At most one snapshot in flight; a synchronous snapshot
(close, Optimize Library), reopening a library and dropping the Journal first wait for it, so an
older snapshot can never replace a newer one. On-disk format unchanged.
Tests (crates/catalog/src/tests_background.rs, a gated store holds the worker mid-flight and
"crashes" by copying the files): edits during a background snapshot survive crashes at each
point and the trimmed log holds exactly seq N+1.. in order; failure loses nothing and backs
off; sync snapshot waits (no regression); drop waits; real FsStore with a leftover partial
temp file; a proptest over interleavings of edits / start / land / finish / sync snapshot /
crash. Mutations checked: trimming the log to empty, or not waiting in `snapshot`, fail them.
Engine: `threshold_compaction_runs_in_the_background` (crash copy mid-run, reopen without close).
`library.info` -> persistence gains snapshotRunning, failedSnapshots, maxBlockingMs and
lastSnapshot.{background, blockingMs}.
persist_bench, same session (load avg ~160), UI-thread time per threshold compaction:
85k photos: synchronous 0.85-1.7 s (median 1.25 / 1.34 s) -> background: blocking median
45 / 74 ms (clone 1.3-154 ms + log rewrite 9-23 ms); worker 1.2-2.4 s; 230-380
single-op appends during it at median 4-6 ms (same as without)
10k photos: synchronous 72-233 ms -> blocking median 12.5 / 14 ms
Co-Authored-By: Claude Opus 5.5 <[email protected]>
4.9 KiB
4.9 KiB
Control protocol
lightcraft --control 7980 (or LIGHTCRAFT_CONTROL_PORT=7980) starts a JSON-lines server on
127.0.0.1:7980 (loopback only). One request per line, one reply per line, in order:
→ {"id": 1, "method": "engine.execute", "params": {"command": "photo.rate", "params": {"rating": 4}}}
← {"id": 1, "ok": true, "result": null}
← {"id": 2, "ok": false, "error": "unknown command `nope`"}
Requests are answered on the UI thread between frames (timeout 60 s). The MCP server's connect
mode (mcp.md) is a thin layer over this channel. Implementation:
crates/ui-egui/src/control.rs (methods) and apps/lightcraft/src/control_server.rs (transport).
Methods
| Method | Params | Result |
|---|---|---|
engine.execute (alias ui.menu.invoke) |
{command, params?} |
Run any engine or UI command (see engine.commands) |
engine.commands |
— | Engine + UI commands: id, label, menu, shortcut, params doc, enabled |
ui.menu.list |
— | Menu entries (flat: id, label, menu path, shortcut, enabled) |
ui.menu.tree |
— | The menu bar as shown (File … Help): items {id, params?, label, shortcut?, enabled, checked?}, separators, submenus — the model behind the native macOS menu bar and the in-window menus |
ui.inspect |
— | UI state, window, canvas/image rects, scroll: {grid, filmstrip} (scroll offsets in points, null until drawn), active photo, selection, perf (frameMs = layout, logicMs = per-frame logic before it, updateMs = both, maxUpdateMs, fps, render queue …), status, memory (bytes per cache, see library.memory; plus stage caches and textures), export: {running: {total, done, current} | null, last} |
ui.widgets |
{filter?} |
On-screen widgets {id, rect: [x, y, w, h]} (screen points) |
ui.clickWidget / ui.dragWidget / ui.hoverWidget |
{id, count?, fx?, fy?} / {id, toX?, toY?, dx?, dy?, steps?} / {id, fx?, fy?} |
Real egui input on a widget (hover: the pointer rests on it, e.g. for preset/profile previews) |
ui.move / ui.click / ui.drag |
{x, y, count?, button?} / {x, y, toX, toY, steps?} |
Raw pointer input, screen points |
ui.pointer |
{events: [{kind: down|drag|up, x, y}], alt?, shift?, cmd?} |
Gesture in normalized image coordinates (Detail view) |
ui.key |
{key, cmd?, shift?, alt?, ctrl?} |
Key press |
ui.text |
{text} |
Text input |
ui.scroll |
{dx, dy} |
Mouse wheel |
ui.set |
partial UI state, e.g. {"view": "detail"} |
Resulting UI state |
ui.dialog.confirm / ui.dialog.cancel |
— | Close the open dialog |
ui.resize |
{width, height} |
Resize the window |
ui.screenshot |
{path?, headless?} |
{path, width, height} once the frame (with finished renders) is captured. headless: true draws the UI on the CPU (no compositor needed); a windowed capture that gets no frame within 2 s falls back to headless automatically |
engine.execute {command: "app.export", params} |
export params (see docs/mcp.md), plus preset, dir / path, ids, background |
Writes the files and returns {files}; with background: true (what the Export dialog and menus use) it returns {background: true, total} at once and the batch runs on a worker thread — poll ui.inspect → export |
ui.render |
{id?, size?, path?} |
Render a photo (PNG to path), {width, height} |
app.quit |
— | Close the app |
Headless rendering (no window, no GPU)
The egui UI can be rasterized on the CPU (crates/ui-egui/src/softpaint.rs, driven by
crates/ui-egui/src/headless.rs): same tessellation, gamma-space premultiplied blending and
scissor clipping as egui's GPU backends, so the image matches the window (minus GPU dithering).
- In the running app:
{"method": "ui.screenshot", "params": {"path": "a.png", "headless": true}}. The UI is drawn into an offscreen context from the app's logic tick, which keeps running when the window is occluded or the display sleeps/locks. Windowed screenshots fall back to this after 2 s. - Without any app window:
lightcraft-cli snapshotruns a whole app session headlessly and answers the same control requests (same handler) from a JSON-lines script:
lightcraft-cli snapshot --demo -o grid.png --size 1600x1000 [--scale 2]
lightcraft-cli snapshot --library DIR --script tour.jsonl -o shot.png
tour.jsonl holds one request per line (# comments allowed), e.g.
{"method": "ui.set", "params": {"view": "detail", "right": "edit", "openSections": ["optics"]}}
then {"method": "ui.screenshot"}. Replies are printed to stdout. A ui.screenshot without
path writes -o (then OUT-2.png, OUT-3.png, …); ui.settle {timeoutMs?} waits until no
renders are in flight. Each request runs frames until it is answered and its injected input
(clicks, keys, drags) has played out. Widget ids for ui.clickWidget come from ui.widgets
(e.g. button:upright-auto). A 1600×1000 demo snapshot takes ~1–2 s (debug build).