mirror of
https://github.com/storytold/effectcraft.git
synced 2026-10-14 19:27:05 -04:00
bflatasticandClaude Opus 5.5
67305fbcd1
What: the Composition panel's Resolution (Auto, Full, Half, Third, Quarter, Custom) is now a
property of the composition (`Comp::resolution`, moved with its `Resolution` type from the UI
into the project crate) instead of one global viewer setting. Switching to another comp shows
that comp's resolution, switching back shows the first one's again, and saving and reopening the
project keeps them. `view.res.full|half|third|quarter|custom` and the new engine command
`view.res.auto` set it (headless too, with an optional `comp`); `ui.set {viewer: {res}}` sets
the active comp's and `ui.inspect` still reports `ui.viewer.res`; `comp.info` reports
`resolution` and `resolutionFactor`; View ▸ Resolution's check marks come from the engine.
Why: After Effects saves each composition's Resolution with the project (CompItem.resolutionFactor
is a per-comp attribute). As there, changing it marks the project modified but adds no undo step:
undo, redo and History jumps carry the current values over to the restored state. It is not part
of the comp's pixels (`Comp::same_pixels` ignores it and `Comp::pixel_form` normalises it), so
changing it keeps the comp's RAM and disk cache keys. Auto is not written (serde default and
skip_serializing_if), so projects saved before open with Auto and older versions read new files.
The magnification is not saved per comp: Adobe's documentation doesn't say it is.
Tests: project `comp_resolution_is_saved_and_old_projects_load_with_auto`,
`resolution_factors_and_scales`; engine
`each_comp_keeps_its_resolution_outside_the_undo_history_and_in_the_file` (switching comps,
no undo step, undo/redo keep it, save/open round trip, menu check marks, Custom dialog);
ui-egui `each_comp_keeps_its_viewer_resolution` (the viewer popup on comp A, comp B through
`ui.set`, back to A, unchanged frame cache identity, save/open). They don't build before the
change (no `Comp::resolution`, no `view.res.auto` engine command).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
130 lines
10 KiB
Markdown
130 lines
10 KiB
Markdown
# Control protocol
|
|
|
|
The desktop app exposes a control channel for agents, tests and the MCP bridge:
|
|
|
|
```sh
|
|
effectcraft --control 9877 # or EFFECTCRAFT_CONTROL_PORT=9877
|
|
```
|
|
|
|
- **Transport:** TCP on `127.0.0.1` only, one JSON object per line in each direction. Connections are
|
|
persistent; requests on one connection are answered in order.
|
|
- **Request:** `{"id": 1, "method": "engine.execute", "params": {...}}` (`id` is echoed back, `params`
|
|
defaults to `{}`).
|
|
- **Reply:** `{"id": 1, "ok": true, "result": ...}` or `{"id": 1, "ok": false, "error": "message"}`.
|
|
- Requests run on the UI thread between frames. Input methods (`ui.click`, `ui.key` and similar)
|
|
reply after the synthetic input has been processed. Methods that need an element that isn't drawn
|
|
yet are retried for a few frames. A request times out after 60 s.
|
|
- **Only requests are read:** every line must be a JSON object with a string `method` (blank lines are
|
|
skipped). Anything else (text that isn't JSON, a JSON array or number, an object without `method`,
|
|
invalid UTF-8, a line longer than 4 MiB) gets one error reply ending in "closing the connection", and
|
|
the server closes the connection, so nothing sent after it runs. An HTTP request (for example a web
|
|
page's cross-origin `fetch` to `127.0.0.1:<port>`) therefore can't smuggle a command in its body: its
|
|
request line is rejected first. At most 16 connections are served at once; further ones get an error
|
|
line and are closed. The port has no authentication, so only enable it while you use it.
|
|
|
|
Quick test:
|
|
|
|
```sh
|
|
printf '{"id":1,"method":"engine.execute","params":{"command":"comp.new","params":{"name":"A"}}}\n' | nc 127.0.0.1 9877
|
|
```
|
|
|
|
## Engine
|
|
|
|
| Method | Params | Result |
|
|
|---|---|---|
|
|
| `engine.execute` | `{command, params?}` | The command's result. Every engine command (`effectcraft-cli commands`) runs exactly as it does headless, so it never opens a dialog: `comp.new {}` creates a default comp. UI-only ids (`tool.*`, `view.*`, `window.*`, `playback.*`, `timeline.*`, `app.*`) fall through to the menu dispatcher. Unknown top-level params are rejected, and the error lists the accepted keys. |
|
|
| `ui.menu.invoke` | `{id, params?}` | Like a menu click: commands with empty params may open their dialog (New Composition, Solid Settings, file pickers). |
|
|
| UI commands (via either) | | `timeline.column {column, visible?}` (av, keys, label, num, comment, switches, modes, parent, in, out, duration, stretch), `timeline.sourceName {value?}`, `timeline.search {query}`, `timeline.reveal.<kind>` / `timeline.revealAdd.<kind>` (the property shortcuts and Shift+shortcut: position, scale, rotation, opacity, anchor, effects, masks, feather, levels, animated, and the double-press sets maskPath, maskOpacity, timeRemap, expressions, material, paint, selected, missingEffects, waveform), `timeline.keyAt.<anchor\|position\|scale\|rotation\|opacity>` (Alt+Shift+A/P/S/R/T), `timeline.twirlSelected` (Ctrl/Cmd+`), `timeline.collapseAll`, `flowchart.options {layers?, effects?, solids?, direction?: lr\|tb, comp?}`, `flowchart.graph {comp?}` → `{nodes, edges}`, `window.workspace {name}` (all After Effects workspaces). |
|
|
| `engine.commands` | `{filter?, enabledOnly?}` | `[{id, label, menu, shortcut, params, enabled, why}]` |
|
|
| `ui.menu.list` | `{}` | The menu tree with enablement. |
|
|
| `render.frame` | `{comp?, time?, max_side?, path?, base64?}` | Renders a comp frame through the session, independent of the viewer zoom or resolution. `comp` is an id or name (default: the active comp), `time` is in comp seconds (default: the CTI), `max_side` caps the longest side (0 = full size). Writes a PNG to `path` (default: a temp file) and returns `{comp, time, width, height, path}`. With `base64: true` it returns `{…, png: "<base64>"}` and writes no file. |
|
|
|
|
Useful query commands for `engine.execute` (they aren't journaled or undoable): `project.summary`,
|
|
`comp.info {comp?}`, `layer.tree {layer, comp?, depth?, time?}` (every node has a `path`),
|
|
`prop.get {layer, path, comp?, time?}`, `editor.state`, `command.list {filter?, enabledOnly?, schemas?}`,
|
|
`command.describe {command}` (params doc and JSON Schema), `app.capabilities` (version, counts,
|
|
export formats, parity summary), `jobs.list` (Progress panel jobs, e.g. the background
|
|
`footage.check` after File ▸ Open).
|
|
|
|
Every engine command is reachable through `engine.execute`, and every interactive widget of every
|
|
panel registers an element id; both are enforced by `crates/ui-egui/tests/ui_agent_audit.rs`.
|
|
|
|
## UI state
|
|
|
|
| Method | Params | Result |
|
|
|---|---|---|
|
|
| `ui.inspect` | `{}` | Window size, pixels per point, fps, serialized `UiState`, playback, time, active comp, selection, open dialog, last render ms. |
|
|
| `ui.elements` | `{prefix?}` | `[{id, label, rect: [x, y, w, h]}]`: the widgets drawn in the last frame, in points. |
|
|
| `ui.set` | `{tool?, workspace?, theme?, focused?, viewer?, timeline?, menuBar?, home?}` | `viewer: {zoom (null = fit), res: Full\|Half\|Third\|Quarter\|Auto, pan: [x,y], grid, rulers, safeMargins, transparencyGrid}`; `timeline: {pps (null = fit), start, graphEditor, showModes, openLayers: [ids], openGroups: [uids]}`. |
|
|
| `ui.panel.show` / `ui.panel.close` | `{panel}` | Panels: `Project`, `Composition`, `Timeline`, `EffectControls`, `EffectsPresets`, `Info`, `Preview`, `Character`, `Paragraph`, `Align` and others. |
|
|
| `ui.playback` | `{action: play\|stop\|toggle\|status}` | `{playing, time, audio, levelsDb, peaksDb}` (audio: an audio preview is running; L/R meter levels and held peaks in dBFS) |
|
|
| `ui.resize` | `{width, height}` | Resizes the window (points). |
|
|
| `ui.focus` | `{}` | Activates the window and takes keyboard focus. Agents normally avoid this, because the app is started without stealing focus. |
|
|
| `app.quit` | `{force?}` | Closes the app. A modified project shows the Save / Don't Save / Cancel prompt first (automation ids `dialog.unsaved.save`, `dialog.unsaved.dontSave`, `dialog.unsaved.cancel`) unless `force: true`. |
|
|
|
|
## Input
|
|
|
|
Points are logical window coordinates. A target is `{id}` (the element's centre; add `fx`/`fy` between
|
|
0 and 1 to pick a spot inside it) or `{x, y}`. `modifiers` is `{shift, alt, command, ctrl}`, where
|
|
`command` is ⌘ on macOS and Ctrl elsewhere.
|
|
|
|
| Method | Params |
|
|
|---|---|
|
|
| `ui.click` | `{id \| x,y, button?: left\|right\|middle, count?, modifiers?}` |
|
|
| `ui.move` | `{id \| x,y}` |
|
|
| `ui.drag` | `{from: target, to: target, steps? (12), modifiers?}`: press, move, release |
|
|
| `ui.scroll` | `{id \| x,y, dx, dy}` |
|
|
| `ui.key` | `{key}`, for example `Space`, `F9`, `Cmd+Z`, `Cmd+Shift+Z`, `PageDown`, `Escape` |
|
|
| `ui.type` | `{text}`: text input into the focused field |
|
|
|
|
## Locating things
|
|
|
|
| Method | Params | Result |
|
|
|---|---|---|
|
|
| `ui.viewer.locate` | `{x, y}` (comp pixels) | `{x, y}` window point |
|
|
| `ui.viewer.hit` | `{id \| x,y}` (window point) | `{x, y}` comp pixel |
|
|
| `ui.timeline.locate` | `{layer (id), time?}` | `{x, y}` window point on the layer bar |
|
|
|
|
## Screenshots
|
|
|
|
| Method | Params | Result |
|
|
|---|---|---|
|
|
| `ui.screenshot` | `{path?, panel?, id?}` | `{path, width, height}`: a PNG of the window, or cropped to one panel or element. |
|
|
|
|
Element ids are stable, for example `tools.Selection`, `panel.Timeline`, `panel.tab.EffectControls`,
|
|
`panel.tab.Timeline.<comp>` (one Timeline tab per open comp; `panel.tab.Timeline` is the shown
|
|
one), `panel.tab.Viewer(<n>)` and `viewers.<n>` (View ▸ New Viewer's other Composition viewers;
|
|
`viewers.0` is the Composition panel while another viewer is active), `header.workspace.Animation`, `project.item.<id>` (a double-click opens the item, Enter
|
|
renames it), `viewer.comp`, `viewer.handle.<layer>.<i>`,
|
|
`timeline.layer.<id>.bar`, `timeline.layer.<id>.nestedMarker.<i>` (a precomp's comp markers), `timeline.layer.<id>.twirl`, `timeline.layer.<id>.row` (click selects,
|
|
Enter or a double-click renames, drag reorders), `timeline.prop.<uid>.stopwatch`,
|
|
`timeline.key.<uid>.<frame>`, `timeline.cti`, `effectControls.prop.<uid>.value` (an angle's
|
|
revolutions are `timeline.prop.<uid>.revolutions`, `effectControls.prop.<uid>.revolutions` and
|
|
`properties.prop.<uid>.revolutions`) and `effects.item.<name>`. The Composition viewer adds `viewer.magnification`, `viewer.resolution`,
|
|
`viewer.roi`, `viewer.grid`, `viewer.channel`, `viewer.exposure`, `viewer.snapshot`,
|
|
`viewer.showSnapshot`, `viewer.fastPreviews` (their popup entries are `viewer.<menu>Item.<n>`),
|
|
`viewer.ruler.top|left|origin`, `viewer.mask.<uid>.vertex.<i>`, `viewer.shapePath.<uid>.vertex.<i>`,
|
|
`viewer.motionPath.<layer>.<key>[.in|.out]`, `viewer.puppetPin.<uid>[.rotate|.scale]`,
|
|
`viewer.freeTransform.handle.<i>` and
|
|
`viewer.regionOfInterest`; the Graph Editor adds `timeline.graph.transformBox[.<i>]`,
|
|
`timeline.graph.snap` and `timeline.graph.reference`. Use `ui.elements` to see what is on screen.
|
|
|
|
Viewer state that agents drive headless too: `view.snapping`, `view.snappingOptions
|
|
{edgesExtended?, collapsedFeatures?, toggle?}` (the Tools bar's Snapping options,
|
|
`header.snappingOptions`: After Effects' Snap Edges Extended and Snap to Features in Collapsed
|
|
Compositions and Text Layers), `view.channel {channel, colorized?}`,
|
|
`view.exposure {stops | delta}`, `view.resetExposure`, `view.takeSnapshot`, `view.showSnapshot`,
|
|
`view.fastPreviewMode {mode}`, `view.res.full|half|third|quarter|auto {comp?}` and
|
|
`view.res.custom {factor, comp?}` (the comp's Resolution, saved with the project but no undo
|
|
step; `ui.set {viewer: {res}}` sets the active comp's, and `ui.inspect` reports it as
|
|
`ui.viewer.res`), `view.setRegionOfInterest {rect}`, `view.addGuide`,
|
|
`view.moveGuide`, `view.removeGuide`; editing: `shape.newPath` (Pen on shape layers),
|
|
`shape.newShape` (the shape tools: a new group in the selected shape layer, else a new shape layer),
|
|
`shape.toolOptions` (the Tools bar's Tool Creates Shape / Mask and Fill and Stroke Options),
|
|
`mask.insertVertex`, `mask.convertVertex`, `mask.deleteVertices`, `path.freeTransform` (masks and
|
|
shape paths by uid), `keys.setSpatialTangents` (motion-path handles) and `keys.transform` (Graph
|
|
Editor transform box, timeline Alt-drag scaling).
|
|
|
|
The MCP server's bridge mode (`effectcraft-cli mcp --bridge 9877`) is a thin client of this protocol;
|
|
see [agents.md](agents.md).
|