docs(maestro): write STRUCTURED-VIEW.md

Contracts the eight transcript row kinds, daemon-vs-client decisions, the
transcript/reduced-state projections and their cursor/replay semantics, the
ACP handshake and declined capabilities, the adapter install command, and
the approval nonce rules (generation, live-only placement, secrecy from the
agent and durable history, single-use/duplicate resolution) and bounded
prompt/cancel liveness.

Amends RESTART.md to state plainly that a structured session's agent is a
daemon child, not tmux-backed: a restart ends the active turn, the
transcript survives in the event log, and pending approvals/nonces do not.

Refs #137
This commit is contained in:
Miguel Palhas
2026-09-05 03:32:42 +01:00
parent 1e2c0f0a38
commit f8973b3802
4 changed files with 386 additions and 5 deletions
+6
View File
@@ -92,6 +92,12 @@ clients — how it opens, the sigils, the key model, the match-and-rank algorith
the two matchers must agree on, and the fixture table they are tested against.
Read it before changing either bar or either matcher.
`docs/STRUCTURED-VIEW.md` is the contract for a structured session (an agent
run as a daemon child over ACP instead of in a tmux pane) — the eight
transcript row kinds, the two projections and their cursor/replay semantics,
the handshake and declined capabilities, and the approval nonce rules. Read
it before changing anything that renders a structured session.
`nix run .#dev` (or `just dev`) brings up the whole local stack under
process-compose: a debug daemon on 127.0.0.1:7071, then Vite on :5173 once the
daemon answers `/api/info`. The SPA proxies `/api/*` back to the daemon, so only
+4 -1
View File
@@ -211,7 +211,10 @@ what to run. The adapter is a child of the daemon: it ends with a `DELETE
itself. A daemon restart ends an in-flight structured turn, but preserves its
worktree, session record, and event-log transcript. The replacement daemon
starts a new adapter, resumes it with `session/load`, and replays the
transcript. Terminal sessions keep surviving restarts through tmux.
transcript. Terminal sessions keep surviving restarts through tmux. See
`docs/STRUCTURED-VIEW.md` for the full contract — row kinds, projections, the
handshake, and the approval and recovery rules — and `docs/RESTART.md` for
what a client observes across the restart.
## The browser UI
+35 -4
View File
@@ -2,10 +2,16 @@
Maestro is upgraded in place: the operator reinstalls the binary and systemd
restarts the daemon under every browser tab and TUI that is attached to it.
tmux keeps the sessions alive across that (see `docs/INSTALL.md`), but clients
lose their sockets. This document is the contract for what a client does about
it — what the daemon publishes, how a client tells a restart from a blip, and
what the user sees while it happens.
tmux keeps **terminal** sessions alive across that (see `docs/INSTALL.md`),
but clients lose their sockets either way. This document is the contract for
what a client does about it — what the daemon publishes, how a client tells a
restart from a blip, and what the user sees while it happens.
A **structured** session (`docs/STRUCTURED-VIEW.md`) has no tmux pane behind
it to survive on: its agent is a child of the daemon process itself, so a
restart takes it down with the daemon. See
[Structured sessions](#structured-sessions) below for exactly what that
costs and what it doesn't.
## What the daemon publishes
@@ -68,6 +74,31 @@ resolves, so a blocked client is never a trapped one. It clears itself the
moment a probe succeeds — there is no "retry" button, because the client is
already retrying.
## Structured sessions
Everything above describes a **client's** view of a restart, and it holds for
a structured session exactly as written — the same three verdicts, the same
status indicator, the same overlay. What differs is what survives on the
**daemon** side, because a structured session's agent is a daemon child, not
a tmux pane:
- A restart **ends the active turn**. There is no adapter process left to
finish it, and no tmux pane keeping it alive underneath.
- The **transcript survives**, in the per-session event log
(`docs/STRUCTURED-VIEW.md#persistence-recovery-and-dead-sessions`) — a
graceful shutdown marks the session for one resume attempt, and the next
daemon replays that log into a fresh adapter connection via `session/load`.
An ungraceful death (a crash, `kill -9`) leaves no resume marker, and the
session stays dead permanently.
- **Pending approvals and their nonces do not survive.** Both live only in
the old daemon's memory; any ACP permission request still mid-flight when
the daemon exits is answered `Cancelled` on the way down, and a resumed
session starts with no pending approval at all.
`docs/STRUCTURED-VIEW.md` is the full contract for all of this — the event
log's shape, the recovery marker, and exactly which kind of session death is
resumable.
## Timing
- **Grace window** — 1200 ms between the socket dropping and the overlay
+341
View File
@@ -0,0 +1,341 @@
# STRUCTURED-VIEW.md — the structured session contract
The contract for a structured session (`view: Structured`, #133): an agent run
as a daemon child over the Agent Client Protocol (ACP) instead of in a tmux
pane. Alongside `docs/WORKTREES.md`, `docs/SIDEBAR.md` and `docs/RESTART.md`,
read this before changing anything that renders a structured session — both
the web and TUI renderers are written against it.
It covers the eight transcript row kinds, what the daemon decides versus what
a client decides, the two projections and their delta/cursor semantics, the
ACP handshake and the capabilities it declines, the adapter's install
command, and the approval nonce rules. **Non-goal: the detached runner.** It
does not exist yet, and nothing here describes it.
## What a structured session is
A terminal session's agent runs inside a tmux pane; a structured session's
agent runs as a subprocess of the daemon itself, speaking ACP over stdio. The
daemon is the ACP **client**; the adapter process, `claude-code-acp`, is the
ACP **agent** — it wraps Claude Code's own SDK and translates the model's
tool calls into ACP `session/update` notifications instead of drawing them to
a pane. `crates/maestro-acp` owns the daemon's side of that connection;
`crates/maestro-daemon/src/projection.rs` folds what it receives into the two
projections below.
The child is a real child of the daemon process: same process group, marked
with `PR_SET_PDEATHSIG` so it dies with the daemon, and ended by
`AcpSession::terminate` (`session/cancel`, a short grace, `SIGTERM`, then
`SIGKILL`) from both `DELETE /api/sessions/:id` and daemon shutdown.
`docs/WORKTREES.md`'s kill-guard rule — an unforced kill refused while the
session's worktree holds unsaved work — applies to a structured session
exactly as it does to a terminal one; only what gets terminated differs (an
ACP handle instead of a tmux pane).
## The adapter
`ADAPTER_BIN` (`claude-code-acp`) is resolved by walking `PATH` — the
session's own `PATH` env var if the spawn request set one, the daemon's
otherwise — and taking the first executable file named `claude-code-acp` on
it. Install it once per host:
```sh
npm install -g --prefix ~/.local @zed-industries/claude-code-acp
```
That is `maestro_acp::INSTALL_COMMAND`, and it is not just documentation: it
is baked into `AcpError::AdapterMissing`'s message. When the adapter cannot
be found, spawn still creates the session — **Dead**, with that install
command as its `activity_detail` — so the sidebar names the fix instead of
the session silently never appearing (see `docs/INSTALL.md` for the full
install writeup, including the systemd `PATH` caveat). The adapter also needs
a logged-in Claude Code (`claude /login`); a missing login surfaces the same
way, via the adapter's own stderr tail in the failure message.
The handshake is bounded by `HANDSHAKE_TIMEOUT` (60s) — long enough for a
slow machine to boot a node process and the Claude Code SDK behind it, never
long enough to hang a spawn indefinitely. Any handshake failure — missing
adapter, early exit, a protocol error, or the timeout — leaves no orphaned
process: `spawn` closes and reaps the child itself before returning the
error.
## The ACP handshake
`initialize` is where the daemon states what kind of client it is. It
declares, deliberately, almost nothing:
| Capability | Declared | Why |
| --- | --- | --- |
| `fs.readTextFile` | `false` | The adapter uses Claude Code's own Read tool instead of asking the daemon to read a file on its behalf — the same tool a terminal-view Claude Code session uses. |
| `fs.writeTextFile` | `false` | Same reasoning, for Write. |
| `terminal` | `false` | The adapter runs Claude Code's own Bash tool rather than asking the daemon to open a terminal for it. |
Declining all three keeps the daemon out of the file/terminal surface
entirely — the adapter already has direct access to the session's `cwd`, and
routing every read, write, and shell command through the daemon would only
add a hop with no new capability. Every other capability namespace ACP
defines (session extensions, auth, elicitation) is simply left at its
default; the daemon does not advertise support because `claude-code-acp` has
never asked for it, not because it was weighed and refused.
The one client method that *is* implemented for real is
`session/request_permission` (#139) — see [Approvals](#approvals) below.
Before #139 it was a stub that declined every request; that stub is gone now
that a live implementation exists.
`session/new` opens a fresh ACP session on every ordinary spawn. A respawn
after a daemon restart instead sets `SpawnConfig::load_session(id)` and runs
`session/load` with the ACP session id the original handshake returned
(`Session::acp_session_id` on the wire) — the adapter must advertise
`agent_capabilities.load_session`, or the handshake fails outright naming
that. Whatever history the adapter streams back while `session/load` is in
flight is dropped on arrival, deliberately: the daemon's own event log
already holds that history, and forwarding it too would duplicate every row
in the projections below.
## The eight transcript row kinds
`TranscriptRowKind` (`maestro-protocol`) is the alphabet a structured
transcript is written in. Every row has a stable `id` (so later deltas can
address it), a `kind`, and a `data: Value` that preserves the underlying ACP
payload — clients depend on ACP's *shape* without depending on ACP as a
crate.
| Kind | Carries | Row identity |
| --- | --- | --- |
| `User` | An echoed `user_message_chunk` — the prompt text as ACP saw it. | `user:<message_id>`, or `user:<seq>` when the chunk carries no id. |
| `AgentText` | An `agent_message_chunk` — the agent's visible reply, streamed. | `agent:<message_id>` / `agent:<seq>`. Later chunks for the same id are appended into `data.content.text` in place, not appended as new rows. |
| `AgentThought` | An `agent_thought_chunk` — the model's reasoning stream. | `thought:<message_id>` / `thought:<seq>`, coalesced the same way as `AgentText`. |
| `ToolCall` | A `tool_call` / `tool_call_update` pair whose content carries no diff. | `tool:<tool_call_id>` — one row for the call's whole life; updates patch it in place, they never create a second row. |
| `ToolDiff` | The same tool-call machinery, when the tool's content includes a `Diff` block. | Same `tool:<tool_call_id>` identity as `ToolCall`; the *kind* flips between the two as the tool's own content changes, not the row. |
| `Approval` | A daemon-originated permission request (`RequestPermissionRequest`), serialized without its nonce. | `approval:<uuid>` — minted fresh per request, unrelated to the ACP request's own id. |
| `Plan` | A `plan` update — the agent's current step list. | The fixed id `"plan"`. Each new plan **replaces** the row wholesale; entries are not diffed against the previous plan. |
| `Notice` | The raw JSON of any `session/update` variant the projection does not special-case. | `notice:<seq>`. |
`Notice` is the projection's forward-compatibility seam: ACP adds
notification variants over time, and an unrecognised one becomes a generic
row instead of being silently dropped or making the daemon reject the whole
stream. A client that does not know what to do with a `Notice` can render it
as an opaque, collapsed line — that is a reasonable floor, not a client bug.
Rendering is entirely a client concern. The projection's job stops at
folding ACP into these eight shapes; markdown formatting, diff rendering,
tool-card treatment, and how a `Notice` is presented are all renderer
choices, not daemon ones.
## Daemon decisions vs. client decisions
**The daemon decides:**
- Which ACP `session/update` becomes which row kind, and under what id — the
fold in `projection.rs` is the one place this logic lives, so the web and
TUI renderers can never disagree about it.
- Turn lifecycle: when a turn begins, and whether an ACP response, a cancel,
or a failure ends it (`TurnState.responding` / `.thinking` /
`.active_tool_calls` are derived, not renderer-computed).
- Approval nonce minting, secrecy, and the single-active-approval rule (see
below) — a client never sees an ACP permission request before the daemon
has already turned it into an `Approval` row and a nonce.
- What happens to an ACP update variant nobody has taught the daemon about
yet (falls back to `Notice` rather than erroring).
- Persistence and replay: what is written to the event log, in what order,
and how a restart resumes it.
**A client decides:**
- How each row kind is drawn — markdown for `AgentText`/`AgentThought`, a
diff view for `ToolDiff`, a tool card for `ToolCall`, a checklist for
`Plan`, and a floor presentation for `Notice`.
- How `responding` vs. `thinking` are distinguished visually, if at all.
- How the operator is asked to resolve an `Approval` row — the daemon only
exposes the three-way decision (`AllowOnce` / `AllowForSession` / `Deny`),
never the UI that solicits it.
- Scrolling, collapsing, and every other presentation-only behaviour.
## The two projections
`Projection` (daemon-only) is folded once from the durable ACP stream and
exposes two independent views, both defined in `maestro-protocol`:
- **`TranscriptSnapshot { cursor, rows }`** — the complete ordered list of
transcript rows built so far, plus the log position (`cursor`) it reflects.
It is intentionally always the *whole* transcript, not a windowed one: a
tool row patched much later must still find its original entry.
- **`ReducedState`** — one whole-session snapshot folded over every event
seen: `turn` (the per-turn indicators above), `pending_approval`, `plan`,
`usage`/`context`, `model`, and `mode`. Like `TranscriptSnapshot`, it is
always full-history, never a delta — a client that only wants "is a turn
active right now" never has to replay anything to answer that.
Changes to the transcript travel as `TranscriptDelta`, tagged `type`:
`Append { row }` (a new row), `Patch { row }` (an existing row, addressed by
`id`, replaced wholesale), and `Remove { id }`. `Remove` is defined on the
wire for forward compatibility; nothing in the current projection emits it —
every row, once appended, is only ever patched or left alone.
### Cursor and replay semantics
The event log's own line position is the cursor: 1-based, shared by ACP
notifications and daemon-generated rows alike (`sidecar.rs`'s
`PersistedEvent` envelope is what lets an `Approval` row share a cursor
sequence with raw ACP JSON without pretending to be ACP traffic). Two ways to
read it:
- **`WS /api/structured/:id`** — on connect, sends a `TranscriptSnapshot` and
a `ReducedState` covering the whole log as it stands, then streams
`ProjectionUpdate { transcript, state }` for everything that happens next.
There is no client-supplied resume cursor on this socket: every connection
gets the full snapshot, not a gap-filled continuation. A client that needs
history from before it was connected uses the replay endpoint below, not
this socket.
- **`GET /api/sessions/:id/structured/replay`** — cursor-bounded pagination
over the durable log, returning one `StructuredReplayEvent` (`cursor`,
`transcript`, `state`) per matching line. `?since=<n>` returns the first
page after `n` (`since=0` is the whole log); `?before=<n>` returns the
page immediately before `n`, still in ascending order — passing both is a
`400`. `?limit` defaults to 100 and is clamped to 500. Each call re-folds
the entire log from scratch to compute the state at every returned cursor,
so a page reflects exactly the state *at* its cursor, not an
approximation — at the cost of redoing the whole fold per request. A
session that is not a live structured session (terminal, or a dead
adapter) answers `409 not_structured` from either route rather than a
coherent-looking but stale reply.
## Approvals
An ACP `session/request_permission` callback becomes an `Approval` row plus a
pending decision, entirely on the daemon side:
- **Generation.** The daemon mints the nonce itself — a fresh
`Uuid::new_v4()` string per request — rather than using anything ACP's own
request carries. ACP permission callbacks have no request id of their own
that would double as one.
- **Live-only placement.** The nonce lives in exactly one place: the
in-memory `pending_approvals` map keyed by nonce, and its mirror in
`ReducedState.pending_approval.nonce` on the live wire. It is never part of
the `Approval` row's `data` — the row is `RequestPermissionRequest`
serialized *without* the nonce.
- **Secrecy from the agent and from durable history.** The adapter itself
never learns the nonce — it is a daemon-to-client secret, never sent back
over the ACP connection. It is likewise never written to the event log:
restoring an `Approval` row from disk (`Projection::restore_approval`)
produces a historical row with no `pending_approval` in state, because the
nonce that would make it "pending" never made it to disk in the first
place. A restart therefore cannot resume a pending approval — see
[Persistence](#persistence-recovery-and-dead-sessions).
- **One live approval at a time.** A new permission request cancels whatever
was already pending (`cancel_pending_approvals`) before minting its own
nonce — ACP serializes requests through one connection, so this only ever
fires if an adapter nonetheless overlaps two, and the newest one wins.
- **Single-use and duplicate resolution.**
`POST /api/sessions/:id/structured/approvals/:nonce` **always answers
`204`** for a known, live structured session — whether the nonce is the
one currently pending, one already consumed by an earlier answer, or one
that never existed. Only an unknown or non-structured *session* answers
otherwise (`404` / `409`). This is deliberate: two clients racing to answer the same
approval must not turn a harmless duplicate click into a client-visible
error. Whichever `POST` reaches the daemon first removes the nonce from
`pending_approvals` and is the one whose decision actually reaches the
adapter; every later `POST` for that nonce is a no-op `204`.
- **Cancellation vs. denial.** A turn cancel or a session shutdown resolves
any pending approval with ACP's `Cancelled` outcome, not a `Deny` decision
— the adapter should read those as "the operation was withdrawn," not "the
operator refused it."
- **Decision mapping.** `AllowOnce` and `AllowForSession` select the ACP
option of kind `AllowOnce` / `AllowAlways` from the request's own option
list. `Deny` prefers a one-shot refusal (`RejectOnce`) and falls back to a
permanent one (`RejectAlways`) only when that is the sole rejection option
offered; if neither is present, it answers `Cancelled`.
## Prompt and cancel: bounded liveness
`POST /api/sessions/:id/structured/prompt` and `.../cancel` are serialized
against each other through one per-session `operation` lock
(`AcpEventStream::operation`), because ACP permission callbacks carry no
per-request id — the daemon's own notion of "the active turn" is the only
thing a callback can be attributed to, so at most one turn may ever be
in flight, and a cancelled turn must fully settle before the next prompt is
allowed to claim the daemon's idea of "the active turn."
- A prompt claims the turn and dispatches it to ACP as one operation. While a
turn is active, a second prompt gets `409 turn_in_flight` rather than
queuing — one active turn per session, never a hidden queue of prompts.
- Cancel sends `session/cancel`, clears any pending approval as `Cancelled`,
and ends the turn's client-visible lifecycle immediately. If the cancelled
turn was still *dispatched* to ACP (waiting on its response), the daemon
cannot simply forget it: ACP has not confirmed the cancel yet, and a new
prompt claiming "the active turn" while the old one is still outstanding
would make the next permission callback ambiguous between the two.
- So a cancelled-but-still-dispatched turn must **settle within
`CANCEL_GRACE` (500ms)** — either ACP acknowledges the prompt (however it
resolves) or that grace period runs out. If it settles in time, the next
prompt proceeds normally. If it does not, the daemon does not guess: it
marks the whole structured session dead (`fail_cancelled_acp_turn`) rather
than risk a callback resolving against the wrong turn. This is the bounded
half of "bounded liveness" — a structured session's prompt/cancel dance
either resolves promptly or the session is declared unusable, never left
in an indefinite ambiguous state.
## Persistence, recovery, and dead sessions
A structured session's durable state is three files next to its sidecar
(`<id>.json`) in the sidecar directory:
- **`<id>.events.jsonl`** — the unified event log. Each line is either a raw
ACP `SessionNotification` (kept in its original JSON so old logs stay
readable) or a `{"maestroEvent": "approval", "row": …}` envelope for a
daemon-generated row. Both share the same 1-based line-position cursor. A
line a hard kill left half-written is terminated before the next append, so
it keeps its position and is skipped as malformed rather than corrupting
what follows.
- **`<id>.recover`** — an empty marker file. Its only meaning is "the daemon
that wrote this shut down gracefully while this session was running, and
the next daemon may attempt to resume it." It is consumed (deleted) the
moment a recovery attempt reads it — one recovery attempt per graceful
shutdown, never a standing flag.
Three ways a structured session ends up `Dead`, each with different
persistence consequences:
1. **The adapter exits, or the log append fails, while the daemon keeps
running.** `mark_session_dead` applies the same natural-exit worktree rule
as a terminal session's own pane exiting (`docs/WORKTREES.md`): a pristine
worktree is removed, one holding unsaved work is left on disk. No recovery
marker is written. This death is permanent — a later daemon restart's
`recover()` finds the sidecar, sees no marker, and leaves the session
`Dead` without attempting to resume it.
2. **A cancelled turn fails to settle within `CANCEL_GRACE`.** Same
consequence as (1): the session is marked dead through the same path, no
marker, no future resumption.
3. **Graceful daemon shutdown**, while the session was still `Running`.
`mark_session_dead_after_shutdown` marks the session dead **without**
applying the worktree exit rule — the tree is kept exactly as it is,
pristine or not, because there is nothing "natural" about this exit — and
writes the `.recover` marker. The sidecar on disk therefore says `Dead`,
but is eligible for resumption.
Recovery, on the next daemon's `recover()`: for every `Structured` sidecar
not already adopted, the marker is consumed. When one was present and the
sidecar carries a stored `acp_session_id`, the daemon spawns a fresh adapter
with `SpawnConfig::load_session(acp_session_id)`, replays the
entire durable event log into a new `Projection` (rebuilding the transcript
and reduced state exactly as they were, with any restored `Approval` row
landing as history — never as a re-armed pending approval, since its nonce
never reached disk), and only then reopens the log for further appends. What
the adapter streams back while its own `session/load` is in flight is
discarded rather than appended, so the replayed history is not duplicated
(see [The ACP handshake](#the-acp-handshake)). Success flips the session back
to `Running`/`Active`; failure (no stored `acp_session_id`, or the handshake
itself failing) leaves it `Dead` with the failure reason as
`activity_detail` — the same "tell the operator what to fix" pattern as a
missing adapter binary.
Pending approvals and their nonces never survive any of this. They live only
in the in-memory `pending_approvals` map and the live `ReducedState`, both
gone the instant the daemon that held them exits. Whatever ACP permission
request was mid-flight when the old daemon died was already answered
`Cancelled` on that daemon's own shutdown path, so a resumed session always
starts with no pending approval.
See `docs/RESTART.md` for what a *client* observes across a daemon restart,
now that structured sessions exist alongside terminal ones.