maestro/docs/STRUCTURED-VIEW.md

# STRUCTURED-VIEW.md — the structured session contract

The contract for a structured session (`view: Structured`, #133): an agent run
as a daemon child over the Agent Client Protocol (ACP) instead of in a tmux
pane. Alongside `docs/WORKTREES.md`, `docs/SIDEBAR.md` and `docs/RESTART.md`,
read this before changing anything that renders a structured session — both
the web and TUI renderers are written against it.

It covers the eight transcript row kinds, what the daemon decides versus what
a client decides, the two projections and their delta/cursor semantics, the
ACP handshake and the capabilities it declines, the adapter's install
command, and the approval nonce rules. **Non-goal: the detached runner.** It
does not exist yet, and nothing here describes it.

## What a structured session is

A terminal session's agent runs inside a tmux pane; a structured session's
agent runs as a subprocess of the daemon itself, speaking ACP over stdio. The
daemon is the ACP **client**; the adapter process, `claude-code-acp`, is the
ACP **agent** — it wraps Claude Code's own SDK and translates the model's
tool calls into ACP `session/update` notifications instead of drawing them to
a pane. `crates/maestro-acp` owns the daemon's side of that connection;
`crates/maestro-daemon/src/projection.rs` folds what it receives into the two
projections below.

The child is a real child of the daemon process: same process group, marked
with `PR_SET_PDEATHSIG` so it dies with the daemon, and ended by
`AcpSession::terminate` (`session/cancel`, a short grace, `SIGTERM`, then
`SIGKILL`) from both `DELETE /api/sessions/:id` and daemon shutdown.
`docs/WORKTREES.md`'s kill-guard rule — an unforced kill refused while the
session's worktree holds unsaved work — applies to a structured session
exactly as it does to a terminal one; only what gets terminated differs (an
ACP handle instead of a tmux pane).

## The adapter

`ADAPTER_BIN` (`claude-code-acp`) is resolved by walking `PATH` — the
session's own `PATH` env var if the spawn request set one, the daemon's
otherwise — and taking the first executable file named `claude-code-acp` on
it. Install it once per host:

```sh
npm install -g --prefix ~/.local @zed-industries/claude-code-acp
```

That is `maestro_acp::INSTALL_COMMAND`, and it is not just documentation: it
is baked into `AcpError::AdapterMissing`'s message. When the adapter cannot
be found, spawn still creates the session — **Dead**, with that install
command as its `activity_detail` — so the sidebar names the fix instead of
the session silently never appearing (see `docs/INSTALL.md` for the full
install writeup, including the systemd `PATH` caveat). The adapter also needs
a logged-in Claude Code (`claude /login`); a missing login surfaces the same
way, via the adapter's own stderr tail in the failure message.

The handshake is bounded by `HANDSHAKE_TIMEOUT` (60s) — long enough for a
slow machine to boot a node process and the Claude Code SDK behind it, never
long enough to hang a spawn indefinitely. Any handshake failure — missing
adapter, early exit, a protocol error, or the timeout — leaves no orphaned
process: `spawn` closes and reaps the child itself before returning the
error.

## The ACP handshake

`initialize` is where the daemon states what kind of client it is. It
declares, deliberately, almost nothing:

| Capability | Declared | Why |
| --- | --- | --- |
| `fs.readTextFile` | `false` | The adapter uses Claude Code's own Read tool instead of asking the daemon to read a file on its behalf — the same tool a terminal-view Claude Code session uses. |
| `fs.writeTextFile` | `false` | Same reasoning, for Write. |
| `terminal` | `false` | The adapter runs Claude Code's own Bash tool rather than asking the daemon to open a terminal for it. |

Declining all three keeps the daemon out of the file/terminal surface
entirely — the adapter already has direct access to the session's `cwd`, and
routing every read, write, and shell command through the daemon would only
add a hop with no new capability. Every other capability namespace ACP
defines (session extensions, auth, elicitation) is simply left at its
default; the daemon does not advertise support because `claude-code-acp` has
never asked for it, not because it was weighed and refused.

The one client method that *is* implemented for real is
`session/request_permission` (#139) — see [Approvals](#approvals) below.
Before #139 it was a stub that declined every request; that stub is gone now
that a live implementation exists.

`session/new` opens a fresh ACP session on every ordinary spawn. A respawn
after a daemon restart instead sets `SpawnConfig::load_session(id)` and runs
`session/load` with the ACP session id the original handshake returned
(`Session::acp_session_id` on the wire) — the adapter must advertise
`agent_capabilities.load_session`, or the handshake fails outright naming
that. Whatever history the adapter streams back while `session/load` is in
flight is dropped on arrival, deliberately: the daemon's own event log
already holds that history, and forwarding it too would duplicate every row
in the projections below.

## The eight transcript row kinds

`TranscriptRowKind` (`maestro-protocol`) is the alphabet a structured
transcript is written in. Every row has a stable `id` (so later deltas can
address it), a `kind`, and a `data: Value` that preserves the underlying ACP
payload — clients depend on ACP's *shape* without depending on ACP as a
crate.

| Kind | Carries | Row identity |
| --- | --- | --- |
| `User` | An echoed `user_message_chunk` — the prompt text as ACP saw it. | `user:<message_id>`, or `user:<seq>` when the chunk carries no id. |
| `AgentText` | An `agent_message_chunk` — the agent's visible reply, streamed. | `agent:<message_id>` / `agent:<seq>`. Later chunks for the same id are appended into `data.content.text` in place, not appended as new rows. |
| `AgentThought` | An `agent_thought_chunk` — the model's reasoning stream. | `thought:<message_id>` / `thought:<seq>`, coalesced the same way as `AgentText`. |
| `ToolCall` | A `tool_call` / `tool_call_update` pair whose content carries no diff. | `tool:<tool_call_id>` — one row for the call's whole life; updates patch it in place, they never create a second row. |
| `ToolDiff` | The same tool-call machinery, when the tool's content includes a `Diff` block. | Same `tool:<tool_call_id>` identity as `ToolCall`; the *kind* flips between the two as the tool's own content changes, not the row. |
| `Approval` | A daemon-originated permission request (`RequestPermissionRequest`), serialized without its nonce. | `approval:<uuid>` — minted fresh per request, unrelated to the ACP request's own id. |
| `Plan` | A `plan` update — the agent's current step list. | The fixed id `"plan"`. Each new plan **replaces** the row wholesale; entries are not diffed against the previous plan. |
| `Notice` | The raw JSON of any `session/update` variant the projection does not special-case. | `notice:<seq>`. |

`Notice` is the projection's forward-compatibility seam: ACP adds
notification variants over time, and an unrecognised one becomes a generic
row instead of being silently dropped or making the daemon reject the whole
stream. A client that does not know what to do with a `Notice` can render it
as an opaque, collapsed line — that is a reasonable floor, not a client bug.

Rendering is entirely a client concern. The projection's job stops at
folding ACP into these eight shapes; markdown formatting, diff rendering,
tool-card treatment, and how a `Notice` is presented are all renderer
choices, not daemon ones.

## Daemon decisions vs. client decisions

**The daemon decides:**

- Which ACP `session/update` becomes which row kind, and under what id — the
  fold in `projection.rs` is the one place this logic lives, so the web and
  TUI renderers can never disagree about it.
- Turn lifecycle: when a turn begins, and whether an ACP response, a cancel,
  or a failure ends it (`TurnState.responding` / `.thinking` /
  `.active_tool_calls` are derived, not renderer-computed).
- Approval nonce minting, secrecy, and the single-active-approval rule (see
  below) — a client never sees an ACP permission request before the daemon
  has already turned it into an `Approval` row and a nonce.
- What happens to an ACP update variant nobody has taught the daemon about
  yet (falls back to `Notice` rather than erroring).
- Persistence and replay: what is written to the event log, in what order,
  and how a restart resumes it.

**A client decides:**

- How each row kind is drawn — markdown for `AgentText`/`AgentThought`, a
  diff view for `ToolDiff`, a tool card for `ToolCall`, a checklist for
  `Plan`, and a floor presentation for `Notice`.
- How `responding` vs. `thinking` are distinguished visually, if at all.
- How the operator is asked to resolve an `Approval` row — the daemon only
  exposes the three-way decision (`AllowOnce` / `AllowForSession` / `Deny`),
  never the UI that solicits it.
- Scrolling, collapsing, and every other presentation-only behaviour.

## The two projections

`Projection` (daemon-only) is folded once from the durable ACP stream and
exposes two independent views, both defined in `maestro-protocol`:

- **`TranscriptSnapshot { cursor, rows }`** — the complete ordered list of
  transcript rows built so far, plus the log position (`cursor`) it reflects.
  It is intentionally always the *whole* transcript, not a windowed one: a
  tool row patched much later must still find its original entry.
- **`ReducedState`** — one whole-session snapshot folded over every event
  seen: `turn` (the per-turn indicators above), `pending_approval`, `plan`,
  `usage`/`context`, `model`, and `mode`. Like `TranscriptSnapshot`, it is
  always full-history, never a delta — a client that only wants "is a turn
  active right now" never has to replay anything to answer that.

Changes to the transcript travel as `TranscriptDelta`, tagged `type`:
`Append { row }` (a new row), `Patch { row }` (an existing row, addressed by
`id`, replaced wholesale), and `Remove { id }`. `Remove` is defined on the
wire for forward compatibility; nothing in the current projection emits it —
every row, once appended, is only ever patched or left alone.

### Cursor and replay semantics

The event log's own line position is the cursor: 1-based, shared by ACP
notifications and daemon-generated rows alike (`sidecar.rs`'s
`PersistedEvent` envelope is what lets an `Approval` row share a cursor
sequence with raw ACP JSON without pretending to be ACP traffic). Two ways to
read it:

- **`WS /api/structured/:id`** — on connect, sends a `TranscriptSnapshot` and
  a `ReducedState` covering the whole log as it stands, then streams
  `ProjectionUpdate { transcript, state }` for everything that happens next.
  There is no client-supplied resume cursor on this socket: every connection
  gets the full snapshot, not a gap-filled continuation. A client that needs
  history from before it was connected uses the replay endpoint below, not
  this socket.
- **`GET /api/sessions/:id/structured/replay`** — cursor-bounded pagination
  over the durable log, returning one `StructuredReplayEvent` (`cursor`,
  `transcript`, `state`) per matching line. `?since=<n>` returns the first
  page after `n` (`since=0` is the whole log); `?before=<n>` returns the
  page immediately before `n`, still in ascending order — passing both is a
  `400`. `?limit` defaults to 100 and is clamped to 500. Each call re-folds
  the entire log from scratch to compute the state at every returned cursor,
  so a page reflects exactly the state *at* its cursor, not an
  approximation — at the cost of redoing the whole fold per request. A
  session that is not a live structured session (terminal, or a dead
  adapter) answers `409 not_structured` from either route rather than a
  coherent-looking but stale reply.

## Approvals

An ACP `session/request_permission` callback becomes an `Approval` row plus a
pending decision, entirely on the daemon side:

- **Generation.** The daemon mints the nonce itself — a fresh
  `Uuid::new_v4()` string per request — rather than using anything ACP's own
  request carries. ACP permission callbacks have no request id of their own
  that would double as one.
- **Live-only placement.** The nonce lives in exactly one place: the
  in-memory `pending_approvals` map keyed by nonce, and its mirror in
  `ReducedState.pending_approval.nonce` on the live wire. It is never part of
  the `Approval` row's `data` — the row is `RequestPermissionRequest`
  serialized *without* the nonce.
- **Secrecy from the agent and from durable history.** The adapter itself
  never learns the nonce — it is a daemon-to-client secret, never sent back
  over the ACP connection. It is likewise never written to the event log:
  restoring an `Approval` row from disk (`Projection::restore_approval`)
  produces a historical row with no `pending_approval` in state, because the
  nonce that would make it "pending" never made it to disk in the first
  place. A restart therefore cannot resume a pending approval — see
  [Persistence](#persistence-recovery-and-dead-sessions).
- **One live approval at a time.** A new permission request cancels whatever
  was already pending (`cancel_pending_approvals`) before minting its own
  nonce — ACP serializes requests through one connection, so this only ever
  fires if an adapter nonetheless overlaps two, and the newest one wins.
- **Single-use and duplicate resolution.**
  `POST /api/sessions/:id/structured/approvals/:nonce` **always answers
  `204`** for a known, live structured session — whether the nonce is the
  one currently pending, one already consumed by an earlier answer, or one
  that never existed. Only an unknown or non-structured *session* answers
  otherwise (`404` / `409`). This is deliberate: two clients racing to answer the same
  approval must not turn a harmless duplicate click into a client-visible
  error. Whichever `POST` reaches the daemon first removes the nonce from
  `pending_approvals` and is the one whose decision actually reaches the
  adapter; every later `POST` for that nonce is a no-op `204`.
- **Cancellation vs. denial.** A turn cancel or a session shutdown resolves
  any pending approval with ACP's `Cancelled` outcome, not a `Deny` decision
  — the adapter should read those as "the operation was withdrawn," not "the
  operator refused it."
- **Decision mapping.** `AllowOnce` and `AllowForSession` select the ACP
  option of kind `AllowOnce` / `AllowAlways` from the request's own option
  list. `Deny` prefers a one-shot refusal (`RejectOnce`) and falls back to a
  permanent one (`RejectAlways`) only when that is the sole rejection option
  offered; if neither is present, it answers `Cancelled`.

## Prompt and cancel: bounded liveness

`POST /api/sessions/:id/structured/prompt` and `.../cancel` are serialized
against each other through one per-session `operation` lock
(`AcpEventStream::operation`), because ACP permission callbacks carry no
per-request id — the daemon's own notion of "the active turn" is the only
thing a callback can be attributed to, so at most one turn may ever be
in flight, and a cancelled turn must fully settle before the next prompt is
allowed to claim the daemon's idea of "the active turn."

- A prompt claims the turn and dispatches it to ACP as one operation. While a
  turn is active, a second prompt gets `409 turn_in_flight` rather than
  queuing — one active turn per session, never a hidden queue of prompts.
- Cancel sends `session/cancel`, clears any pending approval as `Cancelled`,
  and ends the turn's client-visible lifecycle immediately. If the cancelled
  turn was still *dispatched* to ACP (waiting on its response), the daemon
  cannot simply forget it: ACP has not confirmed the cancel yet, and a new
  prompt claiming "the active turn" while the old one is still outstanding
  would make the next permission callback ambiguous between the two.
- So a cancelled-but-still-dispatched turn must **settle within
  `CANCEL_GRACE` (500ms)** — either ACP acknowledges the prompt (however it
  resolves) or that grace period runs out. If it settles in time, the next
  prompt proceeds normally. If it does not, the daemon does not guess: it
  marks the whole structured session dead (`fail_cancelled_acp_turn`) rather
  than risk a callback resolving against the wrong turn. This is the bounded
  half of "bounded liveness" — a structured session's prompt/cancel dance
  either resolves promptly or the session is declared unusable, never left
  in an indefinite ambiguous state.

## Persistence, recovery, and dead sessions

A structured session's durable state is three files next to its sidecar
(`<id>.json`) in the sidecar directory:

- **`<id>.events.jsonl`** — the unified event log. Each line is either a raw
  ACP `SessionNotification` (kept in its original JSON so old logs stay
  readable) or a `{"maestroEvent": "approval", "row": …}` envelope for a
  daemon-generated row. Both share the same 1-based line-position cursor. A
  line a hard kill left half-written is terminated before the next append, so
  it keeps its position and is skipped as malformed rather than corrupting
  what follows.
- **`<id>.recover`** — an empty marker file. Its only meaning is "the daemon
  that wrote this shut down gracefully while this session was running, and
  the next daemon may attempt to resume it." It is consumed (deleted) the
  moment a recovery attempt reads it — one recovery attempt per graceful
  shutdown, never a standing flag.

Three ways a structured session ends up `Dead`, each with different
persistence consequences:

1. **The adapter exits, or the log append fails, while the daemon keeps
   running.** `mark_session_dead` applies the same natural-exit worktree rule
   as a terminal session's own pane exiting (`docs/WORKTREES.md`): a pristine
   worktree is removed, one holding unsaved work is left on disk. No recovery
   marker is written. This death is permanent — a later daemon restart's
   `recover()` finds the sidecar, sees no marker, and leaves the session
   `Dead` without attempting to resume it.
2. **A cancelled turn fails to settle within `CANCEL_GRACE`.** Same
   consequence as (1): the session is marked dead through the same path, no
   marker, no future resumption.
3. **Graceful daemon shutdown**, while the session was still `Running`.
   `mark_session_dead_after_shutdown` marks the session dead **without**
   applying the worktree exit rule — the tree is kept exactly as it is,
   pristine or not, because there is nothing "natural" about this exit — and
   writes the `.recover` marker. The sidecar on disk therefore says `Dead`,
   but is eligible for resumption.

Recovery, on the next daemon's `recover()`: for every `Structured` sidecar
not already adopted, the marker is consumed. When one was present and the
sidecar carries a stored `acp_session_id`, the daemon spawns a fresh adapter
with `SpawnConfig::load_session(acp_session_id)`, replays the
entire durable event log into a new `Projection` (rebuilding the transcript
and reduced state exactly as they were, with any restored `Approval` row
landing as history — never as a re-armed pending approval, since its nonce
never reached disk), and only then reopens the log for further appends. What
the adapter streams back while its own `session/load` is in flight is
discarded rather than appended, so the replayed history is not duplicated
(see [The ACP handshake](#the-acp-handshake)). Success flips the session back
to `Running`/`Active`; failure (no stored `acp_session_id`, or the handshake
itself failing) leaves it `Dead` with the failure reason as
`activity_detail` — the same "tell the operator what to fix" pattern as a
missing adapter binary.

Pending approvals and their nonces never survive any of this. They live only
in the in-memory `pending_approvals` map and the live `ReducedState`, both
gone the instant the daemon that held them exits. Whatever ACP permission
request was mid-flight when the old daemon died was already answered
`Cancelled` on that daemon's own shutdown path, so a resumed session always
starts with no pending approval.

See `docs/RESTART.md` for what a *client* observes across a daemon restart,
now that structured sessions exist alongside terminal ones.