Corrects recovery-as-two-operations, tmux-only survival on restart, replay's durable-vs-live-state gap, snapshot-vs-replay purpose, and the three-file persistence count. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
6.4 KiB
Restart and upgrade contract
Maestro is upgraded in place: the operator reinstalls the binary and systemd
restarts the daemon under every browser tab and TUI that is attached to it.
tmux keeps terminal sessions alive across that (see docs/INSTALL.md),
but clients lose their sockets either way. This document is the contract for
what a client does about it — what the daemon publishes, how a client tells a
restart from a blip, and what the user sees while it happens.
A structured session (docs/STRUCTURED-VIEW.md) has no tmux pane behind
it to survive on: its agent is a child of the daemon process itself, so a
restart takes it down with the daemon. See
Structured sessions below for exactly what that
costs and what it doesn't.
What the daemon publishes
GET /api/info carries two fields beyond identity and presets:
| field | meaning |
|---|---|
build |
{ commit, commit_date, dirty } — the commit the binary was compiled from |
instance |
uuid regenerated on every daemon start |
build is stamped at compile time by crates/maestro-daemon/build.rs, which
runs git in the checkout it is compiled in. The Nix build has no .git in the
crane-filtered source tree, so flake.nix passes the flake's own revision as
MAESTRO_BUILD_COMMIT / MAESTRO_BUILD_DATE / MAESTRO_BUILD_DIRTY. A build
with neither source reports empty strings — clients render that as
"unknown release" and fall back to instance alone.
instance is the one that never goes missing. It says this process, so two
replies that differ on it are two runs of the daemon, even when the commit did
not change (a plain systemctl restart, or a crash).
How a client reads them
Three verdicts, from comparing the reply before an outage with the one after
(compareIdentity in web/src/daemonLink.ts, mirrored by
crates/maestro-tui/src/daemon_link.rs):
- upgraded — both commits are known and differ. New server code.
- restarted — commits match (or are unknown) and
instancediffers. - same — neither changed; the socket blipped, the daemon never moved.
Only upgraded is special. A browser tab is holding a bundle that the upgraded
daemon no longer serves, so it reloads the page rather than driving new server
code from an old bundle. The TUI is its own binary — a reload would not replace
it — so it stays up, reconnects, and says which release it is now talking to;
the user restarts maestro tui when they want the new client.
What the user sees
The status indicator lives in the sidebar footer, left of the spawn button, in both clients. Its colour is the link state, and it names the running release — commit hash and commit date — on hover (web) or in the status bar (TUI).
| state | colour | meaning |
|---|---|---|
connecting |
amber | first probe hasn't landed |
online |
green | daemon answered; same process, or a resolved new one |
reconnecting |
amber | socket dropped, still inside the grace window |
down |
red | unreachable past the grace window — the UI blocks |
updating |
red | answered on a new commit; the page is reloading |
updating is web-only, because reloading is web-only. The TUI goes straight
back to online after an upgrade and puts updated: <hash> (restart tui) on
its status bar instead, until the user restarts it.
The restart overlay is up for exactly down and updating. It is not
dismissable and it swallows the app's keyboard shortcuts: everything behind it
would act on a daemon that is not there. In the TUI the quit binding still
resolves, so a blocked client is never a trapped one. It clears itself the
moment a probe succeeds — there is no "retry" button, because the client is
already retrying.
Structured sessions
Everything above describes a client's view of a restart, and it holds for a structured session exactly as written — the same three verdicts, the same status indicator, the same overlay. What differs is what survives on the daemon side, because a structured session's agent is a daemon child, not a tmux pane:
- A restart ends the active turn. There is no adapter process left to finish it, and no tmux pane keeping it alive underneath.
- The transcript survives, in the per-session event log
(
docs/STRUCTURED-VIEW.md#persistence-recovery-and-dead-sessions) — a graceful shutdown marks the session for one resume attempt. Recovery is two separate operations, not one: the next daemon callssession/loadso the adapter restores its own ACP session, but whatever history the adapter emits during that call is discarded, not replayed into anything — the daemon already has that history on disk. Separately, the daemon rebuilds its own projection by folding the persisted event log back through it. An ungraceful death (a crash,kill -9) leaves no resume marker, and the session stays dead permanently. - Pending approvals and their nonces do not survive. Both live only in
the old daemon's memory; any ACP permission request still mid-flight when
the daemon exits is answered
Cancelledon the way down, and a resumed session starts with no pending approval at all.
docs/STRUCTURED-VIEW.md is the full contract for all of this — the event
log's shape, the recovery marker, and exactly which kind of session death is
resumable.
Timing
- Grace window — 1200 ms between the socket dropping and the overlay appearing. A wifi hiccup or a laptop waking up should not throw a modal.
- Poll cadence —
GET /api/infoevery 700 ms while unreachable, flat, not backed off: a restart takes a second or two and the point is to unlock as soon as it is over. The events socket keeps its own exponential backoff for the session stream; the two are independent, and the probe is what decides state.
The events socket dropping is the trigger, not a timer: the daemon cannot warn anyone it is about to die, so its socket closing is the earliest signal there is. A reconnect of that socket does not by itself clear the state — only a probe does, because the socket carries no identity and reconnecting to an upgraded daemon must reload rather than resume.