maestro/docs/RESTART.md

# Restart and upgrade contract

Maestro is upgraded in place: the operator reinstalls the binary and systemd
restarts the daemon under every browser tab and TUI that is attached to it.
tmux keeps the sessions alive across that (see `docs/INSTALL.md`), but clients
lose their sockets. This document is the contract for what a client does about
it — what the daemon publishes, how a client tells a restart from a blip, and
what the user sees while it happens.

## What the daemon publishes

`GET /api/info` carries two fields beyond identity and presets:

| field      | meaning                                                        |
| ---------- | -------------------------------------------------------------- |
| `build`    | `{ commit, commit_date, dirty }` — the commit the binary was compiled from |
| `instance` | uuid regenerated on every daemon start                          |

`build` is stamped at compile time by `crates/maestro-daemon/build.rs`, which
runs `git` in the checkout it is compiled in. The Nix build has no `.git` in the
crane-filtered source tree, so `flake.nix` passes the flake's own revision as
`MAESTRO_BUILD_COMMIT` / `MAESTRO_BUILD_DATE` / `MAESTRO_BUILD_DIRTY`. A build
with neither source reports empty strings — clients render that as
"unknown release" and fall back to `instance` alone.

`instance` is the one that never goes missing. It says *this process*, so two
replies that differ on it are two runs of the daemon, even when the commit did
not change (a plain `systemctl restart`, or a crash).

## How a client reads them

Three verdicts, from comparing the reply before an outage with the one after
(`compareIdentity` in `web/src/daemonLink.ts`, mirrored by
`crates/maestro-tui/src/daemon_link.rs`):

- **upgraded** — both commits are known and differ. New server code.
- **restarted** — commits match (or are unknown) and `instance` differs.
- **same** — neither changed; the socket blipped, the daemon never moved.

Only *upgraded* is special. A browser tab is holding a bundle that the upgraded
daemon no longer serves, so it reloads the page rather than driving new server
code from an old bundle. The TUI is its own binary — a reload would not replace
it — so it stays up, reconnects, and says which release it is now talking to;
the user restarts `maestro tui` when they want the new client.

## What the user sees

The **status indicator** lives in the sidebar footer, left of the spawn button,
in both clients. Its colour is the link state, and it names the running release
— commit hash and commit date — on hover (web) or in the status bar (TUI).

| state          | colour | meaning                                              |
| -------------- | ------ | ---------------------------------------------------- |
| `connecting`   | amber  | first probe hasn't landed                            |
| `online`       | green  | daemon answered; same process, or a resolved new one |
| `reconnecting` | amber  | socket dropped, still inside the grace window        |
| `down`         | red    | unreachable past the grace window — the UI blocks    |
| `updating`     | red    | answered on a new commit; the page is reloading      |

`updating` is web-only, because reloading is web-only. The TUI goes straight
back to `online` after an upgrade and puts `updated: <hash> (restart tui)` on
its status bar instead, until the user restarts it.

The **restart overlay** is up for exactly `down` and `updating`. It is not
dismissable and it swallows the app's keyboard shortcuts: everything behind it
would act on a daemon that is not there. In the TUI the quit binding still
resolves, so a blocked client is never a trapped one. It clears itself the
moment a probe succeeds — there is no "retry" button, because the client is
already retrying.

## What survives, and what comes back dormant

A daemon restart leaves tmux alone, so recovery re-adopts every
`maestro-<uuid>` session tmux still holds and the operator sees no change
beyond the reconnect above.

A **host reboot** is different: it kills the tmux server, so there is nothing
to adopt while every session's sidecar is still on disk. Recovery makes a
second pass over those sidecars and registers each as a `Dormant` session
(#162) — id, name, cwd, command, worktree and `created_at` intact. A dormant
session has no pane and no PTY bridge: it cannot be attached to, typed into or
resized, and neither watcher pass touches it.

A dormant session is brought back with `POST /api/sessions/:id/resume` (#166),
which respawns tmux under the **same** uuid and sidecar — `metadata.name`,
`created_at`, `env` and worktree ownership all survive, and only the tmux
session and the PTY bridge are new. The command is the matching spawn preset's
`resume_command` with `{agent_session}` filled in from the session's own
`agent_session_id`, so an agent comes back inside its old conversation; a
session with no matching preset, no `resume_command`, or no reported
conversation id is respawned on the command it was created with. `Session::command`
always keeps that original command, so a second resume resolves the same preset.
Resume is refused with `409` for a session that is already running
(`already_running`) and for one whose `cwd` has disappeared (`cwd_missing`) —
neither is something a respawn could fix. The other thing to do with a dormant
session is dismiss it, which removes the sidecar so the next recovery does not
bring it back.

## What recovery does with them: `resume_on_boot`

Whether a recovered-but-dead session is left dormant at all is the operator's
call, per host, via `[daemon].resume_on_boot` (#167):

| value       | what recovery does                                                    |
| ----------- | --------------------------------------------------------------------- |
| `"dormant"` | **Default.** Register each one dormant and wait for the operator.      |
| `"auto"`    | Resume each one during recovery, itself.                               |
| `"never"`   | Prune the sidecars instead — a reboot is a clean slate.                |

`dormant` is the default because resuming agents unattended is a foot-gun:
they come back on whatever flags they were spawned with, seconds after a
reboot, with nobody watching. `auto` exists for hosts where that is known to
be safe, and it is opted into rather than inherited.

`auto` calls the same `resume` the client's `POST /api/sessions/:id/resume`
does — same uuid, same sidecar, the preset's `resume_command` — so there is
one resume path and not two. A session that fails to resume is logged and
stays dormant; the rest still come back.

`never` prunes the same way #163 does for a missing `cwd`: the sidecar is
deleted and nothing is registered, so the session is not listed and the next
recovery does not find it. Whatever worktree it owned is left on disk.

The policy only decides the fate of sessions with no live pane. A session
tmux still holds is adopted `Running` by the first pass under every policy,
including `never`.

Recovery logs the policy that ran and the number of sessions it acted on, so
a host that resumes agents unattended says so in its own journal.

Attaching to a session that has no bridge answers `409 not_attachable`, not
`404 unknown_session`: the session exists, and a client that cannot tell "gone"
from "not running" cannot offer the resume.

`SessionStatus` is a wire type, so from this build forward it deserializes
leniently on both clients: a spelling the client does not know reads as `Dead`
rather than failing the whole session. That leniency lives in the client's own
binary, so it covers spellings added *after* this build, not this one — a CLI
or TUI still running an older build fails to parse a `Dormant` session, and
with it the whole session list, until it is upgraded. Clients render a dormant session as known-but-not-running —
see `docs/SIDEBAR.md` for the card and `docs/COMMAND-BAR.md` for how it ranks.

## Timing

- **Grace window** — 1200 ms between the socket dropping and the overlay
  appearing. A wifi hiccup or a laptop waking up should not throw a modal.
- **Poll cadence** — `GET /api/info` every 700 ms while unreachable, flat, not
  backed off: a restart takes a second or two and the point is to unlock as soon
  as it is over. The events socket keeps its own exponential backoff for the
  session stream; the two are independent, and the probe is what decides state.

The events socket dropping is the trigger, not a timer: the daemon cannot warn
anyone it is about to die, so its socket closing is the earliest signal there
is. A reconnect of that socket does **not** by itself clear the state — only a
probe does, because the socket carries no identity and reconnecting to an
upgraded daemon must reload rather than resume.