Compare commits

..
Author SHA1 Message Date
Miguel Palhas 87f1e73dec fix(maestro): replace agent_session_id when a report names a new one
A resume issues a new conversation id, so write-once left the stored id
pointing at the pre-resume conversation and every later resume reattached
it. Rewrite the sidecar when the reported id differs from the stored one;
a repeated report of the same id still writes nothing.

Closes #176
2026-09-04 09:26:16 +01:00
Miguel Palhas aebd10c2da merge: guard resume against kill and second resume (#175)
Closes #175
2026-09-04 09:24:35 +01:00
Miguel Palhas ae3091e2a2 fix(maestro): guard resume against kill and second resume
The respawn runs with the session map unlocked, so two takes of the
lock straddle it. A kill landing in that window deleted the sidecar
before resume wrote it back, leaving a file for a session that was
gone — the next recovery registered it as a dormant ghost. Resume now
deletes the sidecar in the branch where it finds its own record gone.

Two resumes both passed the `Running` check for the same reason. The
record is marked in-flight under the first lock, so the second request
is refused with `already_running` instead of losing a race for the
FIFO and reporting a tmux failure. Every path out of resume clears the
marker.

Both races are driven deterministically in the tests: the manager's
tmux spawn is parked mid-resume while the racing kill or resume runs.

Closes #175
2026-09-04 09:19:59 +01:00
Miguel Palhas 1366c2d430 merge: always probe tmux during recovery (#174)
Closes #174
2026-09-04 09:15:45 +01:00
Miguel Palhas d050df7d3d fix(maestro): always probe tmux during recovery
#164 skipped the tmux probe whenever the stored boot id differed from the
current one, assuming a changed boot id proves the tmux server is gone. It
does not: a restored state directory, a VM or container snapshot rollback,
or a truncated boot_id file all change it while tmux keeps running with
live sessions. Every one of those sessions was then registered Dormant and
stranded — resume respawns `maestro-<id>`, which collides with the name
tmux still holds, so it failed 502 forever.

Recovery now always runs `tmux list-sessions`, the single source of truth
for liveness the rest of the daemon assumes. The boot id keeps its other
job: telling a reboot from a restart in recovery's log.

The #164 test asserted the old behaviour with tmux still alive, so it is
replaced by one asserting the session is adopted Running, plus one covering
a real reboot — boot id changed and the tmux server dead — still going
dormant.

Refs #174
2026-09-04 09:13:15 +01:00
Miguel Palhas e51a7007a6 merge: resume_on_boot policy (#167)
Closes #167
2026-09-04 09:10:28 +01:00
Miguel PalhasandClaude Opus 5 02bf077c03 feat(maestro): add [daemon].resume_on_boot policy
Recovery now has a stated policy for sessions whose tmux pane is gone but
whose sidecar survived: dormant (default), auto, or never. auto goes down
the same SessionManager::resume the client's POST takes, so there is one
resume path. never prunes the sidecars the way #163 does for a missing cwd.

The default is dormant because resuming agents unattended seconds after a
reboot is a foot-gun; auto is opted into per host. Recovery logs which
policy ran and how many sessions it acted on.

Refs #167

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 09:06:51 +01:00
Miguel Palhas a793727bb9 merge: resume a dormant session in place (#166)
Closes #166
2026-09-04 08:59:37 +01:00
Miguel Palhas bb6b885c68 feat(maestro): add POST /api/sessions/:id/resume
A dormant session comes back under the same uuid and sidecar, so its
name, created_at, env and worktree ownership survive; only the tmux
session and the PTY bridge are new. The command is the matching preset's
resume_command with {agent_session} filled in, falling back to the
command the session was spawned with.

Refused with 409 for a session that is already running or whose cwd is
gone. Attaching to a session with no bridge now answers 409
not_attachable instead of 404 unknown_session, so a client can tell a
resumable session from one that does not exist.
2026-09-04 08:53:06 +01:00
Miguel Palhas db28827647 merge: clear the worktree on a settled natural exit (#172)
Also brings origin/main's web fixes onto the integration branch, which the
issue branch had already merged.

Closes #172
2026-09-04 08:51:42 +01:00
Miguel Palhas 6ac137d432 feat(maestro): add the resume route path and preset command builder
The template's {agent_session} is filled from the session's own
conversation id; a template that asks for an id the session never
reported yields nothing rather than a `--resume` with no argument.
2026-09-04 08:48:30 +01:00
18 changed files with 1590 additions and 42 deletions
+1
View File
@@ -14,6 +14,7 @@ maestro daemon tmux wrapper + session API + SPA on 127.0.0.1:7070
├── DELETE /api/sessions/:id kill
├── POST /api/sessions/:id/input type text into a session
├── POST /api/sessions/:id/prompt an agent reports its first prompt (#131)
├── POST /api/sessions/:id/resume respawn a dormant session, same id (#166)
├── GET /api/info hostname, version, home dir, spawn presets
├── GET /api/dirs directory listing (spawn autocomplete)
├── GET /api/recent-dirs spawn history
+97
View File
@@ -129,6 +129,80 @@ pub struct DaemonSection {
/// `Some(0)` disables history entirely (no log written, no recents read).
#[serde(default)]
pub spawn_history_days: Option<u32>,
/// What recovery does with sessions that outlived their tmux server —
/// after a host reboot, or any restart that finds a sidecar with no pane
/// (#167). Default [`ResumeOnBoot::Dormant`].
#[serde(default, deserialize_with = "deserialize_resume_on_boot")]
pub resume_on_boot: Option<ResumeOnBoot>,
}
impl DaemonSection {
/// The configured `[daemon].resume_on_boot`, or the safe default:
/// recovered-but-dead sessions wait for the operator.
pub fn resume_on_boot(&self) -> ResumeOnBoot {
self.resume_on_boot.unwrap_or_default()
}
}
/// `[daemon].resume_on_boot` — what recovery does with a session whose tmux
/// pane is gone but whose sidecar is still on disk (#167).
///
/// Resuming agents unattended seconds after a reboot is a foot-gun: they run
/// with whatever flags they were spawned with, with nobody watching. So the
/// default waits for a human, and `auto` is opted into per host.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub enum ResumeOnBoot {
/// Register each one `Dormant` and wait for the operator to resume it.
#[default]
Dormant,
/// Resume each one during recovery, on the same path a client's
/// `POST /api/sessions/:id/resume` takes.
Auto,
/// Prune the sidecars instead: a reboot is a clean slate.
Never,
}
/// Every `[daemon].resume_on_boot` name and its variant, in documentation
/// order.
pub const RESUME_ON_BOOT_POLICIES: &[(&str, ResumeOnBoot)] = &[
("dormant", ResumeOnBoot::Dormant),
("auto", ResumeOnBoot::Auto),
("never", ResumeOnBoot::Never),
];
impl ResumeOnBoot {
/// The name this policy is spelled with in the config file and in logs.
pub fn as_str(&self) -> &'static str {
match self {
ResumeOnBoot::Dormant => "dormant",
ResumeOnBoot::Auto => "auto",
ResumeOnBoot::Never => "never",
}
}
}
fn deserialize_resume_on_boot<'de, D>(deserializer: D) -> Result<Option<ResumeOnBoot>, D::Error>
where
D: Deserializer<'de>,
{
let raw = Option::<String>::deserialize(deserializer)?;
raw.map(|name| {
RESUME_ON_BOOT_POLICIES
.iter()
.find(|(n, _)| *n == name)
.map(|(_, policy)| *policy)
.ok_or_else(|| {
let known = RESUME_ON_BOOT_POLICIES
.iter()
.map(|(n, _)| format!("{n:?}"))
.collect::<Vec<_>>()
.join(", ");
D::Error::custom(format!(
"invalid [daemon].resume_on_boot {name:?}; expected one of {known}"
))
})
})
.transpose()
}
/// `[tui]` — `maestro tui` settings (NAP-275 epic, phase 2 NAP-277).
@@ -775,4 +849,27 @@ bind_addr = "127.0.0.1:9090"
);
assert_eq!(cfg.resolve_daemon_url(None), "http://explicit:1");
}
#[test]
fn resume_on_boot_defaults_to_dormant_and_parses_every_policy() {
assert_eq!(
MaestroConfigFile::default().daemon.resume_on_boot(),
ResumeOnBoot::Dormant,
"an absent policy is the safe one"
);
for (name, expected) in RESUME_ON_BOOT_POLICIES {
let cfg = file_with(&format!("[daemon]\nresume_on_boot = \"{name}\"\n"));
assert_eq!(cfg.daemon.resume_on_boot(), *expected, "{name}");
assert_eq!(expected.as_str(), *name);
}
}
#[test]
fn resume_on_boot_rejects_an_unknown_policy() {
let err = toml::from_str::<MaestroConfigFile>("[daemon]\nresume_on_boot = \"maybe\"\n")
.expect_err("unknown policy must not silently fall back");
let msg = err.to_string();
assert!(msg.contains("resume_on_boot"), "{msg}");
assert!(msg.contains("\"auto\""), "{msg}");
}
}
+7 -4
View File
@@ -2,10 +2,13 @@
//!
//! `/proc/sys/kernel/random/boot_id` is fixed for the life of the kernel and
//! changes on every boot. [`SessionManager::recover`] compares it against the
//! value it stored on the previous run: a difference means every session from
//! before is provably dead, so the tmux probe can be skipped and those
//! sessions registered dormant directly instead of relying on tmux reporting
//! an empty list.
//! value it stored on the previous run and logs which of the two it was.
//!
//! It does not decide liveness (#174). A boot id changes without the tmux
//! server dying whenever the state directory is restored, a VM or container
//! snapshot is rolled back, or the `boot_id` file is truncated — so recovery
//! always probes `tmux list-sessions` and the boot id only says whether a
//! server that did not answer is coming back.
use std::path::Path;
+4 -1
View File
@@ -87,7 +87,10 @@ async fn run(config_override: Option<PathBuf>) -> Result<(), Box<dyn Error>> {
.with_boot_id(
config.boot_id_path(),
std::sync::Arc::new(crate::boot_id::ProcBootId),
);
)
// #167: recovery needs the presets to resolve a resumed session's command,
// so `auto` runs the same resume a client's POST would.
.with_resume_policy(config.resume_on_boot, config.spawn_presets.clone());
// Spawned sessions learn where to report back (#103). Derived from the
// configured bind address; set before any spawn can happen.
manager.set_daemon_url(format!("http://{}", config.bind_addr));
+35
View File
@@ -97,6 +97,10 @@ pub struct DaemonConfig {
/// never binds it itself; it serves it so the web client and the TUI read
/// one knob.
pub leader: String,
/// What recovery does with sessions whose tmux pane is gone but whose
/// sidecar survives (#167): `[daemon].resume_on_boot`, default
/// [`maestro_config::ResumeOnBoot::Dormant`].
pub resume_on_boot: maestro_config::ResumeOnBoot,
}
impl DaemonConfig {
@@ -149,6 +153,8 @@ pub fn load_config(path: &Path, env: EnvOverrides) -> Result<DaemonConfig, Confi
// validation (#22/#26).
let spawn_presets = file.spawn.resolve()?;
let resume_on_boot = daemon.resume_on_boot();
let hostname = daemon.hostname.unwrap_or_else(system_hostname);
let home_dir = dirs::home_dir()
.map(|p| p.to_string_lossy().into_owned())
@@ -209,6 +215,7 @@ pub fn load_config(path: &Path, env: EnvOverrides) -> Result<DaemonConfig, Confi
project_scan_depth,
hooks_auto_install: file.hooks.auto_install(),
leader: file.tui.leader().to_string(),
resume_on_boot,
})
}
@@ -289,6 +296,34 @@ auto_install = false
assert!(!cfg.hooks_auto_install);
}
#[test]
fn resume_on_boot_defaults_to_dormant_and_reads_the_file() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("config.toml");
std::fs::write(&path, "").unwrap();
assert_eq!(
load_config(&path, EnvOverrides::default())
.unwrap()
.resume_on_boot,
maestro_config::ResumeOnBoot::Dormant
);
std::fs::write(
&path,
r#"
[daemon]
resume_on_boot = "auto"
"#,
)
.unwrap();
assert_eq!(
load_config(&path, EnvOverrides::default())
.unwrap()
.resume_on_boot,
maestro_config::ResumeOnBoot::Auto
);
}
#[test]
fn empty_file_defaults_everything() {
let dir = tempfile::tempdir().unwrap();
+1
View File
@@ -25,6 +25,7 @@ pub mod proc;
pub mod projects;
pub mod pty;
pub mod release;
pub mod resume;
pub mod session;
pub mod sidecar;
pub mod spawn_history;
+35
View File
@@ -52,6 +52,7 @@ pub fn router(state: LocalApiState) -> Router {
.route(routes::patterns::SESSION_ACTIVITY, post(report_activity))
.route(routes::patterns::SESSION_INPUT, post(send_input))
.route(routes::patterns::SESSION_FOCUS, post(set_focus))
.route(routes::patterns::SESSION_RESUME, post(resume_session))
.route(
routes::patterns::SESSION,
delete(kill_session).patch(rename_session),
@@ -170,6 +171,9 @@ fn session_error_status(code: &str) -> StatusCode {
// The session outlived its pane: nothing to type into, and nothing
// the daemon can do about it.
"not_running" => StatusCode::CONFLICT,
// Resume refusals (#166): the session exists and the request was
// well-formed, it just cannot be brought back in this state.
"already_running" | "cwd_missing" => StatusCode::CONFLICT,
_ => StatusCode::BAD_GATEWAY,
}
}
@@ -368,6 +372,27 @@ async fn set_focus(
}
}
/// `POST /api/sessions/:id/resume` (#166): respawn a session that has no pane
/// under its own id and sidecar, so its name, `created_at`, env and worktree
/// ownership survive. Answers with the updated session, which also goes out as
/// `SessionUpdated` to every events subscriber.
///
/// Refused with `409` when the session is already running or its `cwd` is
/// gone; `404` when there is no such session.
async fn resume_session(State(s): State<LocalApiState>, Path(id): Path<Uuid>) -> Response {
match s.manager.resume(id, &s.presets).await {
Ok(session) => (StatusCode::OK, Json(session)).into_response(),
Err(e) => {
let info = e.to_error_info();
error_response(
session_error_status(&info.code),
info.code.as_str(),
info.message.as_str(),
)
}
}
}
fn error_response(status: StatusCode, code: &str, message: &str) -> Response {
(status, Json(ApiError::new(code, message))).into_response()
}
@@ -410,6 +435,16 @@ async fn attach_ws(
ws: axum::extract::WebSocketUpgrade,
) -> Result<Response, Response> {
let Some((replay, rx)) = s.manager.subscribe_pty_with_replay(id).await else {
// #166: a session the daemon knows but has no bridge for — a dormant
// one — is not missing, it is not running. Reporting it as unknown
// left the client unable to tell "gone" from "resumable".
if s.manager.has(id).await {
return Err(error_response(
StatusCode::CONFLICT,
"not_attachable",
"session has no live pane; resume it first",
));
}
return Err(error_response(
StatusCode::NOT_FOUND,
"unknown_session",
+99
View File
@@ -0,0 +1,99 @@
//! Choosing the command a resumed session comes back on (#166).
//!
//! A session records the command it was spawned with. Bringing it back means
//! running that preset's `resume_command` (#165) instead, so an agent picks up
//! its old conversation rather than starting a fresh one. Everything else
//! about the session — id, sidecar, name, `created_at`, env, worktree — is
//! untouched, so only the command is decided here.
use maestro_protocol::{Session, SpawnPreset};
/// The command to respawn `session` with.
///
/// The preset is matched on the command the session was spawned with, which is
/// what the sidecar holds. A session with no matching preset, a preset with no
/// `resume_command`, or a `resume_command` needing an agent session id the
/// session never reported all fall back to the original command: respawning
/// what the operator asked for is always honest, where a half-filled resume
/// template is not.
pub fn resume_command(session: &Session, presets: &[SpawnPreset]) -> String {
presets
.iter()
.find(|p| p.command == session.command)
.and_then(|p| p.resume_command_for(session.agent_session_id.as_deref()))
.unwrap_or_else(|| session.command.clone())
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashMap;
fn presets() -> Vec<SpawnPreset> {
vec![
SpawnPreset {
id: "claude".into(),
label: "Claude".into(),
command: "claude".into(),
resume_command: Some("claude --resume {agent_session}".into()),
},
SpawnPreset {
id: "zsh".into(),
label: "Shell".into(),
command: "zsh".into(),
resume_command: None,
},
]
}
fn session(command: &str, agent_session_id: Option<&str>) -> Session {
Session {
id: uuid::Uuid::new_v4(),
cwd: "/tmp".into(),
command: command.into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
status: maestro_protocol::SessionStatus::Dormant,
created_at: chrono::Utc::now(),
foreground: None,
activity: maestro_protocol::ActivityState::Idle,
activity_detail: None,
last_activity_at: chrono::Utc::now(),
activity_since: chrono::Utc::now(),
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: agent_session_id.map(str::to_string),
}
}
#[test]
fn uses_the_matching_presets_resume_command() {
assert_eq!(
resume_command(&session("claude", Some("conv-7")), &presets()),
"claude --resume conv-7"
);
}
#[test]
fn falls_back_when_the_agent_session_id_is_unknown() {
assert_eq!(
resume_command(&session("claude", None), &presets()),
"claude"
);
}
#[test]
fn falls_back_for_a_preset_without_a_resume_command() {
assert_eq!(resume_command(&session("zsh", None), &presets()), "zsh");
}
#[test]
fn falls_back_when_no_preset_matches() {
assert_eq!(
resume_command(&session("bash --noprofile", Some("conv-7")), &presets()),
"bash --noprofile"
);
}
}
+252 -21
View File
@@ -12,6 +12,7 @@ use std::sync::Arc;
use std::time::Duration;
use chrono::Utc;
use maestro_config::ResumeOnBoot;
use maestro_protocol::{
ActivityState, CatalogEvent, CreateSessionRequest, ErrorInfo, NameSource, Session,
SessionStatus, WorktreeSpec, NAME_KEY, NAME_SOURCE_KEY,
@@ -86,6 +87,11 @@ pub struct SessionRecord {
/// (`POST /.../focus`), `None` while unfocused. Daemon-local; never rides
/// the wire.
pub focused_since: Option<std::time::Instant>,
/// Set for as long as a [`SessionManager::resume`] is between its two
/// takes of the map lock (#175). The spawn happens with the lock released,
/// so the `Running` check alone lets a second request through; this is the
/// state the second one is refused on. Daemon-local; never rides the wire.
pub resuming: bool,
/// Sticky acknowledgement applied by [`crate::activity::apply_focus_suppression`]:
/// once the pane has been focused long enough to clear an `AwaitingInput`
/// glow, this holds through focus loss until the session does new work.
@@ -112,6 +118,15 @@ pub enum SessionError {
/// a stray keystroke on an attach socket, which is dropped.
#[error("session is no longer running: {0}")]
NotRunning(Uuid),
/// A resume (#166) was asked for a session that is already running. There
/// is nothing to bring back, and respawning would strand the pane that is
/// already there.
#[error("session is already running: {0}")]
AlreadyRunning(Uuid),
/// A resume (#166) was asked for a session whose working directory is
/// gone. Nothing can be resumed in a directory that no longer exists.
#[error("session working directory no longer exists: {0}")]
CwdMissing(PathBuf),
/// The kill was refused (#79): the session's worktree holds unsaved work.
/// Nothing happened — the session keeps running and the tree is untouched,
/// so it stays the operator's handle on their own unfinished work.
@@ -130,6 +145,8 @@ impl SessionError {
SessionError::InvalidEnv(_) => "invalid_env",
SessionError::NotRunning(_) => "not_running",
SessionError::KillGuarded(_) => "kill_guarded",
SessionError::AlreadyRunning(_) => "already_running",
SessionError::CwdMissing(_) => "cwd_missing",
};
let unsaved_work = match self {
SessionError::KillGuarded(b) => Some(*b),
@@ -184,6 +201,11 @@ struct Inner {
/// reboot detection is unwired, which behaves exactly like a boot id
/// that could never be read: always fall back to probing tmux.
boot_id: Option<BootIdConfig>,
/// What recovery does with a session whose pane is gone (#167), and the
/// presets an `auto` resume resolves its command from. Defaults to
/// [`ResumeOnBoot::Dormant`] with no presets: register and wait.
resume_policy: ResumeOnBoot,
resume_presets: Vec<maestro_protocol::SpawnPreset>,
}
/// Where recovery persists the boot id it last saw, and how it reads the
@@ -242,6 +264,8 @@ impl SessionManager {
viewports: Mutex::new(Viewports::new()),
next_viewport: AtomicU64::new(0),
boot_id: None,
resume_policy: ResumeOnBoot::default(),
resume_presets: Vec::new(),
}),
}
}
@@ -286,6 +310,26 @@ impl SessionManager {
self
}
/// Wire the `[daemon].resume_on_boot` policy (#167) and the spawn presets
/// an `auto` resume resolves each session's command from. Builder, called
/// once right after construction like the other `with_*` methods. Without
/// it, recovery behaves as [`ResumeOnBoot::Dormant`].
///
/// # Panics
///
/// If the handle has already been cloned.
pub fn with_resume_policy(
mut self,
policy: ResumeOnBoot,
presets: Vec<maestro_protocol::SpawnPreset>,
) -> Self {
let inner = Arc::get_mut(&mut self.inner)
.expect("with_resume_policy must be called before the SessionManager is cloned");
inner.resume_policy = policy;
inner.resume_presets = presets;
self
}
/// Set the daemon's own HTTP base URL (#103), exported to every session
/// spawned afterwards as `MAESTRO_URL`. Unlike the `with_*` builders this
/// is a plain setter: the daemon runner learns its bound address after
@@ -556,6 +600,7 @@ impl SessionManager {
naming: false,
focused_since: None,
awaiting_suppressed: false,
resuming: false,
},
);
}
@@ -586,6 +631,141 @@ impl SessionManager {
Ok(session)
}
/// Bring a session that has no pane back to life (#166).
///
/// The session comes back under the **same** uuid and the same sidecar, so
/// `metadata.name`, `created_at`, `env` and worktree ownership all survive
/// — only the tmux session and the PTY bridge are new. That is the whole
/// point: spawning a fresh session at the same path instead produces a
/// stranger with no worktree to clean up on kill.
///
/// The command is [`crate::resume::resume_command`]: the matching preset's
/// `resume_command` (#165) with the session's own agent conversation id
/// substituted in, falling back to the command the session was spawned
/// with. `Session::command` keeps the original either way, so a second
/// resume still resolves the same preset.
///
/// Refused for a session that is already running, and for one whose `cwd`
/// has disappeared — neither is something a respawn could fix.
pub async fn resume(
&self,
id: Uuid,
presets: &[maestro_protocol::SpawnPreset],
) -> Result<Session, SessionError> {
// The spawn below runs with the map lock released, so the `Running`
// check alone is not enough to keep two resumes apart (#175): both
// would pass it, both would open a `PtyBridge` on the same FIFO, and
// the loser would report a tmux failure instead of the conflict it
// is. The record is marked in-flight under this first lock, and every
// path out of here clears it again.
let mut session = {
let mut sessions = self.inner.sessions.lock().await;
let rec = sessions.get_mut(&id).ok_or(SessionError::NotFound(id))?;
if rec.session.status == SessionStatus::Running || rec.resuming {
return Err(SessionError::AlreadyRunning(id));
}
rec.resuming = true;
rec.session.clone()
};
if !session.cwd.exists() {
self.clear_resuming(id).await;
return Err(SessionError::CwdMissing(session.cwd.clone()));
}
let command = crate::resume::resume_command(&session, presets);
// Same fifo path as the original spawn: it is derived from the id, and
// `mkfifo` tolerates the stale node the previous run left behind.
let fifo_path = self.inner.fifo_dir.join(format!("{id}.fifo"));
let bridge =
PtyBridge::start_with_replay(fifo_path.clone(), self.inner.replay_bytes).await?;
let mut session_env = sanitize_env(&session.env)?;
session_env.insert("MAESTRO_SESSION_ID".into(), id.to_string());
if let Some(url) = self
.inner
.daemon_url
.lock()
.expect("daemon_url mutex")
.clone()
{
session_env.insert("MAESTRO_URL".into(), url);
}
let spawned = {
let tmux = self.inner.tmux.clone();
let cwd = session.cwd.clone();
let command = command.clone();
let sink = pipe_pane_command(&fifo_path);
tmux_blocking(move || tmux.new_session_piped(id, &cwd, &command, &session_env, &sink))
.await
};
if let Err(e) = spawned {
// `bridge` drops here, closing the FIFO reader. Nothing else was
// touched, so the session stays exactly as dormant as it was.
self.clear_resuming(id).await;
return Err(SessionError::Tmux(e));
}
session.status = SessionStatus::Running;
// A resumed session is producing its initial output again; the
// watcher refines this on its next tick, exactly as after a spawn.
session.foreground = None;
session.activity = ActivityState::Active;
session.activity_detail = None;
session.last_activity_at = Utc::now();
session.activity_since = Utc::now();
session.current_cwd = None;
session.last_focused_at = None;
if let Err(e) = self.inner.sidecar.write(&session) {
// tmux already spawned; kill it rather than leave a pane the
// sidecar does not describe.
let tmux = self.inner.tmux.clone();
let _ = tmux_blocking(move || tmux.kill_session(id)).await;
self.clear_resuming(id).await;
return Err(SessionError::Sidecar(e));
}
{
let mut sessions = self.inner.sessions.lock().await;
let Some(rec) = sessions.get_mut(&id) else {
// Killed while we were spawning. Undo the pane instead of
// leaving one behind with no record of it — and delete the
// sidecar the write above just put back (#175). The kill
// deleted it before that write landed, so leaving it would
// have the next recovery register a dormant ghost for a
// session that is gone and whose worktree is already removed.
drop(sessions);
let tmux = self.inner.tmux.clone();
let _ = tmux_blocking(move || tmux.kill_session(id)).await;
if let Err(e) = self.inner.sidecar.delete(id) {
tracing::warn!(error = ?e, %id, "resume: failed to delete the sidecar of a session killed mid-resume");
}
return Err(SessionError::NotFound(id));
};
rec.session = session.clone();
rec.fifo_path = fifo_path;
rec.bridge = Some(bridge);
rec.reported_activity = None;
rec.settled = false;
rec.focused_since = None;
rec.awaiting_suppressed = false;
rec.resuming = false;
}
let _ = self.inner.events.send(CatalogEvent::SessionUpdated {
session: session.clone(),
});
Ok(session)
}
/// Drop the in-flight marker [`Self::resume`] set under its first lock
/// (#175), so a refused or failed resume can be retried. A no-op when the
/// record is already gone: a kill that landed mid-resume took the marker
/// with it.
async fn clear_resuming(&self, id: Uuid) {
if let Some(rec) = self.inner.sessions.lock().await.get_mut(&id) {
rec.resuming = false;
}
}
/// Best-effort removal of a worktree created earlier in `create` when a
/// later step fails. No-op when the request had no worktree.
fn rollback_worktree(&self, base: &Option<PathBuf>, path: &Option<PathBuf>) {
@@ -702,20 +882,20 @@ impl SessionManager {
/// that no longer exists are still adopted — the spec does not require
/// us to repair them.
///
/// #164: when the kernel's boot id has changed since the last run, the
/// tmux probe below is skipped entirely — a reboot kills the tmux server,
/// so every previous session is already dead and there is nothing tmux
/// could truthfully adopt. [`Self::register_dormant`] then registers all
/// of them directly.
/// #164/#174: the kernel's boot id says whether the host rebooted since
/// the last run, but it never decides which sessions are alive —
/// `tmux list-sessions` does, always, which is what the rest of the
/// daemon assumes. A boot id can change without the tmux server dying
/// (a restored state directory, a VM or container snapshot restore, a
/// truncated `boot_id` file); skipping the probe then registered every
/// live session `Dormant`, and resume could never bring one back because
/// the tmux name was still taken. The boot id is now only a signal that a
/// tmux server which did not answer is not coming back, and it names
/// itself in recovery's log.
pub async fn recover(&self) -> Result<usize, SessionError> {
tokio::fs::create_dir_all(&self.inner.fifo_dir).await?;
let rebooted = self.check_boot_id().await;
let live = if rebooted {
// #164: every session from the previous boot is provably dead —
// no tmux server survives a reboot — so there is nothing to gain
// from asking it and no session it could honestly report back.
Vec::new()
} else {
let live = {
let tmux = self.inner.tmux.clone();
tmux_blocking(move || tmux.list_sessions()).await?
};
@@ -777,6 +957,7 @@ impl SessionManager {
naming: false,
focused_since: None,
awaiting_suppressed: false,
resuming: false,
},
);
let _ = self
@@ -795,7 +976,9 @@ impl SessionManager {
///
/// A missing stored value (first run) or an unreadable current one
/// (non-Linux, restricted `/proc`, unwired in this build) both read as
/// "no reboot", which is today's behaviour: fall back to probing tmux.
/// "no reboot". Either way the tmux probe runs (#174): the answer only
/// tells recovery whether a tmux server that did not answer is coming
/// back, never which sessions are alive.
async fn check_boot_id(&self) -> bool {
let Some(cfg) = &self.inner.boot_id else {
return false;
@@ -842,7 +1025,8 @@ impl SessionManager {
/// `rebooted` (#164) only changes what gets logged: a detected reboot
/// names the count even when it is zero, so the operator sees that
/// recovery told a reboot from a restart instead of a log that stays
/// silent either way.
/// silent either way. It never decides membership — everything here got
/// here by tmux not holding it (#174).
async fn register_dormant(&self, rebooted: bool) {
let stored = match self.inner.sidecar.load_all() {
Ok(stored) => stored,
@@ -853,6 +1037,7 @@ impl SessionManager {
};
let mut registered = Vec::new();
let mut pruned = 0usize;
let mut discarded = 0usize;
{
let mut sessions = self.inner.sessions.lock().await;
for mut session in stored {
@@ -867,6 +1052,16 @@ impl SessionManager {
pruned += 1;
continue;
}
if self.inner.resume_policy == ResumeOnBoot::Never {
// #167: this host wants a reboot to be a clean slate, so
// the sidecar goes the way #163's does — nothing is
// registered and the next recovery finds nothing.
if let Err(e) = self.inner.sidecar.delete(session.id) {
tracing::warn!(error = ?e, id = %session.id, "recover: failed to prune sidecar under resume_on_boot=never");
}
discarded += 1;
continue;
}
let id = session.id;
session.status = SessionStatus::Dormant;
session.foreground = None;
@@ -889,6 +1084,7 @@ impl SessionManager {
naming: false,
focused_since: None,
awaiting_suppressed: false,
resuming: false,
},
);
registered.push(session);
@@ -908,12 +1104,45 @@ impl SessionManager {
if pruned > 0 {
tracing::info!(pruned, "pruned sidecars whose cwd no longer exists");
}
let resumable: Vec<Uuid> = registered.iter().map(|s| s.id).collect();
for session in registered {
let _ = self
.inner
.events
.send(CatalogEvent::SessionAdded { session });
}
// #167: say which policy ran and how many sessions it acted on, so a
// host that resumes agents unattended says so in its own log. The
// match is exhaustive on purpose: a fourth policy must decide what it
// does here rather than inherit whichever arm happened to be last.
match self.inner.resume_policy {
ResumeOnBoot::Dormant => tracing::info!(
policy = ResumeOnBoot::Dormant.as_str(),
sessions = resumable.len(),
"resume_on_boot: sessions left dormant for the operator"
),
ResumeOnBoot::Never => tracing::info!(
policy = ResumeOnBoot::Never.as_str(),
sessions = discarded,
"resume_on_boot: pruned the sidecars of sessions with no live pane"
),
ResumeOnBoot::Auto => {
tracing::info!(
policy = ResumeOnBoot::Auto.as_str(),
sessions = resumable.len(),
"resume_on_boot: resuming sessions with no live pane"
);
let presets = self.inner.resume_presets.clone();
for id in resumable {
// The same call the client's POST /api/sessions/:id/resume
// makes — one resume path, not two.
if let Err(e) = self.resume(id, &presets).await {
tracing::warn!(error = %e, id = %id, "resume_on_boot=auto: resume failed; session stays dormant");
}
}
}
}
}
/// Forward keystrokes to a session's pane. A dead or dormant session is
@@ -1076,10 +1305,12 @@ impl SessionManager {
/// and stays silent on the event stream at steady state.
///
/// `agent_session_id` (#161) is the agent's own conversation id, parsed
/// by the caller from the hook payload's `session_id` field. It is
/// written once — the first report that carries one wins — and the
/// sidecar is only rewritten on that transition, never on every report,
/// since an agent's session id never changes mid-session.
/// by the caller from the hook payload's `session_id` field. The
/// sidecar is rewritten only when the reported id differs from the
/// stored one — a repeated report of the same id is pure churn from the
/// hook's `PreToolUse` cadence and writes nothing, but a resume (#176)
/// issues a new conversation id and must replace the stale one, or
/// every later resume reattaches the pre-resume conversation.
pub async fn report_activity(
&self,
id: Uuid,
@@ -1101,16 +1332,16 @@ impl SessionManager {
state,
self.inner.report_ttl,
));
let newly_identified =
rec.session.agent_session_id.is_none() && agent_session_id.is_some();
if newly_identified {
let id_changed =
agent_session_id.is_some() && rec.session.agent_session_id != agent_session_id;
if id_changed {
rec.session.agent_session_id = agent_session_id;
self.inner
.sidecar
.write(&rec.session)
.map_err(SessionError::Sidecar)?;
}
(changed || newly_identified).then(|| rec.session.clone())
(changed || id_changed).then(|| rec.session.clone())
};
if let Some(session) = updated {
let _ = self
+31 -4
View File
@@ -187,10 +187,30 @@ async fn a_reported_session_id_is_recorded_on_the_session() {
assert_eq!(s.agent_session_id.as_deref(), Some("conv-abc123"));
}
/// #161: a session id does not change mid-session, so once set it is never
/// overwritten by a later report — even one that names a different id.
/// #161: repeated reports of the same id are pure churn from the hook's
/// `PreToolUse` cadence — the sidecar must not be rewritten for them.
#[tokio::test]
async fn a_reported_session_id_is_written_once_and_never_replaced() {
async fn a_repeated_report_of_the_same_session_id_writes_nothing() {
let fx = fixture(Duration::from_secs(300)).await;
let id = spawn_sleeping(&fx.manager).await;
fx.manager
.report_activity(id, ActivityState::Active, None, Some("conv-first".into()))
.await
.unwrap();
fx.manager
.report_activity(id, ActivityState::Active, None, Some("conv-first".into()))
.await
.unwrap();
let s = observed(&fx.manager, id).await;
assert_eq!(s.agent_session_id.as_deref(), Some("conv-first"));
}
/// #176: a resume issues a new conversation id. The stored id must move to
/// it, or every later resume reattaches the stale pre-resume conversation.
#[tokio::test]
async fn a_reported_session_id_is_replaced_when_it_differs() {
let fx = fixture(Duration::from_secs(300)).await;
let id = spawn_sleeping(&fx.manager).await;
@@ -204,7 +224,14 @@ async fn a_reported_session_id_is_written_once_and_never_replaced() {
.unwrap();
let s = observed(&fx.manager, id).await;
assert_eq!(s.agent_session_id.as_deref(), Some("conv-first"));
assert_eq!(s.agent_session_id.as_deref(), Some("conv-second"));
let stored = fx
.sidecar
.read(id)
.expect("sidecar read")
.expect("sidecar present");
assert_eq!(stored.agent_session_id.as_deref(), Some("conv-second"));
}
#[tokio::test]
+68
View File
@@ -94,6 +94,74 @@ async fn rejects_unknown_session() {
assert!(format!("{err}").contains("404"));
}
/// #166: a session the daemon knows but has no bridge for is not unknown, it
/// is not running. `404 unknown_session` made "gone" and "resumable" look the
/// same to a client that could have offered a resume.
#[tokio::test]
async fn dormant_session_is_not_attachable_rather_than_unknown() {
let h = spawn_daemon_ws().await;
let cwd = TempDir::new().unwrap();
let session = h
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.unwrap();
// The host reboot: tmux is gone, the sidecar is not, and a fresh manager
// registers the session dormant.
h._fx.tmux.tmux.kill_server().unwrap();
let dormant = SessionManager::new(
std::sync::Arc::new(h._fx.tmux.tmux.clone()),
maestro_daemon::sidecar::SidecarStore::new(h._fx.sidecar_dir()),
maestro_daemon::worktree::WorktreeManager::new(None),
h._fx.fifo_dir(),
);
dormant.recover().await.unwrap();
let dormant_addr = serve_manager(dormant).await;
let err = connect_attach(dormant_addr, session.id).await.unwrap_err();
let tokio_tungstenite::tungstenite::Error::Http(resp) = err else {
panic!("want an HTTP rejection, got {err}");
};
assert_eq!(resp.status().as_u16(), 409);
let body: maestro_protocol::ApiError =
serde_json::from_slice(resp.body().as_deref().expect("error body")).unwrap();
assert_eq!(body.error.code, "not_attachable");
// A session that really is unknown still reads as unknown.
let err = connect_attach(dormant_addr, Uuid::new_v4())
.await
.unwrap_err();
assert!(format!("{err}").contains("404"));
}
/// Serve `manager` on its own listener, for tests that need a second daemon
/// over the same tmux/sidecar state.
async fn serve_manager(manager: SessionManager) -> SocketAddr {
let router = local_api::router(local_api::state_for(
manager,
"test-host",
"/home/test",
Vec::new(),
std::sync::Arc::new(maestro_daemon::projects::ProjectScanner::disabled()),
maestro_config::DEFAULT_LEADER,
));
let listener = TcpListener::bind(SocketAddr::from((Ipv4Addr::LOCALHOST, 0)))
.await
.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
let _ = axum::serve(listener, router).await;
});
addr
}
#[tokio::test]
async fn pty_round_trip_through_real_tmux() {
let h = spawn_daemon_ws().await;
+593
View File
@@ -0,0 +1,593 @@
//! End-to-end tests for `POST /api/sessions/:id/resume` (#166) and the
//! manager call behind it, against real tmux.
//!
//! The point of resume is that the session comes back under the *same* uuid
//! and sidecar — name, `created_at`, env and worktree ownership intact — with
//! only the tmux session and the PTY bridge new.
mod common;
use std::collections::HashMap;
use std::net::{Ipv4Addr, SocketAddr};
use std::sync::Arc;
use std::time::Duration;
use common::{wait_for_async, DaemonFixture};
use maestro_daemon::{local_api, SessionError, SessionManager};
use maestro_protocol::{CreateSessionRequest, Session, SessionStatus, SpawnPreset};
use tempfile::TempDir;
use tokio::net::TcpListener;
use tokio::sync::oneshot;
use uuid::Uuid;
/// A fresh manager over the same tmux server, sidecars and fifos — what the
/// daemon builds after a restart.
fn revive(fx: &DaemonFixture) -> SessionManager {
SessionManager::new(
Arc::new(fx.tmux.tmux.clone()),
maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir()),
maestro_daemon::worktree::WorktreeManager::new(None),
fx.fifo_dir(),
)
}
/// Create a session, then simulate the host reboot that leaves it dormant:
/// the tmux server dies, the sidecar stays, a new manager recovers.
async fn dormant_session(
fx: &DaemonFixture,
cwd: &TempDir,
command: &str,
name: &str,
) -> (SessionManager, Session) {
let mut metadata = HashMap::new();
metadata.insert(
maestro_protocol::NAME_KEY.to_string(),
serde_json::Value::String(name.into()),
);
let session = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: command.into(),
env: HashMap::from([("MAESTRO_TEST_ENV".to_string(), "kept".to_string())]),
metadata,
worktree: None,
})
.await
.expect("create");
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive(fx);
revived.recover().await.expect("recover");
(revived, session)
}
fn marker_preset(command: &str, marker_dir: &TempDir) -> Vec<SpawnPreset> {
vec![SpawnPreset {
id: "agent".into(),
label: "Agent".into(),
command: command.into(),
resume_command: Some(format!(
"sh -c 'touch {}/resumed-{{agent_session}}; sleep 30'",
marker_dir.path().display()
)),
}]
}
/// The keystone: a dormant session comes back running under its own id, with
/// its name, `created_at`, env and sidecar untouched.
#[tokio::test]
async fn resume_brings_a_dormant_session_back_under_the_same_id() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let (manager, original) = dormant_session(&fx, &cwd, "sleep 30", "long-runner").await;
assert_eq!(
manager.status(original.id).await,
Some(SessionStatus::Dormant)
);
let resumed = manager.resume(original.id, &[]).await.expect("resume");
assert_eq!(resumed.id, original.id, "same uuid");
assert_eq!(resumed.status, SessionStatus::Running);
assert_eq!(resumed.created_at, original.created_at);
assert_eq!(resumed.cwd, original.cwd);
assert_eq!(resumed.command, original.command);
assert_eq!(resumed.env, original.env);
assert_eq!(
resumed
.metadata
.get(maestro_protocol::NAME_KEY)
.and_then(|v| v.as_str()),
Some("long-runner")
);
// tmux really holds it again, and the sidecar is the same file, updated.
let live = fx.tmux.tmux.list_sessions().expect("list");
assert_eq!(live, vec![original.id]);
let stored = maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir())
.read(original.id)
.expect("read sidecar")
.expect("sidecar still there");
assert_eq!(stored.status, SessionStatus::Running);
assert_eq!(stored.created_at, original.created_at);
// And it is attachable again, which a dormant session never was.
assert!(manager
.subscribe_pty_with_replay(original.id)
.await
.is_some());
}
/// The worktree the session owns stays the session's, so killing the resumed
/// session still cleans the tree up.
#[tokio::test]
async fn resume_keeps_worktree_ownership() {
let fx = DaemonFixture::new();
let repo = TempDir::new().unwrap();
common::init_repo(repo.path());
let session = fx
.manager
.create(CreateSessionRequest {
cwd: repo.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: Some(maestro_protocol::WorktreeSpec {
base_repo: repo.path().to_path_buf(),
branch: "feature".into(),
create_branch: true,
}),
})
.await
.expect("create");
let worktree = session.worktree.clone().expect("worktree");
fx.tmux.tmux.kill_server().expect("kill tmux server");
let manager = revive(&fx);
manager.recover().await.expect("recover");
let resumed = manager.resume(session.id, &[]).await.expect("resume");
assert_eq!(resumed.worktree, Some(worktree.clone()));
assert_eq!(
resumed.cwd, worktree.path,
"resumed inside its own worktree"
);
}
/// The command comes from the preset's `resume_command`, with the session's
/// own agent conversation id (#161) substituted in.
#[tokio::test]
async fn resume_runs_the_presets_resume_command() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let markers = TempDir::new().unwrap();
let presets = marker_preset("sleep 30", &markers);
let (manager, session) = dormant_session(&fx, &cwd, "sleep 30", "agentic").await;
// The agent reported its conversation id before the reboot, so it is on
// the sidecar and the resume template can be filled in.
fx.manager
.report_activity(
session.id,
maestro_protocol::ActivityState::Idle,
None,
Some("conv-42".into()),
)
.await
.ok();
// Re-read the sidecar the report updated.
let manager = {
drop(manager);
let m = revive(&fx);
m.recover().await.expect("recover");
m
};
manager.resume(session.id, &presets).await.expect("resume");
let marker = markers.path().join("resumed-conv-42");
wait_for_async(Duration::from_secs(5), || {
let marker = marker.clone();
async move { marker.exists() }
})
.await;
}
/// A session that is already running has nothing to bring back, and
/// respawning would strand the pane that is already there.
#[tokio::test]
async fn resume_refuses_a_running_session() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let session = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
let err = fx
.manager
.resume(session.id, &[])
.await
.expect_err("already running");
assert!(matches!(err, SessionError::AlreadyRunning(id) if id == session.id));
assert_eq!(err.to_error_info().code, "already_running");
}
/// Nothing can be resumed in a directory that no longer exists.
#[tokio::test]
async fn resume_refuses_when_the_cwd_is_gone() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let (manager, session) = dormant_session(&fx, &cwd, "sleep 30", "homeless").await;
std::fs::remove_dir_all(cwd.path()).unwrap();
let err = manager
.resume(session.id, &[])
.await
.expect_err("cwd is gone");
assert_eq!(err.to_error_info().code, "cwd_missing");
assert_eq!(
manager.status(session.id).await,
Some(SessionStatus::Dormant),
"a refused resume leaves the session exactly as it was"
);
}
#[tokio::test]
async fn resume_of_an_unknown_session_is_not_found() {
let fx = DaemonFixture::new();
let err = fx
.manager
.resume(Uuid::new_v4(), &[])
.await
.expect_err("unknown");
assert_eq!(err.to_error_info().code, "not_found");
}
// ---------------------------------------------------------------------------
// HTTP surface
// ---------------------------------------------------------------------------
struct Harness {
addr: SocketAddr,
_shutdown: oneshot::Sender<()>,
}
async fn serve(manager: SessionManager, presets: Vec<SpawnPreset>) -> Harness {
let router = local_api::router(local_api::state_for(
manager,
"test-host",
"/home/test",
presets,
Arc::new(maestro_daemon::projects::ProjectScanner::disabled()),
maestro_config::DEFAULT_LEADER,
));
let listener = TcpListener::bind(SocketAddr::from((Ipv4Addr::LOCALHOST, 0)))
.await
.unwrap();
let addr = listener.local_addr().unwrap();
let (tx, rx) = oneshot::channel::<()>();
tokio::spawn(async move {
let _ = axum::serve(listener, router)
.with_graceful_shutdown(async move {
let _ = rx.await;
})
.await;
});
Harness {
addr,
_shutdown: tx,
}
}
#[tokio::test]
async fn post_resume_returns_the_revived_session() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let (manager, session) = dormant_session(&fx, &cwd, "sleep 30", "web-resumed").await;
let h = serve(manager, Vec::new()).await;
let resp = reqwest::Client::new()
.post(format!(
"http://{}{}",
h.addr,
maestro_protocol::routes::session_resume(session.id)
))
.send()
.await
.unwrap();
assert_eq!(resp.status(), reqwest::StatusCode::OK);
let body: Session = resp.json().await.unwrap();
assert_eq!(body.id, session.id);
assert_eq!(body.status, SessionStatus::Running);
}
#[tokio::test]
async fn post_resume_on_a_running_session_is_a_conflict() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let session = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
let h = serve(fx.manager.clone(), Vec::new()).await;
let resp = reqwest::Client::new()
.post(format!(
"http://{}{}",
h.addr,
maestro_protocol::routes::session_resume(session.id)
))
.send()
.await
.unwrap();
assert_eq!(resp.status(), reqwest::StatusCode::CONFLICT);
let err: maestro_protocol::ApiError = resp.json().await.unwrap();
assert_eq!(err.error.code, "already_running");
}
#[tokio::test]
async fn post_resume_on_an_unknown_session_is_404() {
let fx = DaemonFixture::new();
let h = serve(fx.manager.clone(), Vec::new()).await;
let resp = reqwest::Client::new()
.post(format!(
"http://{}{}",
h.addr,
maestro_protocol::routes::session_resume(Uuid::new_v4())
))
.send()
.await
.unwrap();
assert_eq!(resp.status(), reqwest::StatusCode::NOT_FOUND);
}
// ---------------------------------------------------------------------------
// Races (#175)
// ---------------------------------------------------------------------------
/// Real tmux, with the *first* `new_session_piped` held inside the call until
/// the test releases it. Later calls pass straight through, so a second resume
/// racing the first never blocks on a gate the first one owns.
///
/// This is what makes the two races below deterministic rather than timing
/// dependent: the resume under test is parked in exactly the window the bug
/// lives in, and the racing operation runs to completion before it moves.
#[derive(Debug)]
struct GatedSpawn {
inner: maestro_daemon::tmux::Tmux,
entered: tokio::sync::mpsc::UnboundedSender<()>,
release: std::sync::Mutex<std::sync::mpsc::Receiver<()>>,
gated: std::sync::atomic::AtomicBool,
}
/// The test's end of a [`GatedSpawn`].
struct Gate {
entered: tokio::sync::mpsc::UnboundedReceiver<()>,
release: std::sync::mpsc::SyncSender<()>,
}
impl Gate {
/// Wait until the gated spawn is parked mid-resume.
async fn wait_entered(&mut self) {
tokio::time::timeout(Duration::from_secs(10), self.entered.recv())
.await
.expect("resume reached the tmux spawn")
.expect("gate alive");
}
fn release(&self) {
self.release.send(()).expect("gate receiver alive");
}
}
fn gated_panes(inner: maestro_daemon::tmux::Tmux) -> (Arc<GatedSpawn>, Gate) {
let (entered_tx, entered_rx) = tokio::sync::mpsc::unbounded_channel();
let (release_tx, release_rx) = std::sync::mpsc::sync_channel(1);
(
Arc::new(GatedSpawn {
inner,
entered: entered_tx,
release: std::sync::Mutex::new(release_rx),
gated: std::sync::atomic::AtomicBool::new(true),
}),
Gate {
entered: entered_rx,
release: release_tx,
},
)
}
impl maestro_daemon::panes::Panes for GatedSpawn {
fn new_session_piped(
&self,
id: Uuid,
cwd: &std::path::Path,
command: &str,
session_env: &std::collections::BTreeMap<String, String>,
sink_command: &str,
) -> Result<(), maestro_daemon::tmux::TmuxError> {
if self.gated.swap(false, std::sync::atomic::Ordering::SeqCst) {
let _ = self.entered.send(());
let _ = self.release.lock().expect("gate mutex").recv();
}
self.inner
.new_session_piped(id, cwd, command, session_env, sink_command)
}
fn kill_session(&self, id: Uuid) -> Result<(), maestro_daemon::tmux::TmuxError> {
self.inner.kill_session(id)
}
fn list_sessions(&self) -> Result<Vec<Uuid>, maestro_daemon::tmux::TmuxError> {
self.inner.list_sessions()
}
fn pane_info(
&self,
id: Uuid,
) -> Result<Option<maestro_daemon::tmux::PaneInfo>, maestro_daemon::tmux::TmuxError> {
self.inner.pane_info(id)
}
fn send_keys(&self, id: Uuid, bytes: &[u8]) -> Result<(), maestro_daemon::tmux::TmuxError> {
self.inner.send_keys(id, bytes)
}
fn pipe_pane(&self, id: Uuid, sink: &str) -> Result<(), maestro_daemon::tmux::TmuxError> {
self.inner.pipe_pane(id, sink)
}
fn resize_window(
&self,
id: Uuid,
cols: u16,
rows: u16,
) -> Result<(), maestro_daemon::tmux::TmuxError> {
self.inner.resize_window(id, cols, rows)
}
fn capture_pane(&self, id: Uuid) -> Result<Option<Vec<u8>>, maestro_daemon::tmux::TmuxError> {
self.inner.capture_pane(id)
}
}
/// A dormant session whose manager's tmux spawn can be parked by the test.
async fn dormant_gated(fx: &DaemonFixture, cwd: &TempDir) -> (SessionManager, Session, Gate) {
let session = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
fx.tmux.tmux.kill_server().expect("kill tmux server");
let (panes, gate) = gated_panes(fx.tmux.tmux.clone());
let manager = SessionManager::new(
panes,
maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir()),
maestro_daemon::worktree::WorktreeManager::new(None),
fx.fifo_dir(),
);
manager.recover().await.expect("recover");
assert_eq!(
manager.status(session.id).await,
Some(SessionStatus::Dormant)
);
(manager, session, gate)
}
/// A kill that lands while a resume is inside the tmux spawn must not leave
/// the sidecar behind. The kill deletes it; the resume writes it back a moment
/// later and then discovers its own record is gone. The sidecar it resurrected
/// would otherwise make the next recovery register a ghost dormant session for
/// a session that was killed.
#[tokio::test]
async fn a_kill_during_resume_leaves_no_sidecar_behind() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let (manager, session, mut gate) = dormant_gated(&fx, &cwd).await;
let id = session.id;
let resuming = tokio::spawn({
let manager = manager.clone();
async move { manager.resume(id, &[]).await }
});
gate.wait_entered().await;
manager.kill(id, false).await.expect("kill mid-resume");
gate.release();
let err = resuming.await.unwrap().expect_err("record is gone");
assert!(matches!(err, SessionError::NotFound(got) if got == id));
assert!(
maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir())
.read(id)
.expect("read sidecar")
.is_none(),
"the killed session's sidecar must not be resurrected by the losing resume"
);
// And so the next recovery registers no ghost.
let after = revive(&fx);
after.recover().await.expect("recover");
assert!(
after.snapshot().await.is_empty(),
"a killed session must not come back dormant"
);
}
/// Two resumes of the same session must not both proceed. The second is
/// refused on the state — `already_running` — not on its tmux spawn colliding
/// with the first one's.
#[tokio::test]
async fn a_second_resume_is_refused_while_the_first_is_in_flight() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let (manager, session, mut gate) = dormant_gated(&fx, &cwd).await;
let id = session.id;
let resuming = tokio::spawn({
let manager = manager.clone();
async move { manager.resume(id, &[]).await }
});
gate.wait_entered().await;
let err = manager
.resume(id, &[])
.await
.expect_err("a resume is already in flight");
assert!(matches!(err, SessionError::AlreadyRunning(got) if got == id));
assert_eq!(err.to_error_info().code, "already_running");
gate.release();
let first = resuming.await.unwrap().expect("the first resume wins");
assert_eq!(first.status, SessionStatus::Running);
assert_eq!(
fx.tmux.tmux.list_sessions().expect("list"),
vec![id],
"exactly one pane for the session"
);
}
/// A refused resume must not leave the session marked in-flight: once the
/// reason for the refusal is gone, resuming works.
#[tokio::test]
async fn a_refused_resume_can_be_retried() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let (manager, session) = dormant_session(&fx, &cwd, "sleep 30", "retried").await;
std::fs::remove_dir_all(cwd.path()).unwrap();
let err = manager
.resume(session.id, &[])
.await
.expect_err("cwd is gone");
assert_eq!(err.to_error_info().code, "cwd_missing");
std::fs::create_dir_all(cwd.path()).unwrap();
let resumed = manager.resume(session.id, &[]).await.expect("resume");
assert_eq!(resumed.status, SessionStatus::Running);
}
@@ -0,0 +1,161 @@
//! `[daemon].resume_on_boot` (#167): what recovery does with sessions whose
//! tmux pane is gone but whose sidecar survived, against real tmux.
//!
//! The default is deliberately the timid one. Resuming agents unattended
//! seconds after a reboot is a foot-gun, so `auto` is opted into per host and
//! `never` throws the sidecars away instead.
mod common;
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use common::{wait_for_async, DaemonFixture};
use maestro_config::ResumeOnBoot;
use maestro_daemon::SessionManager;
use maestro_protocol::{CreateSessionRequest, Session, SessionStatus, SpawnPreset};
use tempfile::TempDir;
/// A fresh manager over the same tmux server, sidecars and fifos — what the
/// daemon builds after a restart — carrying `policy`.
fn revive(fx: &DaemonFixture, policy: ResumeOnBoot, presets: Vec<SpawnPreset>) -> SessionManager {
SessionManager::new(
Arc::new(fx.tmux.tmux.clone()),
maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir()),
maestro_daemon::worktree::WorktreeManager::new(None),
fx.fifo_dir(),
)
.with_resume_policy(policy, presets)
}
/// A preset whose `resume_command` touches a marker file, so a test can see
/// that the resumed session ran the resume command rather than the original.
fn marker_preset(command: &str, marker_dir: &TempDir) -> Vec<SpawnPreset> {
vec![SpawnPreset {
id: "agent".into(),
label: "Agent".into(),
command: command.into(),
resume_command: Some(format!(
"sh -c 'touch {}/resumed; sleep 30'",
marker_dir.path().display()
)),
}]
}
async fn create(fx: &DaemonFixture, cwd: &TempDir, command: &str) -> Session {
fx.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: command.into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create")
}
/// The default: a reboot leaves the session listed but not running, waiting
/// for the operator.
#[tokio::test]
async fn dormant_is_the_default_and_leaves_the_session_waiting() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let session = create(&fx, &cwd, "sleep 30").await;
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive(&fx, ResumeOnBoot::Dormant, Vec::new());
revived.recover().await.expect("recover");
assert_eq!(
revived.status(session.id).await,
Some(SessionStatus::Dormant)
);
assert!(fx
.sidecar_dir()
.join(format!("{}.json", session.id))
.exists());
assert!(
fx.tmux.tmux.list_sessions().expect("list").is_empty(),
"nothing was respawned"
);
}
/// `auto` brings each one back itself, down the same path a client's resume
/// takes: same uuid, same sidecar, the preset's `resume_command`.
#[tokio::test]
async fn auto_resumes_every_dormant_session_during_recovery() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let markers = TempDir::new().unwrap();
let session = create(&fx, &cwd, "sleep 30").await;
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive(&fx, ResumeOnBoot::Auto, marker_preset("sleep 30", &markers));
revived.recover().await.expect("recover");
assert_eq!(
revived.status(session.id).await,
Some(SessionStatus::Running),
"auto resumed it"
);
assert_eq!(
fx.tmux.tmux.list_sessions().expect("list"),
vec![session.id],
"back under the same uuid"
);
let stored = maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir())
.read(session.id)
.expect("read sidecar")
.expect("same sidecar");
assert_eq!(stored.created_at, session.created_at);
assert_eq!(stored.command, "sleep 30", "the original command is kept");
// It really ran the preset's resume command, not the spawn command.
let marker = markers.path().join("resumed");
wait_for_async(Duration::from_secs(5), || {
let marker = marker.clone();
async move { marker.exists() }
})
.await;
assert!(marker.exists(), "resume_command ran");
}
/// `never` makes a reboot a clean slate: the sidecars go, and the session is
/// not listed at all.
#[tokio::test]
async fn never_prunes_the_sidecars_instead() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let session = create(&fx, &cwd, "sleep 30").await;
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive(&fx, ResumeOnBoot::Never, Vec::new());
revived.recover().await.expect("recover");
assert!(!revived.has(session.id).await, "not listed");
assert!(
!fx.sidecar_dir()
.join(format!("{}.json", session.id))
.exists(),
"sidecar pruned"
);
assert!(fx.tmux.tmux.list_sessions().expect("list").is_empty());
}
/// The policy only decides what happens to sessions that lost their pane. A
/// session tmux still holds is adopted running by the first pass under every
/// policy — including `never`, which must not delete a live session's sidecar.
#[tokio::test]
async fn the_policy_leaves_sessions_tmux_still_holds_alone() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let live = create(&fx, &cwd, "sleep 30").await;
let revived = revive(&fx, ResumeOnBoot::Never, Vec::new());
assert_eq!(revived.recover().await.expect("recover"), 1);
assert_eq!(revived.status(live.id).await, Some(SessionStatus::Running));
assert!(fx.sidecar_dir().join(format!("{}.json", live.id)).exists());
}
+42 -8
View File
@@ -326,12 +326,14 @@ fn revive_with_boot_id(
)
}
/// #164: a changed boot id means every session from the previous boot is
/// already dead, so recovery must not trust tmux to say so — it skips the
/// probe outright. Proven by leaving the tmux session alive: if the probe
/// ran, it would come back `Running`, not `Dormant`.
/// #174: a boot id can change without the tmux server dying — a restored
/// state directory, a VM or container snapshot restore, a truncated
/// `boot_id` file. `tmux list-sessions` stays the single source of truth for
/// liveness, so a session tmux still holds is adopted `Running` no matter
/// what the boot id says. Registering it dormant instead stranded it: resume
/// would collide with the tmux name still in use and fail forever.
#[tokio::test]
async fn recover_skips_tmux_probe_when_boot_id_changed() {
async fn recover_adopts_live_session_when_boot_id_changed() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let s = fx
@@ -349,15 +351,15 @@ async fn recover_skips_tmux_probe_when_boot_id_changed() {
let boot_id_path = fx.state_dir.path().join("boot_id");
std::fs::write(&boot_id_path, "previous-boot").unwrap();
// tmux session is still alive — a probe would find it if one ran.
// tmux session is still alive: the probe runs and finds it.
let revived = revive_with_boot_id(&fx, boot_id_path.clone(), Some("current-boot"));
let adopted = revived.recover().await.expect("recover");
assert_eq!(adopted, 0, "reboot detected: nothing adopted live");
assert_eq!(adopted, 1, "tmux still holds the session; adopt it");
let snap = revived.snapshot().await;
assert_eq!(snap.len(), 1);
assert_eq!(snap[0].id, s.id);
assert_eq!(snap[0].status, SessionStatus::Dormant);
assert_eq!(snap[0].status, SessionStatus::Running);
// The current boot id is now the stored one, for the next comparison.
assert_eq!(
@@ -366,6 +368,38 @@ async fn recover_skips_tmux_probe_when_boot_id_changed() {
);
}
/// #164, still true after #174: a real reboot takes the tmux server with it,
/// so the probe comes back empty and every sidecar is registered dormant.
#[tokio::test]
async fn recover_registers_dormant_when_boot_id_changed_and_tmux_gone() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let s = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
let boot_id_path = fx.state_dir.path().join("boot_id");
std::fs::write(&boot_id_path, "previous-boot").unwrap();
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive_with_boot_id(&fx, boot_id_path, Some("current-boot"));
let adopted = revived.recover().await.expect("recover");
assert_eq!(adopted, 0, "no tmux server survived the reboot");
let snap = revived.snapshot().await;
assert_eq!(snap.len(), 1);
assert_eq!(snap[0].id, s.id);
assert_eq!(snap[0].status, SessionStatus::Dormant);
}
/// #164: an unchanged boot id is a plain restart — recovery probes tmux as
/// before and adopts the still-live session `Running`.
#[tokio::test]
+64
View File
@@ -536,6 +536,29 @@ pub struct SpawnPreset {
pub resume_command: Option<String>,
}
/// Placeholder a preset's `resume_command` may carry for the agent's own
/// conversation id (#165), substituted from [`Session::agent_session_id`].
pub const AGENT_SESSION_PLACEHOLDER: &str = "{agent_session}";
impl SpawnPreset {
/// The command that resumes a session spawned from this preset, with
/// [`AGENT_SESSION_PLACEHOLDER`] filled in from `agent_session`.
///
/// `None` when the preset has no `resume_command` — a plain shell has
/// nothing to resume — and also when the template asks for an agent
/// session id the session never reported: `claude --resume` with an empty
/// id resumes nothing, so the caller is better off respawning the original
/// command than running a broken one.
pub fn resume_command_for(&self, agent_session: Option<&str>) -> Option<String> {
let template = self.resume_command.as_deref()?;
if !template.contains(AGENT_SESSION_PLACEHOLDER) {
return Some(template.to_string());
}
let id = agent_session.filter(|s| !s.is_empty())?;
Some(template.replace(AGENT_SESSION_PLACEHOLDER, id))
}
}
/// Build provenance of the running daemon: the commit it was compiled
/// from and when that commit was authored.
///
@@ -785,6 +808,47 @@ pub enum ClientControl {
mod tests {
use super::*;
fn preset(resume: Option<&str>) -> SpawnPreset {
SpawnPreset {
id: "claude".into(),
label: "Claude".into(),
command: "claude".into(),
resume_command: resume.map(str::to_string),
}
}
#[test]
fn resume_command_substitutes_the_agent_session() {
let p = preset(Some("claude --resume {agent_session}"));
assert_eq!(
p.resume_command_for(Some("abc-123")).as_deref(),
Some("claude --resume abc-123")
);
}
#[test]
fn resume_command_without_a_placeholder_ignores_the_agent_session() {
let p = preset(Some("opencode --continue"));
assert_eq!(
p.resume_command_for(None).as_deref(),
Some("opencode --continue")
);
}
/// A template that wants an id the session never reported would run
/// `claude --resume` against nothing; the caller respawns instead.
#[test]
fn resume_command_is_none_without_the_id_it_asks_for() {
let p = preset(Some("claude --resume {agent_session}"));
assert!(p.resume_command_for(None).is_none());
assert!(p.resume_command_for(Some("")).is_none());
}
#[test]
fn a_preset_without_a_resume_command_has_none() {
assert!(preset(None).resume_command_for(Some("abc")).is_none());
}
#[test]
fn version_compatibility_is_major_only() {
assert!(is_compatible(PROTOCOL_VERSION));
+15
View File
@@ -66,6 +66,12 @@ pub fn session_focus(id: Uuid) -> String {
format!("{SESSIONS}/{id}/focus")
}
/// `POST /api/sessions/:id/resume` (#166): respawn a dormant session under its
/// own id, keeping its sidecar, name, worktree and `created_at`.
pub fn session_resume(id: Uuid) -> String {
format!("{SESSIONS}/{id}/resume")
}
/// `WS /api/attach/:id` — raw PTY bytes.
pub fn attach(id: Uuid) -> String {
format!("{API_PREFIX}/attach/{id}")
@@ -79,6 +85,7 @@ pub mod patterns {
pub const SESSION_INPUT: &str = "/api/sessions/:id/input";
pub const SESSION_PROMPT: &str = "/api/sessions/:id/prompt";
pub const SESSION_FOCUS: &str = "/api/sessions/:id/focus";
pub const SESSION_RESUME: &str = "/api/sessions/:id/resume";
pub const ATTACH: &str = "/api/attach/:id";
}
@@ -202,6 +209,10 @@ mod tests {
session_focus(id()),
"/api/sessions/6f1a7c0e-0f5e-4a2b-9a3d-6b1c2d3e4f50/focus"
);
assert_eq!(
session_resume(id()),
"/api/sessions/6f1a7c0e-0f5e-4a2b-9a3d-6b1c2d3e4f50/resume"
);
assert_eq!(
attach(id()),
"/api/attach/6f1a7c0e-0f5e-4a2b-9a3d-6b1c2d3e4f50"
@@ -236,6 +247,10 @@ mod tests {
patterns::SESSION_FOCUS.trim_end_matches(":id/focus"),
session(id()).trim_end_matches(&id().to_string())
);
assert_eq!(
patterns::SESSION_RESUME.trim_end_matches(":id/resume"),
session(id()).trim_end_matches(&id().to_string())
);
assert_eq!(
patterns::ATTACH.trim_end_matches(":id"),
attach(id()).trim_end_matches(&id().to_string())
+20
View File
@@ -243,6 +243,26 @@ Pick `/tmp` if the daemon must outlive your login session and you can't run
tmux sockets with it). The env var `MAESTRO_DAEMON_TMUX_TMPDIR` takes
precedence over the file.
## What happens to sessions after a reboot
A host reboot kills the tmux server, so every session the daemon knew is dead
while its record on disk survives. What recovery does with those is
`[daemon].resume_on_boot`:
```toml
[daemon]
resume_on_boot = "dormant" # "dormant" (default) | "auto" | "never"
```
- `dormant` — the sessions are listed but not running, and you resume the ones
you want from the web UI or the TUI.
- `auto` — the daemon resumes them all itself during startup. Only pick this
on a host where restarting every agent unattended is safe: they come back on
the flags they were spawned with, with nobody watching.
- `never` — the records are deleted and a reboot is a clean slate.
See `docs/RESTART.md` for the full contract.
## What the systemd unit looks like
The wizard writes a *user* unit (`systemctl --user`), since `maestro daemon`
+65 -4
View File
@@ -75,13 +75,74 @@ A daemon restart leaves tmux alone, so recovery re-adopts every
beyond the reconnect above.
A **host reboot** is different: it kills the tmux server, so there is nothing
to adopt while every session's sidecar is still on disk. Recovery makes a
to adopt while every session's sidecar is still on disk. Recovery always asks
tmux — `tmux list-sessions` is the single source of truth for liveness, and the
kernel's `boot_id` only tells a reboot from a restart in the log (#164, #174).
A boot id changes without the tmux server dying whenever the state directory is
restored, a VM or container snapshot is rolled back, or the `boot_id` file is
truncated; a session tmux still holds is adopted `Running` in every one of
those cases. Recovery makes a
second pass over those sidecars and registers each as a `Dormant` session
(#162) — id, name, cwd, command, worktree and `created_at` intact. A dormant
session has no pane and no PTY bridge: it cannot be attached to, typed into or
resized, and neither watcher pass touches it. It is a record of what was
running, and the only thing left to do with one is dismiss it, which removes
the sidecar so the next recovery does not bring it back.
resized, and neither watcher pass touches it.
A dormant session is brought back with `POST /api/sessions/:id/resume` (#166),
which respawns tmux under the **same** uuid and sidecar — `metadata.name`,
`created_at`, `env` and worktree ownership all survive, and only the tmux
session and the PTY bridge are new. The command is the matching spawn preset's
`resume_command` with `{agent_session}` filled in from the session's own
`agent_session_id`, so an agent comes back inside its old conversation; a
session with no matching preset, no `resume_command`, or no reported
conversation id is respawned on the command it was created with. `Session::command`
always keeps that original command, so a second resume resolves the same preset.
Resume is refused with `409` for a session that is already running
(`already_running`) and for one whose `cwd` has disappeared (`cwd_missing`) —
neither is something a respawn could fix. A session with a resume already in
flight is refused as `already_running` too (#175): the respawn happens with
the session map unlocked, so the record is marked in-flight before it starts
and the second request is answered on that state rather than on its tmux spawn
colliding with the first one's. A resume that loses a race with a kill takes
its own sidecar back off disk, so the killed session does not return as a
dormant ghost on the next recovery. The other thing to do with a dormant
session is dismiss it, which removes the sidecar so the next recovery does not
bring it back.
## What recovery does with them: `resume_on_boot`
Whether a recovered-but-dead session is left dormant at all is the operator's
call, per host, via `[daemon].resume_on_boot` (#167):
| value | what recovery does |
| ----------- | --------------------------------------------------------------------- |
| `"dormant"` | **Default.** Register each one dormant and wait for the operator. |
| `"auto"` | Resume each one during recovery, itself. |
| `"never"` | Prune the sidecars instead — a reboot is a clean slate. |
`dormant` is the default because resuming agents unattended is a foot-gun:
they come back on whatever flags they were spawned with, seconds after a
reboot, with nobody watching. `auto` exists for hosts where that is known to
be safe, and it is opted into rather than inherited.
`auto` calls the same `resume` the client's `POST /api/sessions/:id/resume`
does — same uuid, same sidecar, the preset's `resume_command` — so there is
one resume path and not two. A session that fails to resume is logged and
stays dormant; the rest still come back.
`never` prunes the same way #163 does for a missing `cwd`: the sidecar is
deleted and nothing is registered, so the session is not listed and the next
recovery does not find it. Whatever worktree it owned is left on disk.
The policy only decides the fate of sessions with no live pane. A session
tmux still holds is adopted `Running` by the first pass under every policy,
including `never`.
Recovery logs the policy that ran and the number of sessions it acted on, so
a host that resumes agents unattended says so in its own journal.
Attaching to a session that has no bridge answers `409 not_attachable`, not
`404 unknown_session`: the session exists, and a client that cannot tell "gone"
from "not running" cannot offer the resume.
`SessionStatus` is a wire type, so from this build forward it deserializes
leniently on both clients: a spelling the client does not know reads as `Dead`