Compare commits

...
Author SHA1 Message Date
Miguel Palhas 87f1e73dec fix(maestro): replace agent_session_id when a report names a new one
A resume issues a new conversation id, so write-once left the stored id
pointing at the pre-resume conversation and every later resume reattached
it. Rewrite the sidecar when the reported id differs from the stored one;
a repeated report of the same id still writes nothing.

Closes #176
2026-09-04 09:26:16 +01:00
Miguel Palhas aebd10c2da merge: guard resume against kill and second resume (#175)
Closes #175
2026-09-04 09:24:35 +01:00
Miguel Palhas ae3091e2a2 fix(maestro): guard resume against kill and second resume
The respawn runs with the session map unlocked, so two takes of the
lock straddle it. A kill landing in that window deleted the sidecar
before resume wrote it back, leaving a file for a session that was
gone — the next recovery registered it as a dormant ghost. Resume now
deletes the sidecar in the branch where it finds its own record gone.

Two resumes both passed the `Running` check for the same reason. The
record is marked in-flight under the first lock, so the second request
is refused with `already_running` instead of losing a race for the
FIFO and reporting a tmux failure. Every path out of resume clears the
marker.

Both races are driven deterministically in the tests: the manager's
tmux spawn is parked mid-resume while the racing kill or resume runs.

Closes #175
2026-09-04 09:19:59 +01:00
Miguel Palhas 1366c2d430 merge: always probe tmux during recovery (#174)
Closes #174
2026-09-04 09:15:45 +01:00
Miguel Palhas d050df7d3d fix(maestro): always probe tmux during recovery
#164 skipped the tmux probe whenever the stored boot id differed from the
current one, assuming a changed boot id proves the tmux server is gone. It
does not: a restored state directory, a VM or container snapshot rollback,
or a truncated boot_id file all change it while tmux keeps running with
live sessions. Every one of those sessions was then registered Dormant and
stranded — resume respawns `maestro-<id>`, which collides with the name
tmux still holds, so it failed 502 forever.

Recovery now always runs `tmux list-sessions`, the single source of truth
for liveness the rest of the daemon assumes. The boot id keeps its other
job: telling a reboot from a restart in recovery's log.

The #164 test asserted the old behaviour with tmux still alive, so it is
replaced by one asserting the session is adopted Running, plus one covering
a real reboot — boot id changed and the tmux server dead — still going
dormant.

Refs #174
2026-09-04 09:13:15 +01:00
Miguel Palhas e51a7007a6 merge: resume_on_boot policy (#167)
Closes #167
2026-09-04 09:10:28 +01:00
Miguel PalhasandClaude Opus 5 02bf077c03 feat(maestro): add [daemon].resume_on_boot policy
Recovery now has a stated policy for sessions whose tmux pane is gone but
whose sidecar survived: dormant (default), auto, or never. auto goes down
the same SessionManager::resume the client's POST takes, so there is one
resume path. never prunes the sidecars the way #163 does for a missing cwd.

The default is dormant because resuming agents unattended seconds after a
reboot is a foot-gun; auto is opted into per host. Recovery logs which
policy ran and how many sessions it acted on.

Refs #167

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 09:06:51 +01:00
Miguel Palhas a793727bb9 merge: resume a dormant session in place (#166)
Closes #166
2026-09-04 08:59:37 +01:00
Miguel Palhas bb6b885c68 feat(maestro): add POST /api/sessions/:id/resume
A dormant session comes back under the same uuid and sidecar, so its
name, created_at, env and worktree ownership survive; only the tmux
session and the PTY bridge are new. The command is the matching preset's
resume_command with {agent_session} filled in, falling back to the
command the session was spawned with.

Refused with 409 for a session that is already running or whose cwd is
gone. Attaching to a session with no bridge now answers 409
not_attachable instead of 404 unknown_session, so a client can tell a
resumable session from one that does not exist.
2026-09-04 08:53:06 +01:00
Miguel Palhas db28827647 merge: clear the worktree on a settled natural exit (#172)
Also brings origin/main's web fixes onto the integration branch, which the
issue branch had already merged.

Closes #172
2026-09-04 08:51:42 +01:00
Miguel Palhas 6ac137d432 feat(maestro): add the resume route path and preset command builder
The template's {agent_session} is filled from the session's own
conversation id; a template that asks for an id the session never
reported yields nothing rather than a `--resume` with no argument.
2026-09-04 08:48:30 +01:00
Miguel Palhas ec6345cc32 Merge remote-tracking branch 'origin/main' into reboot/172-worktree-field 2026-09-04 08:48:30 +01:00
Miguel PalhasandClaude Sonnet 5 b05b014e84 fix(maestro): clear worktree from the session clone on natural exit
mark_session_dead cleared blocked_work on the local session clone
written to the sidecar, but not worktree — so a pristine tree the
daemon just removed stayed named in the sidecar, and recovery from
that sidecar advertised a branch and path that no longer exist.

Refs #172.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-04 08:48:24 +01:00
Miguel Palhas aa5e0551b9 merge: per-preset resume_command (#165)
Closes #165
2026-09-04 08:43:11 +01:00
Miguel Palhas ca2b60c17e merge: detect host reboot via boot_id (#164)
Closes #164
2026-09-04 08:43:11 +01:00
Miguel Palhas d548d0b24e feat(maestro): skip the tmux probe on a detected host reboot
Recovery could not tell a daemon restart from a host reboot: both
looked like "some sessions were not in tmux list-sessions", so the
tmux probe ran unconditionally and the logs said nothing useful when
it was actually a reboot.

Record /proc/sys/kernel/random/boot_id in the state directory on
every recover(). When it differs from the stored value, every session
from the previous boot is provably dead, so the tmux probe is skipped
and the second recovery pass registers them dormant directly, logging
"host rebooted, sessions dormant" with the count. A missing stored
value or an unreadable boot id (non-Linux, restricted /proc) falls
back to today's probe-based recovery.

The boot id source is injected via a BootIdSource trait so tests can
simulate a changed or unreadable boot id without rebooting the
machine.

Refs #164
2026-09-04 08:40:10 +01:00
Miguel Palhas aa34d975a9 test(maestro): assert the agent session id survives recover()
The test named for recovery only read the sidecar back; nothing covered
the id reaching the recovered session, which is what a resume reads.

Also narrows RESTART.md's leniency claim: the custom Deserialize lives in
the client's own binary, so an older build still fails on Dormant.
2026-09-04 08:36:43 +01:00
Miguel Palhas a0c97937a1 merge: prune sidecars with a missing cwd (#163)
Closes #163
2026-09-04 08:33:29 +01:00
Miguel Palhas 9728b10ffd feat(maestro): add preset resume commands 2026-09-04 08:33:08 +01:00
Miguel Palhas c004b5d1b6 feat(maestro): prune sidecars whose cwd no longer exists
recover()'s dormant-registration pass (#162) now deletes a sidecar
instead of registering it dormant when the directory it points at is
gone. Nothing can be resumed there, so the honest move is to remove
the record rather than keep an unactionable one around (#163).

Sidecars whose cwd still exists are unaffected, regardless of age.
2026-09-04 08:30:39 +01:00
Miguel Palhas c51d5b8f7b merge: persist agent session id (#161)
Closes #161
2026-09-04 08:27:49 +01:00
Miguel Palhas 1b2b392710 merge: dormant session recovery (#162)
Closes #162
2026-09-04 08:25:44 +01:00
Miguel PalhasandClaude Opus 5 72644dd564 feat(maestro): register dormant sessions on recovery
A host reboot kills the tmux server, so `SessionManager::recover` iterated an
empty list and every session vanished from the API while its sidecar sat
untouched on disk. A second recovery pass now reads the sidecars tmux had
nothing to adopt and registers each as `SessionStatus::Dormant` — a record of
what was running, with no pane and no PTY bridge behind it.

`SessionStatus` deserializes leniently now, like `ActivityState`: an unknown
spelling reads as `Dead` rather than failing the whole `Session`.

Refs #162

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 08:22:52 +01:00
Miguel Palhas fa846000e7 feat(maestro): persist the agent's own session id on the sidecar (#161)
The Claude Code hook already receives a session_id in every payload, but
the activity route dropped it and the daemon had no field to store it on.
Parse it out of the hook body, add Session.agent_session_id, and write it
through to the sidecar once — the first report that carries one wins, and
later reports never overwrite it since an agent's own id never changes
mid-session.

This is the input to a follow-up POST /api/sessions/:id/resume.
2026-09-04 08:22:51 +01:00
39 changed files with 2708 additions and 46 deletions
+1
View File
@@ -14,6 +14,7 @@ maestro daemon tmux wrapper + session API + SPA on 127.0.0.1:7070
├── DELETE /api/sessions/:id kill
├── POST /api/sessions/:id/input type text into a session
├── POST /api/sessions/:id/prompt an agent reports its first prompt (#131)
├── POST /api/sessions/:id/resume respawn a dormant session, same id (#166)
├── GET /api/info hostname, version, home dir, spawn presets
├── GET /api/dirs directory listing (spawn autocomplete)
├── GET /api/recent-dirs spawn history
+2
View File
@@ -188,6 +188,7 @@ fn print_session_table(out: &mut impl Write, sessions: &[Session], info: &Daemon
let status = match s.status {
SessionStatus::Running => "running",
SessionStatus::Dead => "dead",
SessionStatus::Dormant => "dormant",
};
let activity = match s.activity {
ActivityState::Active => "active",
@@ -252,6 +253,7 @@ mod tests {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
}
}
@@ -28,6 +28,7 @@ async fn info_carries_version_and_presets() {
id: "claude".into(),
label: "Claude".into(),
command: "claude".into(),
resume_command: None,
}],
..Default::default()
});
+140
View File
@@ -129,6 +129,80 @@ pub struct DaemonSection {
/// `Some(0)` disables history entirely (no log written, no recents read).
#[serde(default)]
pub spawn_history_days: Option<u32>,
/// What recovery does with sessions that outlived their tmux server —
/// after a host reboot, or any restart that finds a sidecar with no pane
/// (#167). Default [`ResumeOnBoot::Dormant`].
#[serde(default, deserialize_with = "deserialize_resume_on_boot")]
pub resume_on_boot: Option<ResumeOnBoot>,
}
impl DaemonSection {
/// The configured `[daemon].resume_on_boot`, or the safe default:
/// recovered-but-dead sessions wait for the operator.
pub fn resume_on_boot(&self) -> ResumeOnBoot {
self.resume_on_boot.unwrap_or_default()
}
}
/// `[daemon].resume_on_boot` — what recovery does with a session whose tmux
/// pane is gone but whose sidecar is still on disk (#167).
///
/// Resuming agents unattended seconds after a reboot is a foot-gun: they run
/// with whatever flags they were spawned with, with nobody watching. So the
/// default waits for a human, and `auto` is opted into per host.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub enum ResumeOnBoot {
/// Register each one `Dormant` and wait for the operator to resume it.
#[default]
Dormant,
/// Resume each one during recovery, on the same path a client's
/// `POST /api/sessions/:id/resume` takes.
Auto,
/// Prune the sidecars instead: a reboot is a clean slate.
Never,
}
/// Every `[daemon].resume_on_boot` name and its variant, in documentation
/// order.
pub const RESUME_ON_BOOT_POLICIES: &[(&str, ResumeOnBoot)] = &[
("dormant", ResumeOnBoot::Dormant),
("auto", ResumeOnBoot::Auto),
("never", ResumeOnBoot::Never),
];
impl ResumeOnBoot {
/// The name this policy is spelled with in the config file and in logs.
pub fn as_str(&self) -> &'static str {
match self {
ResumeOnBoot::Dormant => "dormant",
ResumeOnBoot::Auto => "auto",
ResumeOnBoot::Never => "never",
}
}
}
fn deserialize_resume_on_boot<'de, D>(deserializer: D) -> Result<Option<ResumeOnBoot>, D::Error>
where
D: Deserializer<'de>,
{
let raw = Option::<String>::deserialize(deserializer)?;
raw.map(|name| {
RESUME_ON_BOOT_POLICIES
.iter()
.find(|(n, _)| *n == name)
.map(|(_, policy)| *policy)
.ok_or_else(|| {
let known = RESUME_ON_BOOT_POLICIES
.iter()
.map(|(n, _)| format!("{n:?}"))
.collect::<Vec<_>>()
.join(", ");
D::Error::custom(format!(
"invalid [daemon].resume_on_boot {name:?}; expected one of {known}"
))
})
})
.transpose()
}
/// `[tui]` — `maestro tui` settings (NAP-275 epic, phase 2 NAP-277).
@@ -313,16 +387,19 @@ pub fn builtin_presets() -> Vec<SpawnPreset> {
id: "claude".into(),
label: "Claude".into(),
command: "claude".into(),
resume_command: Some("claude --resume {agent_session}".into()),
},
SpawnPreset {
id: "opencode".into(),
label: "opencode".into(),
command: "opencode".into(),
resume_command: None,
},
SpawnPreset {
id: "zsh".into(),
label: "Shell".into(),
command: "zsh".into(),
resume_command: None,
},
]
}
@@ -334,6 +411,7 @@ pub fn custom_preset() -> SpawnPreset {
id: CUSTOM_PRESET_ID.into(),
label: "Custom…".into(),
command: String::new(),
resume_command: None,
}
}
@@ -507,6 +585,18 @@ auto_install = false
assert!(custom.command.is_empty());
}
#[test]
fn builtin_claude_preset_resumes_by_agent_session() {
let claude = builtin_presets()
.into_iter()
.find(|preset| preset.id == "claude")
.expect("Claude preset");
assert_eq!(
claude.resume_command.as_deref(),
Some("claude --resume {agent_session}")
);
}
#[test]
fn spawn_presets_override_replaces_builtins() {
// Present list replaces the built-ins entirely; custom still appended.
@@ -515,6 +605,7 @@ auto_install = false
id: "fish".into(),
label: "fish".into(),
command: "fish".into(),
resume_command: None,
}]),
..Default::default()
};
@@ -551,6 +642,7 @@ auto_install = false
id: " ".into(),
label: "x".into(),
command: "x".into(),
resume_command: None,
}]),
..Default::default()
};
@@ -561,6 +653,7 @@ auto_install = false
id: "x".into(),
label: "x".into(),
command: "".into(),
resume_command: None,
}]),
..Default::default()
};
@@ -578,11 +671,13 @@ auto_install = false
id: "a".into(),
label: "a".into(),
command: "a".into(),
resume_command: None,
},
SpawnPreset {
id: "a".into(),
label: "a2".into(),
command: "a2".into(),
resume_command: None,
},
]),
..Default::default()
@@ -597,6 +692,7 @@ auto_install = false
id: "custom".into(),
label: "Custom".into(),
command: "whatever".into(),
resume_command: None,
}]),
..Default::default()
};
@@ -632,6 +728,27 @@ presets = [
assert_eq!(ids, ["claude", "zsh", "custom"]);
}
#[test]
fn spawn_preset_parses_resume_command_from_issue_example() {
let cfg = toml::from_str::<MaestroConfigFile>(
r#"
[spawn]
presets = [
{ id = "claude", command = "claude --dangerously-skip-permissions", resume_command = "claude --dangerously-skip-permissions --resume {agent_session}" }
]
"#,
)
.expect("issue #165 example parses");
let preset = &cfg.spawn.presets.expect("configured presets")[0];
assert_eq!(preset.id, "claude");
assert_eq!(preset.command, "claude --dangerously-skip-permissions");
assert_eq!(
preset.resume_command.as_deref(),
Some("claude --dangerously-skip-permissions --resume {agent_session}")
);
}
#[test]
fn spawn_project_scan_fields_default_to_none() {
// Absent [spawn].project_* → None; the daemon applies its own defaults.
@@ -732,4 +849,27 @@ bind_addr = "127.0.0.1:9090"
);
assert_eq!(cfg.resolve_daemon_url(None), "http://explicit:1");
}
#[test]
fn resume_on_boot_defaults_to_dormant_and_parses_every_policy() {
assert_eq!(
MaestroConfigFile::default().daemon.resume_on_boot(),
ResumeOnBoot::Dormant,
"an absent policy is the safe one"
);
for (name, expected) in RESUME_ON_BOOT_POLICIES {
let cfg = file_with(&format!("[daemon]\nresume_on_boot = \"{name}\"\n"));
assert_eq!(cfg.daemon.resume_on_boot(), *expected, "{name}");
assert_eq!(expected.as_str(), *name);
}
}
#[test]
fn resume_on_boot_rejects_an_unknown_policy() {
let err = toml::from_str::<MaestroConfigFile>("[daemon]\nresume_on_boot = \"maybe\"\n")
.expect_err("unknown policy must not silently fall back");
let msg = err.to_string();
assert!(msg.contains("resume_on_boot"), "{msg}");
assert!(msg.contains("\"auto\""), "{msg}");
}
}
+65
View File
@@ -0,0 +1,65 @@
//! Tells a daemon restart from a host reboot (#164).
//!
//! `/proc/sys/kernel/random/boot_id` is fixed for the life of the kernel and
//! changes on every boot. [`SessionManager::recover`] compares it against the
//! value it stored on the previous run and logs which of the two it was.
//!
//! It does not decide liveness (#174). A boot id changes without the tmux
//! server dying whenever the state directory is restored, a VM or container
//! snapshot is rolled back, or the `boot_id` file is truncated — so recovery
//! always probes `tmux list-sessions` and the boot id only says whether a
//! server that did not answer is coming back.
use std::path::Path;
/// Where the daemon reads the running kernel's boot id. Behind a trait so
/// tests can simulate a changed (or unreadable) boot id without rebooting the
/// machine.
pub trait BootIdSource: std::fmt::Debug + Send + Sync + 'static {
/// Current boot id, or `None` if it can't be determined (non-Linux,
/// restricted `/proc`, missing or empty file).
fn current(&self) -> Option<String>;
}
/// Production source: `/proc/sys/kernel/random/boot_id`.
#[derive(Debug, Default)]
pub struct ProcBootId;
impl BootIdSource for ProcBootId {
fn current(&self) -> Option<String> {
read_boot_id(Path::new("/proc/sys/kernel/random/boot_id"))
}
}
fn read_boot_id(path: &Path) -> Option<String> {
let contents = std::fs::read_to_string(path).ok()?;
let trimmed = contents.trim();
(!trimmed.is_empty()).then(|| trimmed.to_string())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn reads_trimmed_contents() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("boot_id");
std::fs::write(&path, "abc-123\n").unwrap();
assert_eq!(read_boot_id(&path).as_deref(), Some("abc-123"));
}
#[test]
fn missing_file_returns_none() {
let dir = tempfile::tempdir().unwrap();
assert_eq!(read_boot_id(&dir.path().join("missing")), None);
}
#[test]
fn empty_file_returns_none() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("boot_id");
std::fs::write(&path, "\n").unwrap();
assert_eq!(read_boot_id(&path), None);
}
}
+8 -1
View File
@@ -83,7 +83,14 @@ async fn run(config_override: Option<PathBuf>) -> Result<(), Box<dyn Error>> {
.with_spawn_history(crate::spawn_history::SpawnHistory::new(
config.spawn_history_path(),
config.spawn_history_days,
));
))
.with_boot_id(
config.boot_id_path(),
std::sync::Arc::new(crate::boot_id::ProcBootId),
)
// #167: recovery needs the presets to resolve a resumed session's command,
// so `auto` runs the same resume a client's POST would.
.with_resume_policy(config.resume_on_boot, config.spawn_presets.clone());
// Spawned sessions learn where to report back (#103). Derived from the
// configured bind address; set before any spawn can happen.
manager.set_daemon_url(format!("http://{}", config.bind_addr));
+41
View File
@@ -97,6 +97,10 @@ pub struct DaemonConfig {
/// never binds it itself; it serves it so the web client and the TUI read
/// one knob.
pub leader: String,
/// What recovery does with sessions whose tmux pane is gone but whose
/// sidecar survives (#167): `[daemon].resume_on_boot`, default
/// [`maestro_config::ResumeOnBoot::Dormant`].
pub resume_on_boot: maestro_config::ResumeOnBoot,
}
impl DaemonConfig {
@@ -112,6 +116,12 @@ impl DaemonConfig {
pub fn spawn_history_path(&self) -> PathBuf {
self.state_dir.join("spawn-history.jsonl")
}
/// Where recovery records the kernel boot id it saw last (#164):
/// `<state_dir>/boot_id`.
pub fn boot_id_path(&self) -> PathBuf {
self.state_dir.join("boot_id")
}
}
/// Env-var overrides applied by [`load_config`]. Tests pass explicit
@@ -143,6 +153,8 @@ pub fn load_config(path: &Path, env: EnvOverrides) -> Result<DaemonConfig, Confi
// validation (#22/#26).
let spawn_presets = file.spawn.resolve()?;
let resume_on_boot = daemon.resume_on_boot();
let hostname = daemon.hostname.unwrap_or_else(system_hostname);
let home_dir = dirs::home_dir()
.map(|p| p.to_string_lossy().into_owned())
@@ -203,6 +215,7 @@ pub fn load_config(path: &Path, env: EnvOverrides) -> Result<DaemonConfig, Confi
project_scan_depth,
hooks_auto_install: file.hooks.auto_install(),
leader: file.tui.leader().to_string(),
resume_on_boot,
})
}
@@ -283,6 +296,34 @@ auto_install = false
assert!(!cfg.hooks_auto_install);
}
#[test]
fn resume_on_boot_defaults_to_dormant_and_reads_the_file() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("config.toml");
std::fs::write(&path, "").unwrap();
assert_eq!(
load_config(&path, EnvOverrides::default())
.unwrap()
.resume_on_boot,
maestro_config::ResumeOnBoot::Dormant
);
std::fs::write(
&path,
r#"
[daemon]
resume_on_boot = "auto"
"#,
)
.unwrap();
assert_eq!(
load_config(&path, EnvOverrides::default())
.unwrap()
.resume_on_boot,
maestro_config::ResumeOnBoot::Auto
);
}
#[test]
fn empty_file_defaults_everything() {
let dir = tempfile::tempdir().unwrap();
+2
View File
@@ -12,6 +12,7 @@ pub mod attach;
/// Naming a session after its first prompt (#131).
#[cfg(feature = "haiku-autoname")]
pub mod autoname;
pub mod boot_id;
pub mod branches;
pub mod cli;
pub mod config;
@@ -24,6 +25,7 @@ pub mod proc;
pub mod projects;
pub mod pty;
pub mod release;
pub mod resume;
pub mod session;
pub mod sidecar;
pub mod spawn_history;
+40 -1
View File
@@ -52,6 +52,7 @@ pub fn router(state: LocalApiState) -> Router {
.route(routes::patterns::SESSION_ACTIVITY, post(report_activity))
.route(routes::patterns::SESSION_INPUT, post(send_input))
.route(routes::patterns::SESSION_FOCUS, post(set_focus))
.route(routes::patterns::SESSION_RESUME, post(resume_session))
.route(
routes::patterns::SESSION,
delete(kill_session).patch(rename_session),
@@ -170,6 +171,9 @@ fn session_error_status(code: &str) -> StatusCode {
// The session outlived its pane: nothing to type into, and nothing
// the daemon can do about it.
"not_running" => StatusCode::CONFLICT,
// Resume refusals (#166): the session exists and the request was
// well-formed, it just cannot be brought back in this state.
"already_running" | "cwd_missing" => StatusCode::CONFLICT,
_ => StatusCode::BAD_GATEWAY,
}
}
@@ -307,7 +311,11 @@ async fn report_activity(
Path(id): Path<Uuid>,
Json(body): Json<ReportActivityRequest>,
) -> Response {
match s.manager.report_activity(id, body.state, body.detail).await {
match s
.manager
.report_activity(id, body.state, body.detail, body.session_id)
.await
{
Ok(()) => StatusCode::NO_CONTENT.into_response(),
Err(e) => {
let info = e.to_error_info();
@@ -364,6 +372,27 @@ async fn set_focus(
}
}
/// `POST /api/sessions/:id/resume` (#166): respawn a session that has no pane
/// under its own id and sidecar, so its name, `created_at`, env and worktree
/// ownership survive. Answers with the updated session, which also goes out as
/// `SessionUpdated` to every events subscriber.
///
/// Refused with `409` when the session is already running or its `cwd` is
/// gone; `404` when there is no such session.
async fn resume_session(State(s): State<LocalApiState>, Path(id): Path<Uuid>) -> Response {
match s.manager.resume(id, &s.presets).await {
Ok(session) => (StatusCode::OK, Json(session)).into_response(),
Err(e) => {
let info = e.to_error_info();
error_response(
session_error_status(&info.code),
info.code.as_str(),
info.message.as_str(),
)
}
}
}
fn error_response(status: StatusCode, code: &str, message: &str) -> Response {
(status, Json(ApiError::new(code, message))).into_response()
}
@@ -406,6 +435,16 @@ async fn attach_ws(
ws: axum::extract::WebSocketUpgrade,
) -> Result<Response, Response> {
let Some((replay, rx)) = s.manager.subscribe_pty_with_replay(id).await else {
// #166: a session the daemon knows but has no bridge for — a dormant
// one — is not missing, it is not running. Reporting it as unknown
// left the client unable to tell "gone" from "resumable".
if s.manager.has(id).await {
return Err(error_response(
StatusCode::CONFLICT,
"not_attachable",
"session has no live pane; resume it first",
));
}
return Err(error_response(
StatusCode::NOT_FOUND,
"unknown_session",
+99
View File
@@ -0,0 +1,99 @@
//! Choosing the command a resumed session comes back on (#166).
//!
//! A session records the command it was spawned with. Bringing it back means
//! running that preset's `resume_command` (#165) instead, so an agent picks up
//! its old conversation rather than starting a fresh one. Everything else
//! about the session — id, sidecar, name, `created_at`, env, worktree — is
//! untouched, so only the command is decided here.
use maestro_protocol::{Session, SpawnPreset};
/// The command to respawn `session` with.
///
/// The preset is matched on the command the session was spawned with, which is
/// what the sidecar holds. A session with no matching preset, a preset with no
/// `resume_command`, or a `resume_command` needing an agent session id the
/// session never reported all fall back to the original command: respawning
/// what the operator asked for is always honest, where a half-filled resume
/// template is not.
pub fn resume_command(session: &Session, presets: &[SpawnPreset]) -> String {
presets
.iter()
.find(|p| p.command == session.command)
.and_then(|p| p.resume_command_for(session.agent_session_id.as_deref()))
.unwrap_or_else(|| session.command.clone())
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashMap;
fn presets() -> Vec<SpawnPreset> {
vec![
SpawnPreset {
id: "claude".into(),
label: "Claude".into(),
command: "claude".into(),
resume_command: Some("claude --resume {agent_session}".into()),
},
SpawnPreset {
id: "zsh".into(),
label: "Shell".into(),
command: "zsh".into(),
resume_command: None,
},
]
}
fn session(command: &str, agent_session_id: Option<&str>) -> Session {
Session {
id: uuid::Uuid::new_v4(),
cwd: "/tmp".into(),
command: command.into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
status: maestro_protocol::SessionStatus::Dormant,
created_at: chrono::Utc::now(),
foreground: None,
activity: maestro_protocol::ActivityState::Idle,
activity_detail: None,
last_activity_at: chrono::Utc::now(),
activity_since: chrono::Utc::now(),
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: agent_session_id.map(str::to_string),
}
}
#[test]
fn uses_the_matching_presets_resume_command() {
assert_eq!(
resume_command(&session("claude", Some("conv-7")), &presets()),
"claude --resume conv-7"
);
}
#[test]
fn falls_back_when_the_agent_session_id_is_unknown() {
assert_eq!(
resume_command(&session("claude", None), &presets()),
"claude"
);
}
#[test]
fn falls_back_for_a_preset_without_a_resume_command() {
assert_eq!(resume_command(&session("zsh", None), &presets()), "zsh");
}
#[test]
fn falls_back_when_no_preset_matches() {
assert_eq!(
resume_command(&session("bash --noprofile", Some("conv-7")), &presets()),
"bash --noprofile"
);
}
}
+472 -23
View File
@@ -12,6 +12,7 @@ use std::sync::Arc;
use std::time::Duration;
use chrono::Utc;
use maestro_config::ResumeOnBoot;
use maestro_protocol::{
ActivityState, CatalogEvent, CreateSessionRequest, ErrorInfo, NameSource, Session,
SessionStatus, WorktreeSpec, NAME_KEY, NAME_SOURCE_KEY,
@@ -19,6 +20,7 @@ use maestro_protocol::{
use tokio::sync::{broadcast, Mutex};
use uuid::Uuid;
use crate::boot_id::BootIdSource;
use crate::panes::Panes;
use crate::pty::{pipe_pane_command, PtyBridge};
use crate::sidecar::{SidecarError, SidecarStore};
@@ -62,7 +64,10 @@ pub struct SessionRecord {
/// `base_repo` from the original `WorktreeSpec`. Needed because `git
/// worktree remove` is invoked from the base repo.
pub worktree_base_repo: Option<PathBuf>,
pub bridge: PtyBridge,
/// The PTY bridge feeding this session's attach stream. `None` for a
/// `Dormant` session (#162): there is no pane behind it, so there is
/// nothing to pipe and nothing to attach to.
pub bridge: Option<PtyBridge>,
/// Agent-reported activity (#105) still within its TTL. While in force it
/// outranks the activity pass's own inference; when it lapses, inference
/// resumes and [`Session::activity_detail`] is cleared with it. This
@@ -82,6 +87,11 @@ pub struct SessionRecord {
/// (`POST /.../focus`), `None` while unfocused. Daemon-local; never rides
/// the wire.
pub focused_since: Option<std::time::Instant>,
/// Set for as long as a [`SessionManager::resume`] is between its two
/// takes of the map lock (#175). The spawn happens with the lock released,
/// so the `Running` check alone lets a second request through; this is the
/// state the second one is refused on. Daemon-local; never rides the wire.
pub resuming: bool,
/// Sticky acknowledgement applied by [`crate::activity::apply_focus_suppression`]:
/// once the pane has been focused long enough to clear an `AwaitingInput`
/// glow, this holds through focus loss until the session does new work.
@@ -108,6 +118,15 @@ pub enum SessionError {
/// a stray keystroke on an attach socket, which is dropped.
#[error("session is no longer running: {0}")]
NotRunning(Uuid),
/// A resume (#166) was asked for a session that is already running. There
/// is nothing to bring back, and respawning would strand the pane that is
/// already there.
#[error("session is already running: {0}")]
AlreadyRunning(Uuid),
/// A resume (#166) was asked for a session whose working directory is
/// gone. Nothing can be resumed in a directory that no longer exists.
#[error("session working directory no longer exists: {0}")]
CwdMissing(PathBuf),
/// The kill was refused (#79): the session's worktree holds unsaved work.
/// Nothing happened — the session keeps running and the tree is untouched,
/// so it stays the operator's handle on their own unfinished work.
@@ -126,6 +145,8 @@ impl SessionError {
SessionError::InvalidEnv(_) => "invalid_env",
SessionError::NotRunning(_) => "not_running",
SessionError::KillGuarded(_) => "kill_guarded",
SessionError::AlreadyRunning(_) => "already_running",
SessionError::CwdMissing(_) => "cwd_missing",
};
let unsaved_work = match self {
SessionError::KillGuarded(b) => Some(*b),
@@ -176,6 +197,30 @@ struct Inner {
/// client that reconnects never collides with the registration its
/// previous socket has not finished releasing.
next_viewport: AtomicU64,
/// Boot-id tracking for `recover` (#164). `None` — the default — means
/// reboot detection is unwired, which behaves exactly like a boot id
/// that could never be read: always fall back to probing tmux.
boot_id: Option<BootIdConfig>,
/// What recovery does with a session whose pane is gone (#167), and the
/// presets an `auto` resume resolves its command from. Defaults to
/// [`ResumeOnBoot::Dormant`] with no presets: register and wait.
resume_policy: ResumeOnBoot,
resume_presets: Vec<maestro_protocol::SpawnPreset>,
}
/// Where recovery persists the boot id it last saw, and how it reads the
/// current one. See [`crate::boot_id`].
struct BootIdConfig {
path: PathBuf,
source: Arc<dyn BootIdSource>,
}
impl std::fmt::Debug for BootIdConfig {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("BootIdConfig")
.field("path", &self.path)
.finish_non_exhaustive()
}
}
impl SessionManager {
@@ -218,6 +263,9 @@ impl SessionManager {
events,
viewports: Mutex::new(Viewports::new()),
next_viewport: AtomicU64::new(0),
boot_id: None,
resume_policy: ResumeOnBoot::default(),
resume_presets: Vec::new(),
}),
}
}
@@ -247,6 +295,41 @@ impl SessionManager {
self
}
/// Wire boot-id tracking for `recover` (#164). Builder, called once right
/// after construction like the other `with_*` methods. Without it,
/// `recover` always falls back to probing tmux — the same as a boot id
/// that could never be read.
///
/// # Panics
///
/// If the handle has already been cloned.
pub fn with_boot_id(mut self, path: PathBuf, source: Arc<dyn BootIdSource>) -> Self {
let inner = Arc::get_mut(&mut self.inner)
.expect("with_boot_id must be called before the SessionManager is cloned");
inner.boot_id = Some(BootIdConfig { path, source });
self
}
/// Wire the `[daemon].resume_on_boot` policy (#167) and the spawn presets
/// an `auto` resume resolves each session's command from. Builder, called
/// once right after construction like the other `with_*` methods. Without
/// it, recovery behaves as [`ResumeOnBoot::Dormant`].
///
/// # Panics
///
/// If the handle has already been cloned.
pub fn with_resume_policy(
mut self,
policy: ResumeOnBoot,
presets: Vec<maestro_protocol::SpawnPreset>,
) -> Self {
let inner = Arc::get_mut(&mut self.inner)
.expect("with_resume_policy must be called before the SessionManager is cloned");
inner.resume_policy = policy;
inner.resume_presets = presets;
self
}
/// Set the daemon's own HTTP base URL (#103), exported to every session
/// spawned afterwards as `MAESTRO_URL`. Unlike the `with_*` builders this
/// is a plain setter: the daemon runner learns its bound address after
@@ -291,7 +374,10 @@ impl SessionManager {
/// unknown to this daemon.
pub async fn subscribe_pty(&self, id: Uuid) -> Option<broadcast::Receiver<Vec<u8>>> {
let sessions = self.inner.sessions.lock().await;
sessions.get(&id).map(|r| r.bridge.subscribe())
sessions
.get(&id)
.and_then(|r| r.bridge.as_ref())
.map(|b| b.subscribe())
}
/// Subscribe to a session's PTY broadcast and return the replay buffer
@@ -310,19 +396,26 @@ impl SessionManager {
let (result, updated) = {
let mut sessions = self.inner.sessions.lock().await;
let rec = sessions.get_mut(&id)?;
rec.session.last_focused_at = Some(Utc::now());
let updated = rec.session.clone();
let SessionRecord {
session, bridge, ..
} = &mut *rec;
// #162: a dormant session has no bridge and no pane — nothing was
// ever piped for it, so there is no stream to hand back and the
// attach reads as unknown. Checked before anything is stamped.
let bridge = bridge.as_ref()?;
session.last_focused_at = Some(Utc::now());
let updated = session.clone();
// For a live session, tmux's own screen beats the byte tail: the
// tail starts mid-stream, so a full-screen program (an agent TUI)
// replayed from it renders as garbage until something forces a
// repaint. A dead session has no pane left to capture, so it
// keeps the byte tail.
let alive = rec.session.status == SessionStatus::Running;
let alive = session.status == SessionStatus::Running;
// The capture shells out to tmux while the replay lock is held,
// which parks the reader for the length of one tmux round-trip.
// That only happens on attach, and the alternative is a seam in
// the stream.
let result = rec.bridge.subscribe_with_snapshot(|| {
let result = bridge.subscribe_with_snapshot(|| {
if !alive {
return None;
}
@@ -478,6 +571,7 @@ impl SessionManager {
// fresh tree has nothing in the way anyway.
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
};
if let Err(e) = self.inner.sidecar.write(&session) {
@@ -499,13 +593,14 @@ impl SessionManager {
fifo_path,
worktree_path,
worktree_base_repo,
bridge,
bridge: Some(bridge),
reported_activity: None,
settled: false,
#[cfg(feature = "haiku-autoname")]
naming: false,
focused_since: None,
awaiting_suppressed: false,
resuming: false,
},
);
}
@@ -536,6 +631,141 @@ impl SessionManager {
Ok(session)
}
/// Bring a session that has no pane back to life (#166).
///
/// The session comes back under the **same** uuid and the same sidecar, so
/// `metadata.name`, `created_at`, `env` and worktree ownership all survive
/// — only the tmux session and the PTY bridge are new. That is the whole
/// point: spawning a fresh session at the same path instead produces a
/// stranger with no worktree to clean up on kill.
///
/// The command is [`crate::resume::resume_command`]: the matching preset's
/// `resume_command` (#165) with the session's own agent conversation id
/// substituted in, falling back to the command the session was spawned
/// with. `Session::command` keeps the original either way, so a second
/// resume still resolves the same preset.
///
/// Refused for a session that is already running, and for one whose `cwd`
/// has disappeared — neither is something a respawn could fix.
pub async fn resume(
&self,
id: Uuid,
presets: &[maestro_protocol::SpawnPreset],
) -> Result<Session, SessionError> {
// The spawn below runs with the map lock released, so the `Running`
// check alone is not enough to keep two resumes apart (#175): both
// would pass it, both would open a `PtyBridge` on the same FIFO, and
// the loser would report a tmux failure instead of the conflict it
// is. The record is marked in-flight under this first lock, and every
// path out of here clears it again.
let mut session = {
let mut sessions = self.inner.sessions.lock().await;
let rec = sessions.get_mut(&id).ok_or(SessionError::NotFound(id))?;
if rec.session.status == SessionStatus::Running || rec.resuming {
return Err(SessionError::AlreadyRunning(id));
}
rec.resuming = true;
rec.session.clone()
};
if !session.cwd.exists() {
self.clear_resuming(id).await;
return Err(SessionError::CwdMissing(session.cwd.clone()));
}
let command = crate::resume::resume_command(&session, presets);
// Same fifo path as the original spawn: it is derived from the id, and
// `mkfifo` tolerates the stale node the previous run left behind.
let fifo_path = self.inner.fifo_dir.join(format!("{id}.fifo"));
let bridge =
PtyBridge::start_with_replay(fifo_path.clone(), self.inner.replay_bytes).await?;
let mut session_env = sanitize_env(&session.env)?;
session_env.insert("MAESTRO_SESSION_ID".into(), id.to_string());
if let Some(url) = self
.inner
.daemon_url
.lock()
.expect("daemon_url mutex")
.clone()
{
session_env.insert("MAESTRO_URL".into(), url);
}
let spawned = {
let tmux = self.inner.tmux.clone();
let cwd = session.cwd.clone();
let command = command.clone();
let sink = pipe_pane_command(&fifo_path);
tmux_blocking(move || tmux.new_session_piped(id, &cwd, &command, &session_env, &sink))
.await
};
if let Err(e) = spawned {
// `bridge` drops here, closing the FIFO reader. Nothing else was
// touched, so the session stays exactly as dormant as it was.
self.clear_resuming(id).await;
return Err(SessionError::Tmux(e));
}
session.status = SessionStatus::Running;
// A resumed session is producing its initial output again; the
// watcher refines this on its next tick, exactly as after a spawn.
session.foreground = None;
session.activity = ActivityState::Active;
session.activity_detail = None;
session.last_activity_at = Utc::now();
session.activity_since = Utc::now();
session.current_cwd = None;
session.last_focused_at = None;
if let Err(e) = self.inner.sidecar.write(&session) {
// tmux already spawned; kill it rather than leave a pane the
// sidecar does not describe.
let tmux = self.inner.tmux.clone();
let _ = tmux_blocking(move || tmux.kill_session(id)).await;
self.clear_resuming(id).await;
return Err(SessionError::Sidecar(e));
}
{
let mut sessions = self.inner.sessions.lock().await;
let Some(rec) = sessions.get_mut(&id) else {
// Killed while we were spawning. Undo the pane instead of
// leaving one behind with no record of it — and delete the
// sidecar the write above just put back (#175). The kill
// deleted it before that write landed, so leaving it would
// have the next recovery register a dormant ghost for a
// session that is gone and whose worktree is already removed.
drop(sessions);
let tmux = self.inner.tmux.clone();
let _ = tmux_blocking(move || tmux.kill_session(id)).await;
if let Err(e) = self.inner.sidecar.delete(id) {
tracing::warn!(error = ?e, %id, "resume: failed to delete the sidecar of a session killed mid-resume");
}
return Err(SessionError::NotFound(id));
};
rec.session = session.clone();
rec.fifo_path = fifo_path;
rec.bridge = Some(bridge);
rec.reported_activity = None;
rec.settled = false;
rec.focused_since = None;
rec.awaiting_suppressed = false;
rec.resuming = false;
}
let _ = self.inner.events.send(CatalogEvent::SessionUpdated {
session: session.clone(),
});
Ok(session)
}
/// Drop the in-flight marker [`Self::resume`] set under its first lock
/// (#175), so a refused or failed resume can be retried. A no-op when the
/// record is already gone: a kill that landed mid-resume took the marker
/// with it.
async fn clear_resuming(&self, id: Uuid) {
if let Some(rec) = self.inner.sessions.lock().await.get_mut(&id) {
rec.resuming = false;
}
}
/// Best-effort removal of a worktree created earlier in `create` when a
/// later step fails. No-op when the request had no worktree.
fn rollback_worktree(&self, base: &Option<PathBuf>, path: &Option<PathBuf>) {
@@ -651,8 +881,20 @@ impl SessionManager {
/// and start a `PtyBridge`. Sessions whose sidecar references a `cwd`
/// that no longer exists are still adopted — the spec does not require
/// us to repair them.
///
/// #164/#174: the kernel's boot id says whether the host rebooted since
/// the last run, but it never decides which sessions are alive —
/// `tmux list-sessions` does, always, which is what the rest of the
/// daemon assumes. A boot id can change without the tmux server dying
/// (a restored state directory, a VM or container snapshot restore, a
/// truncated `boot_id` file); skipping the probe then registered every
/// live session `Dormant`, and resume could never bring one back because
/// the tmux name was still taken. The boot id is now only a signal that a
/// tmux server which did not answer is not coming back, and it names
/// itself in recovery's log.
pub async fn recover(&self) -> Result<usize, SessionError> {
tokio::fs::create_dir_all(&self.inner.fifo_dir).await?;
let rebooted = self.check_boot_id().await;
let live = {
let tmux = self.inner.tmux.clone();
tmux_blocking(move || tmux.list_sessions()).await?
@@ -683,6 +925,7 @@ impl SessionManager {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
},
};
let fifo_path = self.inner.fifo_dir.join(format!("{id}.fifo"));
@@ -707,13 +950,14 @@ impl SessionManager {
// directory blocks that branch forever (#54 review).
worktree_path: session.worktree.as_ref().map(|w| w.path.clone()),
worktree_base_repo: session.worktree.as_ref().map(|w| w.base_repo.clone()),
bridge,
bridge: Some(bridge),
reported_activity: None,
settled: false,
#[cfg(feature = "haiku-autoname")]
naming: false,
focused_since: None,
awaiting_suppressed: false,
resuming: false,
},
);
let _ = self
@@ -722,16 +966,192 @@ impl SessionManager {
.send(CatalogEvent::SessionAdded { session });
adopted += 1;
}
self.register_dormant(rebooted).await;
Ok(adopted)
}
/// Forward keystrokes to a session's pane. A dead session is read-only —
/// its pane is gone, so this is a no-op rather than a tmux failure the
/// client would have to interpret.
/// Compare the running kernel's boot id against the one recovery stored
/// last time (#164), then persist the current one for next time. Returns
/// `true` only when both are known and differ — a host reboot.
///
/// A missing stored value (first run) or an unreadable current one
/// (non-Linux, restricted `/proc`, unwired in this build) both read as
/// "no reboot". Either way the tmux probe runs (#174): the answer only
/// tells recovery whether a tmux server that did not answer is coming
/// back, never which sessions are alive.
async fn check_boot_id(&self) -> bool {
let Some(cfg) = &self.inner.boot_id else {
return false;
};
let Some(current) = cfg.source.current() else {
return false;
};
let stored = tokio::fs::read_to_string(&cfg.path)
.await
.ok()
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty());
let rebooted = stored.is_some_and(|s| s != current);
if let Some(parent) = cfg.path.parent() {
let _ = tokio::fs::create_dir_all(parent).await;
}
if let Err(e) = tokio::fs::write(&cfg.path, &current).await {
tracing::warn!(error = ?e, path = %cfg.path.display(), "recover: failed to persist boot id");
}
rebooted
}
/// Second recovery pass (#162): register every sidecar tmux had nothing to
/// adopt as a [`SessionStatus::Dormant`] session.
///
/// A host reboot kills the tmux server, so the first pass runs over an
/// empty list and every session would otherwise disappear from the API
/// while its sidecar sat untouched on disk. A dormant record keeps the id,
/// name, cwd, command, worktree and `created_at` — everything the sidecar
/// holds — so the operator still has a record of what was running. It has
/// no tmux session and no [`PtyBridge`] behind it: it cannot be attached
/// to, cannot be typed into, and is skipped by both watcher passes. The
/// only thing left to do with one is dismiss it, which removes the sidecar
/// so the next recovery does not bring it back.
///
/// Activity is not carried over: whatever the session was doing, it is not
/// doing it now. The rest of the sidecar is left as written.
///
/// #163: a sidecar whose `cwd` no longer exists is pruned instead —
/// nothing can be resumed in a directory that is gone, so the honest move
/// is to delete the sidecar rather than register a record no operator can
/// act on. A sidecar whose `cwd` still exists is kept regardless of age.
///
/// `rebooted` (#164) only changes what gets logged: a detected reboot
/// names the count even when it is zero, so the operator sees that
/// recovery told a reboot from a restart instead of a log that stays
/// silent either way. It never decides membership — everything here got
/// here by tmux not holding it (#174).
async fn register_dormant(&self, rebooted: bool) {
let stored = match self.inner.sidecar.load_all() {
Ok(stored) => stored,
Err(e) => {
tracing::warn!(error = ?e, "recover: reading sidecars for dormant sessions failed");
return;
}
};
let mut registered = Vec::new();
let mut pruned = 0usize;
let mut discarded = 0usize;
{
let mut sessions = self.inner.sessions.lock().await;
for mut session in stored {
if sessions.contains_key(&session.id) {
// Adopted live by the first pass; its sidecar is on disk too.
continue;
}
if !session.cwd.exists() {
if let Err(e) = self.inner.sidecar.delete(session.id) {
tracing::warn!(error = ?e, id = %session.id, "recover: failed to prune sidecar with missing cwd");
}
pruned += 1;
continue;
}
if self.inner.resume_policy == ResumeOnBoot::Never {
// #167: this host wants a reboot to be a clean slate, so
// the sidecar goes the way #163's does — nothing is
// registered and the next recovery finds nothing.
if let Err(e) = self.inner.sidecar.delete(session.id) {
tracing::warn!(error = ?e, id = %session.id, "recover: failed to prune sidecar under resume_on_boot=never");
}
discarded += 1;
continue;
}
let id = session.id;
session.status = SessionStatus::Dormant;
session.foreground = None;
session.activity = ActivityState::Idle;
session.activity_detail = None;
sessions.insert(
id,
SessionRecord {
session: session.clone(),
fifo_path: self.inner.fifo_dir.join(format!("{id}.fifo")),
// Still known, so a dismissal cleans the worktree up —
// the same reason recovery restores it for a live
// session (#54 review).
worktree_path: session.worktree.as_ref().map(|w| w.path.clone()),
worktree_base_repo: session.worktree.as_ref().map(|w| w.base_repo.clone()),
bridge: None,
reported_activity: None,
settled: false,
#[cfg(feature = "haiku-autoname")]
naming: false,
focused_since: None,
awaiting_suppressed: false,
resuming: false,
},
);
registered.push(session);
}
}
if rebooted {
tracing::info!(
dormant = registered.len(),
"host rebooted, sessions dormant"
);
} else if !registered.is_empty() {
tracing::info!(
dormant = registered.len(),
"registered sessions with no live tmux as dormant"
);
}
if pruned > 0 {
tracing::info!(pruned, "pruned sidecars whose cwd no longer exists");
}
let resumable: Vec<Uuid> = registered.iter().map(|s| s.id).collect();
for session in registered {
let _ = self
.inner
.events
.send(CatalogEvent::SessionAdded { session });
}
// #167: say which policy ran and how many sessions it acted on, so a
// host that resumes agents unattended says so in its own log. The
// match is exhaustive on purpose: a fourth policy must decide what it
// does here rather than inherit whichever arm happened to be last.
match self.inner.resume_policy {
ResumeOnBoot::Dormant => tracing::info!(
policy = ResumeOnBoot::Dormant.as_str(),
sessions = resumable.len(),
"resume_on_boot: sessions left dormant for the operator"
),
ResumeOnBoot::Never => tracing::info!(
policy = ResumeOnBoot::Never.as_str(),
sessions = discarded,
"resume_on_boot: pruned the sidecars of sessions with no live pane"
),
ResumeOnBoot::Auto => {
tracing::info!(
policy = ResumeOnBoot::Auto.as_str(),
sessions = resumable.len(),
"resume_on_boot: resuming sessions with no live pane"
);
let presets = self.inner.resume_presets.clone();
for id in resumable {
// The same call the client's POST /api/sessions/:id/resume
// makes — one resume path, not two.
if let Err(e) = self.resume(id, &presets).await {
tracing::warn!(error = %e, id = %id, "resume_on_boot=auto: resume failed; session stays dormant");
}
}
}
}
}
/// Forward keystrokes to a session's pane. A dead or dormant session is
/// read-only — its pane is gone, so this is a no-op rather than a tmux
/// failure the client would have to interpret.
pub async fn send_keys(&self, id: Uuid, bytes: &[u8]) -> Result<(), SessionError> {
match self.status(id).await {
None => return Err(SessionError::NotFound(id)),
Some(SessionStatus::Dead) => return Ok(()),
Some(SessionStatus::Dead | SessionStatus::Dormant) => return Ok(()),
Some(SessionStatus::Running) => {}
}
let tmux = self.inner.tmux.clone();
@@ -744,13 +1164,15 @@ impl SessionManager {
/// Type `text` into a session's pane, with an Enter after it when
/// `submit` is set.
///
/// Unlike [`Self::send_keys`], a dead session is an error: the caller
/// asked for text to land somewhere, and a silent success would let a
/// script believe an agent had been told something it never heard.
/// Unlike [`Self::send_keys`], a dead or dormant session is an error: the
/// caller asked for text to land somewhere, and a silent success would let
/// a script believe an agent had been told something it never heard.
pub async fn send_input(&self, id: Uuid, text: &str, submit: bool) -> Result<(), SessionError> {
match self.status(id).await {
None => return Err(SessionError::NotFound(id)),
Some(SessionStatus::Dead) => return Err(SessionError::NotRunning(id)),
Some(SessionStatus::Dead | SessionStatus::Dormant) => {
return Err(SessionError::NotRunning(id))
}
Some(SessionStatus::Running) => {}
}
let tmux = self.inner.tmux.clone();
@@ -768,12 +1190,12 @@ impl SessionManager {
Ok(())
}
/// Resize a session's window. No-op for a dead session, as with
/// Resize a session's window. No-op for a dead or dormant session, as with
/// [`Self::send_keys`].
pub async fn resize(&self, id: Uuid, cols: u16, rows: u16) -> Result<(), SessionError> {
match self.status(id).await {
None => return Err(SessionError::NotFound(id)),
Some(SessionStatus::Dead) => return Ok(()),
Some(SessionStatus::Dead | SessionStatus::Dormant) => return Ok(()),
Some(SessionStatus::Running) => {}
}
let tmux = self.inner.tmux.clone();
@@ -881,11 +1303,20 @@ impl SessionManager {
/// A report that changes the effective state restarts `activity_since`
/// (#104's clock); re-reporting an unchanged state only extends the TTL
/// and stays silent on the event stream at steady state.
///
/// `agent_session_id` (#161) is the agent's own conversation id, parsed
/// by the caller from the hook payload's `session_id` field. The
/// sidecar is rewritten only when the reported id differs from the
/// stored one — a repeated report of the same id is pure churn from the
/// hook's `PreToolUse` cadence and writes nothing, but a resume (#176)
/// issues a new conversation id and must replace the stale one, or
/// every later resume reattaches the pre-resume conversation.
pub async fn report_activity(
&self,
id: Uuid,
state: ActivityState,
detail: Option<String>,
agent_session_id: Option<String>,
) -> Result<(), SessionError> {
let updated = {
let mut sessions = self.inner.sessions.lock().await;
@@ -901,7 +1332,16 @@ impl SessionManager {
state,
self.inner.report_ttl,
));
changed.then(|| rec.session.clone())
let id_changed =
agent_session_id.is_some() && rec.session.agent_session_id != agent_session_id;
if id_changed {
rec.session.agent_session_id = agent_session_id;
self.inner
.sidecar
.write(&rec.session)
.map_err(SessionError::Sidecar)?;
}
(changed || id_changed).then(|| rec.session.clone())
};
if let Some(session) = updated {
let _ = self
@@ -1174,7 +1614,11 @@ async fn run_activity_pass(inner: &Arc<Inner>) {
rec.session.activity_detail = None;
}
}
let idle = rec.bridge.idle_for();
// Only `Running` sessions reach this pass, and every one of them
// has a bridge; a dormant session has neither pane nor bridge.
let Some(idle) = rec.bridge.as_ref().map(|b| b.idle_for()) else {
continue;
};
// #105: while a report is in force it supplies the state and
// nothing here may overwrite it; only an unreported session is
// decided by inference.
@@ -1292,8 +1736,10 @@ async fn run_blocked_work_pass(inner: &Arc<Inner>) {
/// alone and stays attached to the dead record for a later forced dismissal
/// (#81), which leaves it on disk too.
///
/// Dead sessions do not survive a daemon restart: [`SessionManager::recover`]
/// adopts only what tmux still holds.
/// A dead session does not come back as dead across a daemon restart: its
/// sidecar survives, so [`SessionManager::recover`]'s second pass registers it
/// as [`SessionStatus::Dormant`] (#162) — accurate either way, since neither
/// has a pane behind it.
async fn mark_session_dead(inner: &Arc<Inner>, id: Uuid) {
let flipped = {
let mut sessions = inner.sessions.lock().await;
@@ -1302,7 +1748,9 @@ async fn mark_session_dead(inner: &Arc<Inner>, id: Uuid) {
record.session.status = SessionStatus::Dead;
// Release anyone attached; the scrollback survives for a later
// attach.
record.bridge.seal();
if let Some(bridge) = record.bridge.as_mut() {
bridge.seal();
}
let session = record.session.clone();
let worktree = record
.worktree_base_repo
@@ -1323,6 +1771,7 @@ async fn mark_session_dead(inner: &Arc<Inner>, id: Uuid) {
// The tree is settled (removed); a stale "in the way" would
// describe something the session no longer holds.
session.blocked_work = None;
session.worktree = None;
let mut sessions = inner.sessions.lock().await;
if let Some(record) = sessions.get_mut(&id) {
record.session.worktree = None;
+112 -5
View File
@@ -18,6 +18,7 @@ use uuid::Uuid;
struct Fixture {
manager: SessionManager,
sidecar: SidecarStore,
_tmux: TmuxFixture,
_state_dir: TempDir,
}
@@ -25,15 +26,17 @@ struct Fixture {
async fn fixture(ttl: Duration) -> Fixture {
let tmux = TmuxFixture::new();
let state_dir = TempDir::new().unwrap();
let sidecar = SidecarStore::new(state_dir.path().join("sessions"));
let manager = SessionManager::new(
Arc::new(tmux.tmux.clone()),
SidecarStore::new(state_dir.path().join("sessions")),
sidecar.clone(),
WorktreeManager::new(None),
state_dir.path().join("fifos"),
)
.with_activity_report_ttl(ttl);
Fixture {
manager,
sidecar,
_tmux: tmux,
_state_dir: state_dir,
}
@@ -67,7 +70,7 @@ async fn report_on_unknown_session_is_not_found() {
let fx = fixture(Duration::from_secs(300)).await;
let err = fx
.manager
.report_activity(Uuid::new_v4(), ActivityState::Idle, None)
.report_activity(Uuid::new_v4(), ActivityState::Idle, None, None)
.await
.unwrap_err();
assert!(matches!(err, maestro_daemon::SessionError::NotFound(_)));
@@ -83,6 +86,7 @@ async fn report_sets_error_state_and_detail() {
id,
ActivityState::Error,
Some("cargo build failed: link error".into()),
None,
)
.await
.unwrap();
@@ -101,7 +105,7 @@ async fn replacing_a_report_replaces_both_state_and_detail() {
let id = spawn_sleeping(&fx.manager).await;
fx.manager
.report_activity(id, ActivityState::Error, Some("first failure".into()))
.report_activity(id, ActivityState::Error, Some("first failure".into()), None)
.await
.unwrap();
fx.manager
@@ -109,6 +113,7 @@ async fn replacing_a_report_replaces_both_state_and_detail() {
id,
ActivityState::AwaitingInput,
Some("needs approval".into()),
None,
)
.await
.unwrap();
@@ -125,7 +130,7 @@ async fn detail_is_capped_server_side() {
let long = "x".repeat(10_000);
fx.manager
.report_activity(id, ActivityState::Error, Some(long))
.report_activity(id, ActivityState::Error, Some(long), None)
.await
.unwrap();
@@ -144,7 +149,12 @@ async fn reported_state_survives_watcher_ticks_until_the_ttl_then_inference_resu
fx.manager.start_liveness_watcher(Duration::from_millis(50));
fx.manager
.report_activity(id, ActivityState::Error, Some("agent died mid-run".into()))
.report_activity(
id,
ActivityState::Error,
Some("agent died mid-run".into()),
None,
)
.await
.unwrap();
@@ -162,3 +172,100 @@ async fn reported_state_survives_watcher_ticks_until_the_ttl_then_inference_resu
})
.await;
}
#[tokio::test]
async fn a_reported_session_id_is_recorded_on_the_session() {
let fx = fixture(Duration::from_secs(300)).await;
let id = spawn_sleeping(&fx.manager).await;
fx.manager
.report_activity(id, ActivityState::Active, None, Some("conv-abc123".into()))
.await
.unwrap();
let s = observed(&fx.manager, id).await;
assert_eq!(s.agent_session_id.as_deref(), Some("conv-abc123"));
}
/// #161: repeated reports of the same id are pure churn from the hook's
/// `PreToolUse` cadence — the sidecar must not be rewritten for them.
#[tokio::test]
async fn a_repeated_report_of_the_same_session_id_writes_nothing() {
let fx = fixture(Duration::from_secs(300)).await;
let id = spawn_sleeping(&fx.manager).await;
fx.manager
.report_activity(id, ActivityState::Active, None, Some("conv-first".into()))
.await
.unwrap();
fx.manager
.report_activity(id, ActivityState::Active, None, Some("conv-first".into()))
.await
.unwrap();
let s = observed(&fx.manager, id).await;
assert_eq!(s.agent_session_id.as_deref(), Some("conv-first"));
}
/// #176: a resume issues a new conversation id. The stored id must move to
/// it, or every later resume reattaches the stale pre-resume conversation.
#[tokio::test]
async fn a_reported_session_id_is_replaced_when_it_differs() {
let fx = fixture(Duration::from_secs(300)).await;
let id = spawn_sleeping(&fx.manager).await;
fx.manager
.report_activity(id, ActivityState::Active, None, Some("conv-first".into()))
.await
.unwrap();
fx.manager
.report_activity(id, ActivityState::Active, None, Some("conv-second".into()))
.await
.unwrap();
let s = observed(&fx.manager, id).await;
assert_eq!(s.agent_session_id.as_deref(), Some("conv-second"));
let stored = fx
.sidecar
.read(id)
.expect("sidecar read")
.expect("sidecar present");
assert_eq!(stored.agent_session_id.as_deref(), Some("conv-second"));
}
#[tokio::test]
async fn a_reported_session_id_survives_recovery_from_the_sidecar() {
let fx = fixture(Duration::from_secs(300)).await;
let id = spawn_sleeping(&fx.manager).await;
fx.manager
.report_activity(id, ActivityState::Active, None, Some("conv-abc123".into()))
.await
.unwrap();
let stored = fx
.sidecar
.read(id)
.expect("sidecar read")
.expect("sidecar present");
assert_eq!(stored.agent_session_id.as_deref(), Some("conv-abc123"));
// The sidecar holding the id is only half the claim: a restarted daemon
// has to carry it back onto the recovered session, which is what a resume
// reads. Build a parallel manager over the same tmux server and state
// dirs and recover through it.
let revived = SessionManager::new(
Arc::new(fx._tmux.tmux.clone()),
fx.sidecar.clone(),
WorktreeManager::new(None),
fx._state_dir.path().join("fifos"),
);
revived.recover().await.expect("recover");
let recovered = observed(&revived, id).await;
assert_eq!(
recovered.agent_session_id.as_deref(),
Some("conv-abc123"),
"the recovered session must carry the agent session id, not just the sidecar"
);
}
+68
View File
@@ -94,6 +94,74 @@ async fn rejects_unknown_session() {
assert!(format!("{err}").contains("404"));
}
/// #166: a session the daemon knows but has no bridge for is not unknown, it
/// is not running. `404 unknown_session` made "gone" and "resumable" look the
/// same to a client that could have offered a resume.
#[tokio::test]
async fn dormant_session_is_not_attachable_rather_than_unknown() {
let h = spawn_daemon_ws().await;
let cwd = TempDir::new().unwrap();
let session = h
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.unwrap();
// The host reboot: tmux is gone, the sidecar is not, and a fresh manager
// registers the session dormant.
h._fx.tmux.tmux.kill_server().unwrap();
let dormant = SessionManager::new(
std::sync::Arc::new(h._fx.tmux.tmux.clone()),
maestro_daemon::sidecar::SidecarStore::new(h._fx.sidecar_dir()),
maestro_daemon::worktree::WorktreeManager::new(None),
h._fx.fifo_dir(),
);
dormant.recover().await.unwrap();
let dormant_addr = serve_manager(dormant).await;
let err = connect_attach(dormant_addr, session.id).await.unwrap_err();
let tokio_tungstenite::tungstenite::Error::Http(resp) = err else {
panic!("want an HTTP rejection, got {err}");
};
assert_eq!(resp.status().as_u16(), 409);
let body: maestro_protocol::ApiError =
serde_json::from_slice(resp.body().as_deref().expect("error body")).unwrap();
assert_eq!(body.error.code, "not_attachable");
// A session that really is unknown still reads as unknown.
let err = connect_attach(dormant_addr, Uuid::new_v4())
.await
.unwrap_err();
assert!(format!("{err}").contains("404"));
}
/// Serve `manager` on its own listener, for tests that need a second daemon
/// over the same tmux/sidecar state.
async fn serve_manager(manager: SessionManager) -> SocketAddr {
let router = local_api::router(local_api::state_for(
manager,
"test-host",
"/home/test",
Vec::new(),
std::sync::Arc::new(maestro_daemon::projects::ProjectScanner::disabled()),
maestro_config::DEFAULT_LEADER,
));
let listener = TcpListener::bind(SocketAddr::from((Ipv4Addr::LOCALHOST, 0)))
.await
.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
let _ = axum::serve(listener, router).await;
});
addr
}
#[tokio::test]
async fn pty_round_trip_through_real_tmux() {
let h = spawn_daemon_ws().await;
+27
View File
@@ -526,6 +526,33 @@ async fn post_activity_reports_error_with_detail() {
assert_eq!(s.activity_detail.as_deref(), Some("cargo build failed"));
}
/// #161: a reported `session_id` lands on the session as `agent_session_id`.
#[tokio::test]
async fn post_activity_records_the_reported_session_id() {
let h = spawn("naps-laptop").await;
let id = spawn_via_manager(&h).await;
let resp = reqwest::Client::new()
.post(url(h.addr, &format!("/api/sessions/{id}/activity")))
.json(&serde_json::json!({
"state": "Active",
"session_id": "conv-abc123"
}))
.send()
.await
.unwrap();
assert_eq!(resp.status(), reqwest::StatusCode::NO_CONTENT);
let sessions: Vec<Session> = reqwest::get(url(h.addr, "/api/sessions"))
.await
.unwrap()
.json()
.await
.unwrap();
let s = sessions.iter().find(|s| s.id == id).unwrap();
assert_eq!(s.agent_session_id.as_deref(), Some("conv-abc123"));
}
#[tokio::test]
async fn post_activity_to_unknown_session_returns_404() {
let h = spawn("naps-laptop").await;
+593
View File
@@ -0,0 +1,593 @@
//! End-to-end tests for `POST /api/sessions/:id/resume` (#166) and the
//! manager call behind it, against real tmux.
//!
//! The point of resume is that the session comes back under the *same* uuid
//! and sidecar — name, `created_at`, env and worktree ownership intact — with
//! only the tmux session and the PTY bridge new.
mod common;
use std::collections::HashMap;
use std::net::{Ipv4Addr, SocketAddr};
use std::sync::Arc;
use std::time::Duration;
use common::{wait_for_async, DaemonFixture};
use maestro_daemon::{local_api, SessionError, SessionManager};
use maestro_protocol::{CreateSessionRequest, Session, SessionStatus, SpawnPreset};
use tempfile::TempDir;
use tokio::net::TcpListener;
use tokio::sync::oneshot;
use uuid::Uuid;
/// A fresh manager over the same tmux server, sidecars and fifos — what the
/// daemon builds after a restart.
fn revive(fx: &DaemonFixture) -> SessionManager {
SessionManager::new(
Arc::new(fx.tmux.tmux.clone()),
maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir()),
maestro_daemon::worktree::WorktreeManager::new(None),
fx.fifo_dir(),
)
}
/// Create a session, then simulate the host reboot that leaves it dormant:
/// the tmux server dies, the sidecar stays, a new manager recovers.
async fn dormant_session(
fx: &DaemonFixture,
cwd: &TempDir,
command: &str,
name: &str,
) -> (SessionManager, Session) {
let mut metadata = HashMap::new();
metadata.insert(
maestro_protocol::NAME_KEY.to_string(),
serde_json::Value::String(name.into()),
);
let session = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: command.into(),
env: HashMap::from([("MAESTRO_TEST_ENV".to_string(), "kept".to_string())]),
metadata,
worktree: None,
})
.await
.expect("create");
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive(fx);
revived.recover().await.expect("recover");
(revived, session)
}
fn marker_preset(command: &str, marker_dir: &TempDir) -> Vec<SpawnPreset> {
vec![SpawnPreset {
id: "agent".into(),
label: "Agent".into(),
command: command.into(),
resume_command: Some(format!(
"sh -c 'touch {}/resumed-{{agent_session}}; sleep 30'",
marker_dir.path().display()
)),
}]
}
/// The keystone: a dormant session comes back running under its own id, with
/// its name, `created_at`, env and sidecar untouched.
#[tokio::test]
async fn resume_brings_a_dormant_session_back_under_the_same_id() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let (manager, original) = dormant_session(&fx, &cwd, "sleep 30", "long-runner").await;
assert_eq!(
manager.status(original.id).await,
Some(SessionStatus::Dormant)
);
let resumed = manager.resume(original.id, &[]).await.expect("resume");
assert_eq!(resumed.id, original.id, "same uuid");
assert_eq!(resumed.status, SessionStatus::Running);
assert_eq!(resumed.created_at, original.created_at);
assert_eq!(resumed.cwd, original.cwd);
assert_eq!(resumed.command, original.command);
assert_eq!(resumed.env, original.env);
assert_eq!(
resumed
.metadata
.get(maestro_protocol::NAME_KEY)
.and_then(|v| v.as_str()),
Some("long-runner")
);
// tmux really holds it again, and the sidecar is the same file, updated.
let live = fx.tmux.tmux.list_sessions().expect("list");
assert_eq!(live, vec![original.id]);
let stored = maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir())
.read(original.id)
.expect("read sidecar")
.expect("sidecar still there");
assert_eq!(stored.status, SessionStatus::Running);
assert_eq!(stored.created_at, original.created_at);
// And it is attachable again, which a dormant session never was.
assert!(manager
.subscribe_pty_with_replay(original.id)
.await
.is_some());
}
/// The worktree the session owns stays the session's, so killing the resumed
/// session still cleans the tree up.
#[tokio::test]
async fn resume_keeps_worktree_ownership() {
let fx = DaemonFixture::new();
let repo = TempDir::new().unwrap();
common::init_repo(repo.path());
let session = fx
.manager
.create(CreateSessionRequest {
cwd: repo.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: Some(maestro_protocol::WorktreeSpec {
base_repo: repo.path().to_path_buf(),
branch: "feature".into(),
create_branch: true,
}),
})
.await
.expect("create");
let worktree = session.worktree.clone().expect("worktree");
fx.tmux.tmux.kill_server().expect("kill tmux server");
let manager = revive(&fx);
manager.recover().await.expect("recover");
let resumed = manager.resume(session.id, &[]).await.expect("resume");
assert_eq!(resumed.worktree, Some(worktree.clone()));
assert_eq!(
resumed.cwd, worktree.path,
"resumed inside its own worktree"
);
}
/// The command comes from the preset's `resume_command`, with the session's
/// own agent conversation id (#161) substituted in.
#[tokio::test]
async fn resume_runs_the_presets_resume_command() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let markers = TempDir::new().unwrap();
let presets = marker_preset("sleep 30", &markers);
let (manager, session) = dormant_session(&fx, &cwd, "sleep 30", "agentic").await;
// The agent reported its conversation id before the reboot, so it is on
// the sidecar and the resume template can be filled in.
fx.manager
.report_activity(
session.id,
maestro_protocol::ActivityState::Idle,
None,
Some("conv-42".into()),
)
.await
.ok();
// Re-read the sidecar the report updated.
let manager = {
drop(manager);
let m = revive(&fx);
m.recover().await.expect("recover");
m
};
manager.resume(session.id, &presets).await.expect("resume");
let marker = markers.path().join("resumed-conv-42");
wait_for_async(Duration::from_secs(5), || {
let marker = marker.clone();
async move { marker.exists() }
})
.await;
}
/// A session that is already running has nothing to bring back, and
/// respawning would strand the pane that is already there.
#[tokio::test]
async fn resume_refuses_a_running_session() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let session = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
let err = fx
.manager
.resume(session.id, &[])
.await
.expect_err("already running");
assert!(matches!(err, SessionError::AlreadyRunning(id) if id == session.id));
assert_eq!(err.to_error_info().code, "already_running");
}
/// Nothing can be resumed in a directory that no longer exists.
#[tokio::test]
async fn resume_refuses_when_the_cwd_is_gone() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let (manager, session) = dormant_session(&fx, &cwd, "sleep 30", "homeless").await;
std::fs::remove_dir_all(cwd.path()).unwrap();
let err = manager
.resume(session.id, &[])
.await
.expect_err("cwd is gone");
assert_eq!(err.to_error_info().code, "cwd_missing");
assert_eq!(
manager.status(session.id).await,
Some(SessionStatus::Dormant),
"a refused resume leaves the session exactly as it was"
);
}
#[tokio::test]
async fn resume_of_an_unknown_session_is_not_found() {
let fx = DaemonFixture::new();
let err = fx
.manager
.resume(Uuid::new_v4(), &[])
.await
.expect_err("unknown");
assert_eq!(err.to_error_info().code, "not_found");
}
// ---------------------------------------------------------------------------
// HTTP surface
// ---------------------------------------------------------------------------
struct Harness {
addr: SocketAddr,
_shutdown: oneshot::Sender<()>,
}
async fn serve(manager: SessionManager, presets: Vec<SpawnPreset>) -> Harness {
let router = local_api::router(local_api::state_for(
manager,
"test-host",
"/home/test",
presets,
Arc::new(maestro_daemon::projects::ProjectScanner::disabled()),
maestro_config::DEFAULT_LEADER,
));
let listener = TcpListener::bind(SocketAddr::from((Ipv4Addr::LOCALHOST, 0)))
.await
.unwrap();
let addr = listener.local_addr().unwrap();
let (tx, rx) = oneshot::channel::<()>();
tokio::spawn(async move {
let _ = axum::serve(listener, router)
.with_graceful_shutdown(async move {
let _ = rx.await;
})
.await;
});
Harness {
addr,
_shutdown: tx,
}
}
#[tokio::test]
async fn post_resume_returns_the_revived_session() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let (manager, session) = dormant_session(&fx, &cwd, "sleep 30", "web-resumed").await;
let h = serve(manager, Vec::new()).await;
let resp = reqwest::Client::new()
.post(format!(
"http://{}{}",
h.addr,
maestro_protocol::routes::session_resume(session.id)
))
.send()
.await
.unwrap();
assert_eq!(resp.status(), reqwest::StatusCode::OK);
let body: Session = resp.json().await.unwrap();
assert_eq!(body.id, session.id);
assert_eq!(body.status, SessionStatus::Running);
}
#[tokio::test]
async fn post_resume_on_a_running_session_is_a_conflict() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let session = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
let h = serve(fx.manager.clone(), Vec::new()).await;
let resp = reqwest::Client::new()
.post(format!(
"http://{}{}",
h.addr,
maestro_protocol::routes::session_resume(session.id)
))
.send()
.await
.unwrap();
assert_eq!(resp.status(), reqwest::StatusCode::CONFLICT);
let err: maestro_protocol::ApiError = resp.json().await.unwrap();
assert_eq!(err.error.code, "already_running");
}
#[tokio::test]
async fn post_resume_on_an_unknown_session_is_404() {
let fx = DaemonFixture::new();
let h = serve(fx.manager.clone(), Vec::new()).await;
let resp = reqwest::Client::new()
.post(format!(
"http://{}{}",
h.addr,
maestro_protocol::routes::session_resume(Uuid::new_v4())
))
.send()
.await
.unwrap();
assert_eq!(resp.status(), reqwest::StatusCode::NOT_FOUND);
}
// ---------------------------------------------------------------------------
// Races (#175)
// ---------------------------------------------------------------------------
/// Real tmux, with the *first* `new_session_piped` held inside the call until
/// the test releases it. Later calls pass straight through, so a second resume
/// racing the first never blocks on a gate the first one owns.
///
/// This is what makes the two races below deterministic rather than timing
/// dependent: the resume under test is parked in exactly the window the bug
/// lives in, and the racing operation runs to completion before it moves.
#[derive(Debug)]
struct GatedSpawn {
inner: maestro_daemon::tmux::Tmux,
entered: tokio::sync::mpsc::UnboundedSender<()>,
release: std::sync::Mutex<std::sync::mpsc::Receiver<()>>,
gated: std::sync::atomic::AtomicBool,
}
/// The test's end of a [`GatedSpawn`].
struct Gate {
entered: tokio::sync::mpsc::UnboundedReceiver<()>,
release: std::sync::mpsc::SyncSender<()>,
}
impl Gate {
/// Wait until the gated spawn is parked mid-resume.
async fn wait_entered(&mut self) {
tokio::time::timeout(Duration::from_secs(10), self.entered.recv())
.await
.expect("resume reached the tmux spawn")
.expect("gate alive");
}
fn release(&self) {
self.release.send(()).expect("gate receiver alive");
}
}
fn gated_panes(inner: maestro_daemon::tmux::Tmux) -> (Arc<GatedSpawn>, Gate) {
let (entered_tx, entered_rx) = tokio::sync::mpsc::unbounded_channel();
let (release_tx, release_rx) = std::sync::mpsc::sync_channel(1);
(
Arc::new(GatedSpawn {
inner,
entered: entered_tx,
release: std::sync::Mutex::new(release_rx),
gated: std::sync::atomic::AtomicBool::new(true),
}),
Gate {
entered: entered_rx,
release: release_tx,
},
)
}
impl maestro_daemon::panes::Panes for GatedSpawn {
fn new_session_piped(
&self,
id: Uuid,
cwd: &std::path::Path,
command: &str,
session_env: &std::collections::BTreeMap<String, String>,
sink_command: &str,
) -> Result<(), maestro_daemon::tmux::TmuxError> {
if self.gated.swap(false, std::sync::atomic::Ordering::SeqCst) {
let _ = self.entered.send(());
let _ = self.release.lock().expect("gate mutex").recv();
}
self.inner
.new_session_piped(id, cwd, command, session_env, sink_command)
}
fn kill_session(&self, id: Uuid) -> Result<(), maestro_daemon::tmux::TmuxError> {
self.inner.kill_session(id)
}
fn list_sessions(&self) -> Result<Vec<Uuid>, maestro_daemon::tmux::TmuxError> {
self.inner.list_sessions()
}
fn pane_info(
&self,
id: Uuid,
) -> Result<Option<maestro_daemon::tmux::PaneInfo>, maestro_daemon::tmux::TmuxError> {
self.inner.pane_info(id)
}
fn send_keys(&self, id: Uuid, bytes: &[u8]) -> Result<(), maestro_daemon::tmux::TmuxError> {
self.inner.send_keys(id, bytes)
}
fn pipe_pane(&self, id: Uuid, sink: &str) -> Result<(), maestro_daemon::tmux::TmuxError> {
self.inner.pipe_pane(id, sink)
}
fn resize_window(
&self,
id: Uuid,
cols: u16,
rows: u16,
) -> Result<(), maestro_daemon::tmux::TmuxError> {
self.inner.resize_window(id, cols, rows)
}
fn capture_pane(&self, id: Uuid) -> Result<Option<Vec<u8>>, maestro_daemon::tmux::TmuxError> {
self.inner.capture_pane(id)
}
}
/// A dormant session whose manager's tmux spawn can be parked by the test.
async fn dormant_gated(fx: &DaemonFixture, cwd: &TempDir) -> (SessionManager, Session, Gate) {
let session = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
fx.tmux.tmux.kill_server().expect("kill tmux server");
let (panes, gate) = gated_panes(fx.tmux.tmux.clone());
let manager = SessionManager::new(
panes,
maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir()),
maestro_daemon::worktree::WorktreeManager::new(None),
fx.fifo_dir(),
);
manager.recover().await.expect("recover");
assert_eq!(
manager.status(session.id).await,
Some(SessionStatus::Dormant)
);
(manager, session, gate)
}
/// A kill that lands while a resume is inside the tmux spawn must not leave
/// the sidecar behind. The kill deletes it; the resume writes it back a moment
/// later and then discovers its own record is gone. The sidecar it resurrected
/// would otherwise make the next recovery register a ghost dormant session for
/// a session that was killed.
#[tokio::test]
async fn a_kill_during_resume_leaves_no_sidecar_behind() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let (manager, session, mut gate) = dormant_gated(&fx, &cwd).await;
let id = session.id;
let resuming = tokio::spawn({
let manager = manager.clone();
async move { manager.resume(id, &[]).await }
});
gate.wait_entered().await;
manager.kill(id, false).await.expect("kill mid-resume");
gate.release();
let err = resuming.await.unwrap().expect_err("record is gone");
assert!(matches!(err, SessionError::NotFound(got) if got == id));
assert!(
maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir())
.read(id)
.expect("read sidecar")
.is_none(),
"the killed session's sidecar must not be resurrected by the losing resume"
);
// And so the next recovery registers no ghost.
let after = revive(&fx);
after.recover().await.expect("recover");
assert!(
after.snapshot().await.is_empty(),
"a killed session must not come back dormant"
);
}
/// Two resumes of the same session must not both proceed. The second is
/// refused on the state — `already_running` — not on its tmux spawn colliding
/// with the first one's.
#[tokio::test]
async fn a_second_resume_is_refused_while_the_first_is_in_flight() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let (manager, session, mut gate) = dormant_gated(&fx, &cwd).await;
let id = session.id;
let resuming = tokio::spawn({
let manager = manager.clone();
async move { manager.resume(id, &[]).await }
});
gate.wait_entered().await;
let err = manager
.resume(id, &[])
.await
.expect_err("a resume is already in flight");
assert!(matches!(err, SessionError::AlreadyRunning(got) if got == id));
assert_eq!(err.to_error_info().code, "already_running");
gate.release();
let first = resuming.await.unwrap().expect("the first resume wins");
assert_eq!(first.status, SessionStatus::Running);
assert_eq!(
fx.tmux.tmux.list_sessions().expect("list"),
vec![id],
"exactly one pane for the session"
);
}
/// A refused resume must not leave the session marked in-flight: once the
/// reason for the refusal is gone, resuming works.
#[tokio::test]
async fn a_refused_resume_can_be_retried() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let (manager, session) = dormant_session(&fx, &cwd, "sleep 30", "retried").await;
std::fs::remove_dir_all(cwd.path()).unwrap();
let err = manager
.resume(session.id, &[])
.await
.expect_err("cwd is gone");
assert_eq!(err.to_error_info().code, "cwd_missing");
std::fs::create_dir_all(cwd.path()).unwrap();
let resumed = manager.resume(session.id, &[]).await.expect("resume");
assert_eq!(resumed.status, SessionStatus::Running);
}
@@ -0,0 +1,161 @@
//! `[daemon].resume_on_boot` (#167): what recovery does with sessions whose
//! tmux pane is gone but whose sidecar survived, against real tmux.
//!
//! The default is deliberately the timid one. Resuming agents unattended
//! seconds after a reboot is a foot-gun, so `auto` is opted into per host and
//! `never` throws the sidecars away instead.
mod common;
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use common::{wait_for_async, DaemonFixture};
use maestro_config::ResumeOnBoot;
use maestro_daemon::SessionManager;
use maestro_protocol::{CreateSessionRequest, Session, SessionStatus, SpawnPreset};
use tempfile::TempDir;
/// A fresh manager over the same tmux server, sidecars and fifos — what the
/// daemon builds after a restart — carrying `policy`.
fn revive(fx: &DaemonFixture, policy: ResumeOnBoot, presets: Vec<SpawnPreset>) -> SessionManager {
SessionManager::new(
Arc::new(fx.tmux.tmux.clone()),
maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir()),
maestro_daemon::worktree::WorktreeManager::new(None),
fx.fifo_dir(),
)
.with_resume_policy(policy, presets)
}
/// A preset whose `resume_command` touches a marker file, so a test can see
/// that the resumed session ran the resume command rather than the original.
fn marker_preset(command: &str, marker_dir: &TempDir) -> Vec<SpawnPreset> {
vec![SpawnPreset {
id: "agent".into(),
label: "Agent".into(),
command: command.into(),
resume_command: Some(format!(
"sh -c 'touch {}/resumed; sleep 30'",
marker_dir.path().display()
)),
}]
}
async fn create(fx: &DaemonFixture, cwd: &TempDir, command: &str) -> Session {
fx.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: command.into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create")
}
/// The default: a reboot leaves the session listed but not running, waiting
/// for the operator.
#[tokio::test]
async fn dormant_is_the_default_and_leaves_the_session_waiting() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let session = create(&fx, &cwd, "sleep 30").await;
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive(&fx, ResumeOnBoot::Dormant, Vec::new());
revived.recover().await.expect("recover");
assert_eq!(
revived.status(session.id).await,
Some(SessionStatus::Dormant)
);
assert!(fx
.sidecar_dir()
.join(format!("{}.json", session.id))
.exists());
assert!(
fx.tmux.tmux.list_sessions().expect("list").is_empty(),
"nothing was respawned"
);
}
/// `auto` brings each one back itself, down the same path a client's resume
/// takes: same uuid, same sidecar, the preset's `resume_command`.
#[tokio::test]
async fn auto_resumes_every_dormant_session_during_recovery() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let markers = TempDir::new().unwrap();
let session = create(&fx, &cwd, "sleep 30").await;
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive(&fx, ResumeOnBoot::Auto, marker_preset("sleep 30", &markers));
revived.recover().await.expect("recover");
assert_eq!(
revived.status(session.id).await,
Some(SessionStatus::Running),
"auto resumed it"
);
assert_eq!(
fx.tmux.tmux.list_sessions().expect("list"),
vec![session.id],
"back under the same uuid"
);
let stored = maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir())
.read(session.id)
.expect("read sidecar")
.expect("same sidecar");
assert_eq!(stored.created_at, session.created_at);
assert_eq!(stored.command, "sleep 30", "the original command is kept");
// It really ran the preset's resume command, not the spawn command.
let marker = markers.path().join("resumed");
wait_for_async(Duration::from_secs(5), || {
let marker = marker.clone();
async move { marker.exists() }
})
.await;
assert!(marker.exists(), "resume_command ran");
}
/// `never` makes a reboot a clean slate: the sidecars go, and the session is
/// not listed at all.
#[tokio::test]
async fn never_prunes_the_sidecars_instead() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let session = create(&fx, &cwd, "sleep 30").await;
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive(&fx, ResumeOnBoot::Never, Vec::new());
revived.recover().await.expect("recover");
assert!(!revived.has(session.id).await, "not listed");
assert!(
!fx.sidecar_dir()
.join(format!("{}.json", session.id))
.exists(),
"sidecar pruned"
);
assert!(fx.tmux.tmux.list_sessions().expect("list").is_empty());
}
/// The policy only decides what happens to sessions that lost their pane. A
/// session tmux still holds is adopted running by the first pass under every
/// policy — including `never`, which must not delete a live session's sidecar.
#[tokio::test]
async fn the_policy_leaves_sessions_tmux_still_holds_alone() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let live = create(&fx, &cwd, "sleep 30").await;
let revived = revive(&fx, ResumeOnBoot::Never, Vec::new());
assert_eq!(revived.recover().await.expect("recover"), 1);
assert_eq!(revived.status(live.id).await, Some(SessionStatus::Running));
assert!(fx.sidecar_dir().join(format!("{}.json", live.id)).exists());
}
@@ -9,6 +9,7 @@ use std::sync::Arc;
use std::time::Duration;
use common::{init_bare_repo, init_repo, push_all_to_origin, wait_for_async, DaemonFixture};
use maestro_daemon::sidecar::SidecarStore;
use maestro_daemon::SessionError;
use maestro_protocol::CreateSessionRequest;
use maestro_protocol::{CatalogEvent, SessionStatus, WorktreeSpec};
@@ -140,6 +141,390 @@ async fn recover_adopts_existing_tmux_sessions() {
assert_eq!(snap[0].command, "sleep 30");
}
/// #162: a host reboot kills the tmux server, so recovery finds nothing to
/// adopt while every sidecar is still on disk. Those sessions come back as
/// `Dormant` — a record of what was running — rather than vanishing from the
/// API and leaving the operator with nothing.
#[tokio::test]
async fn recover_registers_orphaned_sidecars_as_dormant() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let mut metadata = HashMap::new();
metadata.insert(
maestro_protocol::NAME_KEY.to_string(),
serde_json::Value::String("after-reboot".into()),
);
let s = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata,
worktree: None,
})
.await
.expect("create");
// The reboot: the tmux server is gone, the sidecar is untouched.
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive(&fx);
let adopted = revived.recover().await.expect("recover");
assert_eq!(adopted, 0, "no live tmux session left to adopt");
let snap = revived.snapshot().await;
assert_eq!(snap.len(), 1, "the sidecar's session is still listed");
assert_eq!(snap[0].id, s.id);
assert_eq!(snap[0].status, SessionStatus::Dormant);
assert_eq!(snap[0].command, "sleep 30");
assert_eq!(snap[0].cwd, s.cwd);
assert_eq!(snap[0].created_at, s.created_at);
assert_eq!(
snap[0]
.metadata
.get(maestro_protocol::NAME_KEY)
.and_then(|v| v.as_str()),
Some("after-reboot"),
);
}
/// #162: a session tmux still holds is adopted `Running` by the first pass and
/// must not be re-registered as dormant by the second — the sidecar is on disk
/// for both.
#[tokio::test]
async fn recover_does_not_mark_adopted_sessions_dormant() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let live = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create live");
let orphan = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 31".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create orphan");
// Only the second session loses its pane.
fx.tmux.tmux.kill_session(orphan.id).expect("kill session");
let revived = revive(&fx);
assert_eq!(revived.recover().await.expect("recover"), 1);
let mut snap = revived.snapshot().await;
snap.sort_by_key(|s| s.command.clone());
assert_eq!(snap.len(), 2);
assert_eq!(snap[0].id, live.id);
assert_eq!(snap[0].status, SessionStatus::Running);
assert_eq!(snap[1].id, orphan.id);
assert_eq!(snap[1].status, SessionStatus::Dormant);
}
/// #163: a sidecar whose `cwd` no longer exists is pruned instead of
/// registered dormant — the worktree it pointed at is gone, so there is
/// nothing left to resume.
#[tokio::test]
async fn recover_deletes_sidecar_when_cwd_gone() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let cwd_path = cwd.path().to_path_buf();
let s = fx
.manager
.create(CreateSessionRequest {
cwd: cwd_path.clone(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
fx.tmux.tmux.kill_server().expect("kill tmux server");
drop(cwd);
std::fs::remove_dir_all(&cwd_path).ok();
assert!(!cwd_path.exists());
let revived = revive(&fx);
let adopted = revived.recover().await.expect("recover");
assert_eq!(adopted, 0);
assert!(!revived.has(s.id).await);
assert!(!fx.sidecar_dir().join(format!("{}.json", s.id)).exists());
}
/// #163: a sidecar whose `cwd` still exists is left alone regardless of age
/// — it is registered dormant like any other orphan.
#[tokio::test]
async fn recover_keeps_dormant_sidecar_when_cwd_exists() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let s = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive(&fx);
revived.recover().await.expect("recover");
assert!(revived.has(s.id).await);
assert!(fx.sidecar_dir().join(format!("{}.json", s.id)).exists());
let snap = revived.snapshot().await;
assert_eq!(snap[0].status, SessionStatus::Dormant);
}
/// #164: a fake [`maestro_daemon::boot_id::BootIdSource`] lets tests simulate
/// a changed (or unreadable) boot id without rebooting the machine.
#[derive(Debug)]
struct FakeBootId(Option<String>);
impl maestro_daemon::boot_id::BootIdSource for FakeBootId {
fn current(&self) -> Option<String> {
self.0.clone()
}
}
fn revive_with_boot_id(
fx: &DaemonFixture,
boot_id_path: PathBuf,
current: Option<&str>,
) -> maestro_daemon::SessionManager {
maestro_daemon::SessionManager::new(
Arc::new(fx.tmux.tmux.clone()),
maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir()),
maestro_daemon::worktree::WorktreeManager::new(None),
fx.fifo_dir(),
)
.with_boot_id(
boot_id_path,
Arc::new(FakeBootId(current.map(String::from))),
)
}
/// #174: a boot id can change without the tmux server dying — a restored
/// state directory, a VM or container snapshot restore, a truncated
/// `boot_id` file. `tmux list-sessions` stays the single source of truth for
/// liveness, so a session tmux still holds is adopted `Running` no matter
/// what the boot id says. Registering it dormant instead stranded it: resume
/// would collide with the tmux name still in use and fail forever.
#[tokio::test]
async fn recover_adopts_live_session_when_boot_id_changed() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let s = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
let boot_id_path = fx.state_dir.path().join("boot_id");
std::fs::write(&boot_id_path, "previous-boot").unwrap();
// tmux session is still alive: the probe runs and finds it.
let revived = revive_with_boot_id(&fx, boot_id_path.clone(), Some("current-boot"));
let adopted = revived.recover().await.expect("recover");
assert_eq!(adopted, 1, "tmux still holds the session; adopt it");
let snap = revived.snapshot().await;
assert_eq!(snap.len(), 1);
assert_eq!(snap[0].id, s.id);
assert_eq!(snap[0].status, SessionStatus::Running);
// The current boot id is now the stored one, for the next comparison.
assert_eq!(
std::fs::read_to_string(&boot_id_path).unwrap().trim(),
"current-boot"
);
}
/// #164, still true after #174: a real reboot takes the tmux server with it,
/// so the probe comes back empty and every sidecar is registered dormant.
#[tokio::test]
async fn recover_registers_dormant_when_boot_id_changed_and_tmux_gone() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let s = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
let boot_id_path = fx.state_dir.path().join("boot_id");
std::fs::write(&boot_id_path, "previous-boot").unwrap();
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive_with_boot_id(&fx, boot_id_path, Some("current-boot"));
let adopted = revived.recover().await.expect("recover");
assert_eq!(adopted, 0, "no tmux server survived the reboot");
let snap = revived.snapshot().await;
assert_eq!(snap.len(), 1);
assert_eq!(snap[0].id, s.id);
assert_eq!(snap[0].status, SessionStatus::Dormant);
}
/// #164: an unchanged boot id is a plain restart — recovery probes tmux as
/// before and adopts the still-live session `Running`.
#[tokio::test]
async fn recover_probes_tmux_when_boot_id_unchanged() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let s = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
let boot_id_path = fx.state_dir.path().join("boot_id");
std::fs::write(&boot_id_path, "same-boot").unwrap();
let revived = revive_with_boot_id(&fx, boot_id_path, Some("same-boot"));
let adopted = revived.recover().await.expect("recover");
assert_eq!(adopted, 1);
let snap = revived.snapshot().await;
assert_eq!(snap[0].id, s.id);
assert_eq!(snap[0].status, SessionStatus::Running);
}
/// #164: no stored boot id (first run ever, or an older build that never
/// wrote one) means there is nothing to compare against — falls back to
/// today's probe-based recovery rather than assuming a reboot.
#[tokio::test]
async fn recover_falls_back_to_probe_when_no_stored_boot_id() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
fx.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
let boot_id_path = fx.state_dir.path().join("boot_id");
assert!(!boot_id_path.exists());
let revived = revive_with_boot_id(&fx, boot_id_path.clone(), Some("current-boot"));
let adopted = revived.recover().await.expect("recover");
assert_eq!(adopted, 1, "no stored boot id; falls back to probing tmux");
assert_eq!(
std::fs::read_to_string(&boot_id_path).unwrap().trim(),
"current-boot"
);
}
/// #164: an unreadable current boot id (non-Linux, restricted `/proc`) falls
/// back to today's behaviour rather than failing startup or assuming a
/// reboot.
#[tokio::test]
async fn recover_falls_back_to_probe_when_boot_id_unreadable() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
fx.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
let boot_id_path = fx.state_dir.path().join("boot_id");
std::fs::write(&boot_id_path, "stored-boot").unwrap();
let revived = revive_with_boot_id(&fx, boot_id_path, None);
let adopted = revived.recover().await.expect("recover");
assert_eq!(adopted, 1, "boot id unreadable; falls back to probing tmux");
}
/// #162: dismissing a dormant session is the one thing that can be done to it
/// — no tmux call to fail, and the sidecar goes with it so the next recovery
/// does not resurrect it.
#[tokio::test]
async fn dormant_session_can_be_dismissed() {
let fx = DaemonFixture::new();
let cwd = TempDir::new().unwrap();
let s = fx
.manager
.create(CreateSessionRequest {
cwd: cwd.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: None,
})
.await
.expect("create");
fx.tmux.tmux.kill_server().expect("kill tmux server");
let revived = revive(&fx);
revived.recover().await.expect("recover");
revived.kill(s.id, false).await.expect("dismiss");
assert!(!revived.has(s.id).await);
assert!(!fx.sidecar_dir().join(format!("{}.json", s.id)).exists());
}
/// A second `SessionManager` over the same dirs and the same tmux server —
/// what a daemon restart looks like from inside a test.
fn revive(fx: &DaemonFixture) -> maestro_daemon::SessionManager {
maestro_daemon::SessionManager::new(
Arc::new(fx.tmux.tmux.clone()),
maestro_daemon::sidecar::SidecarStore::new(fx.sidecar_dir()),
maestro_daemon::worktree::WorktreeManager::new(None),
fx.fifo_dir(),
)
}
/// Regression for NAP-253: the very first bytes a freshly-spawned command
/// writes must reach the replay buffer. Before the fix, `pipe-pane` was a
/// separate tmux invocation that ran after `new-session` + an async FIFO
@@ -954,6 +1339,65 @@ async fn natural_exit_of_a_pristine_worktree_removes_it() {
);
}
/// #172: `mark_session_dead` clears the worktree off the in-memory record
/// when a pristine tree settles, but the local `session` clone it writes to
/// the sidecar and broadcasts must be cleared the same way — otherwise the
/// sidecar keeps naming a tree the daemon just removed, and a recovery from
/// that sidecar advertises a branch and path that no longer exist.
#[tokio::test]
async fn natural_exit_of_a_pristine_worktree_clears_it_from_the_sidecar() {
let fx = DaemonFixture::new();
let repo = TempDir::new().unwrap();
init_repo(repo.path());
let wt_path = repo.path().join("worktrees").join("feat").join("sidecar");
let s = fx
.manager
.create(CreateSessionRequest {
cwd: repo.path().to_path_buf(),
command: "sleep 30".into(),
env: HashMap::new(),
metadata: HashMap::new(),
worktree: Some(WorktreeSpec {
base_repo: repo.path().to_path_buf(),
branch: "feat/sidecar".into(),
create_branch: true,
}),
})
.await
.expect("create");
// Push everything so the branch has no unpushed commits: fully pristine.
let origin = TempDir::new().unwrap();
init_bare_repo(origin.path());
push_all_to_origin(repo.path(), origin.path());
fx.tmux.tmux.kill_session(s.id).expect("kill tmux directly");
fx.manager.start_liveness_watcher(Duration::from_millis(50));
wait_for_async(Duration::from_secs(3), || async {
fx.manager.status(s.id).await == Some(SessionStatus::Dead)
})
.await;
wait_for_async(Duration::from_secs(3), || async {
fx.manager.is_settled(s.id).await == Some(true)
})
.await;
assert!(
!wt_path.exists(),
"a pristine worktree leaves nothing behind"
);
let sidecar = SidecarStore::new(fx.sidecar_dir())
.read(s.id)
.expect("sidecar read")
.expect("sidecar exists");
assert!(
sidecar.worktree.is_none(),
"the sidecar must not keep naming a worktree the daemon already removed"
);
}
fn run_git_in(dir: &std::path::Path, args: &[&str]) {
let out = Command::new("git")
.arg("-C")
+1
View File
@@ -32,6 +32,7 @@ fn sample(id: Uuid) -> Session {
current_cwd: Some(PathBuf::from("/tmp/work/crates")),
blocked_work: None,
last_focused_at: None,
agent_session_id: Some("conv-abc123".into()),
}
}
+10 -2
View File
@@ -84,15 +84,23 @@ if [ "$state" = "Prompt" ]; then
>/dev/null 2>&1 || true
exit 0
fi
body="{\"state\":\"$state\"}"
fields="\"state\":\"$state\""
# StopFailure payloads carry the outcome in "error". Only a plain enum
# spelling is reported as the detail — anything else (quotes, backslashes,
# prose) would risk breaking the JSON body, and the spelling already names
# the failure.
detail=$(printf '%s' "$payload" | sed -n 's/.*"error"[[:space:]]*:[[:space:]]*"\([a-z_]*\)".*/\1/p')
if [ -n "$detail" ]; then
body="{\"state\":\"$state\",\"detail\":\"$detail\"}"
fields="$fields,\"detail\":\"$detail\""
fi
# Every Claude Code hook payload carries its own conversation id (#161).
# Constrained to a safe charset for the same reason as "detail" above: it is
# spliced into the JSON body unescaped.
session_id=$(printf '%s' "$payload" | sed -n 's/.*"session_id"[[:space:]]*:[[:space:]]*"\([A-Za-z0-9_-]*\)".*/\1/p')
if [ -n "$session_id" ]; then
fields="$fields,\"session_id\":\"$session_id\""
fi
body="{$fields}"
curl -fsS --connect-timeout 1 --max-time 2 \
-H 'Content-Type: application/json' \
-d "$body" \
+35
View File
@@ -128,6 +128,41 @@ fn script_reports_the_error_spelling_as_detail() {
assert!(request.contains(r#""detail":"rate_limit""#), "{request}");
}
/// #161: the payload's own `session_id` — the agent's conversation id — is
/// parsed out and forwarded in the activity body, so the daemon can persist
/// it on the session.
#[test]
fn script_forwards_the_payloads_session_id_in_the_activity_body() {
let request = capture_request(|url| {
sh(
HOOK_SCRIPT,
&["Active"],
&owned_envs(&session_envs(&url)),
br#"{"session_id":"conv-abc123","hook_event_name":"PreToolUse"}"#,
)
});
assert!(request.contains(r#""state":"Active""#), "{request}");
assert!(
request.contains(r#""session_id":"conv-abc123""#),
"{request}"
);
}
/// A payload without a `session_id` (or one with an unexpected shape) must
/// never produce a broken body — no `session_id` field is posted.
#[test]
fn script_posts_no_session_id_when_the_payload_has_none() {
let request = capture_request(|url| {
sh(
HOOK_SCRIPT,
&["AwaitingInput"],
&owned_envs(&session_envs(&url)),
br#"{"hook_event_name":"Stop","stop_hook_active":false}"#,
)
});
assert!(!request.contains("session_id"), "{request}");
}
/// An `error` value that is not a plain enum spelling (or a payload without
/// one at all) must never produce a broken body — no detail is posted.
#[test]
+1
View File
@@ -284,6 +284,7 @@ mod tests {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
}
}
+137 -1
View File
@@ -116,6 +116,13 @@ pub struct Session {
/// sorts a never-focused session after every session that has one.
#[serde(default)]
pub last_focused_at: Option<DateTime<Utc>>,
/// The conversation id the agent running in this session reported for
/// itself (#161), parsed from a hook's `session_id` field. Written once,
/// from the first activity report that carries one, and left alone after
/// — an agent's own session id does not change mid-session. `None` for
/// sessions that never run an agent (plain shells) or predate the field.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub agent_session_id: Option<String>,
}
/// Default for [`Session::last_activity_at`] when an older payload omits it.
@@ -123,10 +130,38 @@ fn epoch() -> DateTime<Utc> {
DateTime::from_timestamp(0, 0).expect("unix epoch is a valid timestamp")
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
/// Lifecycle of a session as the daemon knows it.
///
/// Deserializes leniently, like [`ActivityState`] (#162): an unknown spelling
/// from a newer daemon reads as [`SessionStatus::Dead`] instead of failing the
/// whole `Session`. `Dead` is the safe degrade — an unrecognised session is
/// one this client must not treat as having a live pane to type into.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
pub enum SessionStatus {
Running,
Dead,
/// Known from its sidecar, with no tmux session behind it (#162). The
/// daemon registers these at recovery time for every sidecar tmux had
/// nothing to adopt — after a host reboot, that is every session there
/// was. There is no pane and no PTY bridge: a dormant session is a record
/// of what was running, listed so the operator still has it, and the only
/// thing that can be done to it is dismissing it.
Dormant,
}
impl<'de> Deserialize<'de> for SessionStatus {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
let s = String::deserialize(deserializer)?;
Ok(match s.as_str() {
"Running" => Self::Running,
"Dormant" => Self::Dormant,
// Known or not, any other spelling degrades to the inert variant.
_ => Self::Dead,
})
}
}
/// Inferred session activity, orthogonal to [`SessionStatus`] (NAP-340). The
@@ -184,6 +219,11 @@ pub struct ReportActivityRequest {
/// The daemon caps its length server-side.
#[serde(default)]
pub detail: Option<String>,
/// The agent's own conversation id (#161), forwarded from the hook
/// payload's `session_id` field. `None` for hook events that don't carry
/// one, or hooks predating this field.
#[serde(default)]
pub session_id: Option<String>,
}
/// Reject an unknown [`ActivityState`] spelling instead of degrading it to
@@ -489,8 +529,34 @@ impl ApiError {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct SpawnPreset {
pub id: String,
#[serde(default)]
pub label: String,
pub command: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub resume_command: Option<String>,
}
/// Placeholder a preset's `resume_command` may carry for the agent's own
/// conversation id (#165), substituted from [`Session::agent_session_id`].
pub const AGENT_SESSION_PLACEHOLDER: &str = "{agent_session}";
impl SpawnPreset {
/// The command that resumes a session spawned from this preset, with
/// [`AGENT_SESSION_PLACEHOLDER`] filled in from `agent_session`.
///
/// `None` when the preset has no `resume_command` — a plain shell has
/// nothing to resume — and also when the template asks for an agent
/// session id the session never reported: `claude --resume` with an empty
/// id resumes nothing, so the caller is better off respawning the original
/// command than running a broken one.
pub fn resume_command_for(&self, agent_session: Option<&str>) -> Option<String> {
let template = self.resume_command.as_deref()?;
if !template.contains(AGENT_SESSION_PLACEHOLDER) {
return Some(template.to_string());
}
let id = agent_session.filter(|s| !s.is_empty())?;
Some(template.replace(AGENT_SESSION_PLACEHOLDER, id))
}
}
/// Build provenance of the running daemon: the commit it was compiled
@@ -742,6 +808,47 @@ pub enum ClientControl {
mod tests {
use super::*;
fn preset(resume: Option<&str>) -> SpawnPreset {
SpawnPreset {
id: "claude".into(),
label: "Claude".into(),
command: "claude".into(),
resume_command: resume.map(str::to_string),
}
}
#[test]
fn resume_command_substitutes_the_agent_session() {
let p = preset(Some("claude --resume {agent_session}"));
assert_eq!(
p.resume_command_for(Some("abc-123")).as_deref(),
Some("claude --resume abc-123")
);
}
#[test]
fn resume_command_without_a_placeholder_ignores_the_agent_session() {
let p = preset(Some("opencode --continue"));
assert_eq!(
p.resume_command_for(None).as_deref(),
Some("opencode --continue")
);
}
/// A template that wants an id the session never reported would run
/// `claude --resume` against nothing; the caller respawns instead.
#[test]
fn resume_command_is_none_without_the_id_it_asks_for() {
let p = preset(Some("claude --resume {agent_session}"));
assert!(p.resume_command_for(None).is_none());
assert!(p.resume_command_for(Some("")).is_none());
}
#[test]
fn a_preset_without_a_resume_command_has_none() {
assert!(preset(None).resume_command_for(Some("abc")).is_none());
}
#[test]
fn version_compatibility_is_major_only() {
assert!(is_compatible(PROTOCOL_VERSION));
@@ -862,6 +969,7 @@ mod tests {
current_cwd: Some(PathBuf::from("/home/naps62/projects/maestro/crates")),
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
}
}
@@ -1031,6 +1139,7 @@ mod tests {
let full = ReportActivityRequest {
state: ActivityState::AwaitingInput,
detail: Some("waiting on permission".into()),
session_id: Some("conv-abc123".into()),
};
assert_eq!(full, round_trip(&full));
@@ -1059,6 +1168,33 @@ mod tests {
serde_json::to_value(SessionStatus::Dead).unwrap(),
serde_json::json!("Dead")
);
assert_eq!(
serde_json::to_value(SessionStatus::Dormant).unwrap(),
serde_json::json!("Dormant")
);
}
// #162: a session with no tmux behind it still rides the wire whole.
#[test]
fn dormant_session_round_trips() {
let mut s = sample_session();
s.status = SessionStatus::Dormant;
assert_eq!(s, round_trip(&s));
}
// #162: a `SessionStatus` this client does not know must degrade to the
// inert variant, not fail the whole `Session` the way a derived
// `Deserialize` would.
#[test]
fn unknown_session_status_degrades_to_dead() {
assert_eq!(
serde_json::from_value::<SessionStatus>(serde_json::json!("Hibernating")).unwrap(),
SessionStatus::Dead
);
let mut value = serde_json::to_value(sample_session()).unwrap();
value["status"] = serde_json::json!("Hibernating");
let s: Session = serde_json::from_value(value).expect("session survives unknown status");
assert_eq!(s.status, SessionStatus::Dead);
}
#[test]
+15
View File
@@ -66,6 +66,12 @@ pub fn session_focus(id: Uuid) -> String {
format!("{SESSIONS}/{id}/focus")
}
/// `POST /api/sessions/:id/resume` (#166): respawn a dormant session under its
/// own id, keeping its sidecar, name, worktree and `created_at`.
pub fn session_resume(id: Uuid) -> String {
format!("{SESSIONS}/{id}/resume")
}
/// `WS /api/attach/:id` — raw PTY bytes.
pub fn attach(id: Uuid) -> String {
format!("{API_PREFIX}/attach/{id}")
@@ -79,6 +85,7 @@ pub mod patterns {
pub const SESSION_INPUT: &str = "/api/sessions/:id/input";
pub const SESSION_PROMPT: &str = "/api/sessions/:id/prompt";
pub const SESSION_FOCUS: &str = "/api/sessions/:id/focus";
pub const SESSION_RESUME: &str = "/api/sessions/:id/resume";
pub const ATTACH: &str = "/api/attach/:id";
}
@@ -202,6 +209,10 @@ mod tests {
session_focus(id()),
"/api/sessions/6f1a7c0e-0f5e-4a2b-9a3d-6b1c2d3e4f50/focus"
);
assert_eq!(
session_resume(id()),
"/api/sessions/6f1a7c0e-0f5e-4a2b-9a3d-6b1c2d3e4f50/resume"
);
assert_eq!(
attach(id()),
"/api/attach/6f1a7c0e-0f5e-4a2b-9a3d-6b1c2d3e4f50"
@@ -236,6 +247,10 @@ mod tests {
patterns::SESSION_FOCUS.trim_end_matches(":id/focus"),
session(id()).trim_end_matches(&id().to_string())
);
assert_eq!(
patterns::SESSION_RESUME.trim_end_matches(":id/resume"),
session(id()).trim_end_matches(&id().to_string())
);
assert_eq!(
patterns::ATTACH.trim_end_matches(":id"),
attach(id()).trim_end_matches(&id().to_string())
+1
View File
@@ -47,6 +47,7 @@ pub fn session(id: Uuid, cwd: &str, command: &str) -> Session {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
}
}
+6
View File
@@ -1898,6 +1898,7 @@ mod tests {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
})
.collect(),
});
@@ -2079,6 +2080,7 @@ mod tests {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
}],
});
app.dispatch(Action::SidebarNext);
@@ -2106,6 +2108,7 @@ mod tests {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
}
}
@@ -3214,6 +3217,7 @@ mod tests {
current_cwd: None,
blocked_work: None,
last_focused_at: focused,
agent_session_id: None,
};
let older = Utc::now() - chrono::Duration::seconds(60);
app.catalog.apply_event(CatalogEvent::Snapshot {
@@ -3492,6 +3496,7 @@ mod tests {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
}],
});
app.dispatch(Action::KillHighlighted);
@@ -3527,6 +3532,7 @@ mod tests {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
}],
});
}
+1
View File
@@ -223,6 +223,7 @@ mod tests {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
}
}
+22 -2
View File
@@ -194,7 +194,9 @@ pub enum BarOutcome {
pub fn order_sessions_for_bar(sessions: &[Session]) -> Vec<&Session> {
let mut ordered: Vec<&Session> = sessions.iter().collect();
ordered.sort_by(|a, b| {
let dead = |s: &Session| u8::from(s.status == SessionStatus::Dead);
// docs/COMMAND-BAR.md: anything with no live pane — dead or dormant
// (#162) — sorts after the sessions you could actually attach to.
let dead = |s: &Session| u8::from(s.status != SessionStatus::Running);
dead(a)
.cmp(&dead(b))
.then_with(|| {
@@ -319,7 +321,7 @@ impl CommandBar {
.map(|s| BarEntry {
candidate: MatchCandidate::Session(session_candidate(s, home_dir)),
target: BarTarget::Session(s.id),
dead: s.status == SessionStatus::Dead,
dead: s.status != SessionStatus::Running,
chords: Vec::new(),
})
.collect();
@@ -809,6 +811,7 @@ mod tests {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
}
}
@@ -889,6 +892,23 @@ mod tests {
assert!(bar.rows()[1].dead);
}
/// #162: docs/COMMAND-BAR.md ranks by whether you could attach, not by
/// which way the pane went away — a dormant session sorts with the dead.
#[test]
fn dormant_sessions_sort_after_live_ones_however_recent() {
let mut dormant = named(1, "dormant");
dormant.status = SessionStatus::Dormant;
dormant.last_focused_at = Some(at(99));
let live = named(2, "live");
let bar = CommandBar::new(&[dormant, live], &[], "/home/me");
assert_eq!(
ids(bar.rows()),
vec![Uuid::from_u128(2), Uuid::from_u128(1)]
);
assert!(bar.rows()[1].dead);
}
#[test]
fn typing_ranks_through_the_matcher_and_returns_ranges() {
let mut bar = CommandBar::new(
+4
View File
@@ -1705,6 +1705,7 @@ mod tests {
id: "aider".into(),
label: "Aider".into(),
command: "aider".into(),
resume_command: None,
}]);
assert_eq!(
draft.command(),
@@ -1727,11 +1728,13 @@ mod tests {
id: "claude".into(),
label: "Claude".into(),
command: "claude".into(),
resume_command: None,
},
SpawnPreset {
id: "aider".into(),
label: "Aider".into(),
command: "aider".into(),
resume_command: None,
},
]);
assert_eq!(draft.command(), "aider");
@@ -1751,6 +1754,7 @@ mod tests {
id: "claude".into(),
label: "Claude".into(),
command: "claude".into(),
resume_command: None,
}]);
assert!(draft.is_custom());
assert_eq!(draft.command(), "bash -l");
+1
View File
@@ -183,6 +183,7 @@ mod tests {
id: id.into(),
label: id.into(),
command: command.into(),
resume_command: None,
}
}
+41
View File
@@ -268,6 +268,8 @@ impl Sidebar {
fn name_row(&self, s: &Session, width: usize, focused: bool) -> Vec<Span<'static>> {
let glyph = match (s.status, self.attached.contains(&s.id)) {
(SessionStatus::Dead, _) => "✗",
// #162: known from its sidecar, nothing running behind it.
(SessionStatus::Dormant, _) => "◌",
(SessionStatus::Running, true) => "●",
(SessionStatus::Running, false) => "○",
};
@@ -322,6 +324,7 @@ impl Sidebar {
// on line 1; the kill confirm names the specifics.
let blocked = s.blocked_work.filter(|b| !b.is_clear());
let dead = s.status == SessionStatus::Dead;
let dormant = s.status == SessionStatus::Dormant;
let highlighted = self.highlight == Some(SidebarItem::Session(s.id));
let fill = self.card_fill(s, focused);
let base = card_base(fill, focused);
@@ -361,6 +364,9 @@ impl Sidebar {
if dead {
tail.push(Span::styled(" exited", muted));
}
if dormant {
tail.push(Span::styled(" dormant", muted));
}
// docs/SIDEBAR.md: the card names the program, never its arguments.
let command = Span::styled(format!(" {}", command_program(&s.command)), muted);
@@ -500,6 +506,7 @@ mod tests {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
}
}
@@ -1068,6 +1075,39 @@ mod tests {
);
}
/// #162, docs/SIDEBAR.md: a dormant session is known but not running —
/// its own glyph on line 1 and the muted `dormant` fragment on line 2, so
/// it never reads as something that exited.
#[test]
fn dormant_session_reads_as_known_but_not_running() {
let mut s = session_named(1, "claude");
s.status = SessionStatus::Dormant;
s.activity_since = Utc::now() - Duration::seconds(5);
let sb = Sidebar {
sessions: vec![s],
highlight: None,
attached: Default::default(),
pane_session: None,
};
let all = rows(&draw(&sb, 28, 8));
let idx = all.iter().position(|l| l.contains("claude")).unwrap();
assert!(
all[idx].contains('\u{25cc}'),
"expected the dormant glyph in: {:?}",
all[idx]
);
assert!(
all[idx + 1].contains("dormant"),
"expected dormant fragment in: {:?}",
all[idx + 1]
);
assert!(
!all[idx + 1].contains("exited"),
"a dormant session never reads as exited: {:?}",
all[idx + 1]
);
}
/// docs/SIDEBAR.md: line 2 names the program, never its arguments.
#[test]
fn meta_line_drops_the_commands_arguments() {
@@ -1266,6 +1306,7 @@ mod tests {
current_cwd: None,
blocked_work: None,
last_focused_at: None,
agent_session_id: None,
};
let sb = Sidebar {
sessions: vec![s],
+3 -3
View File
@@ -123,9 +123,9 @@ client opening `/api/attach/:id`, and a client reporting focus explicitly at
`POST /api/sessions/:id/focus` when it refocuses a pane that is already open.
**A session that has never been focused sorts after every session that has**,
and among those, most recently created first. Dead sessions sort after live
ones regardless of recency: a session you cannot attach to is not where you
just were.
and among those, most recently created first. Sessions with no live pane —
`Dead`, and `Dormant` (#162) — sort after live ones regardless of recency: a
session you cannot attach to is not where you just were.
Under `>` an empty query lists every command in registry order. Under `~` it
lists recent dirs, most recent first, then the project list.
+20
View File
@@ -243,6 +243,26 @@ Pick `/tmp` if the daemon must outlive your login session and you can't run
tmux sockets with it). The env var `MAESTRO_DAEMON_TMUX_TMPDIR` takes
precedence over the file.
## What happens to sessions after a reboot
A host reboot kills the tmux server, so every session the daemon knew is dead
while its record on disk survives. What recovery does with those is
`[daemon].resume_on_boot`:
```toml
[daemon]
resume_on_boot = "dormant" # "dormant" (default) | "auto" | "never"
```
- `dormant` — the sessions are listed but not running, and you resume the ones
you want from the web UI or the TUI.
- `auto` — the daemon resumes them all itself during startup. Only pick this
on a host where restarting every agent unattended is safe: they come back on
the flags they were spawned with, with nobody watching.
- `never` — the records are deleted and a reboot is a clean slate.
See `docs/RESTART.md` for the full contract.
## What the systemd unit looks like
The wizard writes a *user* unit (`systemctl --user`), since `maestro daemon`
+84
View File
@@ -68,6 +68,90 @@ resolves, so a blocked client is never a trapped one. It clears itself the
moment a probe succeeds — there is no "retry" button, because the client is
already retrying.
## What survives, and what comes back dormant
A daemon restart leaves tmux alone, so recovery re-adopts every
`maestro-<uuid>` session tmux still holds and the operator sees no change
beyond the reconnect above.
A **host reboot** is different: it kills the tmux server, so there is nothing
to adopt while every session's sidecar is still on disk. Recovery always asks
tmux — `tmux list-sessions` is the single source of truth for liveness, and the
kernel's `boot_id` only tells a reboot from a restart in the log (#164, #174).
A boot id changes without the tmux server dying whenever the state directory is
restored, a VM or container snapshot is rolled back, or the `boot_id` file is
truncated; a session tmux still holds is adopted `Running` in every one of
those cases. Recovery makes a
second pass over those sidecars and registers each as a `Dormant` session
(#162) — id, name, cwd, command, worktree and `created_at` intact. A dormant
session has no pane and no PTY bridge: it cannot be attached to, typed into or
resized, and neither watcher pass touches it.
A dormant session is brought back with `POST /api/sessions/:id/resume` (#166),
which respawns tmux under the **same** uuid and sidecar — `metadata.name`,
`created_at`, `env` and worktree ownership all survive, and only the tmux
session and the PTY bridge are new. The command is the matching spawn preset's
`resume_command` with `{agent_session}` filled in from the session's own
`agent_session_id`, so an agent comes back inside its old conversation; a
session with no matching preset, no `resume_command`, or no reported
conversation id is respawned on the command it was created with. `Session::command`
always keeps that original command, so a second resume resolves the same preset.
Resume is refused with `409` for a session that is already running
(`already_running`) and for one whose `cwd` has disappeared (`cwd_missing`) —
neither is something a respawn could fix. A session with a resume already in
flight is refused as `already_running` too (#175): the respawn happens with
the session map unlocked, so the record is marked in-flight before it starts
and the second request is answered on that state rather than on its tmux spawn
colliding with the first one's. A resume that loses a race with a kill takes
its own sidecar back off disk, so the killed session does not return as a
dormant ghost on the next recovery. The other thing to do with a dormant
session is dismiss it, which removes the sidecar so the next recovery does not
bring it back.
## What recovery does with them: `resume_on_boot`
Whether a recovered-but-dead session is left dormant at all is the operator's
call, per host, via `[daemon].resume_on_boot` (#167):
| value | what recovery does |
| ----------- | --------------------------------------------------------------------- |
| `"dormant"` | **Default.** Register each one dormant and wait for the operator. |
| `"auto"` | Resume each one during recovery, itself. |
| `"never"` | Prune the sidecars instead — a reboot is a clean slate. |
`dormant` is the default because resuming agents unattended is a foot-gun:
they come back on whatever flags they were spawned with, seconds after a
reboot, with nobody watching. `auto` exists for hosts where that is known to
be safe, and it is opted into rather than inherited.
`auto` calls the same `resume` the client's `POST /api/sessions/:id/resume`
does — same uuid, same sidecar, the preset's `resume_command` — so there is
one resume path and not two. A session that fails to resume is logged and
stays dormant; the rest still come back.
`never` prunes the same way #163 does for a missing `cwd`: the sidecar is
deleted and nothing is registered, so the session is not listed and the next
recovery does not find it. Whatever worktree it owned is left on disk.
The policy only decides the fate of sessions with no live pane. A session
tmux still holds is adopted `Running` by the first pass under every policy,
including `never`.
Recovery logs the policy that ran and the number of sessions it acted on, so
a host that resumes agents unattended says so in its own journal.
Attaching to a session that has no bridge answers `409 not_attachable`, not
`404 unknown_session`: the session exists, and a client that cannot tell "gone"
from "not running" cannot offer the resume.
`SessionStatus` is a wire type, so from this build forward it deserializes
leniently on both clients: a spelling the client does not know reads as `Dead`
rather than failing the whole session. That leniency lives in the client's own
binary, so it covers spellings added *after* this build, not this one — a CLI
or TUI still running an older build fails to parse a `Dormant` session, and
with it the whole session list, until it is upgraded. Clients render a dormant session as known-but-not-running —
see `docs/SIDEBAR.md` for the card and `docs/COMMAND-BAR.md` for how it ranks.
## Timing
- **Grace window** — 1200 ms between the socket dropping and the overlay
+10 -4
View File
@@ -56,7 +56,7 @@ itself is not.
Line 2 reads left to right:
```
[<duration>] [<detail>] <command> [<diff stat> | exited]
[<duration>] [<detail>] <command> [<diff stat> | exited | dormant]
```
Its minimum content is therefore just the command.
@@ -104,8 +104,14 @@ fragment; it no longer decides whether line 2 exists.
while an agent-reported state is in force (see Precedence): when the report
lapses at its TTL or is replaced, the detail goes with it. The detail
truncates with `…` ahead of the trailing fragments — a long detail never
pushes `<command>`, `<blocked work>`, or `exited` off the line, and neither
client ever lets it wrap.
pushes `<command>`, `<blocked work>`, `exited` or `dormant` off the line, and
neither client ever lets it wrap.
`dormant` (#162) is the counterpart of `exited` for a session the daemon knows
only from its sidecar: recovery found no tmux session behind it, so it has no
pane, no scrollback, and nothing to attach to. It is not an exit — nothing was
observed ending — so the two fragments never both show. The TUI marks the same
state on line 1 with `◌`, next to `✗` for exited and `●`/`○` for running.
`<diff stat>` is the blocked-work breakdown (#83) in diff notation, replacing
the spelled-out `⚠ 3 uncommitted files, 2 unpushed commits`. It renders only
@@ -209,7 +215,7 @@ the cursor's fill while the pane holds the keyboard, and the web's ring on a
highlighted **folder** row — a folder previews nothing, so it has no fill to
step up to.
The duration text inherits its state's colour; the exited and command
The duration text inherits its state's colour; the exited, dormant and command
fragments use the surface's `muted` role, as do the diff stat's `~`, `?` and
`↑` fragments. Only `+` and `-` step outside the surface's muted role, and they
take the `active` and `error` roles of the same surface.
+13
View File
@@ -414,6 +414,19 @@ describe("session cards (#107)", () => {
expect(line2?.textContent).toContain("exited");
});
// #162: a session recovered with no tmux behind it is known but not running,
// and never reads as something that exited.
it("shows dormant on line 2 for dormant sessions", () => {
const s = session({
id: "aaaaaaaa-0000-0000-0000-000000000118",
status: "Dormant",
});
const el = renderSidebar(storeWith([s])).el;
const line2 = rowFor(el, s.id)?.querySelector<HTMLElement>(".card-line2");
expect(line2?.textContent).toContain("dormant");
expect(line2?.textContent).not.toContain("exited");
});
it("carries the state word only in tooltips and aria labels, never as text", () => {
const s = session({
id: "aaaaaaaa-0000-0000-0000-000000000019",
+4
View File
@@ -113,6 +113,9 @@ function activityRailMarkup(slug: string, stateWord: string): string {
function metaLine(s: Session, now: number): string {
const dead = s.status === "Dead";
// #162: known from its sidecar with nothing running behind it — a distinct
// fragment, since it never exited.
const dormant = s.status === "Dormant";
const blockers = sessionBlockers(s);
const slug = activityColorSlug(s.activity ?? "Idle");
const held = activityHeldMs(s.activity_since, now);
@@ -127,6 +130,7 @@ function metaLine(s: Session, now: number): string {
commandFragment(s.command),
blockers ? blockedFragment(blockers) : "",
dead ? `<span class="card-flag">exited</span>` : "",
dormant ? `<span class="card-flag">dormant</span>` : "",
];
return `<span class="card-line2" data-activity="${slug}">${fragments.join("")}</span>`;
}
+14
View File
@@ -99,6 +99,20 @@ describe("empty-query ordering", () => {
]);
});
// #162: docs/COMMAND-BAR.md ranks by whether you could attach, not by which
// way the pane went away.
it("sorts dormant sessions after live ones, like dead ones", () => {
const dormantRecent = session("dormant", {
status: "Dormant",
last_focused_at: "2026-06-01T00:00:00Z",
});
const liveStale = session("live", { last_focused_at: "2020-01-01T00:00:00Z" });
expect(orderSessionsForBar([dormantRecent, liveStale]).map((s) => s.id)).toEqual([
"live",
"dormant",
]);
});
it("treats a null last_focused_at as never focused", () => {
const nulled = session("nulled", { last_focused_at: null });
const stamped = session("stamped", { last_focused_at: "2020-01-01T00:00:00Z" });
+5 -3
View File
@@ -142,8 +142,10 @@ function createdAt(s: Session): number {
* matcher's last tie-break preserves. */
export function orderSessionsForBar(sessions: readonly Session[]): Session[] {
return [...sessions].sort((a, b) => {
const deadA = a.status === "Dead" ? 1 : 0;
const deadB = b.status === "Dead" ? 1 : 0;
// docs/COMMAND-BAR.md: anything with no live pane — dead or dormant (#162)
// — sorts after the sessions you could actually attach to.
const deadA = a.status === "Running" ? 0 : 1;
const deadB = b.status === "Running" ? 0 : 1;
if (deadA !== deadB) return deadA - deadB;
const focusA = focusedAt(a);
@@ -350,7 +352,7 @@ function renderRow(result: BarResult, index: number): HTMLElement {
row.dataset.kind = candidate.kind;
if (candidate.kind === "session") {
row.dataset.sessionId = candidate.session.id;
if (candidate.session.status === "Dead") row.classList.add("is-dead");
if (candidate.session.status !== "Running") row.classList.add("is-dead");
} else if (isActionCandidate(candidate)) {
row.dataset.actionId = candidate.action.id;
} else if (candidate.kind === "command") {
+4 -1
View File
@@ -6,7 +6,10 @@ import { escapeHtml } from "./html.js";
export type Uuid = string;
export type IsoTimestamp = string;
export type SessionStatus = "Running" | "Dead";
/** `Dormant` (#162) is a session the daemon knows only from its sidecar —
* recovered after the tmux server died under it, so there is no pane behind it
* and nothing to attach to. Listed, never live. */
export type SessionStatus = "Running" | "Dead" | "Dormant";
/** Daemon-inferred activity dimension, orthogonal to `SessionStatus` (NAP-340).
* Absent on payloads from daemons that predate the field. `Error` is only set