boot_id change with a live tmux server strands every session #174

Closed
opened 2026-09-04 09:06:49 +01:00 by naps62-yolo · 0 comments
Collaborator

recover skips the tmux probe entirely when the stored boot id differs from the current one, and a test asserts it does so with tmux still alive. The assumption is that a changed boot id means the tmux server cannot exist. That is not true of every case that changes a boot id: a restored state directory, a VM or container snapshot restore, or a truncated boot_id file all produce a "reboot" while the tmux server is still running with live sessions in it.

When that happens every live session is registered Dormant, and it can never come back: resume runs tmux new-session -s maestro-<id>, which collides with the session still holding that name, so it fails 502 forever. The sessions are alive, visible in tmux ls, and unreachable through maestro.

Two ways out, either is acceptable:

  • Keep the probe always, and use the boot id only to skip the wait for a tmux server that is not coming back — the optimisation the issue actually wanted.
  • Have resume adopt or kill a same-named tmux session before spawning, so a stranded session can be recovered rather than being permanently unreachable.

Prefer the first: it keeps tmux list-sessions as the single source of truth for liveness, which is what the rest of the daemon assumes. Cover it with a test that changes the boot id while a tmux session is still alive and asserts the session is adopted Running, not registered Dormant.

Found by the wave-2 integration review of the blitz/reboot-resilience branch.

`recover` skips the tmux probe entirely when the stored boot id differs from the current one, and a test asserts it does so *with tmux still alive*. The assumption is that a changed boot id means the tmux server cannot exist. That is not true of every case that changes a boot id: a restored state directory, a VM or container snapshot restore, or a truncated `boot_id` file all produce a "reboot" while the tmux server is still running with live sessions in it. When that happens every live session is registered `Dormant`, and it can never come back: `resume` runs `tmux new-session -s maestro-<id>`, which collides with the session still holding that name, so it fails `502` forever. The sessions are alive, visible in `tmux ls`, and unreachable through maestro. Two ways out, either is acceptable: - Keep the probe always, and use the boot id only to skip the *wait* for a tmux server that is not coming back — the optimisation the issue actually wanted. - Have `resume` adopt or kill a same-named tmux session before spawning, so a stranded session can be recovered rather than being permanently unreachable. Prefer the first: it keeps `tmux list-sessions` as the single source of truth for liveness, which is what the rest of the daemon assumes. Cover it with a test that changes the boot id while a tmux session is still alive and asserts the session is adopted Running, not registered Dormant. Found by the wave-2 integration review of the `blitz/reboot-resilience` branch.
naps62-yolo added this to the Reboot resilience milestone 2026-09-04 09:06:49 +01:00
naps62-yolo added the difficulty/mediumtype/fixarea/daemon labels 2026-09-04 09:06:49 +01:00
naps62-yolo added a new dependency 2026-09-04 09:06:49 +01:00
This repo is archived. You cannot comment on issues.
1 Participants
Due Date
No due date set.
Reference: naps62/maestro#174