Merge remote-tracking branch 'origin/main' into linear/nap-273-post-nap-245-cleanup-consolidate-duplicated-helpers-across
ci / rust (push) Successful in 4m26s
ci / rust (push) Successful in 4m26s
This commit is contained in:
@@ -0,0 +1,314 @@
|
||||
# Maestro — In-app Update Notifications + Optional Auto-update
|
||||
|
||||
**Date**: 2026-06-18
|
||||
**Status**: Design spike (NAP-331). Produces decisions, not implementation.
|
||||
**Depends on**: NAP-330 (`KillMode=process` — restart without session loss). **Done.**
|
||||
**Touches**: NAP-225 (installer / systemd unit), the "prebuilt binaries for thin hosts" idea.
|
||||
|
||||
## What this is
|
||||
|
||||
Turn today's yolo-local update hack into a first-class, cross-host maestro
|
||||
capability: every client surface ("a newer version is available, X→Y") and an
|
||||
opt-in mechanism to apply the update without dropping sessions.
|
||||
|
||||
This is a design spike. It answers the six questions on NAP-331, then proposes
|
||||
an implementation ticket breakdown. No code ships from this ticket beyond this
|
||||
document.
|
||||
|
||||
## Prior art (the hack we're replacing)
|
||||
|
||||
A yolo-local prototype (out of repo, intentionally uncommitted): a gitea push
|
||||
webhook → a tiny `--user` systemd listener on a private ip:port → worker that
|
||||
hard-resets a dedicated checkout to `origin/main`, `cargo install`s, restarts
|
||||
the daemon (stamping the last-built commit). Works, but it is per-host
|
||||
shell+systemd with no in-app surface and no policy. We keep its shape (separate
|
||||
supervised updater, dedicated checkout, systemd restart) and add: build-commit
|
||||
stamping in the binary, a registry-mediated "latest seen" signal, client
|
||||
surfaces, a config policy block, and a prebuilt-binary path for thin hosts.
|
||||
|
||||
## Key existing infrastructure we build on
|
||||
|
||||
Grounding the design in what already exists in the repo today:
|
||||
|
||||
- **`DaemonInfo.version: String`** (`maestro-protocol`) is *already* advertised
|
||||
by each daemon in `Hello` and fanned out by the registry in
|
||||
`RegistryEvent::Snapshot` / `RegistryEvent::DaemonAdded`. The registry already
|
||||
knows every connected daemon's version. Cross-host version awareness is
|
||||
therefore an *extension of an existing field*, not new plumbing.
|
||||
- **`RegistryEvent` fan-out WS** (`/api/events`, `maestro-registry/src/events.rs`)
|
||||
already server-pushes catalog/presence deltas to every client. A
|
||||
"newest-version-seen" signal rides this same channel.
|
||||
- **`MaestroConfigFile`** (`maestro-config`) is sectioned TOML
|
||||
(`[shared]`, `[daemon]`, `[registry]`, `[web]`, `[tui]`, `[cli]`). An
|
||||
`[autoupdate]` section drops in with the same `skip_serializing_if = is_empty`
|
||||
pattern.
|
||||
- **systemd unit** (`maestro-installer/src/systemd.rs`): `Type=simple`,
|
||||
`ExecStart={exec} daemon`, `KillMode=process` (NAP-330), `Restart=on-failure`,
|
||||
`RestartSec=5`. `systemctl restart maestro` already preserves tmux sessions.
|
||||
- **Version is `0.1.0` workspace-wide, no git hash.** `version.workspace = true`;
|
||||
the only consumer today is a `user_agent` string in `maestro-cli/src/http.rs`.
|
||||
There is **no `build.rs`** in the umbrella crate yet. Stamping the commit is
|
||||
net-new and is the hard prerequisite for everything else.
|
||||
|
||||
---
|
||||
|
||||
## Q1 — Version detection
|
||||
|
||||
**Decision: stamp the build commit at compile time and carry it as a structured
|
||||
`BuildVersion`, with `origin/main` HEAD as the source of truth for "latest" in
|
||||
the self-hosted (cargo-install) path, and a CI-published `manifest.json` for the
|
||||
prebuilt path.**
|
||||
|
||||
### Stamping the binary
|
||||
|
||||
Add a `build.rs` to `crates/maestro/` that emits the git commit (and dirty flag)
|
||||
as compile-time env vars. Two viable approaches:
|
||||
|
||||
- **`vergen`** (or `vergen-gix`): mature, emits `VERGEN_GIT_SHA`,
|
||||
`VERGEN_GIT_COMMIT_TIMESTAMP`, dirty, etc. Heavier dep tree.
|
||||
- **Hand-rolled `build.rs`** running `git rev-parse HEAD` + `git status
|
||||
--porcelain`, with `cargo:rerun-if-changed=.git/HEAD`. Zero deps, ~20 lines.
|
||||
|
||||
Recommend the **hand-rolled build.rs** — the repo already favors minimal deps and
|
||||
real-tool integration tests; we only need three values. Critically, the build
|
||||
script must degrade gracefully: when built from a tarball / crates.io / a
|
||||
checkout with no `.git` (thin host installing a prebuilt), fall back to
|
||||
`option_env!` and emit `unknown`. Never fail the build for a missing git.
|
||||
|
||||
Expose a structured version everywhere `--version` is printed:
|
||||
|
||||
```
|
||||
maestro 0.1.0 (a1b2c3d 2026-06-18, dirty)
|
||||
```
|
||||
|
||||
Add a `BuildVersion { semver, commit, commit_time, dirty }` type
|
||||
(`maestro-protocol`, since it crosses the wire) and a `maestro version --json`
|
||||
subcommand for machine consumers (the updater, tests).
|
||||
|
||||
`DaemonInfo.version` is currently a bare `String` (`"0.1.0"`). **Migrate it to
|
||||
carry the commit** — either widen it to `BuildVersion` (clean, breaks wire compat
|
||||
with older daemons) or keep the `String` and adopt the `"0.1.0 (a1b2c3d)"`
|
||||
display form (compat-friendly, parse on read). Recommend **widening to a struct
|
||||
with `#[serde(default)]`-friendly fields** so old daemons that send a bare semver
|
||||
still deserialize (commit ⇒ `None`/`unknown`). This is the only wire change.
|
||||
|
||||
### Source of truth for "latest"
|
||||
|
||||
Two sources, matching the two apply mechanisms (Q3):
|
||||
|
||||
- **Self-hosted / cargo-install hosts** → `origin/main` HEAD commit. The updater
|
||||
component (Q6) is the only thing that talks to git; it resolves
|
||||
`git ls-remote origin main` (or reads from its dedicated checkout) to learn the
|
||||
latest commit. "Newer" = `latest_commit != my_build_commit` **and** latest is a
|
||||
descendant (avoid offering downgrades on a force-push; `git merge-base
|
||||
--is-ancestor`).
|
||||
- **Thin / prebuilt hosts** → a CI-published `manifest.json`
|
||||
(`{ semver, commit, commit_time, artifacts: { "<target-triple>": { url, sha256 } } }`)
|
||||
at a stable URL. The updater polls/fetches it; "newer" = manifest commit_time >
|
||||
my commit_time.
|
||||
|
||||
Semver (`0.1.0`) alone is **not** sufficient — it is bumped rarely and the same
|
||||
semver spans many commits. Commit identity is the real version key; semver stays
|
||||
the human-facing coarse label.
|
||||
|
||||
---
|
||||
|
||||
## Q2 — Surfaces
|
||||
|
||||
**Decision: the registry computes a `latest_version` across all daemons it has
|
||||
ever seen and broadcasts it on the existing `RegistryEvent` channel; every client
|
||||
compares its local daemon's `BuildVersion` against that and renders a passive
|
||||
"update available X→Y" affordance.**
|
||||
|
||||
### Registry as the version oracle
|
||||
|
||||
The registry already receives every daemon's version. Extend its state with
|
||||
`latest_seen: BuildVersion` = max-by-commit-time across connected daemons,
|
||||
optionally bumped by the external "latest" source (Q1) if the registry is
|
||||
configured to poll it. Add to `RegistryEvent`:
|
||||
|
||||
- `RegistryEvent::Snapshot` gains a `latest_version: Option<BuildVersion>` field.
|
||||
- A new `RegistryEvent::LatestVersion { version }` delta pushed when the max
|
||||
advances (a daemon with a newer build connects, or the poll finds a new commit).
|
||||
|
||||
This means **a phone that only ever talks to the registry still learns that a
|
||||
newer version exists** the moment any single host upgrades — exactly the
|
||||
"broadcast the newest version it has seen" idea from the ticket. No client needs
|
||||
to talk to git.
|
||||
|
||||
### Per-surface rendering
|
||||
|
||||
| Surface | Affordance | Notes |
|
||||
|---|---|---|
|
||||
| CLI `maestro list` | One-line footer: `update available: 0.1.0 (a1b2c3d) → 0.1.0 (e4f5a6b) on 2/3 hosts` | Only when at least one host is behind `latest`. `--quiet` / `MAESTRO_NO_UPDATE_NOTICE` suppresses. |
|
||||
| TUI | Dismissable banner row / status-bar segment | Color-coded; click/hotkey opens a per-host update list. |
|
||||
| Web | Header badge (dot + tooltip) | Subscribes to `RegistryEvent`; badge lists which hosts are behind. |
|
||||
|
||||
All three are **read-only notices** in the notify path. Applying is a separate,
|
||||
explicit action (a CLI subcommand / a confirmed button), never a side effect of
|
||||
viewing the list. Per-host granularity matters: in a federated fleet some hosts
|
||||
upgrade before others, so the surface shows *which* hosts are behind, not a single
|
||||
global flag.
|
||||
|
||||
---
|
||||
|
||||
## Q3 — Apply mechanism
|
||||
|
||||
**Decision: support both, selected per-host by a config/auto-detected
|
||||
`method`. Default `cargo-install` where a toolchain exists, `prebuilt` on thin
|
||||
hosts.**
|
||||
|
||||
- **`cargo-install` (build-on-host).** Mirrors the prototype: dedicated checkout
|
||||
(NOT the user's working tree), `git fetch` + hard-reset to the target commit,
|
||||
`cargo install --path crates/maestro --locked`, then restart. Needs a Rust
|
||||
toolchain; ~tens of seconds; produces a binary matching the host exactly.
|
||||
- **`prebuilt` (CI binaries).** Fetch the artifact for the host's target triple
|
||||
from the Q1 manifest, verify `sha256`, atomically replace `~/.cargo/bin/maestro`
|
||||
(write to temp + `rename`), then restart. Works on a Pi / LXC with no compiler.
|
||||
Requires CI to cross-compile and publish per-triple artifacts + manifest (new CI
|
||||
work, called out as its own ticket).
|
||||
|
||||
`method` resolution order: explicit `[autoupdate].method` → auto-detect (is
|
||||
`cargo` on `PATH` and a source checkout present? → `cargo-install`, else
|
||||
`prebuilt`). Both paths converge on the same **stamp-then-restart** epilogue:
|
||||
record the newly-installed commit where the updater can read it back (so a failed
|
||||
restart doesn't claim success), then `systemctl --user restart maestro`.
|
||||
|
||||
---
|
||||
|
||||
## Q4 — Auto-run policy
|
||||
|
||||
**Decision: opt-in `[autoupdate]` config section, default `off`. Three modes,
|
||||
three trigger styles.**
|
||||
|
||||
```toml
|
||||
[autoupdate]
|
||||
mode = "notify-only" # off | notify-only | auto (default: off)
|
||||
method = "cargo-install" # cargo-install | prebuilt | auto (default: auto)
|
||||
trigger = "poll" # webhook | poll | schedule (default: poll)
|
||||
source = "origin/main" # git ref, or a manifest URL for prebuilt
|
||||
poll_interval = "15m" # for trigger = poll
|
||||
window = "*-*-* 04:00:00" # for trigger = schedule (systemd OnCalendar)
|
||||
```
|
||||
|
||||
- **`off`** — no detection, no notice, no apply. (Detection cost is near-zero, but
|
||||
`off` honors users who want maestro to never phone-home about versions.)
|
||||
- **`notify-only`** — detect + surface (Q2), never apply. The safe default to
|
||||
*recommend* once stamping lands.
|
||||
- **`auto`** — detect + apply via the updater (Q6), respecting `trigger`/`window`.
|
||||
|
||||
Fits `MaestroConfigFile` as a new section with the existing
|
||||
`skip_serializing_if = AutoupdateSection::is_empty` pattern; lives logically near
|
||||
`[daemon]` (the daemon is what gets restarted) but is consumed by the **updater**,
|
||||
not the daemon process (Q6).
|
||||
|
||||
**Triggers:**
|
||||
- `webhook` — the prototype's model: gitea push → updater's tiny HTTP listener.
|
||||
Lowest latency; needs an exposed port on the perimeter.
|
||||
- `poll` — updater wakes on `poll_interval`, checks the Q1 source. No inbound
|
||||
port. **Default** — simplest, works everywhere.
|
||||
- `schedule` — systemd timer (`OnCalendar=window`); apply only inside a quiet
|
||||
window. Best for `auto` mode so restarts don't interrupt active work.
|
||||
|
||||
---
|
||||
|
||||
## Q5 — Restart safety
|
||||
|
||||
**Decision: rely on NAP-330's `KillMode=process` + `systemctl restart`; the
|
||||
updater never kills the daemon directly. Add a pre-restart drain check.**
|
||||
|
||||
NAP-330 already shipped `KillMode=process` on the unit, so `systemctl --user
|
||||
restart maestro` SIGTERMs only the daemon's main process — the tmux server and
|
||||
its sessions (separate process tree) survive, and the fresh daemon re-adopts them
|
||||
on startup. This is the load-bearing guarantee; the spike confirms it is already
|
||||
in place.
|
||||
|
||||
Additions for safety:
|
||||
- The updater calls `systemctl restart` (a supervised swap), **never** `kill`. The
|
||||
binary is replaced on disk first; systemd execs the new one.
|
||||
- **Atomic binary replace** (`prebuilt` path) so a half-written binary can't be
|
||||
exec'd: write to `maestro.new`, `fsync`, `rename` over the target.
|
||||
- **Re-adoption is already the daemon's normal startup behavior** (it lists
|
||||
existing tmux sessions on boot). Verify post-restart the daemon re-appears in the
|
||||
registry with the new `BuildVersion` within a timeout; if not, the updater logs a
|
||||
failure (and, future, could roll back to the prior binary — out of scope here,
|
||||
flagged as a follow-up).
|
||||
- `auto` mode should prefer the `schedule` trigger / a quiet window so a restart
|
||||
never lands mid-keystroke for an attached client. A restart is invisible to
|
||||
persistence but *does* blip live WS streams (clients reconnect); doing it at
|
||||
04:00 is kinder than at noon.
|
||||
|
||||
---
|
||||
|
||||
## Q6 — Trust / security
|
||||
|
||||
**Decision: the self-modifying build+exec logic lives in a separate, minimal
|
||||
`maestro-updater` component supervised by systemd — never inside the daemon
|
||||
process.**
|
||||
|
||||
Rationale: a daemon that can `git reset`, `cargo install`, and exec arbitrary
|
||||
freshly-built code is a remote-code-execution surface. The daemon is the
|
||||
network-facing, multi-client process; it must stay incapable of rewriting its own
|
||||
binary. A supervised restart via systemd is needed anyway (the running process
|
||||
can't replace the file it's executing cleanly), so the separation is free.
|
||||
|
||||
Shape:
|
||||
- **`maestro-updater`** — a small binary (new crate `crates/maestro-updater/`, or a
|
||||
feature-gated subcommand `maestro update`). Owns the dedicated checkout, the
|
||||
fetch/build/fetch-artifact, the atomic replace, and `systemctl restart`. Run as
|
||||
its own `--user` systemd unit/timer, NOT a child of the daemon.
|
||||
- **Trust boundary on the source.** `cargo-install` path trusts whatever
|
||||
`origin/main` resolves to — acceptable only because the operator controls the
|
||||
repo and the perimeter (consistent with maestro's network-agnostic threat model:
|
||||
the perimeter is the VPN/LAN). `prebuilt` path **must** verify `sha256` from the
|
||||
manifest against the downloaded artifact, and the manifest should be fetched over
|
||||
TLS from a trusted origin. Document that turning on `auto` = trusting that
|
||||
source to run code on the host.
|
||||
- **No webhook auth shortcuts.** If `trigger = webhook`, the listener must
|
||||
authenticate the caller (shared secret / HMAC of the gitea payload), bind to the
|
||||
perimeter interface, and only ever *trigger* an update of the configured source —
|
||||
it must not accept a commit/url from the request body (that would let any caller
|
||||
pick the code to run).
|
||||
- **Least privilege.** The updater needs write to `~/.cargo/bin` and the dedicated
|
||||
checkout, and `systemctl --user restart` rights. Nothing more. It does not need
|
||||
the daemon's token or registry credentials for the apply itself (it may read the
|
||||
config for policy only).
|
||||
|
||||
---
|
||||
|
||||
## Proposed implementation ticket breakdown
|
||||
|
||||
In dependency order (each is a separate impl ticket spun off from this spike):
|
||||
|
||||
1. **Build-commit stamping.** `build.rs` in `crates/maestro/`, `BuildVersion`
|
||||
type, `--version` long form + `maestro version --json`. Graceful no-git
|
||||
fallback. *(Prerequisite for everything.)*
|
||||
2. **Wire the commit through `DaemonInfo`.** Widen `DaemonInfo.version` to carry
|
||||
the commit with backward-compatible deserialization; daemon advertises real
|
||||
`BuildVersion` in `Hello`.
|
||||
3. **Registry "latest seen" + `RegistryEvent`.** Compute `latest_version`,
|
||||
add `Snapshot.latest_version` + `RegistryEvent::LatestVersion`.
|
||||
4. **Client surfaces.** CLI `list` footer, TUI banner, web header badge. (Can be
|
||||
three sub-tickets; share the registry signal from #3.)
|
||||
5. **`[autoupdate]` config section.** Schema + parsing + defaults in
|
||||
`maestro-config`. Notify-only behavior end-to-end (no apply yet).
|
||||
6. **`maestro-updater` component — cargo-install path.** Dedicated checkout,
|
||||
fetch/reset/install, atomic stamp, `systemctl restart`. systemd unit/timer via
|
||||
the installer (NAP-225).
|
||||
7. **Prebuilt path + CI manifest.** CI cross-compiles per-triple artifacts,
|
||||
publishes `manifest.json` + `sha256`; updater `prebuilt` method with
|
||||
verification.
|
||||
8. **`auto` mode + triggers.** Wire `webhook` / `poll` / `schedule` to the
|
||||
updater; quiet-window scheduling; post-restart version-confirm check.
|
||||
|
||||
#5 and #6 can proceed in parallel after #1–#4. #7 is the heaviest (new CI) and
|
||||
gates only thin-host auto-update; #1–#6 deliver the full cross-host *notify* story
|
||||
plus auto-update on build-capable hosts.
|
||||
|
||||
## Scope boundary
|
||||
|
||||
- **In scope (this spike):** the decisions above + the ticket breakdown.
|
||||
- **Out of scope:** rollback-on-failed-restart (flagged in Q5), multi-arch CI
|
||||
setup details (lands in #7), and any auth design for a hosted/public manifest
|
||||
CDN — maestro stays network-agnostic; the operator's perimeter is the trust
|
||||
boundary (spec §Auth).
|
||||
Reference in New Issue
Block a user