yolo/agent-skills
3
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(nightshift): calibrated quota verdict + scoped milestone checks

Browse Source 
usage-window.py stores a calibrated window quota and emits an explicit
verdict, so a run can no longer park on an uncalibrated raw token count.
Milestone assess is scoped; full suite moves to the gated push points.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YUXS63P1WCdC6bEKWcnAE
This commit is contained in:
naps62
2026-07-27 12:45:06 +00:00
parent 999f759691
commit 715420d0a9
2 changed files with 207 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/nightshift/SKILL.md
+37 -7
View File
@@ -41,7 +41,7 @@ Then fan out. Agents implementing against frozen signatures can run in parallel
Repeat until done or stopped:
1. **Check the limit** (see below). Park if close.
2. **Assess** — run the project's gate (typecheck + lint + tests). Read what landed since last time.
2. **Assess** — typecheck + lint, plus tests **for what changed this milestone**. Read what landed since last time. The full suite is a push-time check, not a milestone habit — see "Local verification budget" in COMMON.md; you are sharing this machine with other autonomous sessions and running everything every milestone is what OOMs it.
3. **Decide** — one architectural decision, written down. If nothing needs deciding, you are done; say so.
4. **Decompose** into subtasks with **disjoint file ownership**.
5. **Delegate** — subagents, in parallel where files don't collide.
@@ -59,16 +59,43 @@ python3 <skills-root>/nightshift/usage-window.py --json # machine
It sums billable tokens across **every Claude Code session on this machine** — all projects, not just yours, since parallel sessions in other repos share the same window.
**It is a floor, never a ceiling.** Nothing local states the real quota, and the script cannot see Claude Code on another machine, claude.ai web usage, or direct API calls. Real usage is whatever it reports *plus* however much the account is being used elsewhere. If the user works across several machines, treat a comfortable reading with suspicion and take a wider margin.
### Act on the percentage. Never on the raw number.
Because of that gap, the reactive backstop matters as much as the estimate: **an agent that dies on a limit error is telling you the truth this script only guesses at.** Believe it immediately, and park — do not retry into the wall.
**A token count is a numerator.** On its own it means nothing. "4.8 million" sounds enormous and can be 30% of the window.
How to act on it:
The script prints an explicit `VERDICT`. Use it:
- **Window looks heavy** (well into millions, and climbing fast across recent milestones): finish what's running, commit, push, and **park**. Do not dispatch new agents.
- **Parking** = `ScheduleWakeup` for the time `oldestAgesOutInSeconds` reports, plus a margin. Sleeping until the window loosens is strictly better than having three agents killed halfway through their tasks.
- **Before parking, always**: commit, push (parking is one of the deliberate push points, see §8), and write the current state and the next intended step into the log. The run must be resumable by a different session that has none of your context.
| Verdict | Do |
|---|---|
| `clear` (<60%) | Dispatch freely. The window refills — unused capacity is wasted, not saved. |
| `wrap-up` (60–80%) | Keep working. Prefer shorter tasks over long fan-outs, so nothing large is in flight if it tightens. |
| `park` (≥80%) | Finish what's running, commit, push, write the next step down, park. |
| `unknown` | **No quota is calibrated.** Do NOT infer one from magnitude. See below. |
**If the verdict is `unknown`, the correct response is to ask, not to guess.** Ask the user what percentage Claude Code reports (`/status`), then:
```bash
python3 <skills-root>/nightshift/usage-window.py --calibrate 30
```
That stores the implied quota in `~/.claude/nightshift-quota.json` and every later run gets a real percentage. It takes one question. Until then you are flying blind and must not park on a hunch.
> **This section exists because it went wrong.** A run parked itself at what turned out to be 30% of the window, having reasoned that ~4.8M tokens was "well into millions and climbing" — the exact phrasing this skill used to contain. Hours of remaining capacity went unused and the user had to intervene. An unanchored magnitude threshold is not conservatism, it is a made-up number. Parking early is not free: it wastes the window the run was given.
### Both directions are failures
Backing off too late kills agents mid-task. Backing off too early wastes the run. Neither is the safe default — the whole point of measuring is to avoid *guessing* in either direction.
**It is a floor, never a ceiling.** The script cannot see Claude Code on another machine, claude.ai web usage, or direct API calls. Real usage is whatever it reports *plus* however much the account is being used elsewhere. If the user works across several machines, treat a comfortable reading with suspicion — but treat it with suspicion by *asking* or by widening the margin a little, not by inventing a threshold.
Because of that gap, the reactive backstop matters as much as the estimate: **an agent that dies on a limit error is telling you the truth this script only estimates.** Believe it immediately, and park — do not retry into the wall.
Other rules:
- **Parking** = `ScheduleWakeup` for the time `oldestAgesOutInSeconds` reports, plus a margin. Sleeping until the window loosens beats having three agents killed halfway through their tasks.
- **Before parking, always**: commit, push (parking is one of the deliberate push points, see §8), and write the current state and the next intended step into the log. The run must be resumable by a different session with none of your context.
- **If an agent dies on a limit error anyway**: do not immediately retry. Check the tree still passes the gate, commit whatever is green with a message stating plainly that it is **unverified** and what was left half-done, then park.
- **If the user is present, say the number and let them decide.** They can see the real figure; you cannot. A one-line "window at 31%, continuing" costs nothing and catches a miscalibration immediately.
- Never *silently* burn the window to zero. If the user is asleep, they will wake to a stalled run and no explanation.
Scale the check to the work: a run doing small mechanical tasks needs it rarely; a run fanning out three heavy agents per milestone needs it every time.
@@ -83,6 +110,8 @@ What actually works, learned the hard way:
- **Demand verification the task can actually support.** "Tests pass" is not enough for anything a human will look at or listen to. Require a screenshot, a measured number, a browser run. Say plainly when something can only be verified by a human.
- **Model choice**: strongest model for design-heavy or feel-critical work; a cheaper one is fine for mechanical, well-specified changes.
- Instruct them to **commit their own work locally** when it's coherent, so a killed agent loses less — and explicitly **not to push**. A dozen subagent pushes is a dozen CI runs on half-finished work.
- **Tell them not to run the full suite.** Scoped checks on the files they own, nothing more. Five agents each running every test is five copies of the same work and enough memory pressure to kill the run. You run the full suite once, at push time, through `gate.sh`.
- **Cap the fan-out at 3 concurrent subagents, 2 if their tasks compile or test.** More agents is not more throughput on a box this size — it is swap. `<skills-root>/linear-common/scripts/gate.sh --status` shows how much of the machine other sessions are already using; dispatch fewer when it is contended, and remember other `/yolo` and `/nightshift` runs are competing for the same RAM.
## 6. Reviewing what lands
@@ -110,6 +139,7 @@ This is what makes an overnight run reviewable by a human who slept through it.
- All work goes on **one branch** in the worktree. Subtasks commit to it **locally**.
- **Push is a deliberate act, not a milestone habit.** Every push runs CI, and a night of milestone pushes is a night of CI runs on work that was half-finished at the time — noisy, expensive, and it trains the user to ignore the build.
- **Push when:** the run finishes, you park on a limit, or the user asks. That's it.
- **Before each of those pushes**, run the full suite once through the gate: `<skills-root>/linear-common/scripts/gate.sh -- <buildCommand>`. Exit 75 means the machine was busy and it never ran — push and say so plainly in the PR body under what is unverified. Exit 137 is the memory cap, not a failing test.
- **Then open the PR** describing what landed, what is unverified, what you decided and why, and what needs a human. The build log (§7) is most of that text already.
Forge-agnostic: