Files
agent-skills/skills/week-review/SKILL.md
T
Miguel Palhas 2022bbc881 feat(week-review): scan pi/opencode, report model, effort, cost
Scanner covered Claude Code only. It now also reads pi jsonl sessions, the
opencode sqlite store, and Codex (rollout files plus the sqlite thread index
as fallback), and records per-session model, effort level, token counts, tool
errors and cost.

Cost is reported natively by pi and opencode; Claude Code and Codex are
estimated from pricing.json and marked as such, since a subscription seat is
not billed those numbers.

New models.md output ranks (tool, model, effort) by spend with cost per human
turn and a push-back count, and SKILL.md step 4 says how to read it without
turning a regex into a verdict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 21:45:48 +01:00

190 lines
7.8 KiB
Markdown

---
name: week-review
description: Review the past week of agent sessions across Claude Code, Codex, pi and opencode, find recurring friction, judge which models and effort levels earned their cost, and turn it into concrete config or tooling changes. Use when the user asks to review the week, review recent sessions, asks what to improve about their setup, or asks which model or tool is worth the spend. Also picks up carry-over items filed as issues on the agent-skills repo.
user-invocable: true
argument-hint: "[--days N | --since YYYY-MM-DD]"
allowed-tools:
- Read
- Grep
- Glob
- Bash
- Edit
- Write
- WebFetch
- WebSearch
---
# Week review
Find what went wrong repeatedly, fix the cause, file the rest.
The output is changes and issues, not a report. A finding nobody acts on was
not worth the tokens to produce.
## 1. Carry-over first
Read the open issues before scanning anything. Last week's unfinished work is
the highest-value input, and re-deriving it from transcripts wastes a lot of
context.
```sh
source ~/.env.claude
curl -s -H "Authorization: token $GITEA_TOKEN" \
"https://git.naps.pt/api/v1/repos/yolo/agent-skills/issues?state=open&labels=weekly-review&limit=50" \
| python3 -c "import json,sys; [print(f\"#{i['number']} {i['title']}\") for i in json.load(sys.stdin)]"
```
Read the bodies, not just the titles — several carry a design already argued
through, so the run starts from the open question rather than from scratch.
Ask which to take this week. Do not silently re-litigate one the user already
deferred; a deferred item stays open and gets one line in the summary.
## 2. Scan
```sh
python3 <skill-dir>/scripts/scan-sessions.py --days 7 --out <scratch>
```
Covers Claude Code, Codex, pi and opencode in one pass — Claude Code is the
bulk of most weeks, but a friction pattern that only shows up in the other
three is exactly the one nobody has noticed yet. Narrow with
`--tools claude,pi` when a run only needs one of them.
Writes three files:
- `sessions.json` — one record per session: tool, cwd, models, effort levels,
token counts, cost, tool-error count.
- `userturns.txt` — every human turn, grouped, with the tool, model, effort and
cost in each session header.
- `models.md` — spend and friction per (tool, model, effort), for step 4.
It drops subagent transcripts and flags swarm runs — collapse those to a single
line, since one `/code-review ultra` can be 500+ sessions and 40% of the week's
bytes without being 40% of the week's work.
Read `userturns.txt` in full. It is the primary evidence and it is usually
40-100k tokens. Do not sample it.
## 3. Find the friction
Rank by how often the same thing went wrong, not by how annoying any one
instance felt. In order of signal strength:
- **The same correction given more than once**, especially across different
repos. Four separate "stop putting decisions in the spec, use an ADR"
corrections means the rule belongs in global config, not in each repo.
- **A fix that did not hold.** Something declared fixed in one session and
recurring days later. Name both sessions.
- **Crashes, `/compact` confusion, "are you there?", sessions restarted to
rebuild lost state.**
- **Security slips** — a secret echoed, an env value written somewhere it
persists. These outrank everything above on consequence.
- **Anything the user said twice in different words.**
Quote the user verbatim with the date and repo. A finding without a quote is a
guess, and the user can tell.
## 4. Judge the models and effort levels
`models.md` holds one row per (tool, model, effort) with sessions, human turns,
push-back count, tool errors, output tokens, cost, and cost per human turn.
Cost per turn is the honest headline, not total cost. A model that bills three
times as much but reaches the same place in a third of the turns is the cheaper
one, and the raw total will say the opposite.
Read the numbers with these limits in mind:
- **Claude Code and Codex costs are estimated**, from token counts and
`scripts/pricing.json`. A subscription seat is not billed this. Call it
API-equivalent spend every time you quote it. pi and opencode report their
own real cost — those are the only invoice-true numbers in the table.
- **Unpriced models count as zero.** If the unpriced list at the bottom of the
table is long, the ranking is wrong until the rates are added.
- **`push-back` is a regex**, not a verdict. It counts human turns that read
like a correction. Use it to pick which sessions to read, then quote what the
user actually said.
- A single session never establishes that a model is worse. Two rows are
comparable only when they did comparable work.
What the comparison is for:
- **Effort.** Find work that ran at high effort and did not need it — small
mechanical edits, single-file renames — and work that ran too low and came
back with push-back. The fix is a per-task-class default, not a global one.
- **Model and tool choice.** Where the same class of task ran under two models
or two tools in the same week, compare turns-to-done and push-back, not
tokens.
- **Cost concentrated in one repo or one skill.** A skill that reliably costs
ten times the median per turn is a skill to reread, not a model problem.
Recommendations from this step change a default in config; they never end at
"use the cheaper model".
## 5. Check the docs before recommending
Model behaviour changes and last year's advice rots. Before proposing a
prompt, skill, or config change, read the relevant page — do not answer from
memory:
- `platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-opus-5`
- `.../prompting-claude-fable-5`
- `.../claude-prompting-best-practices`
- `code.claude.com/docs/en/memory`
Two findings from these that keep mattering: instructions to verify or
re-check compound badly and should be removed, and prompt style leaks into
output style, so a rule written in dense prose teaches dense prose.
Search for community practice too, and say which source a recommendation came
from.
## 6. Measure before trimming
Always-loaded and on-demand are different budgets, and conflating them
produces wrong advice.
| Always loaded | On demand |
|---|---|
| entry files and every `@import` they pull | skill bodies |
| `~/.claude/rules/*.md` without `paths:` frontmatter | `~/.claude/rules/*.md` **with** `paths:` |
| the first 200 lines of each project's `MEMORY.md` | memory topic files |
| every skill's `name` + `description` | |
Count lines, not words — Anthropic's target is under 200 lines per file.
Splitting one file into `@import`s saves nothing; only deleting content or
adding `paths:` scoping does.
## 7. Apply, then file the rest
Propose a ranked shortlist with an appetite for each. Apply what the user
agrees to, in this repo, and push. For anything deferred or too large, file a
Gitea issue so next week starts from step 1 instead of a re-derivation.
**This repo is public.** Issues must carry no client names, no hostnames, no
secrets, no internal ticket IDs. Describe the shape of the problem, not the
customer it happened at. When quoting the user as evidence, strip identifying
detail first.
Label every issue `weekly-review` (id 37) so step 1 picks it up next run. An
issue filed without it is invisible to the next review.
```sh
source ~/.env.claude
curl -s -X POST -H "Authorization: token $GITEA_TOKEN" \
-H "Content-Type: application/json" \
"https://git.naps.pt/api/v1/repos/yolo/agent-skills/issues" \
-d '{"title":"...","body":"...","labels":[37]}'
```
Close issues that got done this week, with a one-line comment saying what
landed.
## Scope
Config, skills, hooks, prompts, and which model, effort level and tool each
class of work should default to. Not a project status report — the user has
trackers for that. If a week's biggest problem is a product bug, say so in one
line and move on.