Reclassify can leave half a library stale with no way back #247

Open
opened 2026-08-25 12:04:37 +01:00 by naps62-yolo · 0 comments
Owner

All four callers of reclassifypolicies.rs:442, roots.rs:251,
movies.rs:488, series.rs:634 — run after the write has already
committed, and reclassify::apply issues one autocommitted UPDATE per
changed row with no enclosing transaction. The walk is also N+1: one
policy lookup plus one releases fetch per title.

Failure scenario, at the size §5.1 now cites (2000 titles, 10 000 releases):
a policy edit gets halfway, then one row's parsed blob fails to deserialise
(reclassify.rs:210) or SQLite hits the busy timeout. The handler returns 500.
The policy row is already changed, roughly half the verdicts are re-derived
and half are not, and nothing else in the system re-reads that column, so the
stale half stays stale permanently.

§5.1's new paragraph says "The operator is never left reading a verdict computed
under a policy that no longer applies". A 500 here leaves exactly that, and the
only signal is an error on a request whose primary effect succeeded.

Scope: make the re-derivation and the write that triggered it succeed or fail
together, or give the stale half a way to be noticed and repaired. Batching the
updates and dropping the N+1 is worth doing in the same pass, but the recovery
path is the point.

Not urgent for correctness in the common case — #246 measured 0.36s to 2.7s over
that data set — but the failure mode is silent and permanent, which is the
combination §5.1 was written to rule out.

All four callers of `reclassify` — `policies.rs:442`, `roots.rs:251`, `movies.rs:488`, `series.rs:634` — run **after** the write has already committed, and `reclassify::apply` issues one autocommitted `UPDATE` per changed row with no enclosing transaction. The walk is also N+1: one policy lookup plus one releases fetch per title. Failure scenario, at the size §5.1 now cites (2000 titles, 10 000 releases): a policy edit gets halfway, then one row's `parsed` blob fails to deserialise (`reclassify.rs:210`) or SQLite hits the busy timeout. The handler returns 500. The policy row is **already changed**, roughly half the verdicts are re-derived and half are not, and nothing else in the system re-reads that column, so the stale half stays stale permanently. §5.1's new paragraph says "The operator is never left reading a verdict computed under a policy that no longer applies". A 500 here leaves exactly that, and the only signal is an error on a request whose primary effect succeeded. Scope: make the re-derivation and the write that triggered it succeed or fail together, or give the stale half a way to be noticed and repaired. Batching the updates and dropping the N+1 is worth doing in the same pass, but the recovery path is the point. Not urgent for correctness in the common case — #246 measured 0.36s to 2.7s over that data set — but the failure mode is silent and permanent, which is the combination §5.1 was written to rule out.
naps62-yolo added this to the Feedback pass 2 milestone 2026-08-25 12:04:37 +01:00
naps62-yolo added the area/apidifficulty/hardtype/bug labels 2026-08-25 12:04:37 +01:00
naps62-yolo modified the milestone from Feedback pass 2 to Feedback pass 2 follow-ups 2026-08-25 12:16:49 +01:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: yolo/arr#247