Compare commits

...

8 Commits

Author SHA1 Message Date
Miguel Palhas 591cf27dc5 feat(web): say what a pack was abandoned for
A pack that hard-failed at import blacklisted its release, put every
episode back to missing and left the season reading 0/10, with nothing
on screen joining the two. Every fact was already recorded.

The blacklist now carries its reason out of the database: deck rows read
`blacklisted · size` instead of a bare `blacklisted`, and say whether the
policy turned the file down — relaxable for this title — or the release
itself failed, which a retry only repeats. A season whose pack was
abandoned says so on its row and above its deck, with the release name,
when it failed, and what it failed on. A row the blacklist no longer
answers for keeps rendering and claims no reason.

Two defects from the integration review of #211 sit in the same code and
are fixed here: a waived row threw away the rule it now carries and read
a bare `below policy`, and the empty-eligible count called every waived
row force-grabbable, since #211 gave those rows the rule `overridable`
reads.

Verified against a real browser: series detail, both season decks and
their buckets, at 1280 and 390 px.

Refs #227, #211
2026-08-25 12:11:46 +01:00
Miguel Palhas 58a45fc98e Merge #211: let a waiver name the rule it relaxed
Closes #211
2026-08-25 11:47:25 +01:00
Miguel Palhas a0dc07f085 feat(db): let a waiver name the rule it relaxed
`releases` forbade a rule name on anything but a rejection, so §9.3's
deck showed a bare `waived` beside rejections that each named their own,
and §5.7's "watchable but not what was asked" lost the half that says
what was not asked for. Since #210 that is the ordinary outcome of
waiving a size rejection, not a rare one.

0032 rebuilds the table with `CHECK (verdict != 'rejected' OR
rejected_rule IS NOT NULL)`, and the daemon and arr-api's
reclassification both store the waived rule. Existing rows keep NULL and
read as they do today.

`releases` is a parent — `grabs`, `movie_releases`, `episode_releases`
and `season_releases` point at it, three ON DELETE CASCADE — so the
rebuild runs `-- no-transaction` with foreign keys off around one
explicit transaction, per SQLite's own procedure. Verified against a
real database: the pre-0032 binary created and populated it, this build
migrated a copy, and every release row, child row and created_at came
through byte-identical with `PRAGMA foreign_key_check` clean.

Refs #211
2026-08-25 11:44:05 +01:00
Miguel Palhas de6c35cce3 Merge #232: plainer words for grab, blocked, deck
Closes #232
2026-08-25 11:43:55 +01:00
Miguel Palhas 9a413b10ed Merge #246: re-derive verdicts on a policy edit
Closes #246
2026-08-25 11:42:10 +01:00
Miguel Palhas bbb6d2f4a4 feat(api): re-derive verdicts on a policy edit
PUT /api/policies/{id} changed the rule every title under every root
pointing at the policy is judged by, and re-derived nothing, so §9.3's
deck and the daemon's grab gate kept reading verdicts computed under
rules that no longer existed.

Drives #241's walker from a policy id: root by root through
reclassify::root, so the skip rules and the leave-unchanged-rows-alone
rule stay in one place. A rename touches no rule and walks nothing.

Inline still holds at this width. Measured on a release build over 2000
titles and 10 000 stored releases across two roots sharing one policy:
0.36 s when no verdict moves, 2.7 s when all 10 000 do. DESIGN.md §5.1
now names four actions and carries those numbers.

just ci passed through the gate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 11:40:32 +01:00
Miguel Palhas c454b3ec11 Merge #244: normalise stored root paths
Closes #244
2026-08-25 11:29:13 +01:00
Miguel Palhas c4d4ade4da fix(api): normalise stored root paths
#243 normalised the incoming path but compared it against the value read
raw from the database, so a root stored with a trailing separator never
compared equal. Every edit of it -- a policy change included -- took the
relocation branch, where each planned destination is its own source and
the pre-check refuses. That root could not be edited at all.

`update` now normalises both sides, and hands `relocate_root` the
normalised stored value. `path_is_free` normalises the stored side in SQL
and `create` goes through it too, so `/mnt/x` and `/mnt/x/` cannot be two
roots for one directory -- the unique index compares raw strings and
cannot see that.

Migration 0031 strips the separator from rows already written. It skips
any row whose stripped form another row would also hold, rather than
tripping the unique index: a migration that cannot apply stops the daemon
booting, which is worse than two roots naming one directory.

Also from the same review: `undo` recorded only the leaf directory, so a
failed move into `/mnt/media-v2/tv/kids` left `tv` behind. It now records
every level `create_dir_all` materialised, deepest first, and still never
touches one that was already on disk.

Refs #244.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 11:28:35 +01:00
26 changed files with 2001 additions and 100 deletions
@@ -0,0 +1,98 @@
{
"db_name": "SQLite",
"query": "SELECT required_audio AS \"required_audio!: String\",\n dub_blacklist AS \"dub_blacklist!: String\",\n hdr_rules AS \"hdr_rules!: String\",\n size_bands AS \"size_bands!: String\",\n resolution_pref AS \"resolution_pref!: String\",\n source_weights AS \"source_weights!: String\",\n score_weights AS \"score_weights!: String\"\n FROM policies WHERE id = ?",
"describe": {
"columns": [
{
"name": "required_audio!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "required_audio"
}
}
},
{
"name": "dub_blacklist!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "dub_blacklist"
}
}
},
{
"name": "hdr_rules!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "hdr_rules"
}
}
},
{
"name": "size_bands!: String",
"ordinal": 3,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "size_bands"
}
}
},
{
"name": "resolution_pref!: String",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "resolution_pref"
}
}
},
{
"name": "source_weights!: String",
"ordinal": 5,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "source_weights"
}
}
},
{
"name": "score_weights!: String",
"ordinal": 6,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "score_weights"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
false,
false,
false,
false
]
},
"hash": "100e5e9297a9a917eb107c673ac492efe0df0123d9def212d84b88e57aaa0484"
}
@@ -0,0 +1,74 @@
{
"db_name": "SQLite",
"query": "SELECT g.target_id AS \"season_id!: i64\",\n r.name AS \"name!: String\",\n g.infohash AS \"infohash!: String\",\n g.failed_at,\n g.grabbed_at AS \"grabbed_at!: String\"\n FROM grabs g\n JOIN releases r ON r.id = g.release_id\n JOIN seasons s ON s.id = g.target_id\n WHERE g.target_kind = 'season'\n AND g.state = 'failed'\n AND s.series_id = ?\n AND EXISTS (\n SELECT 1 FROM episodes e\n WHERE e.season_id = s.id AND e.wanted\n AND NOT EXISTS (\n SELECT 1 FROM media_files f\n WHERE f.owner_kind = 'episode' AND f.owner_id = e.id\n )\n )\n ORDER BY coalesce(g.failed_at, g.grabbed_at), g.id",
"describe": {
"columns": [
{
"name": "season_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "grabs",
"name": "target_id"
}
}
},
{
"name": "name!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "releases",
"name": "name"
}
}
},
{
"name": "infohash!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "grabs",
"name": "infohash"
}
}
},
{
"name": "failed_at",
"ordinal": 3,
"type_info": "Text",
"origin": {
"Table": {
"table": "grabs",
"name": "failed_at"
}
}
},
{
"name": "grabbed_at!: String",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "grabs",
"name": "grabbed_at"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
true,
false
]
},
"hash": "3f252a992c1bc5b18bb4467d1c4fd4137966a469b134fbeb51fb91f67951cbbe"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT r.id AS \"id!: i64\", r.indexer_id AS \"indexer_id!: i64\", r.guid AS \"guid!: String\", r.name AS \"name!: String\", r.size AS \"size!: i64\", r.seeders, r.publish_date, r.download_url AS \"download_url!: String\", r.parsed AS \"parsed!: serde_json::Value\", r.score, r.verdict, r.rejected_rule FROM releases r JOIN season_releases sr ON sr.release_id = r.id WHERE sr.season_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id",
"query": "SELECT r.id AS \"id!: i64\", r.indexer_id AS \"indexer_id!: i64\", r.guid AS \"guid!: String\", r.name AS \"name!: String\", r.size AS \"size!: i64\", r.seeders, r.publish_date, r.download_url AS \"download_url!: String\", r.parsed AS \"parsed!: serde_json::Value\", r.score, r.verdict, r.rejected_rule, NULL AS \"blacklist_reason?: String\" FROM releases r JOIN movie_releases mr ON mr.release_id = r.id WHERE mr.movie_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id",
"describe": {
"columns": [
{
@@ -134,6 +134,12 @@
"name": "rejected_rule"
}
}
},
{
"name": "blacklist_reason?: String",
"ordinal": 12,
"type_info": "Null",
"origin": "Expression"
}
],
"parameters": {
@@ -151,8 +157,9 @@
false,
true,
true,
true,
true
]
},
"hash": "aaf6f4f7243bffa925fa17c9af3afb91c076ae3976b11cf18eb7583b8ce69a7e"
"hash": "7aa154a1bd54ec84412a1be6d7db51bb65ce9702e3bb1d26b51b93c5fdefa79f"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT r.id AS \"id!: i64\", r.indexer_id AS \"indexer_id!: i64\", r.guid AS \"guid!: String\", r.name AS \"name!: String\", r.size AS \"size!: i64\", r.seeders, r.publish_date, r.download_url AS \"download_url!: String\", r.parsed AS \"parsed!: serde_json::Value\", r.score, r.verdict, r.rejected_rule FROM releases r JOIN episode_releases er ON er.release_id = r.id WHERE er.episode_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id",
"query": "SELECT r.id AS \"id!: i64\", r.indexer_id AS \"indexer_id!: i64\", r.guid AS \"guid!: String\", r.name AS \"name!: String\", r.size AS \"size!: i64\", r.seeders, r.publish_date, r.download_url AS \"download_url!: String\", r.parsed AS \"parsed!: serde_json::Value\", r.score, r.verdict, r.rejected_rule, NULL AS \"blacklist_reason?: String\" FROM releases r JOIN episode_releases er ON er.release_id = r.id WHERE er.episode_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id",
"describe": {
"columns": [
{
@@ -134,6 +134,12 @@
"name": "rejected_rule"
}
}
},
{
"name": "blacklist_reason?: String",
"ordinal": 12,
"type_info": "Null",
"origin": "Expression"
}
],
"parameters": {
@@ -151,8 +157,9 @@
false,
true,
true,
true,
true
]
},
"hash": "f203b69afb44bfb5f906ff2e1ec13915645c35af5349032be9898a697d0d05ba"
"hash": "8b2aa810e679ddde423a5dc5a1a89e0967206e8186b2157bb4b4d11e9a136759"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT normalised_name AS \"normalised_name!: String\", infohash FROM blacklist",
"query": "SELECT normalised_name AS \"normalised_name!: String\",\n infohash,\n reason AS \"reason!: String\"\n FROM blacklist\n ORDER BY id",
"describe": {
"columns": [
{
@@ -24,6 +24,17 @@
"name": "infohash"
}
}
},
{
"name": "reason!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "blacklist",
"name": "reason"
}
}
}
],
"parameters": {
@@ -31,8 +42,9 @@
},
"nullable": [
false,
true
true,
false
]
},
"hash": "071c14544e225001d31c5da60c90d3144fc5a071893c74b2c1eea51bfe00ac97"
"hash": "939252a81cdd0103b2aaa3cf19d5cae6e6e327709dc9e5a2073092b75137e13c"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT r.id AS \"id!: i64\", r.indexer_id AS \"indexer_id!: i64\", r.guid AS \"guid!: String\", r.name AS \"name!: String\", r.size AS \"size!: i64\", r.seeders, r.publish_date, r.download_url AS \"download_url!: String\", r.parsed AS \"parsed!: serde_json::Value\", r.score, r.verdict, r.rejected_rule FROM releases r JOIN movie_releases mr ON mr.release_id = r.id WHERE mr.movie_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id",
"query": "SELECT r.id AS \"id!: i64\", r.indexer_id AS \"indexer_id!: i64\", r.guid AS \"guid!: String\", r.name AS \"name!: String\", r.size AS \"size!: i64\", r.seeders, r.publish_date, r.download_url AS \"download_url!: String\", r.parsed AS \"parsed!: serde_json::Value\", r.score, r.verdict, r.rejected_rule, NULL AS \"blacklist_reason?: String\" FROM releases r JOIN season_releases sr ON sr.release_id = r.id WHERE sr.season_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id",
"describe": {
"columns": [
{
@@ -134,6 +134,12 @@
"name": "rejected_rule"
}
}
},
{
"name": "blacklist_reason?: String",
"ordinal": 12,
"type_info": "Null",
"origin": "Expression"
}
],
"parameters": {
@@ -151,8 +157,9 @@
false,
true,
true,
true,
true
]
},
"hash": "aebbabd41e2086ac37dbd9d2151b6d54cbad525b1d1d63c2a1495c24af44db7f"
"hash": "9cb6a575aa3e0ff377551a324f839c02d0a070277a9ddf4d5749545315746389"
}
@@ -1,10 +1,10 @@
{
"db_name": "SQLite",
"query": "SELECT id FROM roots WHERE path = ? AND id <> ?",
"query": "SELECT id AS \"id!: i64\" FROM roots WHERE policy_id = ?",
"describe": {
"columns": [
{
"name": "id",
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
@@ -16,11 +16,11 @@
}
],
"parameters": {
"Right": 2
"Right": 1
},
"nullable": [
false
]
},
"hash": "fa3d1e4a6cae94780daf8fe20062a107963ba6ab2bcef2c8cfb9a1efbd905b59"
"hash": "a28caf97dfb6a7e6b69543df0693e15c746702c36a0d4068fa70d28b950e7562"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\" FROM roots\n WHERE CASE WHEN rtrim(path, '/') = '' THEN '/' ELSE rtrim(path, '/') END = ?\n AND (? IS NULL OR id <> ?)",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "roots",
"name": "id"
}
}
}
],
"parameters": {
"Right": 3
},
"nullable": [
false
]
},
"hash": "bda8991590c009ca7084fed36fae088f1b6ae12ad95f972724a417711ad37fa1"
}
+16 -7
View File
@@ -201,14 +201,23 @@ both directions — `only_4k` tightens, `allow_english_audio` loosens.
**Stored verdicts follow the effective policy.** A release's verdict is
stamped by the search that found it, and both §9.3's deck and the daemon's
manual-grab gate read that stored column. So the three operator actions that
manual-grab gate read that stored column. So the four operator actions that
change a title's effective policy — editing its overrides, moving it to a
root with a different policy, and pointing a root at a different policy
re-derive the stored verdicts of everything they touch, inline in the same
request. The operator is never left reading a verdict computed under a
policy that no longer applies. Inline is affordable even root-wide:
re-evaluation is pure and in-memory, and a row whose verdict does not move
is not rewritten, so a library-sized repoint costs reads, not writes.
root with a different policy, pointing a root at a different policy, and
editing the contents of a policy some root points at — re-derive the stored
verdicts of everything they touch, inline in the same request. The operator
is never left reading a verdict computed under a policy that no longer
applies.
The fourth is the widest: a repoint moves one library, a policy edit moves
every library sharing the policy. Inline still holds there. Re-evaluation is
pure and in-memory, a row whose verdict does not move is not rewritten, and
the ceiling is the whole database rather than something that grows with the
number of roots — roots partition titles, and a title has exactly one root.
Measured over 2000 titles and 10 000 stored releases split across two roots
sharing one policy: 0.36 s when the edit moves no verdict, 2.7 s in the
pathological case where it moves all 10 000. A rename changes no rule and
re-derives nothing.
### 5.2 Language
+61 -1
View File
@@ -76,7 +76,42 @@ pub struct Release {
pub parsed: serde_json::Value,
pub score: Option<f64>,
pub verdict: Option<String>,
// The rule behind the verdict: the one that killed a `rejected` row, or
// the one a `waived` row relaxed (#211). Null on an `eligible` row, and
// on a `waived` row stored before migration 0032, which could not record
// it. A plain comment, not a doc comment: doc comments here become
// OpenAPI descriptions and would put the generated client in web/ out of
// date, which #227 and #232 own.
pub rejected_rule: Option<String>,
// #227: what the blacklist recorded this release as failing on, when
// `rejected_rule` is `blacklisted`. Null on every other row, and on a
// blacklisted row whose blacklist entry has since gone. A size rejection
// is a policy opinion the operator can relax; a corrupt or mismatched
// release is not, and a bare `blacklisted` reads the same for both.
// Plain comment for the same reason as the field above.
pub blacklist_reason: Option<String>,
}
/// Fill in [`Release::blacklist_reason`] for every deck row the blacklist
/// holds (#227, §6.3).
///
/// The blacklist is keyed on the *normalised* name, which SQL cannot compute,
/// so the match happens here over the whole table — a handful of rows, the
/// same reasoning as [`arr_db::blacklist::Blacklist`] itself.
pub(crate) async fn attach_blacklist_reasons(
pool: &sqlx::SqlitePool,
releases: &mut [Release],
) -> Result<(), ApiError> {
if releases.is_empty() {
return Ok(());
}
let blacklist = arr_db::blacklist::Blacklist::load(pool).await?;
for release in releases.iter_mut() {
release.blacklist_reason = blacklist
.reason_for_candidate(&release.name, &release.download_url)
.map(str::to_owned);
}
Ok(())
}
/// A library file and what it cost to accept it (`DESIGN.md` §5.7).
@@ -644,7 +679,7 @@ pub async fn releases(
Path(id): Path<i64>,
) -> Result<Json<Vec<Release>>, ApiError> {
load_movie(&state, id).await?;
let mut releases = sqlx::query_as!(Release, r#"SELECT r.id AS "id!: i64", r.indexer_id AS "indexer_id!: i64", r.guid AS "guid!: String", r.name AS "name!: String", r.size AS "size!: i64", r.seeders, r.publish_date, r.download_url AS "download_url!: String", r.parsed AS "parsed!: serde_json::Value", r.score, r.verdict, r.rejected_rule FROM releases r JOIN movie_releases mr ON mr.release_id = r.id WHERE mr.movie_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id"#, id)
let mut releases = sqlx::query_as!(Release, r#"SELECT r.id AS "id!: i64", r.indexer_id AS "indexer_id!: i64", r.guid AS "guid!: String", r.name AS "name!: String", r.size AS "size!: i64", r.seeders, r.publish_date, r.download_url AS "download_url!: String", r.parsed AS "parsed!: serde_json::Value", r.score, r.verdict, r.rejected_rule, NULL AS "blacklist_reason?: String" FROM releases r JOIN movie_releases mr ON mr.release_id = r.id WHERE mr.movie_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id"#, id)
.fetch_all(pool(&state)?)
.await?;
let policy = state
@@ -656,6 +691,7 @@ pub async fn releases(
.ok_or(ApiError::NotFound)?
.policy;
rescore(&mut releases, &policy, None, 0)?;
attach_blacklist_reasons(pool(&state)?, &mut releases).await?;
Ok(Json(releases))
}
@@ -1303,6 +1339,10 @@ mod tests {
.expect("releases json");
assert_eq!(releases.len(), 1);
assert_eq!(releases[0]["verdict"], "waived");
// #211: and it says which rule the waiver relaxed, rather than
// sitting in the deck as a bare `waived` beside rejections that each
// name their own.
assert_eq!(releases[0]["rejected_rule"], "size");
}
/// #241: moving a title to a root with a different policy re-derives its
@@ -1413,6 +1453,26 @@ mod tests {
.expect("releases json");
assert_eq!(releases[0]["verdict"], "rejected");
assert_eq!(releases[0]["rejected_rule"], "blacklisted");
// #227: a row the blacklist has no entry for keeps rendering — the
// rule is all the record holds, and no reason is invented for it.
assert_eq!(releases[0]["blacklist_reason"], serde_json::Value::Null);
// With the entry, the row says what it was blacklisted for: a
// corrupt file is not the same decision as a size rejection.
arr_db::blacklist::add(pool, None, name, "no original-language audio")
.await
.expect("blacklist");
let releases: Vec<serde_json::Value> =
reqwest::get(format!("{base}/api/movies/{movie_id}/releases"))
.await
.expect("releases")
.json()
.await
.expect("releases json");
assert_eq!(
releases[0]["blacklist_reason"],
"no original-language audio"
);
}
#[tokio::test]
+332 -9
View File
@@ -173,6 +173,33 @@ struct PolicyColumns {
score_weights: String,
}
/// Everything in a policy row that a verdict depends on — the name is the
/// one column that does not.
#[derive(PartialEq, Eq)]
struct PolicyRules {
required_audio: String,
dub_blacklist: String,
hdr_rules: String,
size_bands: String,
resolution_pref: String,
source_weights: String,
score_weights: String,
}
impl PolicyColumns {
fn rules(&self) -> PolicyRules {
PolicyRules {
required_audio: self.required_audio.clone(),
dub_blacklist: self.dub_blacklist.clone(),
hdr_rules: self.hdr_rules.clone(),
size_bands: self.size_bands.clone(),
resolution_pref: self.resolution_pref.clone(),
source_weights: self.source_weights.clone(),
score_weights: self.score_weights.clone(),
}
}
}
fn column<T: serde::de::DeserializeOwned>(
column: &'static str,
value: &str,
@@ -376,6 +403,8 @@ pub async fn update(
input.validate().map_err(ApiError::Invalid)?;
let name = input.name.trim().to_owned();
let columns = input.into_columns(name)?;
let after = columns.rules();
let before = stored_rules(&state, id).await?;
let result = sqlx::query!(
r#"UPDATE policies SET
name = ?, required_audio = ?, dub_blacklist = ?, hdr_rules = ?,
@@ -405,9 +434,44 @@ pub async fn update(
if result.rows_affected() == 0 {
return Err(ApiError::PolicyNotFound);
}
// Every verdict stored under every root pointing here was reached under
// the rules this write just replaced; §9.3's deck and the grab gate both
// read them (`reclassify`). A rename leaves the rules alone, so it walks
// nothing.
if before.is_none_or(|before| before != after) {
crate::reclassify::policy(&state, id).await?;
}
Ok(Json(load_policy(&state, id).await?))
}
/// The rule columns of a policy as they stand, or `None` when there is no
/// such row. Compared against what the write is about to store, so an edit
/// that only moves the name does not re-derive a library.
async fn stored_rules(state: &AppState, id: i64) -> Result<Option<PolicyRules>, ApiError> {
let row = sqlx::query!(
r#"SELECT required_audio AS "required_audio!: String",
dub_blacklist AS "dub_blacklist!: String",
hdr_rules AS "hdr_rules!: String",
size_bands AS "size_bands!: String",
resolution_pref AS "resolution_pref!: String",
source_weights AS "source_weights!: String",
score_weights AS "score_weights!: String"
FROM policies WHERE id = ?"#,
id
)
.fetch_optional(pool(state)?)
.await?;
Ok(row.map(|row| PolicyRules {
required_audio: row.required_audio,
dub_blacklist: row.dub_blacklist,
hdr_rules: row.hdr_rules,
size_bands: row.size_bands,
resolution_pref: row.resolution_pref,
source_weights: row.source_weights,
score_weights: row.score_weights,
}))
}
#[utoipa::path(
delete, path = "/api/policies/{policy_id}", tag = "policies",
params(("policy_id" = i64, Path, description = "Policy row id")),
@@ -450,7 +514,7 @@ mod tests {
use crate::{router, Upstreams};
use axum::http::StatusCode;
async fn application() -> (tempfile::TempDir, String) {
async fn application() -> (tempfile::TempDir, AppState, String) {
let dir = tempfile::tempdir().expect("tempdir");
let database = arr_db::Db::connect(dir.path().join("arr.db"))
.await
@@ -466,9 +530,9 @@ mod tests {
.await
.expect("bind");
let address = listener.local_addr().expect("address");
let app = router(state);
let app = router(state.clone());
tokio::spawn(async move { axum::serve(listener, app).await.expect("serve") });
(dir, format!("http://{address}"))
(dir, state, format!("http://{address}"))
}
fn valid_input(name: &str) -> serde_json::Value {
@@ -501,7 +565,7 @@ mod tests {
#[tokio::test]
async fn crud_round_trips_a_policy() {
let (_dir, base) = application().await;
let (_dir, _state, base) = application().await;
let created: serde_json::Value = create(&base, valid_input("test policy"))
.await
@@ -557,7 +621,7 @@ mod tests {
#[tokio::test]
async fn a_referenced_policy_refuses_to_die() {
let (_dir, base) = application().await;
let (_dir, _state, base) = application().await;
let roots: Vec<serde_json::Value> = reqwest::get(format!("{base}/api/roots"))
.await
.expect("roots")
@@ -582,7 +646,7 @@ mod tests {
#[tokio::test]
async fn an_unknown_resolution_is_a_422_naming_the_field() {
let (_dir, base) = application().await;
let (_dir, _state, base) = application().await;
let mut payload = valid_input("bad bands");
payload["size_bands"]["1440p"] = serde_json::json!({ "floor_gib": 2, "target_gib": 6, "penalty_points_per_gib_over": 60 });
@@ -596,7 +660,7 @@ mod tests {
#[tokio::test]
async fn every_field_validates_by_name() {
let (_dir, base) = application().await;
let (_dir, _state, base) = application().await;
let with = |patch: &dyn Fn(&mut serde_json::Value)| {
let mut payload = valid_input("validation probe");
patch(&mut payload);
@@ -645,7 +709,7 @@ mod tests {
#[tokio::test]
async fn malformed_json_is_422_not_400_or_500() {
let (_dir, base) = application().await;
let (_dir, _state, base) = application().await;
let client = reqwest::Client::new();
let response = client
.post(format!("{base}/api/policies"))
@@ -667,8 +731,267 @@ mod tests {
#[tokio::test]
async fn a_duplicate_name_conflicts() {
let (_dir, base) = application().await;
let (_dir, _state, base) = application().await;
let response = create(&base, valid_input("Movies — main")).await;
assert_eq!(response.status(), StatusCode::CONFLICT);
}
async fn policy_id_named(base: &str, name: &str) -> i64 {
let policies: Vec<serde_json::Value> = reqwest::get(format!("{base}/api/policies"))
.await
.expect("policies")
.json()
.await
.expect("policies json");
policies
.iter()
.find(|policy| policy["name"] == name)
.and_then(|policy| policy["id"].as_i64())
.expect("policy id")
}
/// A movie under `root_id`, with one English 1080p release stamped
/// `verdict` as a search would have stamped it.
async fn movie_with_release(
state: &AppState,
base: &str,
root_id: i64,
tmdb_id: i64,
verdict: &str,
) -> i64 {
let movie: serde_json::Value = reqwest::Client::new()
.post(format!("{base}/api/movies"))
.json(&serde_json::json!({
"tmdb_id": tmdb_id, "title": format!("Title {tmdb_id}"), "year": 2024,
"original_language": "en", "root_id": root_id, "overrides": {}
}))
.send()
.await
.expect("create movie")
.json()
.await
.expect("movie json");
let movie_id = movie["id"].as_i64().expect("movie id");
let name = format!("Title {tmdb_id} 2024 1080p WEB-DL ENGLISH x264-GROUP");
let release_id = stamped_release(state, &name, 6_i64 * (1 << 30), verdict).await;
sqlx::query("INSERT INTO movie_releases (movie_id, release_id) VALUES (?, ?)")
.bind(movie_id)
.bind(release_id)
.execute(state.database().expect("database").pool())
.await
.expect("movie association");
movie_id
}
/// One release row, verdict stamped by hand. `waived` carries no rule,
/// which the `releases` CHECK allows — only `rejected` needs one.
async fn stamped_release(state: &AppState, name: &str, size: i64, verdict: &str) -> i64 {
let parsed = arr_parse::parse(name);
sqlx::query_scalar::<_, i64>(
"INSERT INTO releases (indexer_id, guid, name, size, seeders, download_url, parsed, score, verdict)
VALUES (7, ?, ?, ?, 40, 'url', ?, 0, ?) RETURNING id",
)
.bind(name)
.bind(name)
.bind(size)
.bind(serde_json::to_string(&parsed).expect("parsed json"))
.bind(verdict)
.fetch_one(state.database().expect("database").pool())
.await
.expect("release")
}
/// A series under `root_id` with a ten-episode season 9 pack stamped
/// `waived`. Under its own policy the pack is eligible, so the stamp
/// only survives if nothing re-derived it.
async fn series_with_waived_pack(state: &AppState, base: &str, root_id: i64) -> i64 {
let client = reqwest::Client::new();
let series: serde_json::Value = client
.post(format!("{base}/api/series"))
.json(&serde_json::json!({
"tmdb_id": 82_728, "title": "Bluey", "year": 2018,
"original_language": "en", "root_id": root_id,
"auto_track": false
}))
.send()
.await
.expect("create series")
.json()
.await
.expect("series json");
let series_id = series["id"].as_i64().expect("series id");
let episodes: Vec<serde_json::Value> = (1..=10)
.map(|number| {
serde_json::json!({
"number": number, "title": format!("Episode {number}"),
"air_date": "2025-01-01"
})
})
.collect();
let season: serde_json::Value = client
.post(format!("{base}/api/series/{series_id}/seasons"))
.json(&serde_json::json!({"number": 9, "episodes": episodes}))
.send()
.await
.expect("create season")
.json()
.await
.expect("season json");
let season_id = season["id"].as_i64().expect("season id");
// Ten episodes in 15 GiB: 1.5 GiB each, inside the 1080p band.
let release_id = stamped_release(
state,
"Bluey S09 1080p WEB-DL ENGLISH x264-GROUP",
15_i64 * (1 << 30),
"waived",
)
.await;
sqlx::query("INSERT INTO season_releases (season_id, release_id) VALUES (?, ?)")
.bind(season_id)
.bind(release_id)
.execute(state.database().expect("database").pool())
.await
.expect("season association");
series_id
}
async fn verdict_at(url: String) -> serde_json::Value {
let releases: Vec<serde_json::Value> = reqwest::get(url)
.await
.expect("releases")
.json()
.await
.expect("releases json");
releases[0]["verdict"].clone()
}
async fn movie_verdict(base: &str, movie_id: i64) -> serde_json::Value {
verdict_at(format!("{base}/api/movies/{movie_id}/releases")).await
}
async fn root_id_of(state: &AppState, kind: &str, audience: &str) -> i64 {
sqlx::query_scalar("SELECT id FROM roots WHERE kind = ? AND audience = ?")
.bind(kind)
.bind(audience)
.fetch_one(state.database().expect("database").pool())
.await
.expect("root id")
}
/// The same policy document with a different required-audio rule, so an
/// English release that was eligible becomes a waiver.
fn requires_portuguese(name: &str) -> serde_json::Value {
let mut payload = valid_input(name);
payload["required_audio"] = serde_json::json!({ "require": "any_of", "langs": ["pt-PT"] });
payload
}
/// #246: editing a policy re-derives the stored verdicts of every title
/// under every root pointing at it (§5.1) — both libraries when two
/// roots share it, and nothing under a root that does not.
#[tokio::test]
async fn editing_a_policy_rederives_every_root_that_points_at_it() {
let (_dir, state, base) = application().await;
let client = reqwest::Client::new();
let shared = policy_id_named(&base, "Movies — main").await;
let main_root = root_id_of(&state, "movie", "main").await;
let kids_root = root_id_of(&state, "movie", "kids").await;
// Two roots on one policy: the edit has two libraries to reach
// rather than one. The path stays, so nothing on disk moves.
let response = client
.put(format!("{base}/api/roots/{kids_root}"))
.json(&serde_json::json!({
"kind": "movie", "audience": "kids",
"path": "/mnt/media/movies/kids", "policy_id": shared,
}))
.send()
.await
.expect("repoint kids root");
assert_eq!(
response.status(),
StatusCode::OK,
"{:?}",
response.text().await
);
let here = movie_with_release(&state, &base, main_root, 693_134, "eligible").await;
let there = movie_with_release(&state, &base, kids_root, 27_205, "eligible").await;
// On the TV main root, a different policy, so the edit must not
// reach it. Stamped against its own policy's answer, so a walk that
// did reach it would show.
let tv_root = root_id_of(&state, "tv", "main").await;
let series_id = series_with_waived_pack(&state, &base, tv_root).await;
let response = client
.put(format!("{base}/api/policies/{shared}"))
.json(&requires_portuguese("Movies — main"))
.send()
.await
.expect("edit policy");
assert_eq!(
response.status(),
StatusCode::OK,
"{:?}",
response.text().await
);
assert_eq!(
movie_verdict(&base, here).await,
"waived",
"the first root's library re-derives"
);
assert_eq!(
movie_verdict(&base, there).await,
"waived",
"and so does the second root's, sharing the policy"
);
assert_eq!(
verdict_at(format!("{base}/api/series/{series_id}/seasons/9/releases")).await,
"waived",
"a root on another policy keeps the verdict it was stamped with"
);
}
/// Renaming a policy changes no rule, so it re-derives nothing — the
/// stamped verdict survives even though the rules would not produce it.
#[tokio::test]
async fn renaming_a_policy_leaves_verdicts_alone() {
let (_dir, state, base) = application().await;
let client = reqwest::Client::new();
let kids = policy_id_named(&base, "Movies — kids").await;
let kids_root = root_id_of(&state, "movie", "kids").await;
// Write the rules through the API once, so the rename that follows
// stores byte-identical rule columns and the only change is the name.
let response = client
.put(format!("{base}/api/policies/{kids}"))
.json(&valid_input("Movies — kids"))
.send()
.await
.expect("normalise policy");
assert_eq!(response.status(), StatusCode::OK);
// Deliberately the wrong answer: these rules make an English release
// a waiver, so a re-derivation would move this row.
let movie = movie_with_release(&state, &base, kids_root, 157_336, "eligible").await;
let response = client
.put(format!("{base}/api/policies/{kids}"))
.json(&valid_input("Movies — children"))
.send()
.await
.expect("rename policy");
assert_eq!(
response.status(),
StatusCode::OK,
"{:?}",
response.text().await
);
assert_eq!(
movie_verdict(&base, movie).await,
"eligible",
"a rename touches no rule, so it walks nothing"
);
}
}
+33 -7
View File
@@ -1,6 +1,7 @@
//! Stored verdicts, re-derived when a title's effective policy changes:
//! an overrides edit (§9.3), a move to a root with a different policy, or a
//! root pointed at a different policy (§5.1).
//! an overrides edit (§9.3), a move to a root with a different policy, a
//! root pointed at a different policy, or an edit to the contents of a
//! policy some root points at (§5.1).
//!
//! A release's verdict is stamped once, by the search that found it. Both the
//! deck and the daemon's manual-grab gate read that stored column, so an
@@ -9,7 +10,8 @@
//! reading `rejected` and the grab would be refused. A root move and a
//! policy repoint invalidate the column the same way, just wider: nothing
//! else re-reads it, so the write that changed the effective policy is the
//! only place the correction can happen.
//! only place the correction can happen. A policy edit invalidates it wider
//! still: every root pointing at that policy, not just one.
//!
//! So the rules run again here, over the releases already attached to the
//! title. This is the same correction the daemon makes when a grab turns out
@@ -176,6 +178,30 @@ pub(crate) async fn root(state: &AppState, root_id: i64) -> Result<(), ApiError>
Ok(())
}
/// Re-evaluate every release of every title under every root that points at
/// one policy, for a `PUT /api/policies/{id}` that changed its rules (§5.1).
///
/// Root by root through [`root`], which is title by title through [`movie`]
/// and [`series`]: one walker, one set of skip rules, one place that decides
/// a row does not need rewriting.
///
/// This is the widest of the four re-derivations — a root repoint moves one
/// library, a policy edit moves every library sharing the policy — but the
/// ceiling is the whole database rather than something that grows with it,
/// since roots partition titles and a title has exactly one root.
pub(crate) async fn policy(state: &AppState, policy_id: i64) -> Result<(), ApiError> {
let root_ids = sqlx::query_scalar!(
r#"SELECT id AS "id!: i64" FROM roots WHERE policy_id = ?"#,
policy_id
)
.fetch_all(pool(state)?)
.await?;
for root_id in root_ids {
root(state, root_id).await?;
}
Ok(())
}
#[allow(clippy::too_many_arguments)]
async fn apply(
state: &AppState,
@@ -208,11 +234,11 @@ async fn apply(
episodes,
runtime_minutes,
);
// `releases` allows a rule name only on a rejected row
// (`CHECK ((verdict = 'rejected') = (rejected_rule IS NOT NULL))`),
// which is also how the daemon writes a waiver.
// A waiver records the rule it relaxed, the same as a rejection
// (#211). Migration 0032 relaxed `releases` to
// `CHECK (verdict != 'rejected' OR rejected_rule IS NOT NULL)` so it
// can, and the daemon writes waivers the same way.
let (verdict, rule) = verdict_columns(&evaluation.verdict);
let rule = if verdict == "rejected" { rule } else { None };
if release.verdict.as_deref() == Some(verdict) && release.rejected_rule == rule {
continue;
}
+46 -25
View File
@@ -50,11 +50,14 @@ struct PlannedRename {
pub(crate) struct Relocation {
performed: Vec<PlannedRename>,
rewrites: Vec<(i64, String)>,
/// The new root directory, set only when this request created it (it did
/// not already exist). [`Self::undo`] removes it, so it is cleaned up
/// exactly when the move it was created for does not complete; a move
/// that commits never calls `undo` and the directory stays.
created_root: Option<PathBuf>,
/// Every directory level this request materialised for the new root,
/// deepest first. `create_dir_all` can make more than one — moving a
/// root to `/mnt/media-v2/tv/kids` when `/mnt/media-v2` is all that
/// exists creates both `tv` and `kids` — and [`Self::undo`] removes all
/// of them, so a failed move leaves nothing behind (#244). Levels that
/// were already on disk are never in this list and are never touched. A
/// move that commits never calls `undo` and the directories stay.
created_dirs: Vec<PathBuf>,
}
/// Whether the destination root is a directory that must already be there.
@@ -179,20 +182,18 @@ async fn relocate_files(
}
}
// Tracked only when this call is the one that created the directory, so
// a failed move can remove it again without ever touching a root path
// that already existed on disk.
let mut created_root: Option<PathBuf> = None;
// Tracked only for the levels this call is the one to create, so a
// failed move can remove them again without ever touching a directory
// that already existed on disk. Recorded before `create_dir_all`, since
// afterwards there is no way to tell which levels it made.
let mut created_dirs: Vec<PathBuf> = Vec::new();
if destination == Destination::Create && !renames.is_empty() {
let already_there = tokio::fs::symlink_metadata(new_root).await.is_ok();
created_dirs = missing_levels(std::path::Path::new(new_root)).await;
if let Err(error) = tokio::fs::create_dir_all(new_root).await {
return Err(ApiError::Filesystem(format!(
"could not create '{new_root}': {error}"
)));
}
if !already_there {
created_root = Some(PathBuf::from(new_root));
}
}
let mut performed: Vec<PlannedRename> = Vec::new();
@@ -209,7 +210,7 @@ async fn relocate_files(
);
continue;
}
Err(error) => return Err(failed(&rename, &error, performed, created_root).await),
Err(error) => return Err(failed(&rename, &error, performed, created_dirs).await),
}
match tokio::fs::rename(&rename.source, &rename.destination).await {
Ok(()) => {
@@ -220,31 +221,47 @@ async fn relocate_files(
);
performed.push(rename);
}
Err(error) => return Err(failed(&rename, &error, performed, created_root).await),
Err(error) => return Err(failed(&rename, &error, performed, created_dirs).await),
}
}
Ok(Relocation {
performed,
rewrites,
created_root,
created_dirs,
})
}
/// The levels of `path` that are not on disk, deepest first — exactly what a
/// following `create_dir_all` will materialise. The walk stops at the first
/// ancestor that exists, so nothing already there is ever listed.
async fn missing_levels(path: &std::path::Path) -> Vec<PathBuf> {
let mut missing = Vec::new();
for ancestor in path.ancestors() {
// `ancestors` ends in an empty path for a relative input; there is
// no level above that to create.
if ancestor.as_os_str().is_empty() || tokio::fs::symlink_metadata(ancestor).await.is_ok() {
break;
}
missing.push(ancestor.to_path_buf());
}
missing
}
/// One rename failed: move back everything that had already moved, remove
/// the new root if this request is the one that created it, and name the
/// folder that stopped the move so the operator knows which title to look at
/// before retrying.
/// every directory level this request created, and name the folder that
/// stopped the move so the operator knows which title to look at before
/// retrying.
async fn failed(
rename: &PlannedRename,
error: &std::io::Error,
performed: Vec<PlannedRename>,
created_root: Option<PathBuf>,
created_dirs: Vec<PathBuf>,
) -> ApiError {
Relocation {
performed,
rewrites: Vec::new(),
created_root,
created_dirs,
}
.undo()
.await;
@@ -297,13 +314,17 @@ impl Relocation {
// `remove_dir` rather than `remove_dir_all`: it only succeeds on an
// empty directory, so anything unexpected left inside it — this
// request's own undo failing, say — is a reason to leave it alone.
if let Some(root) = &self.created_root {
if let Err(error) = tokio::fs::remove_dir(root).await {
// Deepest first, since a parent cannot go while its child is there;
// the first level that will not go stops the walk, because every
// level above it now has content and refusing is the right answer.
for directory in &self.created_dirs {
if let Err(error) = tokio::fs::remove_dir(directory).await {
tracing::warn!(
path = %root.display(),
path = %directory.display(),
%error,
"could not remove the directory created for a move that did not complete"
"could not remove a directory created for a move that did not complete"
);
break;
}
}
}
+228 -9
View File
@@ -152,6 +152,7 @@ pub async fn create(
input.validate().map_err(ApiError::Invalid)?;
input.policy_exists(&state).await?;
let path = normalize_path(&input.path);
path_is_free(&state, None, &path).await?;
let result = sqlx::query!(
"INSERT INTO roots (kind, audience, path, policy_id) VALUES (?, ?, ?, ?)",
input.kind,
@@ -188,17 +189,25 @@ pub async fn update(
input.policy_exists(&state).await?;
let current = load_root(&state, id).await?;
let path = normalize_path(&input.path);
// #244: rows written before #243 can hold a trailing separator, so the
// stored value is normalised too. Comparing a normalised payload against
// a raw stored value means the row can never compare equal: every edit,
// policy changes included, takes the relocation branch, and there each
// planned destination is its own source. The normalised value is what
// `relocate_root` gets as well, so no planned path carries a doubled
// separator.
let current_path = normalize_path(&current.path);
// A path change moves every §7.4 title folder under this root with the
// row (issue #236), the same way changing a title's root moves one
// (#228). Disk first, row second: a failed rename leaves the root row
// alone, so the operator sees the library where its files actually are
// and can retry. A path already taken is refused before any of it, since
// the write would fail afterwards anyway.
let relocation = if path == current.path {
let relocation = if path == current_path {
None
} else {
path_is_free(&state, id, &path).await?;
Some(crate::relocate::relocate_root(&state, id, &current.path, &path).await?)
path_is_free(&state, Some(id), &path).await?;
Some(crate::relocate::relocate_root(&state, id, &current_path, &path).await?)
};
let mut transaction = pool(&state)?.begin().await?;
let written: Result<(), sqlx::Error> = async {
@@ -245,12 +254,25 @@ pub async fn update(
}
/// The unique index on `path` would catch this after the move; catching it
/// first keeps a doomed write from touching the disk at all.
async fn path_is_free(state: &AppState, id: i64, path: &str) -> Result<(), ApiError> {
let taken: Option<i64> =
sqlx::query_scalar!("SELECT id FROM roots WHERE path = ? AND id <> ?", path, id)
.fetch_optional(pool(state)?)
.await?;
/// first keeps a doomed write from touching the disk at all. `except` is the
/// row being updated, or `None` when creating.
///
/// The stored side is normalised in SQL, mirroring [`normalize_path`], so
/// `/mnt/media/x` and `/mnt/media/x/` cannot be two roots for one directory
/// (#244). The unique index cannot see that — it compares the raw strings —
/// and migration 0031 leaves any pair that already collides alone rather
/// than failing to apply, so such a row can still be on disk.
async fn path_is_free(state: &AppState, except: Option<i64>, path: &str) -> Result<(), ApiError> {
let taken: Option<i64> = sqlx::query_scalar!(
r#"SELECT id AS "id!: i64" FROM roots
WHERE CASE WHEN rtrim(path, '/') = '' THEN '/' ELSE rtrim(path, '/') END = ?
AND (? IS NULL OR id <> ?)"#,
path,
except,
except
)
.fetch_optional(pool(state)?)
.await?;
if taken.is_some() {
return Err(ApiError::Conflict(
"a root with this path already exists".into(),
@@ -749,6 +771,17 @@ mod tests {
assert_eq!(response.status(), StatusCode::NO_CONTENT);
}
/// A path written straight into the row, no normalisation — how a root
/// created before #243 could end up holding a trailing separator.
async fn point_root_at_raw(state: &AppState, root_id: i64, path: &str) {
sqlx::query("UPDATE roots SET path = ? WHERE id = ?")
.bind(path)
.bind(root_id)
.execute(state.database().expect("database").pool())
.await
.expect("store the raw path");
}
async fn point_root_at(state: &AppState, root_id: i64, path: &std::path::Path) {
sqlx::query("UPDATE roots SET path = ? WHERE id = ?")
.bind(path.to_str().expect("utf-8 root"))
@@ -1257,6 +1290,192 @@ mod tests {
);
}
/// Issue #244: a root *stored* with a trailing separator — creatable
/// through the API at any point before #243 — could not be edited at
/// all. The payload was normalised and the stored value was not, so no
/// payload compared equal: every edit took the relocation branch, where
/// every planned destination is its own source and the pre-check 409s.
#[tokio::test]
async fn a_root_stored_with_a_trailing_separator_can_still_be_edited() {
let (_dir, state, base) = application().await;
let old = tempfile::tempdir().expect("old root");
let unslashed = old.path().to_str().expect("utf-8").to_owned();
point_root_at_raw(&state, 1, &format!("{unslashed}/")).await;
let id = add_movie(&base, 100, "Dune", 1).await;
let folder = library_folder(&state, id, old.path(), "Dune").await;
// What the settings view sends back: the path exactly as stored,
// separator included, with only the policy changed.
let kids_policy = policy_id_named(&base, "Movies — kids").await;
let mut payload = root_payload(&base, 1, &format!("{unslashed}/")).await;
payload["policy_id"] = kids_policy.into();
let response = reqwest::Client::new()
.put(format!("{base}/api/roots/1"))
.json(&payload)
.send()
.await
.expect("edit the root");
assert_eq!(
response.status(),
StatusCode::OK,
"{:?}",
response.text().await
);
assert!(folder.join("feature.mkv").exists(), "nothing on disk moved");
assert_eq!(
stored_path(&base, 1).await,
unslashed,
"the row is left normalised, so the next edit compares equal too"
);
// And the same payload without the separator is not a relocation
// either.
let response = reqwest::Client::new()
.put(format!("{base}/api/roots/1"))
.json(&root_payload(&base, 1, &unslashed).await)
.send()
.await
.expect("edit the root again");
assert_eq!(response.status(), StatusCode::OK);
assert!(folder.join("feature.mkv").exists(), "still nothing moved");
}
/// Issue #244: a real path change from a root stored with a trailing
/// separator plans from the normalised value, so neither a destination
/// nor a rewritten row carries a doubled separator.
#[tokio::test]
async fn relocating_a_slash_stored_root_plans_no_doubled_separator() {
let (_dir, state, base) = application().await;
let old = tempfile::tempdir().expect("old root");
let home = tempfile::tempdir().expect("home of the new path");
let new = home.path().join("relocated-main");
let unslashed = old.path().to_str().expect("utf-8").to_owned();
point_root_at_raw(&state, 1, &format!("{unslashed}/")).await;
let id = add_movie(&base, 100, "Dune", 1).await;
let folder = library_folder(&state, id, old.path(), "Dune").await;
let response = reqwest::Client::new()
.put(format!("{base}/api/roots/1"))
.json(&root_payload(&base, 1, new.to_str().expect("utf-8")).await)
.send()
.await
.expect("move the root");
assert_eq!(
response.status(),
StatusCode::OK,
"{:?}",
response.text().await
);
assert!(!folder.exists(), "the folder left the old path");
assert!(new.join("Dune").join("feature.mkv").exists());
let paths = file_paths(&state).await;
assert!(
!paths[0].contains("//"),
"no doubled separator in the rewritten row: {}",
paths[0]
);
assert!(
std::path::Path::new(&paths[0]).exists(),
"the rewritten path describes the disk: {}",
paths[0]
);
assert_eq!(stored_path(&base, 1).await, new.to_str().expect("utf-8"));
}
/// Issue #244: `create_dir_all` can materialise more than one level for
/// a root path pointed somewhere fresh. A failed move removes every
/// level it created, not only the leaf — and still nothing that was
/// already on disk.
#[cfg(unix)]
#[tokio::test]
async fn a_failed_move_removes_every_level_it_created() {
use std::os::unix::fs::PermissionsExt;
let (_dir, state, base) = application().await;
let old = tempfile::tempdir().expect("old root");
let home = tempfile::tempdir().expect("home of the new path");
// Three levels below a directory that is already there.
let top = home.path().join("media-v2");
let new = top.join("tv").join("kids");
point_root_at(&state, 1, old.path()).await;
let id = add_movie(&base, 100, "Dune", 1).await;
let stuck = library_folder(&state, id, old.path(), "Dune").await;
// Moving a directory to another parent rewrites its `..`, which
// needs write permission on the directory itself.
tokio::fs::set_permissions(&stuck, std::fs::Permissions::from_mode(0o555))
.await
.expect("freeze the title folder");
let response = reqwest::Client::new()
.put(format!("{base}/api/roots/1"))
.json(&root_payload(&base, 1, new.to_str().expect("utf-8")).await)
.send()
.await
.expect("move the root");
assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR);
tokio::fs::set_permissions(&stuck, std::fs::Permissions::from_mode(0o755))
.await
.expect("thaw the title folder");
assert!(
!top.exists(),
"every level the failed move created is gone, not only the leaf"
);
assert!(
home.path().exists(),
"the level that was already there is left alone"
);
assert!(
stuck.join("feature.mkv").exists(),
"the folder is still where the row says it is"
);
assert_eq!(
stored_path(&base, 1).await,
old.path().to_str().expect("utf-8")
);
}
/// Issue #244: two roots naming one directory, differing only by a
/// trailing separator, are not two roots. The unique index compares raw
/// strings and cannot see it, so the check normalises both sides.
#[tokio::test]
async fn a_slash_stored_path_is_not_free_for_another_root() {
let (_dir, state, base) = application().await;
point_root_at_raw(&state, 2, "/mnt/media/movies/archive/").await;
// An update onto the stripped form of a path another root holds.
let response = reqwest::Client::new()
.put(format!("{base}/api/roots/1"))
.json(&root_payload(&base, 1, "/mnt/media/movies/archive").await)
.send()
.await
.expect("update onto the other root's path");
assert_eq!(response.status(), StatusCode::CONFLICT);
// And a create, which the unique index would have let through.
let policy_ids = first_policy_ids(&base).await;
let mut payload = root_input(policy_ids[0]);
payload["kind"] = serde_json::json!("tv");
payload["audience"] = serde_json::json!("main");
payload["path"] = serde_json::json!("/mnt/media/movies/archive");
let deleted = reqwest::Client::new()
.delete(format!("{base}/api/roots/3"))
.send()
.await
.expect("free the (tv, main) pair");
assert_eq!(deleted.status(), StatusCode::NO_CONTENT);
let response = reqwest::Client::new()
.post(format!("{base}/api/roots"))
.json(&payload)
.send()
.await
.expect("create onto the other root's path");
assert_eq!(response.status(), StatusCode::CONFLICT);
}
/// Issue #236: a path another root already holds is refused before the
/// disk is touched at all.
#[tokio::test]
+10
View File
@@ -156,6 +156,10 @@ pub struct ClassifiedRelease {
pub score_terms: ScoreTerms,
pub verdict: String,
pub rule: Option<String>,
/// What the blacklist recorded this release as failing on (#227, §6.3),
/// when `rule` is `blacklisted`. `None` on every other row. A policy
/// rejection and a bad release both read as `blacklisted` without it.
pub blacklist_reason: Option<String>,
}
#[utoipa::path(
@@ -730,6 +734,11 @@ fn classify(
episodes,
runtime_minutes,
);
// #227: the reason the blacklist holds is what tells a policy rejection
// the operator can relax from a release that should never be retried.
let blacklist_reason = blacklist
.reason_for_candidate(&release.name, &release.download_url)
.map(str::to_owned);
let (verdict, rule) = if blacklist.blocks_candidate(&release.name, &release.download_url) {
("rejected", Some(blacklist::RULE.to_owned()))
} else {
@@ -781,6 +790,7 @@ fn classify(
score_terms: terms,
verdict: verdict.to_owned(),
rule,
blacklist_reason,
})
}
+235 -6
View File
@@ -29,7 +29,9 @@ use chrono::{DateTime, NaiveDate, NaiveTime, Utc};
use serde::{Deserialize, Serialize};
use utoipa::{IntoParams, ToSchema};
use crate::movies::{pool, rescore, Accepted, ApiError, ErrorBody, Release};
use crate::movies::{
attach_blacklist_reasons, pool, rescore, Accepted, ApiError, ErrorBody, Release,
};
use crate::owners::Owner;
use crate::search::tmdb_client;
use crate::state::{AppState, EpisodeCommand, MetadataCommand, SeasonCommand};
@@ -110,6 +112,10 @@ pub struct Season {
/// which is why the row still exists. A conflict for the operator to
/// resolve; nothing was deleted from disk.
pub vanished: bool,
/// #227. The last season pack that downloaded in full and was condemned
/// at import, while the season is still waiting for a file. `None` when
/// no pack was abandoned, or when the gap has since been filled.
pub import_failure: Option<ImportFailure>,
pub episodes: Vec<Episode>,
}
@@ -1039,6 +1045,8 @@ async fn load_seasons(state: &AppState, series_id: i64) -> Result<Vec<Season>, A
.fetch_all(pool(state)?)
.await?;
let mut failures = season_import_failures(state, series_id).await?;
Ok(seasons
.into_iter()
.map(|season| Season {
@@ -1047,6 +1055,7 @@ async fn load_seasons(state: &AppState, series_id: i64) -> Result<Vec<Season>, A
number: season.number,
tracked: season.tracked,
vanished: season.vanished,
import_failure: failures.remove(&season.id),
episodes: episodes
.iter()
.filter(|episode| episode.season_id == season.id)
@@ -1416,7 +1425,7 @@ pub async fn episode_releases(
Path(id): Path<i64>,
) -> Result<Json<Vec<Release>>, ApiError> {
let episode = load_episode(&state, id).await?;
let mut releases = sqlx::query_as!(Release, r#"SELECT r.id AS "id!: i64", r.indexer_id AS "indexer_id!: i64", r.guid AS "guid!: String", r.name AS "name!: String", r.size AS "size!: i64", r.seeders, r.publish_date, r.download_url AS "download_url!: String", r.parsed AS "parsed!: serde_json::Value", r.score, r.verdict, r.rejected_rule FROM releases r JOIN episode_releases er ON er.release_id = r.id WHERE er.episode_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id"#, id)
let mut releases = sqlx::query_as!(Release, r#"SELECT r.id AS "id!: i64", r.indexer_id AS "indexer_id!: i64", r.guid AS "guid!: String", r.name AS "name!: String", r.size AS "size!: i64", r.seeders, r.publish_date, r.download_url AS "download_url!: String", r.parsed AS "parsed!: serde_json::Value", r.score, r.verdict, r.rejected_rule, NULL AS "blacklist_reason?: String" FROM releases r JOIN episode_releases er ON er.release_id = r.id WHERE er.episode_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id"#, id)
.fetch_all(pool(&state)?)
.await?;
let policy = state
@@ -1430,6 +1439,7 @@ pub async fn episode_releases(
let lengths = season_lengths(&state, episode.series_id).await?;
let runtime = series_runtime(&state, episode.series_id).await?;
rescore(&mut releases, &policy, Some(&lengths), runtime)?;
attach_blacklist_reasons(pool(&state)?, &mut releases).await?;
Ok(Json(releases))
}
@@ -1571,7 +1581,7 @@ pub async fn season_releases(
) -> Result<Json<Vec<Release>>, ApiError> {
load_series_row(&state, series_id).await?;
let season_id = load_season_id(&state, series_id, number).await?;
let mut releases = sqlx::query_as!(Release, r#"SELECT r.id AS "id!: i64", r.indexer_id AS "indexer_id!: i64", r.guid AS "guid!: String", r.name AS "name!: String", r.size AS "size!: i64", r.seeders, r.publish_date, r.download_url AS "download_url!: String", r.parsed AS "parsed!: serde_json::Value", r.score, r.verdict, r.rejected_rule FROM releases r JOIN season_releases sr ON sr.release_id = r.id WHERE sr.season_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id"#, season_id)
let mut releases = sqlx::query_as!(Release, r#"SELECT r.id AS "id!: i64", r.indexer_id AS "indexer_id!: i64", r.guid AS "guid!: String", r.name AS "name!: String", r.size AS "size!: i64", r.seeders, r.publish_date, r.download_url AS "download_url!: String", r.parsed AS "parsed!: serde_json::Value", r.score, r.verdict, r.rejected_rule, NULL AS "blacklist_reason?: String" FROM releases r JOIN season_releases sr ON sr.release_id = r.id WHERE sr.season_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id"#, season_id)
.fetch_all(pool(&state)?)
.await?;
let policy = state
@@ -1585,9 +1595,92 @@ pub async fn season_releases(
let lengths = season_lengths(&state, series_id).await?;
let runtime = series_runtime(&state, series_id).await?;
rescore(&mut releases, &policy, Some(&lengths), runtime)?;
attach_blacklist_reasons(pool(&state)?, &mut releases).await?;
Ok(Json(releases))
}
/// A grab for this season that downloaded in full and was then condemned at
/// import (#227, §5.7).
///
/// The torrent stays at 100% in Transmission — §7.3 leaves that lifecycle to
/// the reaper — the release is blacklisted and every episode it was covering
/// reopens as a gap. Nothing on screen connected the two, so the season read
/// `0/10` as though no grab had ever been tried. Every fact is already
/// recorded; this is the row that carries them out.
#[derive(Debug, Clone, Serialize, ToSchema)]
pub struct ImportFailure {
/// The blacklisted release name, as the indexer spelled it.
pub release: String,
/// What the blacklist recorded it as failing on — a policy rule name the
/// operator can relax, or a sentence about the release itself. `None`
/// when no blacklist row answers to either key, which is possible after a
/// blacklist row is cleared by hand; the failure is still true.
pub reason: Option<String>,
/// When the grab entered `failed`, RFC3339. `None` on a row that failed
/// before migration 0030 gave the column a value.
pub failed_at: Option<String>,
}
/// The most recent abandoned pack per season of one series (#227).
///
/// Only seasons still waiting for a file are answered: once the gap is
/// filled, the failure is history and the season has nothing to explain. The
/// blacklist reason is matched in memory because its name key is normalised,
/// which SQL cannot compute.
async fn season_import_failures(
state: &AppState,
series_id: i64,
) -> Result<HashMap<i64, ImportFailure>, ApiError> {
let pool = pool(state)?;
let rows = sqlx::query!(
r#"SELECT g.target_id AS "season_id!: i64",
r.name AS "name!: String",
g.infohash AS "infohash!: String",
g.failed_at,
g.grabbed_at AS "grabbed_at!: String"
FROM grabs g
JOIN releases r ON r.id = g.release_id
JOIN seasons s ON s.id = g.target_id
WHERE g.target_kind = 'season'
AND g.state = 'failed'
AND s.series_id = ?
AND EXISTS (
SELECT 1 FROM episodes e
WHERE e.season_id = s.id AND e.wanted
AND NOT EXISTS (
SELECT 1 FROM media_files f
WHERE f.owner_kind = 'episode' AND f.owner_id = e.id
)
)
ORDER BY coalesce(g.failed_at, g.grabbed_at), g.id"#,
series_id
)
.fetch_all(pool)
.await?;
if rows.is_empty() {
return Ok(HashMap::new());
}
let blacklist = arr_db::blacklist::Blacklist::load(pool).await?;
// Ascending order, so the last row written for a season wins.
Ok(rows
.into_iter()
.map(|row| {
let reason = blacklist
.reason_for_infohash(&row.infohash)
.or_else(|| blacklist.reason_for_name(&row.name))
.map(str::to_owned);
(
row.season_id,
ImportFailure {
release: row.name,
reason,
failed_at: row.failed_at,
},
)
})
.collect())
}
/// Which lane a season's missing episodes take (#182, §6.2).
#[derive(Debug, Clone, Copy, Serialize, ToSchema)]
#[serde(rename_all = "snake_case")]
@@ -1629,6 +1722,11 @@ pub struct SeasonPackState {
/// When a pack sweep for this season last completed. `None` means no
/// pack search has ever run, so an empty deck is pending, not settled.
pub last_pack_search_at: Option<String>,
/// #227. The last pack that downloaded in full and was then condemned at
/// import, while the season is still waiting for a file. A deck that
/// cannot say this leaves the season reading as though nothing was ever
/// tried, with the torrent still sitting at 100% in Transmission.
pub import_failure: Option<ImportFailure>,
}
#[utoipa::path(
@@ -1720,6 +1818,9 @@ pub async fn season_pack_state(
pack_failures: failed.failures,
pack_retry_at: pack_retry_at.map(|at| at.to_rfc3339()),
last_pack_search_at,
import_failure: season_import_failures(&state, series_id)
.await?
.remove(&season_id),
}))
}
@@ -1939,6 +2040,133 @@ mod tests {
response.json().await.expect("season json")
}
async fn seasons_json(base: &str, series_id: i64) -> Vec<serde_json::Value> {
reqwest::get(format!("{base}/api/series/{series_id}/seasons"))
.await
.expect("seasons")
.json()
.await
.expect("seasons json")
}
/// #227, the operator's own report: a pack downloaded in full, §5.7
/// condemned it at import, every episode went back to `missing`, and the
/// season read `0/10` with nothing anywhere saying a grab had been tried.
/// The season row, its deck and the deck's blacklisted release now each
/// carry the failure and the reason it failed on.
#[tokio::test]
async fn an_abandoned_pack_is_visible_on_the_season_and_its_deck() {
let (_dir, state, base) = application().await;
let root_id = tv_root(&state, "main").await;
let series = add_series(&base, root_id, false).await;
let series_id = series["id"].as_i64().expect("id");
let episodes: Vec<serde_json::Value> = (1..=10)
.map(|number| {
serde_json::json!({
"number": number, "title": format!("Episode {number}"),
"air_date": "2025-01-01"
})
})
.collect();
let season = add_season(&base, series_id, 8, serde_json::json!(episodes)).await;
let season_id = season["id"].as_i64().expect("season id");
assert!(
season["import_failure"].is_null(),
"nothing has been grabbed yet"
);
// §4.1: tracking the season is what makes its episodes wanted, and a
// season with no intent has no gap to explain.
let tracked = reqwest::Client::new()
.patch(format!("{base}/api/series/{series_id}/seasons/8"))
.json(&serde_json::json!({"tracked": true}))
.send()
.await
.expect("track the season");
assert_eq!(tracked.status(), StatusCode::OK);
let pool = state.database().expect("database").pool();
let name = "Rick.And.Morty.S08.1080p.WEB-DL.x264-GROUP";
let parsed = arr_parse::parse(name);
let release_id = sqlx::query_scalar::<_, i64>(
"INSERT INTO releases (indexer_id, guid, name, size, seeders, download_url, parsed, score, verdict)
VALUES (7, 'pack', ?, 1000, 50, 'url', ?, 0, 'eligible') RETURNING id",
)
.bind(name)
.bind(serde_json::to_string(&parsed).expect("parsed json"))
.fetch_one(pool)
.await
.expect("release");
sqlx::query("INSERT INTO season_releases (season_id, release_id) VALUES (?, ?)")
.bind(season_id)
.bind(release_id)
.execute(pool)
.await
.expect("association");
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash, state, grabbed_at, failed_at)
VALUES (?, 'season', ?, 'abc123', 'failed', '2026-01-01T00:00:00.000Z', '2026-01-02T00:00:00.000Z')",
)
.bind(release_id)
.bind(season_id)
.execute(pool)
.await
.expect("failed pack grab");
// The failure with no blacklist row yet: still a failure, and the
// reason is simply not known. Rows written before the blacklist
// carried one read this way and must keep rendering.
let seasons = seasons_json(&base, series_id).await;
let row = &seasons[0]["import_failure"];
assert_eq!(row["release"], name);
assert_eq!(row["reason"], serde_json::Value::Null);
assert_eq!(row["failed_at"], "2026-01-02T00:00:00.000Z");
arr_db::blacklist::add(pool, Some("ABC123"), name, "size")
.await
.expect("blacklist");
let seasons = seasons_json(&base, series_id).await;
assert_eq!(seasons[0]["import_failure"]["reason"], "size");
// The deck the season row leads to says the same thing.
let pack_state: serde_json::Value = reqwest::get(format!(
"{base}/api/series/{series_id}/seasons/8/pack-state"
))
.await
.expect("pack state")
.json()
.await
.expect("pack state json");
assert_eq!(pack_state["import_failure"]["release"], name);
assert_eq!(pack_state["import_failure"]["reason"], "size");
// §6.3: the release itself is rejected in the deck, and now names
// what it was blacklisted for — a size rejection the operator can
// relax, not a corrupt file they should leave alone.
let releases: Vec<serde_json::Value> =
reqwest::get(format!("{base}/api/series/{series_id}/seasons/8/releases"))
.await
.expect("releases")
.json()
.await
.expect("releases json");
assert_eq!(releases.len(), 1);
assert_eq!(releases[0]["blacklist_reason"], "size");
// Once the gap is filled the failure is history: the season has
// nothing left to explain and stops saying it.
for episode in seasons[0]["episodes"].as_array().expect("episodes") {
sqlx::query("INSERT INTO media_files (owner_kind, owner_id, path, size) VALUES ('episode', ?, ?, 1)")
.bind(episode["id"].as_i64().expect("episode id"))
.bind(format!("/library/e{}.mkv", episode["number"]))
.execute(pool)
.await
.expect("file on disk");
}
let seasons = seasons_json(&base, series_id).await;
assert!(seasons[0]["import_failure"].is_null());
}
/// The production case behind #210: every pack of a season is under
/// §5.5's per-episode floor, so the deck holds three candidates and
/// nothing is grabbable. Writing `allow_below_floor` turns the
@@ -2014,9 +2242,10 @@ mod tests {
releases[0]["verdict"], "waived",
"a waived grab stays a waiver; nothing here makes it eligible"
);
// The row's rule name goes with the rejection; what survives is the
// dashed `waived` verdict the deck reads (§9.3).
assert!(releases[0]["rejected_rule"].is_null());
// #211: the waiver keeps the rule it relaxed, so the deck names it
// the way it names a rejection (§9.3) instead of showing a bare
// `waived`.
assert_eq!(releases[0]["rejected_rule"], "size");
// The grab the deck's one click sends is now accepted.
let response = reqwest::Client::new()
+24 -1
View File
@@ -1534,10 +1534,17 @@ fn search_query(movie: &PendingMovie) -> String {
)
}
/// The `verdict` and `rejected_rule` columns for a verdict.
///
/// A waiver names the rule it relaxed (#211). §5.7 calls a soft fail
/// "watchable but not what was asked", and which rule was relaxed is the
/// whole content of that sentence, so §9.3's deck can name it the way it
/// names a rejection. Rows written before 0032 hold `NULL` there and stay
/// readable.
fn verdict_columns(verdict: &Verdict) -> (&'static str, Option<String>) {
match verdict {
Verdict::Eligible => ("eligible", None),
Verdict::Waived(_) => ("waived", None),
Verdict::Waived(rule) => ("waived", Some(rule.name())),
Verdict::Rejected(rule) => ("rejected", Some(rule.name())),
}
}
@@ -2029,6 +2036,22 @@ mod tests {
);
}
/// #211: a waiver names the rule it relaxed, the same as a rejection, so
/// §9.3's deck shows what was given up instead of a bare `waived`.
/// Migration 0032 relaxed the constraint that forbade it.
#[test]
fn a_waiver_records_the_rule_it_relaxed() {
assert_eq!(
verdict_columns(&Verdict::Waived(arr_core::Rule::Size)),
("waived", Some("size".to_owned()))
);
assert_eq!(verdict_columns(&Verdict::Eligible), ("eligible", None));
assert_eq!(
verdict_columns(&Verdict::Rejected(arr_core::Rule::RequiredAudio)),
("rejected", Some("required_audio".to_owned()))
);
}
/// Every candidate is cached with its verdict, which is what the manual
/// search view and the attention queues read (§9.3).
#[tokio::test]
@@ -0,0 +1,34 @@
-- Issue #244. A root's path could be stored with a trailing separator until
-- #243 normalised the incoming value. `roots::update` normalises the payload
-- and compares it against the stored value, so such a row never compares
-- equal: every edit, a policy change included, takes the relocation branch,
-- and there every planned destination is its own source. Normalising the
-- payload alone fixed the half that cannot bite; this is the other half.
--
-- `rtrim` strips every trailing separator at once, so '/mnt/x//' normalises
-- in one pass. A bare '/' rtrims to the empty string and is put back, which
-- is what `normalize_path` in arr-api does.
--
-- Guarded, because `roots.path` is UNIQUE and a migration that cannot apply
-- stops the daemon booting -- worse than the bug it fixes. A row is
-- normalised only when no other row shares its normalised path: neither a
-- row already holding the stripped value, nor another trailing-separator row
-- that would strip to the same thing. Every row in such a group is left
-- exactly as it is. That leaves two roots naming one directory, which is a
-- settings mistake for the operator to resolve by hand, not a reason to
-- refuse to boot.
--
-- This cannot introduce a collision either. An updated row's new value is a
-- normalised path no other row normalises to, and a row left alone whose raw
-- path equalled that value would have had the same normalised path, which is
-- the case the guard excludes.
UPDATE roots
SET path = CASE WHEN rtrim(path, '/') = '' THEN '/' ELSE rtrim(path, '/') END
WHERE path <> CASE WHEN rtrim(path, '/') = '' THEN '/' ELSE rtrim(path, '/') END
AND NOT EXISTS (
SELECT 1
FROM roots AS other
WHERE other.id <> roots.id
AND CASE WHEN rtrim(other.path, '/') = '' THEN '/' ELSE rtrim(other.path, '/') END
= CASE WHEN rtrim(roots.path, '/') = '' THEN '/' ELSE rtrim(roots.path, '/') END
);
@@ -0,0 +1,68 @@
-- no-transaction
-- #211: a waived release could not record which rule it relaxed. The old
-- constraint made the rule name an exact synonym for rejection:
--
-- CHECK ((verdict = 'rejected') = (rejected_rule IS NOT NULL))
--
-- so §9.3's deck showed a bare `waived` beside rejected rows that each named
-- their own rule, and §5.7's "watchable but not what was asked" lost the half
-- that says what was not asked for. The relaxed form still demands a rule on
-- a rejection and stops demanding its absence elsewhere.
--
-- SQLite cannot alter a CHECK, so the table is rebuilt (see 0014). Unlike the
-- rebuilds there, `releases` is a parent: `grabs`, `movie_releases`,
-- `episode_releases` and `season_releases` all point at it, three of them
-- ON DELETE CASCADE. Dropping the old table with foreign keys enforced would
-- delete those children (or, for `grabs`, refuse outright), so this follows
-- SQLite's own procedure — foreign keys off, the rebuild in one transaction,
-- foreign keys back on. `PRAGMA foreign_keys` is a no-op inside a
-- transaction, which is why the file opens `-- no-transaction` and manages
-- its own; the migration is still all-or-nothing.
--
-- Rows are copied verbatim. Every existing `waived` row keeps its NULL and
-- goes on reading as it does today; only rows written after this migration
-- carry a waived rule.
PRAGMA foreign_keys = OFF;
BEGIN;
CREATE TABLE releases_new (
id INTEGER PRIMARY KEY,
-- Prowlarr's indexer id. Not a foreign key: indexers live in Prowlarr.
indexer_id INTEGER NOT NULL,
guid TEXT NOT NULL,
name TEXT NOT NULL,
size INTEGER NOT NULL,
seeders INTEGER,
publish_date TEXT,
download_url TEXT NOT NULL,
-- §5.6. What the release name claims, before anything is downloaded.
parsed TEXT NOT NULL CHECK (json_valid(parsed)),
score REAL,
-- §9.3. Three buckets. `rejected_rule` names the rule that killed it so
-- an over-strict filter is visible without reading release names, and
-- §5.7's waiver names the rule it relaxed for the same reason.
verdict TEXT CHECK (verdict IN ('eligible', 'waived', 'rejected')),
rejected_rule TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
UNIQUE (indexer_id, guid),
CHECK (verdict != 'rejected' OR rejected_rule IS NOT NULL)
) STRICT;
INSERT INTO releases_new (
id, indexer_id, guid, name, size, seeders, publish_date, download_url,
parsed, score, verdict, rejected_rule, created_at
)
SELECT
id, indexer_id, guid, name, size, seeders, publish_date, download_url,
parsed, score, verdict, rejected_rule, created_at
FROM releases;
DROP TABLE releases;
ALTER TABLE releases_new RENAME TO releases;
CREATE INDEX releases_verdict ON releases (verdict, score);
COMMIT;
PRAGMA foreign_keys = ON;
+106 -9
View File
@@ -12,7 +12,7 @@
//! infohash, and
//! - **infohash**, because the same torrent is re-listed under a new name.
use std::collections::HashSet;
use std::collections::HashMap;
use sqlx::SqlitePool;
@@ -21,15 +21,21 @@ use sqlx::SqlitePool;
/// rejected row names the rule that killed it" reads the same everywhere.
pub const RULE: &str = "blacklisted";
/// Every blacklist key, loaded once per tick or request.
/// Every blacklist key and the reason it was written under, loaded once per
/// tick or request.
///
/// Household scale: a handful of rows. Loading it whole and matching in
/// memory keeps the check identical for a release that has a database row and
/// one that has only just arrived from an indexer.
///
/// The reason travels with the key because §9.3's deck has to say what a row
/// was blacklisted for (#227): a release the policy rejected on size is one
/// the operator can relax and try again, and a corrupt or mismatched one is
/// not. A bare `blacklisted` makes those two read the same.
#[derive(Debug, Clone, Default)]
pub struct Blacklist {
names: HashSet<String>,
infohashes: HashSet<String>,
names: HashMap<String, String>,
infohashes: HashMap<String, String>,
}
impl Blacklist {
@@ -39,17 +45,30 @@ impl Blacklist {
///
/// If the query fails.
pub async fn load(pool: &SqlitePool) -> Result<Self, sqlx::Error> {
// Oldest first, so a key that hard-failed twice under different
// reasons keeps the first one — the same rule [`add`] applies when it
// refuses to write the second row.
let rows = sqlx::query!(
r#"SELECT normalised_name AS "normalised_name!: String", infohash FROM blacklist"#
r#"SELECT normalised_name AS "normalised_name!: String",
infohash,
reason AS "reason!: String"
FROM blacklist
ORDER BY id"#
)
.fetch_all(pool)
.await?;
let mut blacklist = Self::default();
for row in rows {
blacklist.names.insert(row.normalised_name);
blacklist
.names
.entry(row.normalised_name)
.or_insert_with(|| row.reason.clone());
if let Some(infohash) = row.infohash {
blacklist.infohashes.insert(infohash.to_ascii_lowercase());
blacklist
.infohashes
.entry(infohash.to_ascii_lowercase())
.or_insert(row.reason);
}
}
Ok(blacklist)
@@ -58,14 +77,44 @@ impl Blacklist {
/// Whether this release name has been blacklisted, under any spelling.
#[must_use]
pub fn blocks_name(&self, release_name: &str) -> bool {
self.names.contains(&arr_parse::normalise(release_name))
self.names.contains_key(&arr_parse::normalise(release_name))
}
/// Whether this infohash has been blacklisted. Case-insensitive:
/// Transmission and Torznab disagree on the hex casing.
#[must_use]
pub fn blocks_infohash(&self, infohash: &str) -> bool {
self.infohashes.contains(&infohash.to_ascii_lowercase())
self.infohashes.contains_key(&infohash.to_ascii_lowercase())
}
/// What this release name was blacklisted for, or `None` if it was not.
#[must_use]
pub fn reason_for_name(&self, release_name: &str) -> Option<&str> {
self.names
.get(&arr_parse::normalise(release_name))
.map(String::as_str)
}
/// What this infohash was blacklisted for, or `None` if it was not.
#[must_use]
pub fn reason_for_infohash(&self, infohash: &str) -> Option<&str> {
self.infohashes
.get(&infohash.to_ascii_lowercase())
.map(String::as_str)
}
/// What a candidate was blacklisted for, under either key (#227).
///
/// The name is asked first: it is the key every candidate has, and a
/// `.torrent` URL hides its infohash until the download client fetches
/// it, exactly as [`Blacklist::blocks_candidate`] describes.
#[must_use]
pub fn reason_for_candidate(&self, release_name: &str, download_url: &str) -> Option<&str> {
self.reason_for_name(release_name).or_else(|| {
magnet_infohash(download_url)
.and_then(|hash| self.infohashes.get(&hash))
.map(String::as_str)
})
}
/// Whether a candidate is blacklisted before anything is sent to the
@@ -175,6 +224,54 @@ mod tests {
assert!(!blacklist.blocks_name("Dune Part Two 2024 1080p WEB-DL"));
}
#[tokio::test]
async fn a_key_carries_the_reason_it_was_blacklisted_for() {
let (database, _dir) = database().await;
add(
database.pool(),
Some(HASH),
"Rick.And.Morty.S08.1080p",
"size",
)
.await
.unwrap();
add(
database.pool(),
None,
"Some.Other.Pack.S01",
"no file matches a wanted episode",
)
.await
.unwrap();
let blacklist = Blacklist::load(database.pool()).await.unwrap();
// #227: the deck has to tell a policy rejection from a bad release,
// and the reason is the only thing that says which.
assert_eq!(
blacklist.reason_for_name("Rick And Morty S08 1080p"),
Some("size")
);
assert_eq!(
blacklist.reason_for_infohash(&HASH.to_ascii_uppercase()),
Some("size")
);
assert_eq!(
blacklist.reason_for_name("Some.Other.Pack.S01"),
Some("no file matches a wanted episode")
);
assert_eq!(blacklist.reason_for_name("Never.Failed.S01"), None);
let magnet = format!("magnet:?xt=urn:btih:{HASH}&dn=Renamed.Pack");
assert_eq!(
blacklist.reason_for_candidate("Renamed.Pack", &magnet),
Some("size")
);
assert_eq!(
blacklist.reason_for_candidate("Renamed.Pack", "https://tracker/x.torrent"),
None
);
}
#[tokio::test]
async fn a_second_hard_fail_of_the_same_torrent_adds_no_row() {
let (database, _dir) = database().await;
+258
View File
@@ -347,6 +347,93 @@ mod tests {
assert_eq!(renamed_from_imported, "available");
}
/// The four seeded roots, up to but not including migration 0031, with
/// the given legacy paths written straight into the rows.
async fn roots_before_normalisation(paths: &[(i64, &str)]) -> (tempfile::TempDir, Db) {
let dir = tempfile::tempdir().expect("tempdir");
let db = Db::connect(dir.path().join("arr.db"))
.await
.expect("connect");
MIGRATOR
.run_to(30, db.pool())
.await
.expect("migrations before #244");
for (id, path) in paths {
sqlx::query("UPDATE roots SET path = ? WHERE id = ?")
.bind(path)
.bind(id)
.execute(db.pool())
.await
.expect("legacy path, valid under the unique index");
}
(dir, db)
}
async fn root_path(db: &Db, id: i64) -> String {
sqlx::query_scalar("SELECT path FROM roots WHERE id = ?")
.bind(id)
.fetch_one(db.pool())
.await
.expect("root path")
}
/// #244: a root's path could be stored with a trailing separator until
/// #243 normalised the incoming value, and `roots::update` compares a
/// normalised payload against the stored value — so such a row could
/// never be edited again. Migration 0031 strips the separator, and skips
/// a row whose stripped form another row already holds rather than
/// tripping the unique index and refusing to apply.
#[tokio::test]
async fn root_paths_are_normalised_but_never_onto_a_path_in_use() {
let (_dir, db) = roots_before_normalisation(&[
(1, "/mnt/media/movies/main/"),
(2, "/mnt/collide"),
(3, "/mnt/collide/"),
(4, "/mnt/media/tv/kids///"),
])
.await;
db.migrate()
.await
.expect("0031 applies with a collision present");
assert_eq!(root_path(&db, 1).await, "/mnt/media/movies/main");
assert_eq!(
root_path(&db, 4).await,
"/mnt/media/tv/kids",
"every trailing separator goes in one pass"
);
assert_eq!(root_path(&db, 2).await, "/mnt/collide");
assert_eq!(
root_path(&db, 3).await,
"/mnt/collide/",
"left exactly as it is: normalising it would collide with root 2"
);
}
/// #244: the harder half of the same guard. Two rows that strip to the
/// same path, *neither* of which already holds the stripped value, still
/// have to be left alone — normalising them would collide with each
/// other, and a migration that cannot apply stops the daemon booting.
#[tokio::test]
async fn two_rows_that_would_collide_with_each_other_stop_nothing() {
let (_dir, db) =
roots_before_normalisation(&[(1, "/mnt/one/"), (2, "/mnt/dup/"), (3, "/mnt/dup//")])
.await;
db.migrate()
.await
.expect("0031 applies with a mutually colliding pair present");
assert_eq!(root_path(&db, 2).await, "/mnt/dup/");
assert_eq!(root_path(&db, 3).await, "/mnt/dup//");
assert_eq!(
root_path(&db, 1).await,
"/mnt/one",
"the rest of the table is still normalised"
);
}
/// #155: the table-rebuild migrations (0007, 0014, 0021) run outside a
/// transaction so `PRAGMA foreign_keys = OFF` holds and dropping the old
/// tables does not cascade-delete `movie_releases`/`episode_releases`.
@@ -430,6 +517,177 @@ mod tests {
);
}
/// #211: 0032 rebuilds `releases`, which is a parent — `grabs`,
/// `movie_releases`, `episode_releases` and `season_releases` all point
/// at it, three of them ON DELETE CASCADE. The rebuild runs with foreign
/// keys off so dropping the old table neither cascades those children
/// away nor is refused by `grabs`.
#[tokio::test]
async fn the_releases_rebuild_preserves_children_and_rows() {
let (_dir, db) = a_deck_with_every_child_row().await;
db.migrate().await.expect("remaining migrations");
for (label, query, expected) in [
("releases", "SELECT count(*) FROM releases", 3),
("movie_releases", "SELECT count(*) FROM movie_releases", 1),
(
"episode_releases",
"SELECT count(*) FROM episode_releases",
1,
),
("season_releases", "SELECT count(*) FROM season_releases", 1),
("grabs", "SELECT count(*) FROM grabs", 1),
] {
let rows: i64 = sqlx::query_scalar(query)
.fetch_one(db.pool())
.await
.expect("count");
assert_eq!(rows, expected, "{label} survives the releases rebuild");
}
// An existing waived row keeps its NULL and reads as it did before.
let waived: Option<String> =
sqlx::query_scalar("SELECT rejected_rule FROM releases WHERE guid = 'waived-guid'")
.fetch_one(db.pool())
.await
.expect("waived row");
assert_eq!(waived, None);
let rejected: Option<String> =
sqlx::query_scalar("SELECT rejected_rule FROM releases WHERE guid = 'rejected-guid'")
.fetch_one(db.pool())
.await
.expect("rejected row");
assert_eq!(rejected.as_deref(), Some("size"));
// Foreign keys are back on for the connection the migration used.
let foreign_keys: i64 = sqlx::query_scalar("PRAGMA foreign_keys")
.fetch_one(db.pool())
.await
.expect("foreign_keys");
assert_eq!(foreign_keys, 1);
}
/// A database migrated to just before 0032, holding one release per
/// verdict and one row in every table that points at `releases`.
async fn a_deck_with_every_child_row() -> (tempfile::TempDir, Db) {
let dir = tempfile::tempdir().expect("tempdir");
let db = Db::connect(dir.path().join("arr.db"))
.await
.expect("connect");
MIGRATOR
.run_to(31, db.pool())
.await
.expect("migrations before the releases rebuild");
let movie_id = sqlx::query(
"INSERT INTO movies (tmdb_id, title, root_id)
SELECT 693134, 'Dune Part Two', id FROM roots WHERE kind = 'movie' LIMIT 1",
)
.execute(db.pool())
.await
.expect("movie")
.last_insert_rowid();
let series_id = sqlx::query(
"INSERT INTO series (tmdb_id, title, root_id)
SELECT 82728, 'Bluey', id FROM roots WHERE kind = 'tv' LIMIT 1",
)
.execute(db.pool())
.await
.expect("series")
.last_insert_rowid();
let season_id = sqlx::query("INSERT INTO seasons (series_id, number) VALUES (?, 1)")
.bind(series_id)
.execute(db.pool())
.await
.expect("season")
.last_insert_rowid();
let episode_id =
sqlx::query("INSERT INTO episodes (season_id, number, title) VALUES (?, 1, 'x')")
.bind(season_id)
.execute(db.pool())
.await
.expect("episode")
.last_insert_rowid();
// One row per verdict, including the waived row this issue is about,
// which under the old constraint could only hold NULL.
for (guid, verdict, rule) in [
("eligible-guid", "eligible", None),
("waived-guid", "waived", None),
("rejected-guid", "rejected", Some("size")),
] {
sqlx::query(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, verdict, rejected_rule)
VALUES (1, ?, ?, 1024, 'http://x', '{}', ?, ?)",
)
.bind(guid)
.bind(guid)
.bind(verdict)
.bind(rule)
.execute(db.pool())
.await
.expect("release");
}
let release_id: i64 =
sqlx::query_scalar("SELECT id FROM releases WHERE guid = 'eligible-guid'")
.fetch_one(db.pool())
.await
.expect("release id");
sqlx::query("INSERT INTO movie_releases (movie_id, release_id) VALUES (?, ?)")
.bind(movie_id)
.bind(release_id)
.execute(db.pool())
.await
.expect("movie link");
sqlx::query("INSERT INTO episode_releases (episode_id, release_id) VALUES (?, ?)")
.bind(episode_id)
.bind(release_id)
.execute(db.pool())
.await
.expect("episode link");
sqlx::query("INSERT INTO season_releases (season_id, release_id) VALUES (?, ?)")
.bind(season_id)
.bind(release_id)
.execute(db.pool())
.await
.expect("season link");
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash)
VALUES (?, 'movie', ?, 'infohash-1')",
)
.bind(release_id)
.bind(movie_id)
.execute(db.pool())
.await
.expect("grab");
(dir, db)
}
/// The relaxed constraint (#211) admits a rule on a waiver and still
/// refuses a rejection without one.
#[tokio::test]
async fn a_waiver_may_name_the_rule_it_relaxed() {
let (_dir, db) = fresh().await;
sqlx::query(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, verdict, rejected_rule)
VALUES (1, 'waived', 'Some.Release', 1024, 'http://x', '{}', 'waived', 'size')",
)
.execute(db.pool())
.await
.expect("a waiver names its rule");
sqlx::query(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, verdict)
VALUES (1, 'rejected', 'Some.Release', 1024, 'http://x', '{}', 'rejected')",
)
.execute(db.pool())
.await
.expect_err("a rejection still has to name its rule");
}
#[tokio::test]
async fn seeds_two_tv_roots_with_distinct_policies() {
let (_dir, db) = fresh().await;
+13
View File
@@ -541,6 +541,19 @@
<button type="button" class="control" id="tv-releases-sweep">re-search</button>
</header>
<p class="deck-status readout" id="tv-releases-status" role="status" hidden></p>
<!-- issue 227: a pack that downloaded in full and was condemned at
import (§5.7). It outlives the status line because it is a fact
about the season, not about the request in flight. -->
<section class="deck-notice" id="tv-releases-failure" hidden aria-labelledby="tv-failure-label">
<h3 class="deck-label" id="tv-failure-label">pack abandoned at import</h3>
<p class="notice-release readout" id="tv-failure-release"></p>
<p class="notice-line" id="tv-failure-what"></p>
<p class="notice-line" id="tv-failure-next"></p>
<p class="notice-line dim">
nothing was imported, and the torrent was left where it is: nothing is
deleted early to satisfy the library (§7.3).
</p>
</section>
<div id="tv-buckets"></div>
</main>
+117 -13
View File
@@ -34,6 +34,9 @@ import {
} from "./queues";
import {
type ActionOutcome,
blacklistAdvice,
blacklistClass,
blacklistReasonLabel,
bucketOf,
type FilesOutcome,
formatAudio,
@@ -46,12 +49,14 @@ import {
formatSource,
formatSweepAge,
grabRelease,
type ImportFailure,
libraryFolder,
type MovieRelease,
movieFiles,
movieReleases,
movieSearchState,
overridable,
type PackStateOutcome,
probedAttributeTags,
queueSearch,
removeMovie,
@@ -2237,7 +2242,14 @@ function paintBuckets(dom: BucketsDom, releases: MovieRelease[], actions: Releas
// §9.3: over-strict filters must be visible, not silently absent — and
// where a rule can be waived, the count says so rather than leaving the
// way out folded inside a collapsed bucket.
const forceable = releases.filter(overridable).length;
//
// Only a rejected row needs forcing. A row below policy is offered
// already, one click, whatever its rule — and since #211 gave it a rule
// at all, counting every overridable row here claimed the whole
// collapsed deck had to be forced.
const forceable = releases.filter(
(release) => bucketOf(release) === "rejected" && overridable(release),
).length;
const none = document.createElement("li");
none.className = "rel rel-none readout dim";
none.textContent =
@@ -2379,20 +2391,36 @@ function releaseRow(
}),
);
}
// #227: a blacklisted release was grabbed, downloaded and condemned at
// import. Which of the two things happened decides what the operator does
// next, and `blacklisted` alone reads the same for both.
const blacklisted =
bucket === "rejected" && release.rejected_rule === "blacklisted"
? blacklistClass(release.blacklist_reason)
: null;
if (bucket !== "eligible") {
// A rejected row always names its rule; a waived one cannot — the
// `releases` CHECK allows `rejected_rule` only on a rejection. So a
// waived row says the plainer thing the operator can act on, "below
// policy", rather than the name the record keeps for it.
// §9.3: every row that is not eligible names the rule behind it, waived
// and rejected alike — three waivers for three different reasons read
// identically otherwise, and reading release names to tell them apart is
// the Radarr defect this view exists to fix. #211 gave a waived row the
// rule it relaxed; a row written before it still has none, and says the
// plainer thing alone.
const verdict =
bucket === "waived"
? "below policy"
: release.rejected_rule
? `rejected · ${ruleLabel(release.rejected_rule)}`
: "rejected";
? release.rejected_rule
? `below policy · ${ruleLabel(release.rejected_rule)}`
: "below policy"
: blacklisted
? `blacklisted · ${blacklistReasonLabel(release.blacklist_reason)}`
: release.rejected_rule
? `rejected · ${ruleLabel(release.rejected_rule)}`
: "rejected";
line.append(
chip(verdict, (span) => {
span.dataset.verdict = bucket;
if (blacklisted) {
span.dataset.blacklist = blacklisted;
}
}),
);
}
@@ -2401,6 +2429,13 @@ function releaseRow(
name.className = "rel-name readout";
name.textContent = release.name;
line.append(name);
if (blacklisted) {
const why = document.createElement("span");
why.className = "rel-why readout";
why.dataset.blacklist = blacklisted;
why.textContent = blacklistAdvice(release.blacklist_reason);
line.append(why);
}
item.append(line);
const note = document.createElement("span");
@@ -2884,6 +2919,10 @@ function tvReleasesMain(): TvReleasesView {
const sub = must<HTMLElement>("#tv-releases-sub");
const sweep = must<HTMLButtonElement>("#tv-releases-sweep");
const statusEl = must<HTMLElement>("#tv-releases-status");
const failureEl = must<HTMLElement>("#tv-releases-failure");
const failureRelease = must<HTMLElement>("#tv-failure-release");
const failureWhat = must<HTMLElement>("#tv-failure-what");
const failureNext = must<HTMLElement>("#tv-failure-next");
const dom = buildBucketDom(must<HTMLElement>("#tv-buckets"));
let request: TvDeckRequest | null = null;
@@ -2926,6 +2965,36 @@ function tvReleasesMain(): TvReleasesView {
statusEl.dataset.action = "";
}
/**
* #227: the season's own history, above the candidates. A pack that
* downloaded in full and was condemned at import (§5.7) blacklists the
* release and puts every episode back to `missing`, which leaves the season
* reading `0/10` as though nothing had ever been tried. It sits outside the
* status line because it is a fact about the season, not about the request
* in flight, and it has to survive a sweep that repaints the status.
*/
function paintFailure(failure: ImportFailure | null) {
if (failure === null) {
failureEl.hidden = true;
return;
}
failureEl.dataset.blacklist = blacklistClass(failure.reason);
failureEl.hidden = false;
failureRelease.textContent =
failure.failed_at === null
? failure.release
: `${failure.release} · ${formatSweepAge(failure.failed_at)}`;
// A rule name reads as a preposition — "condemned on size"; a reason
// written as a sentence has to be quoted, not conjugated.
const on =
blacklistClass(failure.reason) === "policy"
? ` on ${blacklistReasonLabel(failure.reason)}`
: "";
const said = blacklistClass(failure.reason) === "release" ? `${failure.reason}. ` : "";
failureWhat.textContent = `downloaded in full, then condemned at import${on}${said}every episode went back to missing and the release is blacklisted.`;
failureNext.textContent = `${blacklistAdvice(failure.reason)}.`;
}
const actions: ReleaseActions = {
reload: () => load(),
notify: (text, tone) => setStatus(text, tone),
@@ -2969,9 +3038,17 @@ function tvReleasesMain(): TvReleasesView {
setStatus(`releases unavailable — ${outcome.detail}`, "fault");
return;
}
if (!paintBuckets(dom, outcome.releases, actions)) {
const painted = paintBuckets(dom, outcome.releases, actions);
// One read of the season's state, shared by the notice and the empty
// verdict below — they answer two questions from the same row.
const pack = await current.target.packState?.();
if (ticket !== sequence || request !== current) {
return;
}
paintFailure(pack?.kind === "state" ? pack.state.import_failure : null);
if (!painted) {
clearBuckets(dom);
await emptyVerdict(current, ticket, sweepIfEmpty);
await emptyVerdict(current, ticket, sweepIfEmpty, pack);
return;
}
setStatus(null);
@@ -2985,8 +3062,13 @@ function tvReleasesMain(): TvReleasesView {
* answers the third; `last_pack_search_at` separates the first two, the
* same way a movie's `last_searched_at` does (#177).
*/
async function emptyVerdict(current: TvDeckRequest, ticket: number, sweepIfEmpty: boolean) {
const outcome = await current.target.packState?.();
async function emptyVerdict(
current: TvDeckRequest,
ticket: number,
sweepIfEmpty: boolean,
known?: PackStateOutcome,
) {
const outcome = known ?? (await current.target.packState?.());
if (ticket !== sequence || request !== current) {
return;
}
@@ -3078,6 +3160,9 @@ function tvReleasesMain(): TvReleasesView {
if (ticket !== sequence || request !== current) {
return;
}
if (state?.kind === "state") {
paintFailure(state.state.import_failure);
}
if (state?.kind === "state" && state.state.last_pack_search_at !== baseline) {
sweep.disabled = false;
await emptyVerdict(current, ticket, false);
@@ -3150,6 +3235,7 @@ function tvReleasesMain(): TvReleasesView {
next.returnTo.hidden = true;
view.hidden = false;
clearBuckets(dom);
paintFailure(null);
sweep.disabled = false;
back.focus();
void load(true);
@@ -3554,6 +3640,24 @@ function seriesMain(tvDeck: TvReleasesView, views: HideableView[]): SeriesView {
});
line.append(disclose, name, track, seasonCountsChip(season));
// #227: the season the operator's report is about read `0/10` with
// nothing saying a pack had been grabbed, downloaded in full and thrown
// out at import. The counts chip beside this one is exactly the number
// that looked like nothing was ever tried.
const failure = season.import_failure;
if (failure) {
const kind = blacklistClass(failure.reason);
const when = failure.failed_at === null ? "" : ` ${formatSweepAge(failure.failed_at)}`;
const told = `a pack for this season downloaded in full and failed at import${when}${failure.release} · ${blacklistAdvice(failure.reason)}`;
line.append(
chip(`import failed · ${blacklistReasonLabel(failure.reason)}`, (span) => {
span.dataset.flag = "import-failed";
span.dataset.blacklist = kind;
span.title = told;
span.setAttribute("aria-label", told);
}),
);
}
// The season-level twin of the episode flag above: gone upstream while
// files under it remained.
if (season.vanished) {
+65
View File
@@ -26,6 +26,12 @@ export interface MovieRelease {
score: number | null;
verdict: string | null;
rejected_rule: string | null;
/**
* What the blacklist recorded this release as failing on (#227, §6.3).
* Null on every row the blacklist does not hold, and on a blacklisted row
* whose entry has since gone — `blacklisted` is then all the record has.
*/
blacklist_reason: string | null;
}
export type ReleasesOutcome =
@@ -113,6 +119,23 @@ export interface SeasonPackState {
pack_failures: number;
pack_retry_at: string | null;
last_pack_search_at: string | null;
import_failure: ImportFailure | null;
}
/**
* A pack that downloaded in full and was condemned at import (#227, §5.7).
*
* The torrent stays where it is — §7.3 hands that lifecycle to the reaper —
* the release is blacklisted and every episode it covered reopens as a gap.
* Nothing on screen joined those facts, so the season read `0/10` as though
* no grab had ever been tried and the operator found out by opening
* Transmission.
*/
export interface ImportFailure {
release: string;
/** A policy rule name, or a sentence about the release. See `blacklistClass`. */
reason: string | null;
failed_at: string | null;
}
export type PackStateOutcome =
@@ -467,6 +490,48 @@ export function ruleLabel(rule: string | null): string {
return RULE_LABEL[rule] ?? rule.replaceAll("_", " ");
}
/**
* What a blacklisting was: the policy turning a file down, or the release
* itself failing (#227).
*
* The blacklist reason is either a policy rule name — the same vocabulary
* `rejected_rule` uses — or a sentence about the release, written where the
* import gave up before any rule was consulted. The two demand opposite
* decisions: a size rejection is the operator's own floor and they can relax
* it, a corrupt or mismatched pack is not theirs to argue with. `unknown` is
* a row the blacklist no longer answers for; nothing is claimed about it.
*/
export type BlacklistClass = "policy" | "release" | "unknown";
export function blacklistClass(reason: string | null): BlacklistClass {
if (reason === null) {
return "unknown";
}
return reason in RULE_LABEL ? "policy" : "release";
}
/** The reason as a chip word: a rule's short label, or the sentence itself. */
export function blacklistReasonLabel(reason: string | null): string {
return reason === null ? "reason not recorded" : (RULE_LABEL[reason] ?? reason);
}
/**
* What the operator does about it, which is the whole difference between the
* two classes — and the sentence #227 exists to put on screen.
*/
export function blacklistAdvice(reason: string | null): string {
switch (blacklistClass(reason)) {
case "policy":
return waiverOverride(reason) === null
? "policy rejected the file — that rule has no per-title relaxation"
: `policy rejected the file — relax ${ruleLabel(reason)} for this title and the next candidate can pass`;
case "release":
return "the release itself failed at import — a retry downloads the same files";
default:
return "blacklisted before the reason was recorded";
}
}
export async function errorDetail(response: Response): Promise<string> {
try {
const body = (await response.json()) as { error?: string };
+7
View File
@@ -5,6 +5,7 @@
import type { MetadataTrailer } from "./movie";
import type {
ActionOutcome,
ImportFailure,
MovieRelease,
PackStateOutcome,
ReleasesOutcome,
@@ -49,6 +50,12 @@ export interface ApiSeason {
tracked: boolean;
/** Gone upstream while a file under it remained — a conflict, not a state. */
vanished: boolean;
/**
* #227: the last pack that downloaded in full and was condemned at import,
* while the season is still waiting for a file. Null once the gap is
* filled — a season with nothing missing has nothing to explain.
*/
import_failure: ImportFailure | null;
episodes: ApiEpisode[];
}
+104
View File
@@ -617,6 +617,48 @@ body {
vertical-align: baseline;
}
/* #227: a pack abandoned at import, above the candidates. A quiet panel, not
an alert: the failure is history the season owes an explanation for, and
the operator opened this deck to grab something, not to be shouted at. The
tone follows the same reading as the row chips — amber for their own policy
floor, red for a release that failed on its own. */
.deck-notice {
margin: 0 0 var(--space-6);
padding: var(--space-3) var(--space-4) var(--space-4);
background: var(--panel);
border: 1px solid var(--line);
border-radius: var(--radius);
}
.deck-notice[data-blacklist="policy"] {
background: oklch(from var(--signal-warn) l c h / 7%);
border-color: oklch(from var(--signal-warn) l c h / 40%);
}
.deck-notice[data-blacklist="release"] {
background: oklch(from var(--signal-fault) l c h / 7%);
border-color: oklch(from var(--signal-fault) l c h / 40%);
}
/* the release name is evidence, in the readout face like every other one */
.notice-release {
margin: 0 0 var(--space-2);
overflow-wrap: anywhere;
font-size: var(--text-xs);
color: var(--ink);
}
.notice-line {
margin: 0;
max-width: 68ch;
font-size: var(--text-sm);
color: var(--ink-muted);
}
.notice-line + .notice-line {
margin-top: var(--space-2);
}
.deck-group {
margin: 0 0 var(--space-8);
}
@@ -644,6 +686,18 @@ body {
color: var(--ink-faint);
}
.deck-notice .deck-label {
margin-bottom: var(--space-2);
}
.deck-notice[data-blacklist="policy"] .deck-label {
color: var(--signal-warn);
}
.deck-notice[data-blacklist="release"] .deck-label {
color: var(--signal-fault);
}
.deck-rows {
margin: 0;
padding: 0;
@@ -719,6 +773,35 @@ body {
color: var(--verdict-rejected);
}
/* #227: a blacklisted release was grabbed, downloaded in full and thrown out
at import, and the two ways that happens want opposite decisions. The
policy turning a file down is the operator's own floor — amber, the hue
this app already gives a gap they can act on. The release itself failing is
the one case in the deck that is genuinely broken, so it takes fault red;
the "never fault red" rule above is about a rejection, and this is not one.
A row whose blacklist entry is gone claims nothing and stays slate. */
.chip[data-blacklist="policy"] {
color: var(--signal-warn);
border-color: oklch(from var(--signal-warn) l c h / 55%);
}
.chip[data-blacklist="release"] {
color: var(--signal-fault);
border-color: oklch(from var(--signal-fault) l c h / 55%);
}
/* a reason written as a sentence is longer than any rule name, and the chips
around it are fixed-width columns that must not be pushed off the line
(§9.3: no horizontal scroll at any viewport). This one chip wraps instead. */
.chip[data-blacklist],
.chip[data-flag="import-failed"] {
min-width: 0;
max-width: 100%;
padding-top: var(--space-1);
padding-bottom: var(--space-1);
overflow-wrap: anywhere;
}
/* media state ramp (§4.2): green on disk, violet downloading, amber wanted
and still missing. Unwanted-missing and parked are nothing-happening and
stay neutral; `parked` exists so a vanished grab never reads as a gap. */
@@ -1199,6 +1282,20 @@ body {
border-bottom: 1px solid oklch(from var(--line) l c h / 45%);
}
/* #227: what the blacklisting means for the next move, on its own line under
the chips. Only a blacklisted row carries it, so the dense list stays dense
everywhere else. */
.rel-why {
flex-basis: 100%;
min-width: 0;
font-size: var(--text-xs);
color: var(--ink-muted);
}
.rel-why[data-blacklist="release"] {
color: oklch(from var(--signal-fault) 0.78 0.1 h);
}
.rel-note {
font-size: var(--text-xs);
color: var(--ink-muted);
@@ -1519,6 +1616,13 @@ body {
font-size: var(--text-xs);
}
/* #227: the season the operator's own report was about read `0/10` beside
this chip's absence. It sits next to the counts because that number is what
looked like nothing had ever been tried. */
.chip[data-flag="import-failed"] {
cursor: help;
}
/* issue 122: gone upstream while its file remained — a conflict, amber dashed */
.chip[data-flag="vanished"] {
color: var(--signal-warn);