Compare commits

...

152 Commits

Author SHA1 Message Date
Miguel Palhas a3186b2305 refactor(arr): chevron icons for provider order 2026-08-25 18:11:11 +01:00
Miguel Palhas 1afd8c1998 refactor(arr): subtitle chip takes shared delete
Drops the hand-rolled arm, disarm, timeout and label swap, and the local
12-grid close glyph that came with it.
2026-08-25 18:11:11 +01:00
Miguel Palhas 1e778afd18 feat(arr): compact armed delete and new glyphs
The subtitle chip needs the settings arm-then-confirm at chip scale: a
chip fits neither the control plate nor the word "confirm". The shared
control grows a compact variant that swaps the glyph instead, rather than
each view keeping a private copy of the idiom.
2026-08-25 18:11:06 +01:00
Miguel Palhas 34e5663920 Merge main into blitz/subtitles
Feedback pass 2 and the size-band work landed on main while this branch
was finishing. Brings them in ahead of the merge back.

# Conflicts:
#	crates/arr-api/src/movies.rs
#	crates/arr-api/src/state.rs
#	crates/arr-daemon/src/main.rs
#	web/src/main.ts
2026-08-25 17:53:18 +01:00
Miguel Palhas 8947febbff fix(web): add series library selector
ci / web (push) Successful in 1m20s
e2e / e2e (push) Successful in 2m8s
ci / rust (push) Failing after 14m52s
2026-08-25 14:24:30 +01:00
Miguel Palhas bd5b00b14a fix(api): create title move destination
ci / web (push) Successful in 34s
ci / rust (push) Failing after 1m48s
e2e / e2e (push) Successful in 1m49s
2026-08-25 13:01:16 +01:00
Miguel Palhas 90efeaa442 Merge milestone 'Feedback pass 2'
ci / web (push) Successful in 30s
ci / rust (push) Failing after 1m40s
e2e / e2e (push) Successful in 2m13s
17 issues: the operator's UI feedback (back button, icon-only controls,
one-line settings rows, plainer words), the root-move and root-path
relocation work, the attention queue's threshold, window, anchor and
liveness rule, reclassify on every action that changes an effective
policy, and the waived rule reaching the deck.

Gate green at 524 tests. Migrations 0030, 0031 and 0032 verified against
a database built through the production upgrade path; 0032's rebuild
preserves rows and children with foreign_key_check clean.
2026-08-25 12:17:01 +01:00
Miguel Palhas 5d80177622 Merge #227: say what a pack was abandoned for
Closes #227
2026-08-25 12:12:34 +01:00
Miguel Palhas 591cf27dc5 feat(web): say what a pack was abandoned for
A pack that hard-failed at import blacklisted its release, put every
episode back to missing and left the season reading 0/10, with nothing
on screen joining the two. Every fact was already recorded.

The blacklist now carries its reason out of the database: deck rows read
`blacklisted · size` instead of a bare `blacklisted`, and say whether the
policy turned the file down — relaxable for this title — or the release
itself failed, which a retry only repeats. A season whose pack was
abandoned says so on its row and above its deck, with the release name,
when it failed, and what it failed on. A row the blacklist no longer
answers for keeps rendering and claims no reason.

Two defects from the integration review of #211 sit in the same code and
are fixed here: a waived row threw away the rule it now carries and read
a bare `below policy`, and the empty-eligible count called every waived
row force-grabbable, since #211 gave those rows the rule `overridable`
reads.

Verified against a real browser: series detail, both season decks and
their buckets, at 1280 and 390 px.

Refs #227, #211
2026-08-25 12:11:46 +01:00
Miguel Palhas 9e445d0398 Merge #240: reconcile design coherence in 5.7 and 9.5
Closes #240
2026-08-25 11:50:56 +01:00
Miguel Palhas 1e30c49a72 Merge #245: pack backoff runs from the failure
Closes #245
2026-08-25 11:50:39 +01:00
Miguel Palhas 55373d228c fix: pack backoff runs from the failure
#239 moved §5.7's attention window to `failed_at` and left §6.2's pack
ladder on `grabbed_at`. A torrent that stalls for weeks before ffprobe
condemns it at import has elapsed the whole ladder the moment it fails,
so the pack lane retried a source that had just failed — the one thing
the backoff exists to prevent.

The ladder now measures from the failure, the same anchor and the same
column §5.7 reads, with `grabbed_at` as the fallback for rows written
before the column existed. All three sites read
`max(coalesce(failed_at, grabbed_at))`, so the `last_failed_at` alias
holds what its name says — including the one the season deck feeds into
`reopens_at` and `pack_retry_at`, which was showing a grab time under a
name §5.7 had redefined.

DESIGN.md §6.2 states the anchor the way §5.7 states its own.

Tests cover a pack grabbed 35 days ago and failed 10 minutes ago on the
targeted lane, the RSS lane and the season deck.

`just ci` through the gate: 519/519 tests pass, web checks clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 11:49:44 +01:00
Miguel Palhas c78c73ec6c docs: reconcile design coherence issues in §5.7 and §9.5
§9.5 restated the pre-#226 decision-bar rule without the 30-day window,
giving the document two versions of the same rule. Make it defer to
§5.7 instead.

§5.7 used two phrasings for the season-queuing predicate in one
paragraph (existence-of-file vs. state check) though both readers
implement the state check; picked the state phrasing throughout. Also
fixed the arithmetically confusing "seen twice ... third face"
sentence, and rewrapped the single unwrapped ~450-character line in
§7.4 to the document's ~78-column width.

Ref #240. Ran full `just ci` through the gate (517 tests, exit 0).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-25 11:49:43 +01:00
Miguel Palhas 58a45fc98e Merge #211: let a waiver name the rule it relaxed
Closes #211
2026-08-25 11:47:25 +01:00
Miguel Palhas a0dc07f085 feat(db): let a waiver name the rule it relaxed
`releases` forbade a rule name on anything but a rejection, so §9.3's
deck showed a bare `waived` beside rejections that each named their own,
and §5.7's "watchable but not what was asked" lost the half that says
what was not asked for. Since #210 that is the ordinary outcome of
waiving a size rejection, not a rare one.

0032 rebuilds the table with `CHECK (verdict != 'rejected' OR
rejected_rule IS NOT NULL)`, and the daemon and arr-api's
reclassification both store the waived rule. Existing rows keep NULL and
read as they do today.

`releases` is a parent — `grabs`, `movie_releases`, `episode_releases`
and `season_releases` point at it, three ON DELETE CASCADE — so the
rebuild runs `-- no-transaction` with foreign keys off around one
explicit transaction, per SQLite's own procedure. Verified against a
real database: the pre-0032 binary created and populated it, this build
migrated a copy, and every release row, child row and created_at came
through byte-identical with `PRAGMA foreign_key_check` clean.

Refs #211
2026-08-25 11:44:05 +01:00
Miguel Palhas de6c35cce3 Merge #232: plainer words for grab, blocked, deck
Closes #232
2026-08-25 11:43:55 +01:00
Miguel Palhas 9a413b10ed Merge #246: re-derive verdicts on a policy edit
Closes #246
2026-08-25 11:42:10 +01:00
Miguel Palhas bbb6d2f4a4 feat(api): re-derive verdicts on a policy edit
PUT /api/policies/{id} changed the rule every title under every root
pointing at the policy is judged by, and re-derived nothing, so §9.3's
deck and the daemon's grab gate kept reading verdicts computed under
rules that no longer existed.

Drives #241's walker from a policy id: root by root through
reclassify::root, so the skip rules and the leave-unchanged-rows-alone
rule stay in one place. A rename touches no rule and walks nothing.

Inline still holds at this width. Measured on a release build over 2000
titles and 10 000 stored releases across two roots sharing one policy:
0.36 s when no verdict moves, 2.7 s when all 10 000 do. DESIGN.md §5.1
now names four actions and carries those numbers.

just ci passed through the gate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 11:40:32 +01:00
Miguel Palhas c454b3ec11 Merge #244: normalise stored root paths
Closes #244
2026-08-25 11:29:13 +01:00
Miguel Palhas c4d4ade4da fix(api): normalise stored root paths
#243 normalised the incoming path but compared it against the value read
raw from the database, so a root stored with a trailing separator never
compared equal. Every edit of it -- a policy change included -- took the
relocation branch, where each planned destination is its own source and
the pre-check refuses. That root could not be edited at all.

`update` now normalises both sides, and hands `relocate_root` the
normalised stored value. `path_is_free` normalises the stored side in SQL
and `create` goes through it too, so `/mnt/x` and `/mnt/x/` cannot be two
roots for one directory -- the unique index compares raw strings and
cannot see that.

Migration 0031 strips the separator from rows already written. It skips
any row whose stripped form another row would also hold, rather than
tripping the unique index: a migration that cannot apply stops the daemon
booting, which is worse than two roots naming one directory.

Also from the same review: `undo` recorded only the leaf directory, so a
failed move into `/mnt/media-v2/tv/kids` left `tv` behind. It now records
every level `create_dir_all` materialised, deepest first, and still never
touches one that was already on disk.

Refs #244.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 11:28:35 +01:00
Miguel Palhas d7e9f710e0 Merge #233: stop offering a forced subtitle grab
Closes #233
2026-08-25 08:47:00 +01:00
Miguel Palhas 097e081f4d fix(arr): drop the forced flag from subtitle grab
§15 gives forced tracks no sidecar name and never lets them
satisfy a want; ranking already rejects every forced candidate
(#222). A grab endpoint accepting `forced: true` had nowhere
coherent to put the result, so the flag and its stale test are
gone (#233).
2026-08-25 08:42:56 +01:00
Miguel Palhas 1126a52bbb Merge #222: one sidecar per language, enforced in the schema
Closes #222
2026-08-25 08:35:58 +01:00
Miguel Palhas 2c5cd85499 Merge #220: configure the OpenAI-compatible endpoint and model
Closes #220
2026-08-25 08:35:54 +01:00
Miguel Palhas a45d8ff86a style(arr): give the OpenAI endpoint its own row
Five fields wrapped the model onto a line of its own and clipped the base
URL placeholder. Paired under one label, the way the translator budgets
already are, and the base URL gets the width it needs.
2026-08-25 08:33:47 +01:00
Miguel Palhas abad9b9cfd fix(arr): name the field, not a reply, in the 422
The backend's Malformed Display describes an answer that came back wrong.
Nothing is sent while validating a settings write, so only the reason
belongs in the message.
2026-08-25 08:33:47 +01:00
Miguel Palhas e467bd47fb feat(arr): add the OpenAI endpoint fields to /settings
Shown only when openai is among available_engines. Placeholders say the
base URL is optional and what a local one looks like.
2026-08-25 08:29:59 +01:00
Miguel Palhas 5aa914f81c refactor(arr): retire the OpenAI bootstrap keys
#216 added translate_openai_model as an explicit stopgap; the database row
replaces it, along with translate_openai_base_url. Only the API key stays
in the environment. deny_unknown_fields makes a config file still carrying
either one a parse error, so the move is visible rather than ignored.
2026-08-25 08:29:59 +01:00
Miguel Palhas e8bc766d4b feat(arr): expose the OpenAI endpoint on /settings
Two more fields on the subtitle settings row, validated on write — a base
URL that does not parse is a 422 naming the field — and pushed into the
cell the running backend and the health lamp both read.
2026-08-25 08:29:59 +01:00
Miguel Palhas c6906bffae feat(arr): make the OpenAI endpoint a live setting
The base URL and model move into a cell the backend re-reads per request,
so an operator can repoint it without a restart. Migration 0028 adds the
two columns; DESIGN.md §15 calls both database rows. Construction never
depends on the API key — llama.cpp serves without one.
2026-08-25 08:29:45 +01:00
Miguel Palhas d358c58844 test(arr): a forced candidate writes no sidecar
Ranking already rejects one (#185); this holds the loop to it at the end
of the pipeline, where the row would appear.
2026-08-25 08:28:56 +01:00
Miguel Palhas 3122d5b0a0 feat(arr): refuse a grab for a satisfied language
The 409 on a second subtitle for one language now comes from §15's
invariant rather than from the sidecar filename, so a provider fetch is
refused even when the language is held by a `.mt.srt` the path check
cannot see. The message names the manual delete as the way to replace
it. `claim_path` stays: two `media_files` rows for one video still
derive the same name from different ids.
2026-08-25 08:28:56 +01:00
Miguel Palhas 29c31beceb feat(arr): one sidecar per language, in the schema
DESIGN.md §15 as amended: a language is satisfied by exactly one
sidecar, and no filename segment distinguishes forced from plain from
SDH. A unique index over sidecar rows says so; embedded rows keep their
own key, since several tracks for one language can legitimately coexist
inside a video.

Existing databases may hold a duplicate from a manual grab that beat the
API's path check, so the migration resolves them rather than failing: a
real subtitle beats a machine translation, and of two of the same kind
the newest wins. The files stay on disk for the manual delete to clean
up.

`record_file` no longer swallows every conflict — only the two that mean
"arr already knows this file".
2026-08-25 08:28:51 +01:00
Miguel Palhas ab001b512f docs(arr): settle the two open §15 questions
Forced and SDH get no sidecar name of their own: one language, one
sidecar, and a forced track is ignored where a plain one exists. That
makes the database constraint fall out rather than needing a scheme.

The OpenAI-compatible backend's base URL and model become database rows.
It is not "OpenAI" — it is any endpoint speaking that shape, llama.cpp
included — so which one is in use is something to try and change, not a
property of the deployment. Only the API key stays in the environment.
2026-08-25 08:17:36 +01:00
Miguel Palhas 84dc5ba27b docs(arr): sharpen the unconfigured provider lamp
Said 'not configured'; the branch is specifically about missing bootstrap
credentials, and after #215 only a provider that needs them can reach it.
2026-08-25 06:51:11 +01:00
Miguel Palhas 60257f3177 Merge #223: offer subtitle delete in the UI
Closes #223
2026-08-25 06:49:39 +01:00
Miguel Palhas b73a58d1a4 Merge #225: cache the remote-command probe
Closes #225
2026-08-25 06:49:39 +01:00
Miguel Palhas 80101e789b Merge #215: drop the unused Podnapisi credentials
Closes #215
2026-08-25 06:49:39 +01:00
Miguel Palhas 73d3f04398 fix(arr): cache the remote-command probe verdict
Reused for 5 minutes (#225) instead of running the configured
command on every /api/health poll.
2026-08-25 06:40:50 +01:00
Miguel Palhas 5f1fffdd97 feat(arr): drop unused podnapisi credentials
Podnapisi's search and download are unauthenticated (#188), so the
ARR_PODNAPISI_USERNAME/PASSWORD fields were config nothing read. §10
keeps bootstrap config an honest list.
2026-08-25 06:40:38 +01:00
Miguel Palhas 687c0c106f feat(arr): delete affordance on subtitle chips
Adds a two-click delete icon to each present-subtitle chip, naming
the sidecar in its title/aria-label. Wires the existing DELETE
/api/subtitles/{id} endpoint and reuses the panel's refresh() so
chips and the manual panel repaint after a delete (#223).
2026-08-25 06:40:26 +01:00
Miguel Palhas 5bc9022046 fix(arr): pass the podnapisi seam to the lamp probes
#200 branched before #205's ARR_PODNAPISI_URL seam was pushed, so its new
broken::SubtitleUpstreams call site was written against the three-argument
subtitle_providers. The merge was textually clean and did not build.
2026-08-25 06:31:27 +01:00
Miguel Palhas 8043ef614a Merge #200: lamp the subtitle upstreams
Closes #200
2026-08-25 06:30:34 +01:00
Miguel Palhas 228db06d83 Merge #221: give an expired candidate a typed error
Closes #221
2026-08-25 06:30:34 +01:00
Miguel Palhas 783a6ba760 Merge #224: clear subtitle attempts on language drop
Closes #224
2026-08-25 06:30:34 +01:00
Miguel Palhas a4422e26e5 style(arr): formatting 2026-08-25 06:28:20 +01:00
Miguel Palhas a836967e32 feat(web): subtitle lane under the signal chain 2026-08-25 06:24:32 +01:00
Miguel Palhas 8c6d4ca577 feat(daemon): fold subtitle lamps into broken notifications 2026-08-25 06:12:22 +01:00
Miguel Palhas be7fa87e74 feat(api): subtitle lamps in the health report 2026-08-25 06:04:30 +01:00
Miguel Palhas 8c5613b247 feat(arr): probe methods for subtitle providers and engines 2026-08-25 05:54:19 +01:00
Miguel Palhas 2357e72113 fix(api): give an expired subtitle candidate a typed error
A grab naming a stale candidate_id now fails as ApiError::SubtitleCandidateExpired (404, code candidate_expired) instead of the generic upstream 503 string the panel had to pattern-match for 'not found'.
2026-08-25 05:35:18 +01:00
Miguel Palhas 7b4def4516 style(arr): cargo fmt 2026-08-25 05:32:25 +01:00
Miguel Palhas 3cc9ab4aff fix(arr): clear subtitle attempts on language drop
Removing a language from wanted_languages left its subtitle_attempts
rows behind, resurrecting stale backoff on re-add (#224).
2026-08-25 05:32:08 +01:00
Miguel Palhas baea1dddbb Merge #204: add the subtitle section to settings
Closes #204
2026-08-25 05:26:19 +01:00
Miguel Palhas fe00220c5b Merge #205: cover the subtitle path in arr-e2e
Closes #205
2026-08-25 05:26:19 +01:00
Miguel Palhas bc43084f74 Merge #217: sync translated subtitles with alass
Closes #217
2026-08-25 05:26:19 +01:00
Miguel Palhas 71c161eff7 ci(arr): install alass for the e2e workflow
No distro package for it; cargo install alass-cli, symlinked to the
name the wrapper actually invokes. Also builds the daemon with
translate-command and triggers on arr-subs changes.
2026-08-25 05:25:37 +01:00
Miguel Palhas 6ee79519ca test(arr-e2e): cover the subtitle path end to end
Cross-process against the real arr binary, real ffmpeg and real alass:
a provider fetch synced and named under DESIGN.md §15, never
re-searched once satisfied; an embedded English track extracted and
translated into a .mt sidecar; an implausible sync kept unsynced and
surfaced in the missing-subtitles queue. Podnapisi and the translation
backend are stubbed at the HTTP/process boundary, never a live tracker
or a live translation API.
2026-08-25 05:25:33 +01:00
Miguel Palhas e47d2c920b feat(arr-e2e): extend the harness for subtitle scenarios
Daemon::spawn_with_env for extra child environment, database_path()
and media_root() accessors so a test can seed a media file directly,
the daemon built with translate-command, and the plausible/farfetched
Podnapisi zip fixtures the subtitle tests download from.
2026-08-25 05:25:28 +01:00
Miguel Palhas 5fc6854705 feat(arr): add an env-only Podnapisi URL seam
ARR_PODNAPISI_URL, same shape as the existing tmdb_url seam: a test
harness can point the provider at a wiremock fake without touching
DESIGN.md §10's config surface.
2026-08-25 05:25:25 +01:00
Miguel Palhas da1a932459 fix(arr): correct alass reference/subtitle argument order
Syncer::run passed [subtitle, video] where alass expects
<reference-file> <incorrect-sub-file>; every real invocation failed
before comparing timings, silently degrading to SyncState::NotRun.
2026-08-25 05:25:19 +01:00
Miguel Palhas 6d665e7c2f feat(arr): render subtitle settings in /settings
Adds wanted languages, translation engine (restricted to
available_engines), remote-command timeout, and an ordered
enable/disable list per provider with daily budgets, wired to
the existing GET/PUT /api/settings/subtitles from #198.
2026-08-25 05:08:58 +01:00
Miguel Palhas 9f0a7de37a fix(arr): sync translated subtitles too 2026-08-25 04:56:35 +01:00
Miguel Palhas e0e7ddf6de docs(arr): correct the subtitle lane's comment
Said translation backends are passed empty; #216 wired them and #219 made
both callers share one set.
2026-08-25 04:55:06 +01:00
Miguel Palhas 78aabe99e5 Merge #202: add the missing-subtitles queue
Closes #202
2026-08-25 04:54:12 +01:00
Miguel Palhas a62a8de8fc Merge #219: apply the remote-command timeout setting
Closes #219
2026-08-25 04:54:12 +01:00
Miguel Palhas b3cca4f692 fix(api): drop subtitle queue gaps for unwanted languages
An attempt row survives after a language leaves wanted_languages —
missing_for (#201) already bounds by the current wanted set, so the
queue reads the same way instead of showing a stale gap forever.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-25 03:16:54 +01:00
Miguel Palhas e338a76c66 fix(arr): wire the command timeout to the settings row 2026-08-25 03:15:33 +01:00
Miguel Palhas 47c7ef6682 fix(arr): re-read the command timeout every batch 2026-08-25 03:15:33 +01:00
Miguel Palhas a780b49ab3 feat(web): missing-subtitles queue alongside no-pt-source
A third lane in the attention queues view: one row per movie or
series with a gap chip per language and why (#186's reasons, plus a
sync alass flagged). Reuses the existing deck-group markup and
missingChipLabel wording so it reads like its two siblings; each
row opens the title's own page, where the #203 fetch/translate
panel already carries the resolving actions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-25 03:11:34 +01:00
Miguel Palhas fb40b35156 feat(api): add the missing-subtitles queue endpoint
GET /api/queues/subtitles, grouped by title with why each language
is a gap (#186's attempt states, plus a sync alass rejected). Series
episodes collapse into one season row when the gap is uniform, the
same restraint §9.5 gives the TV attention queues.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-25 03:03:17 +01:00
Miguel Palhas 4003c3a5a0 fix(arr): gate the shared LLM prompt helpers
#193 moved them out of openai.rs so the command backend could share them,
but left them ungated: with neither LLM backend compiled in they are dead
code. just ci missed it because it lints the workspace with --all-features;
building arr-api alone does not enable arr-subs' features.
2026-08-25 02:50:55 +01:00
Miguel Palhas 9ca3c3ce43 Merge #203: manual subtitle search and translation
Closes #203
2026-08-25 02:49:45 +01:00
Miguel Palhas 100a869c89 Merge #218: delete subtitle sidecars with their file
Closes #218
2026-08-25 02:49:45 +01:00
Miguel Palhas d6fdd3850f Merge #197: budget provider and translator calls
Closes #197
2026-08-25 02:49:45 +01:00
Miguel Palhas 22faad66ca fix(arr): allow the candidate DTO its five bools 2026-08-25 02:49:02 +01:00
Miguel Palhas e850ddff81 feat(arr): manual subtitle fetch and translate
The manual surface §9.3 describes, applied to subtitles (§15). It opens
inline under the file's own row and borrows the release deck's layout
language rather than inventing a second one: fixed-width chips lead, the
release name is secondary, eligible shows and rejected collapses to a
count naming the rule.

The translate lane takes any subtitle already on the file as a source,
including an extracted embedded track — the thing Bazarr cannot do — and
lists only the engines this binary was compiled with.
2026-08-25 02:45:32 +01:00
Miguel Palhas 218c83589e chore(arr): refresh sqlx offline query cache
For #197's budget queries and the settings query's new columns.
2026-08-25 02:40:12 +01:00
Miguel Palhas 3a7d80c295 feat(arr): spend the subtitle budget in the reconcile loop
Providers are charged one unit per download, claimed atomically
right before the call; a provider at its cap is skipped in favour of
the next-ranked candidate rather than failing the whole gap.
Translators are charged the source character count before
translating. Either cap is a queue state, same as a provider's own
429.
2026-08-25 02:40:09 +01:00
Miguel Palhas 4d84625332 feat(arr): token bucket table for subtitle budgets
One row per provider/translator per day; try_spend is a single
atomic upsert so concurrent reconcile closes can't both slip a spend
past the daily cap. No allowance configured reads as unlimited.
2026-08-25 02:40:06 +01:00
Miguel Palhas f49507b7d1 refactor(arr): compute translator billing per call
Backend::characters_billed() polled a cumulative counter that races
under concurrent closes and can't attribute cost to one call. Drop it
in favour of the caller counting source characters before it sends
anything, which #197's budget needs anyway.
2026-08-25 02:40:02 +01:00
Miguel Palhas 9a21649afa feat(arr): report release-name match per candidate
The manual subtitle view shows the facts that decided a row (§9.3), and
release-name match is §15's second ranking tier. The file's own release
name is not otherwise on the wire, so the UI cannot derive it.
2026-08-25 02:29:50 +01:00
Miguel Palhas 2fa74137f9 fix(arr): delete subtitle sidecars with their file
remove_library_files only resolved video paths from media_files, so a
season/episode-scoped delete dropped subtitle_files rows via cascade
but left the .srt sidecars on disk (#218).
2026-08-25 02:29:13 +01:00
Miguel Palhas 7b4cff1874 fix(arr): give the reconcile loop its translators
#196 built SubtitleAction with an empty backend list, correctly: no
translation backend existed when it was written. #216 then built
translation_backends() and wired it into AppState, so the API can translate.
Merged, the reconcile loop still got Vec::new() and its translate step
reported "no engine" in every real deployment.

Neither branch was wrong alone; the gap only exists once both are in.
2026-08-25 02:21:21 +01:00
Miguel Palhas 4110555183 Merge #196: close the subtitle gap in the reconcile loop
Closes #196
2026-08-25 02:20:11 +01:00
Miguel Palhas 987bfa2864 Merge #216: wire translation backends into the daemon
Closes #216
2026-08-25 02:20:07 +01:00
Miguel Palhas 5ee5c56b88 Merge #201: show subtitle state on title detail
Closes #201
2026-08-25 02:20:07 +01:00
Miguel Palhas e684813d0c feat(arr): show subtitle chips on title detail
Per media file: a chip per present language (origin, forced/SDH, sync
flag, dashed for machine translation) and a chip per still-missing
wanted language naming why (§15, #201). Movie and series pages both
wire it in through the new status endpoints.
2026-08-25 02:16:25 +01:00
Miguel Palhas e5092034d3 feat(arr): serve subtitle status for title detail
Exposes per-media-file subtitles and missing wanted languages, with the
attempt reason, so #201's UI has one call per title (movies, episodes)
and one bulk call per series instead of one per episode.
2026-08-25 02:16:20 +01:00
Miguel Palhas c2ca25895f feat(arr): reconcile subtitle gaps in the daemon
Closes each unsatisfied wanted language per DESIGN.md §8/§15: embedded
tracks satisfy for free (recording them as #189 left to this issue),
then provider search + ranked fetch + alass sync + sidecar write, then
immediate machine translation — extracting a text-format embedded track
when that is the only source — and otherwise the reason lands on the
attempt row for the missing-subtitles queue.

Closes run as detached tasks because alass and translation outlive the
25 s reconcile action budget; every outcome is recorded in domain rows
first, so a crash converges on the next tick. Failures back off on the
same §6.2 curve as movie searches; a rate-limited provider is the
'capped' queue state; unreachable providers and translators fold into
the existing §9.5 broken notification, edge-triggered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 02:10:49 +01:00
Miguel Palhas e49bc2736d feat(arr): wire translation backends into daemon
Forward each translate-* feature from arr-daemon to arr-subs, add the
missing OpenAI model bootstrap key, and construct the compiled and
credentialed backends at startup so the translate endpoint stops
answering 503 unconditionally.
2026-08-25 02:01:55 +01:00
Miguel Palhas f1a58c0810 refactor(arr): split api_state out of run
Both #195 and #214 added a builder call to the AppState chain, which pushed
run past the too-many-lines limit. The chain grows a line per upstream the
API learns to talk to, so it gets its own function.
2026-08-25 01:50:17 +01:00
Miguel Palhas 999cb8b08c Merge #214: sync manually grabbed subtitles with alass
Closes #214

# Conflicts:
#	crates/arr-api/src/state.rs
#	crates/arr-daemon/src/main.rs
2026-08-25 01:49:20 +01:00
Miguel Palhas 0d91beff14 Merge #195: write subtitle sidecars, refresh Jellyfin
Closes #195
2026-08-25 01:48:30 +01:00
Miguel Palhas ae604e53d5 Merge #193: remote-command translation backend
Closes #193
2026-08-25 01:48:30 +01:00
Miguel Palhas 07921c9e1e test(arr): drive the command backend via stub scripts 2026-08-25 01:44:52 +01:00
Miguel Palhas 8b92a68edd feat(arr): remote-command translation backend 2026-08-25 01:44:52 +01:00
Miguel Palhas 4af6e0a083 refactor(arr): share the LLM prompt helpers across backends 2026-08-25 01:44:52 +01:00
Miguel Palhas 8baef11c0e feat(arr): refresh jellyfin after a subtitle write
DESIGN.md §7.5's watcher gap applies to a sidecar dropped next to a
file Jellyfin already knows about, same as an imported file. A grab
or translation now makes the same refresh call import does; a
refresh failure logs and never fails the write that already landed.
2026-08-25 01:31:59 +01:00
Miguel Palhas 910d28f639 refactor(arr): give arr-api its own jellyfin client
arr-daemon depends on arr-api, so a handler in arr-api can never
reach the daemon's private JellyfinClient. Move it into arr-api and
attach an instance to AppState, so a manual subtitle write can ask
for the same refresh import already does (#195).
2026-08-25 01:31:54 +01:00
Miguel Palhas f06e0e94bc feat(arr): sync manually grabbed subtitles with alass
The grab handler runs alass before recording the row, replacing the
sidecar with the synced text on acceptance and flagging it as
rejected otherwise (§15). Wires a Syncer into AppState, defaulting
to alass on PATH; the daemon binary points it at config.alass_path.
2026-08-25 01:24:09 +01:00
Miguel Palhas 635a651bee feat(arr): add Syncer::settle as the alass seam
Folds an implausible result and an unusable alass binary into one
SyncState both the grab handler (#199) and reconcile loop (#196)
can record without re-deriving the same match arms.
2026-08-25 01:24:02 +01:00
Miguel Palhas c6cfdff2c9 test(arr): read available_engines from the build
Second test that assumed the empty default feature set. It asserted
available_engines was literally [], which --all-features makes false.
2026-08-25 01:11:08 +01:00
Miguel Palhas f773489fd7 test(arr): make the engine gate test ask the build
Turning on --all-features compiled every translation backend, so the test
asserting that a known-but-uncompiled engine is refused had nothing left to
refuse and failed. It hardcoded "deepl" and a comment that no feature was
on, which stopped being true in the same commit that made CI see it.

It now picks whichever engine this build did not compile, and when all of
them are compiled asserts the complementary truth instead: a compiled
engine is accepted. Meaningful under either feature set.
2026-08-25 01:08:15 +01:00
Miguel Palhas db9ae271cb fix(arr): make CI compile the translation backends
Every translation backend sits behind a default-off cargo feature, and the
gate ran with the default set, so clippy and the test run never saw a line
of arr-subs' openai, deepl or google modules. #191 and #192 each reported
it after verifying their own work by hand. lint and test now pass
--all-features.

Also drops a duplicated wiremock suppression the #188 merge left behind and
puts the module list back in order.
2026-08-25 01:05:50 +01:00
Miguel Palhas 0f0d56ef0e Merge #192: DeepL and Google Translate backends
Closes #192
2026-08-25 01:04:51 +01:00
Miguel Palhas dd1e5d02a3 Merge #191: OpenAI-compatible translation backend
Closes #191
2026-08-25 01:04:47 +01:00
Miguel Palhas f84b04536a Merge #194: sync subtitles with alass
Closes #194
2026-08-25 01:04:43 +01:00
Miguel Palhas 3e5fdc6908 feat(arr): translate through Google Translate 2026-08-25 01:01:17 +01:00
Miguel Palhas 0c07fbaed5 feat(arr): translate through the DeepL API 2026-08-25 01:01:01 +01:00
Miguel Palhas ef0fac4933 feat(arr): let backends report billed characters 2026-08-25 01:00:41 +01:00
Miguel Palhas 82620e940c feat(arr): wrap alass for subtitle sync 2026-08-25 00:50:25 +01:00
Miguel Palhas 8efb9cbf23 feat(arr): OpenAI-compatible translation backend
Behind the existing translate-openai feature. One JSON-in/JSON-out
chat/completions request per batch; refusal, content-filter and
truncated replies fail loudly instead of validating as a short batch.
2026-08-25 00:50:06 +01:00
Miguel Palhas 26bd25e2f6 feat(arr): convert non-SRT grabs instead of refusing
#199 shipped the manual grab before #213 existed, so srt_text answered 422
for any provider not serving SRT. Fetched::to_srt already decodes and
converts, so this is that call. A format with no parser still fails rather
than reaching the disk.

Leaves the alass half of #214 open; that waits on #194.
2026-08-25 00:46:33 +01:00
Miguel Palhas 63472d99cf Merge #199: serve the subtitle API
Closes #199
2026-08-25 00:41:02 +01:00
Miguel Palhas 182285c356 Merge #213: convert non-SRT downloads to SRT
Closes #213
2026-08-25 00:41:02 +01:00
Miguel Palhas faa7a0c056 feat(arr): offer subtitle providers to the API
Provider credentials are bootstrap config and never reach the database
(DESIGN.md §10), so which providers exist is settled once at startup;
which of them a search runs is the `providers_enabled` row the API reads
per request. OpenSubtitles.com cannot be called without a registered API
key, so without one it is not offered at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-24 23:27:08 +01:00
Miguel Palhas 449750e426 feat(arr): serve the subtitle API
Issue #199, DESIGN.md §15 and §9.1. Lists what exists per media file and
per title, runs the enabled providers for one language and returns every
candidate with §9.3's verdict vocabulary — including the rejected ones
naming the rule that killed each — then grabs, translates and deletes.

Inline rather than 202-and-poll like the release deck: a subtitle search
is one or two HTTP calls and nothing persists its candidates, so there
is nothing to come back for. The cost is that a grab repeats the
`forced` and `sdh` facts the search reported, since the server does not
remember them.

Every write ends by marking the language satisfied, whether or not it is
in the wanted set. That is §15's "manual actions bypass the wanted-set
logic": the operator asking for Spanish gets Spanish, and the loop does
not then read it as a gap. A forced track is the exception §15 names — it
covers signs only — so it is recorded and satisfies nothing.

`alass` (#194) does not run yet: a fetched sidecar is recorded unsynced.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-24 23:27:03 +01:00
Miguel Palhas 0c9b9fb796 feat(arr): reopen a language on subtitle delete
DESIGN.md §15 reads satisfaction off the files, so a `satisfied` attempt
row whose sidecar was just deleted by hand is a stale claim that hides
the gap from the reconcile loop's work list. `unsatisfy` withdraws only
that claim: the attempt count and timestamp stay, because the backoff is
a fact about what providers were already asked and a delete does not
un-ask them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-24 23:26:55 +01:00
Miguel Palhas 8450b06e76 feat(arr): convert VTT and ASS downloads to SRT 2026-08-24 23:13:25 +01:00
Miguel Palhas b9f4ee98d5 fix(arr): recover lost fixtures and align provider names
The Podnapisi zip fixtures never reached the branch: a global gitignore
excludes *.zip, so git add skipped them silently and the worker's CI passed
against untracked files. Recovered, with a fixtures .gitignore that keeps
the next binary fixture from vanishing the same way.

Two providers landing in parallel also disagreed on names and on which
crate dependencies each integration-test target uses. PodnapisiProvider is
now Podnapisi, matching OpenSubtitles, and both test targets declare the
dependencies they do not use so the per-target lint stays quiet.
2026-08-24 23:07:46 +01:00
Miguel Palhas 529d7a4ee4 Merge #188: Podnapisi provider
Closes #188

# Conflicts:
#	Cargo.lock
#	crates/arr-subs/Cargo.toml
#	crates/arr-subs/src/error.rs
#	crates/arr-subs/src/lib.rs
2026-08-24 23:06:34 +01:00
Miguel Palhas ee3e2ba4b4 Merge #187: OpenSubtitles.com provider
Closes #187

# Conflicts:
#	Cargo.lock
#	crates/arr-subs/Cargo.toml
2026-08-24 23:04:46 +01:00
Miguel Palhas f617403912 Merge #212: decode fetched subtitles to UTF-8
Closes #212
2026-08-24 23:04:13 +01:00
Miguel Palhas da0c30c606 feat(arr): add opensubtitles.com subtitle provider
Searches by moviehash computed from the media file and by TMDB id
with season/episode for TV, ranks candidates through
arr_core::subs::rank, and downloads under a lazily-fetched user
token. The daily download cap (429/406) surfaces as Error::RateLimited
so the loop can show a queue state. Credentials come from config or
environment only; tests run against wiremock fixtures.
2026-08-24 22:58:48 +01:00
Miguel Palhas 6ec4703edb test(arr): cover the Podnapisi provider 2026-08-24 22:54:13 +01:00
Miguel Palhas 8311307453 feat(arr): fetch subtitles from Podnapisi 2026-08-24 22:54:13 +01:00
Miguel Palhas d8acf30fb7 feat(arr): add zip dep and provider config error 2026-08-24 22:54:13 +01:00
Miguel Palhas 81dd606414 feat(arr): decode fetched subtitles to UTF-8 2026-08-24 22:41:17 +01:00
Miguel Palhas 5d7d881ab9 feat(arr): add Error::Decode for bad subtitle bytes 2026-08-24 22:41:17 +01:00
Miguel Palhas 94d9fc8b89 Merge #190: pluggable subtitle translation
Closes #190
2026-08-24 22:36:13 +01:00
Miguel Palhas 2cee177940 Merge #198: configure subtitles via env and database
Closes #198
2026-08-24 22:36:12 +01:00
Miguel Palhas 340c113007 Merge #189: extract text subtitle streams to SRT
Closes #189
2026-08-24 22:36:12 +01:00
Miguel Palhas 0bdf0103bd feat(arr): extract text subtitle tracks to srt
ffmpeg, spawned and left to die like ffprobe, maps one subtitle stream
and converts it to SRT under §15's sidecar name. Text formats become
legal translation sources; bitmap tracks never extract.
2026-08-24 22:32:26 +01:00
Miguel Palhas e852f42a9c feat(arr): name subtitle codecs and dispositions
SubtitleTrack now carries the codec, split text formats from bitmap
ones per DESIGN.md §15, plus the forced and SDH dispositions ffprobe
reports. Without the forced flag a file carrying only a forced track
read as satisfied for that language.
2026-08-24 22:32:01 +01:00
Miguel Palhas 17d40f3e71 feat(arr): add subtitle bootstrap config 2026-08-24 22:29:18 +01:00
Miguel Palhas 802d2a1208 feat(arr): expose subtitle settings API 2026-08-24 22:28:31 +01:00
Miguel Palhas ed1a74e03f feat(arr): report compiled subtitle engines 2026-08-24 22:28:17 +01:00
Miguel Palhas 1900a82d55 feat(arr): add subtitle_settings table 2026-08-24 22:28:14 +01:00
Miguel Palhas 39ddf65afa style(arr): rustfmt arr-subs 2026-08-24 22:21:40 +01:00
Miguel Palhas e55ce05880 feat(arr): add pluggable subtitle translation layer
The Backend trait plus everything the backends (#191-#193) share, so no
backend can skip it: chunking into character-budgeted batches, rejection
of replies whose cue count or numbering drifted, and reassembly onto the
original timings. Timing data never leaves arr; pt-PT and pt-BR are
distinct targets a backend must refuse rather than conflate.
2026-08-24 22:21:09 +01:00
Miguel Palhas cd304d552c feat(arr): parse and render SRT cues in arr-subs
Translation reassembles translated text onto original timings (§15), so
cues need a structured form. Parsing is tolerant of real files (CRLF,
BOM, missing indices, dot milliseconds); rendering is strict and
renumbers from 1.
2026-08-24 22:21:09 +01:00
Miguel Palhas cdd6133bce fix(arr): close the subtitle ranking seam
#184 and #185 were built in parallel and their candidate types did not
meet. Ranking returned no identity for a candidate, so the winner of a
rank() could not be handed back to Provider::download -- rank() reorders,
so the caller could not recover it by position either.

RankedSubtitle now carries the index of the candidate in the slice it was
given, and Candidate::to_core is the one place the two shapes are mapped:
hash_match against a compared hash, sdh against hearing_impaired, group
against release_group, and the two optional tiebreakers defaulted to sort
last rather than block a candidate.
2026-08-24 22:14:01 +01:00
Miguel Palhas 44c7013e49 Merge #186: persist subtitle files and attempts
Closes #186
2026-08-24 22:06:01 +01:00
Miguel Palhas b0e3905486 Merge #185: rank subtitle candidates in arr-core
Closes #185
2026-08-24 22:06:01 +01:00
Miguel Palhas 3d2b816f9f Merge #184: add arr-subs with the provider trait
Closes #184
2026-08-24 22:06:01 +01:00
Miguel Palhas c3bd009442 feat(arr): add subtitle queries
Recording a subtitle is idempotent on both keys the schema carries: the
sidecar path, and the language an embedded track satisfies, so a second
probe or a re-import converges instead of duplicating. Attempts are
upserted per (media file, language); the work list is every language that
is not satisfied, newest import first, which is the order §15 wants the
daily allowance spent in.
2026-08-24 21:46:13 +01:00
Miguel Palhas 151135b545 feat(arr): add subtitle tables
DESIGN.md §15 needs two shapes: what subtitles exist for a media file,
and what arr has tried per wanted language. An embedded track carries no
path — it is recorded because it satisfies a language, not because there
is a file — and the CHECK constraints tie provider, engine and path to
the origin so an impossible row cannot be written.
2026-08-24 21:46:04 +01:00
Miguel Palhas 89154614e1 feat(arr): rank subtitle candidates in arr-core
Implements DESIGN.md §15 ranking: moviehash > exact release-name >
group/source > rating/downloads, forced always rejected, SDH always
below plain. Verdict mirrors the release path (eligible / rejected(rule)).
2026-08-24 21:43:46 +01:00
Miguel Palhas 8b83ac9511 feat(arr): add arr-subs with the provider trait
The skeleton the subtitles milestone hangs off: provider domain types, an
object-safe Provider trait, the crate's own error type, and the cargo
features the translation backends will sit behind. No provider, no
translation, no ranking — DESIGN.md §15 keeps ranking pure in arr-core.
2026-08-24 21:42:40 +01:00
Miguel Palhas 695cb69f2f Merge #183: DESIGN.md §15 subtitles contract
Closes #183
2026-08-24 21:38:01 +01:00
Miguel Palhas 1401e9e00f docs(arr): add DESIGN.md §15 subtitles contract
Records the subtitle decisions settled during Subtitles milestone
planning: global wanted set (pt + en), embedded-track extraction,
providers, ranking, translation backends, sidecar naming, alass sync,
budgets, and stated non-goals. Points §13 item 9 at §15 and adds
arr-subs to the crate lists.

Closes-Issue: #183

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 21:37:37 +01:00
154 changed files with 25158 additions and 276 deletions
+14 -2
View File
@@ -13,6 +13,7 @@ on:
- 'crates/arr-indexer/**'
- 'crates/arr-meta/**'
- 'crates/arr-probe/**'
- 'crates/arr-subs/**'
- 'crates/arr-api/**'
- 'crates/arr-daemon/**'
- '.gitea/workflows/e2e.yml'
@@ -70,10 +71,21 @@ jobs:
curl -LsSf https://get.nexte.st/latest/linux \
| tar zxf - -C "$HOME/.cargo/bin"
- name: alass
# No distro package; the subtitle sync path (DESIGN.md §15) shells
# out to a binary literally named `alass`, which the crate is not.
run: |
export PATH="$HOME/.cargo/bin:$PATH"
command -v alass >/dev/null || {
cargo install alass-cli --locked
ln -sf "$HOME/.cargo/bin/alass-cli" "$HOME/.cargo/bin/alass"
}
- name: build daemon
# The harness spawns this binary; building it here keeps the compile
# out of the first test's boot window.
run: cargo build -p arr-daemon --bin arr
# out of the first test's boot window. `translate-command` is the
# subtitle e2e scenarios' stub-script translation backend.
run: cargo build -p arr-daemon --bin arr --features translate-command
- name: e2e
env:
@@ -0,0 +1,38 @@
{
"db_name": "SQLite",
"query": "SELECT r.name AS \"name!: String\",\n r.parsed AS \"parsed!: serde_json::Value\"\n FROM episodes e\n JOIN grabs g ON g.target_kind = 'season' AND g.target_id = e.season_id\n JOIN releases r ON r.id = g.release_id\n WHERE e.id = ?\n ORDER BY g.imported_at DESC, g.id DESC LIMIT 1",
"describe": {
"columns": [
{
"name": "name!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "releases",
"name": "name"
}
}
},
{
"name": "parsed!: serde_json::Value",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "releases",
"name": "parsed"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false
]
},
"hash": "05b5757f0fb7e6ab1176387854a14fc8db793871193b6f16e3719d73040e8ce8"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_attempts\n (media_file_id, language, state, attempts, last_attempt_at, last_failure)\n VALUES (?, ?, ?, 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now'), ?)\n ON CONFLICT (media_file_id, language) DO UPDATE\n SET state = excluded.state,\n attempts = subtitle_attempts.attempts + 1,\n last_attempt_at = excluded.last_attempt_at,\n last_failure = excluded.last_failure,\n updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now')",
"describe": {
"columns": [],
"parameters": {
"Right": 4
},
"nullable": []
},
"hash": "08c356bfcd7f7cd7a4bac0bf0ce44ce3276577cb18f32b0c01839150e7827bca"
}
@@ -0,0 +1,21 @@
{
"db_name": "SQLite",
"query": "SELECT EXISTS(SELECT 1 FROM episodes WHERE id = ?) AS \"exists!: bool\"",
"describe": {
"columns": [
{
"name": "exists!: bool",
"ordinal": 0,
"type_info": "Integer",
"origin": "Expression"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "0b40dbc80628244531a044e872238e356ab6bc66567c72008cff506e595932b1"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT wanted_languages FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "wanted_languages",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "wanted_languages"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false
]
},
"hash": "0f94b0f839cfcbe4a70ee5be81f452c88dc610d05042d4af811963c0cbc6113e"
}
@@ -0,0 +1,98 @@
{
"db_name": "SQLite",
"query": "SELECT required_audio AS \"required_audio!: String\",\n dub_blacklist AS \"dub_blacklist!: String\",\n hdr_rules AS \"hdr_rules!: String\",\n size_bands AS \"size_bands!: String\",\n resolution_pref AS \"resolution_pref!: String\",\n source_weights AS \"source_weights!: String\",\n score_weights AS \"score_weights!: String\"\n FROM policies WHERE id = ?",
"describe": {
"columns": [
{
"name": "required_audio!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "required_audio"
}
}
},
{
"name": "dub_blacklist!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "dub_blacklist"
}
}
},
{
"name": "hdr_rules!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "hdr_rules"
}
}
},
{
"name": "size_bands!: String",
"ordinal": 3,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "size_bands"
}
}
},
{
"name": "resolution_pref!: String",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "resolution_pref"
}
}
},
{
"name": "source_weights!: String",
"ordinal": 5,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "source_weights"
}
}
},
{
"name": "score_weights!: String",
"ordinal": 6,
"type_info": "Text",
"origin": {
"Table": {
"table": "policies",
"name": "score_weights"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
false,
false,
false,
false
]
},
"hash": "100e5e9297a9a917eb107c673ac492efe0df0123d9def212d84b88e57aaa0484"
}
@@ -0,0 +1,21 @@
{
"db_name": "SQLite",
"query": "SELECT EXISTS(SELECT 1 FROM series WHERE id = ?) AS \"exists!: bool\"",
"describe": {
"columns": [
{
"name": "exists!: bool",
"ordinal": 0,
"type_info": "Integer",
"origin": "Expression"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "2084928704dbb8a2c81692b9aa3b6edff64745945ea4e88d70bcb139e272c857"
}
@@ -0,0 +1,21 @@
{
"db_name": "SQLite",
"query": "SELECT EXISTS(\n SELECT 1 FROM subtitle_files\n WHERE media_file_id = ? AND language = ? AND forced = 0\n ) AS \"exists!: bool\"",
"describe": {
"columns": [
{
"name": "exists!: bool",
"ordinal": 0,
"type_info": "Integer",
"origin": "Expression"
}
],
"parameters": {
"Right": 2
},
"nullable": [
false
]
},
"hash": "233bae03912e50e6bcde4b24a9d7fbe256dbf626e4e236a06ba08a00643566d3"
}
@@ -0,0 +1,50 @@
{
"db_name": "SQLite",
"query": "SELECT sr.tmdb_id AS \"tmdb_id!: i64\",\n s.number AS \"season!: i64\",\n e.number AS \"episode!: i64\"\n FROM episodes e\n JOIN seasons s ON s.id = e.season_id\n JOIN series sr ON sr.id = s.series_id\n WHERE e.id = ?",
"describe": {
"columns": [
{
"name": "tmdb_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "tmdb_id"
}
}
},
{
"name": "season!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "seasons",
"name": "number"
}
}
},
{
"name": "episode!: i64",
"ordinal": 2,
"type_info": "Integer",
"origin": {
"Table": {
"table": "episodes",
"name": "number"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false
]
},
"hash": "268cb11282b16494330c1655ba3dfcb4f2506686822711f3c6489c8ddd42b0e7"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT wanted_languages AS \"wanted_languages!: String\" FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "wanted_languages!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "wanted_languages"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false
]
},
"hash": "2afeefde38499d722f329d1d472bc89524ce6498ccd4098ba97979e0621b840b"
}
@@ -0,0 +1,122 @@
{
"db_name": "SQLite",
"query": "SELECT m.id AS \"movie_id!: i64\", m.tmdb_id AS \"tmdb_id!: i64\",\n m.title AS \"title!: String\", m.year, m.poster_path,\n mf.id AS \"media_file_id!: i64\", sa.language AS \"language!: String\",\n sa.state AS \"state!: arr_db::SubtitleState\", sa.last_failure\n FROM subtitle_attempts sa\n JOIN media_files mf ON mf.id = sa.media_file_id AND mf.owner_kind = 'movie'\n JOIN movies m ON m.id = mf.owner_id\n WHERE sa.state IN ('failed', 'capped', 'unavailable')\n ORDER BY m.title, sa.language",
"describe": {
"columns": [
{
"name": "movie_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "id"
}
}
},
{
"name": "tmdb_id!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "tmdb_id"
}
}
},
{
"name": "title!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "movies",
"name": "title"
}
}
},
{
"name": "year",
"ordinal": 3,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "year"
}
}
},
{
"name": "poster_path",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "movies",
"name": "poster_path"
}
}
},
{
"name": "media_file_id!: i64",
"ordinal": 5,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
},
{
"name": "language!: String",
"ordinal": 6,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "language"
}
}
},
{
"name": "state!: arr_db::SubtitleState",
"ordinal": 7,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "state"
}
}
},
{
"name": "last_failure",
"ordinal": 8,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "last_failure"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
false,
false,
true,
true,
false,
false,
false,
true
]
},
"hash": "2b67abe4db9e00517fecfa53fbad5bbbc633a9f537eaaaa9d46a445d8c25eb2c"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "UPDATE subtitle_attempts\n SET state = 'wanted',\n updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now')\n WHERE media_file_id = ? AND language = ? AND state = 'satisfied'",
"describe": {
"columns": [],
"parameters": {
"Right": 2
},
"nullable": []
},
"hash": "31bed4c9097d6ea2ed189de48440bd0d882873d27450e5d177baed65c74fd7de"
}
@@ -0,0 +1,74 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\",\n path AS \"path!: String\",\n size AS \"size!: i64\",\n owner_kind AS \"owner_kind!: String\",\n owner_id AS \"owner_id!: i64\"\n FROM media_files WHERE id = ?",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
},
{
"name": "path!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "media_files",
"name": "path"
}
}
},
{
"name": "size!: i64",
"ordinal": 2,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "size"
}
}
},
{
"name": "owner_kind!: String",
"ordinal": 3,
"type_info": "Text",
"origin": {
"Table": {
"table": "media_files",
"name": "owner_kind"
}
}
},
{
"name": "owner_id!: i64",
"ordinal": 4,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "owner_id"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
false,
false
]
},
"hash": "32443dc6e5bdd414340443c5d527b2578bcdeebd053b667db35423d829d4bf10"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_attempts (media_file_id, language)\n SELECT id, ? FROM media_files WHERE probed IS NOT NULL\n ON CONFLICT (media_file_id, language) DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Right": 1
},
"nullable": []
},
"hash": "35ace0417f613340f82522549d70d0dffeb680afb57511cb17fa15b3a571b974"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_files\n (media_file_id, language, origin, provider, candidate_id, engine,\n forced, sdh, synced, sync_rejected, path)\n VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\n ON CONFLICT (path) DO NOTHING\n ON CONFLICT (media_file_id, language, forced, sdh)\n WHERE origin = 'embedded' DO NOTHING\n RETURNING id AS \"id!: i64\"",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "id"
}
}
}
],
"parameters": {
"Right": 11
},
"nullable": [
null
]
},
"hash": "3d48b78768bf8dc2e337d0935889a5654089f38bb7ee2a15b7e19f4ed5540262"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT count(*) AS \"failures!: i64\",\n max(grabbed_at) AS \"last_failed_at?: String\"\n FROM grabs\n WHERE target_kind = 'season' AND target_id = ? AND state = 'failed'",
"query": "SELECT count(*) AS \"failures!: i64\",\n max(coalesce(failed_at, grabbed_at)) AS \"last_failed_at?: String\"\n FROM grabs\n WHERE target_kind = 'season' AND target_id = ? AND state = 'failed'",
"describe": {
"columns": [
{
@@ -24,5 +24,5 @@
true
]
},
"hash": "ea21a91634b441e4cacf693f767549ae5075a56a668e0bf836d85e22d9202019"
"hash": "3e8fdbb8d28441b429d2ef011f206c4fad280f56905b6a85035a142dd592dbec"
}
@@ -0,0 +1,74 @@
{
"db_name": "SQLite",
"query": "SELECT g.target_id AS \"season_id!: i64\",\n r.name AS \"name!: String\",\n g.infohash AS \"infohash!: String\",\n g.failed_at,\n g.grabbed_at AS \"grabbed_at!: String\"\n FROM grabs g\n JOIN releases r ON r.id = g.release_id\n JOIN seasons s ON s.id = g.target_id\n WHERE g.target_kind = 'season'\n AND g.state = 'failed'\n AND s.series_id = ?\n AND EXISTS (\n SELECT 1 FROM episodes e\n WHERE e.season_id = s.id AND e.wanted\n AND NOT EXISTS (\n SELECT 1 FROM media_files f\n WHERE f.owner_kind = 'episode' AND f.owner_id = e.id\n )\n )\n ORDER BY coalesce(g.failed_at, g.grabbed_at), g.id",
"describe": {
"columns": [
{
"name": "season_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "grabs",
"name": "target_id"
}
}
},
{
"name": "name!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "releases",
"name": "name"
}
}
},
{
"name": "infohash!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "grabs",
"name": "infohash"
}
}
},
{
"name": "failed_at",
"ordinal": 3,
"type_info": "Text",
"origin": {
"Table": {
"table": "grabs",
"name": "failed_at"
}
}
},
{
"name": "grabbed_at!: String",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "grabs",
"name": "grabbed_at"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
true,
false
]
},
"hash": "3f252a992c1bc5b18bb4467d1c4fd4137966a469b134fbeb51fb91f67951cbbe"
}
@@ -0,0 +1,38 @@
{
"db_name": "SQLite",
"query": "SELECT r.name AS \"name!: String\",\n r.parsed AS \"parsed!: serde_json::Value\"\n FROM grabs g JOIN releases r ON r.id = g.release_id\n WHERE g.target_kind = ? AND g.target_id = ?\n ORDER BY g.imported_at DESC, g.id DESC LIMIT 1",
"describe": {
"columns": [
{
"name": "name!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "releases",
"name": "name"
}
}
},
{
"name": "parsed!: serde_json::Value",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "releases",
"name": "parsed"
}
}
}
],
"parameters": {
"Right": 2
},
"nullable": [
false,
false
]
},
"hash": "432cf38775bc4d7dee1dd9d009e06638ea57f805127f7634f34b8189e831f461"
}
@@ -0,0 +1,98 @@
{
"db_name": "SQLite",
"query": "SELECT a.media_file_id AS \"media_file_id!: i64\",\n a.language AS \"language!: String\",\n a.state AS \"state!: SubtitleState\",\n a.attempts AS \"attempts!: i64\",\n a.last_attempt_at,\n a.last_failure,\n f.path AS \"media_file_path!: String\"\n FROM subtitle_attempts a\n JOIN media_files f ON f.id = a.media_file_id\n WHERE a.state IN ('wanted', 'failed')\n ORDER BY f.created_at DESC, a.last_attempt_at, a.language\n LIMIT ?",
"describe": {
"columns": [
{
"name": "media_file_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "media_file_id"
}
}
},
{
"name": "language!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "language"
}
}
},
{
"name": "state!: SubtitleState",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "state"
}
}
},
{
"name": "attempts!: i64",
"ordinal": 3,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "attempts"
}
}
},
{
"name": "last_attempt_at",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "last_attempt_at"
}
}
},
{
"name": "last_failure",
"ordinal": 5,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "last_failure"
}
}
},
{
"name": "media_file_path!: String",
"ordinal": 6,
"type_info": "Text",
"origin": {
"Table": {
"table": "media_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
false,
true,
true,
false
]
},
"hash": "47ad9162ba373dfbdf061898f5f148e65de261b494453804db912a919a78c2c5"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT tmdb_id AS \"tmdb_id!: i64\" FROM movies WHERE id = ?",
"describe": {
"columns": [
{
"name": "tmdb_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "tmdb_id"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "4f4f037c956ac6b246503354ff9b59db59e36f9a6d59adc0753b95662c9c630a"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT sf.path AS \"path!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id\n JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id\n JOIN seasons se ON se.id = e.season_id\n WHERE se.series_id = ? AND sf.path IS NOT NULL",
"describe": {
"columns": [
{
"name": "path!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
true
]
},
"hash": "5f3a663eef0318b38cd864a2e06befa20fba22fc2d281295aed81a851a35ea07"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_attempts (media_file_id, language)\n VALUES (?, ?)\n ON CONFLICT (media_file_id, language) DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Right": 2
},
"nullable": []
},
"hash": "6236e1be0e268c678271dd72be52454347da6637f0bf3ecb52dda82cddfe459a"
}
@@ -0,0 +1,158 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\",\n media_file_id AS \"media_file_id!: i64\",\n language AS \"language!: String\",\n origin AS \"origin!: SubtitleOrigin\",\n provider,\n candidate_id,\n engine,\n forced AS \"forced!: bool\",\n sdh AS \"sdh!: bool\",\n synced AS \"synced!: bool\",\n sync_rejected AS \"sync_rejected!: bool\",\n path\n FROM subtitle_files\n WHERE media_file_id = ?\n ORDER BY language, id",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "id"
}
}
},
{
"name": "media_file_id!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "media_file_id"
}
}
},
{
"name": "language!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "language"
}
}
},
{
"name": "origin!: SubtitleOrigin",
"ordinal": 3,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "origin"
}
}
},
{
"name": "provider",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "provider"
}
}
},
{
"name": "candidate_id",
"ordinal": 5,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "candidate_id"
}
}
},
{
"name": "engine",
"ordinal": 6,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "engine"
}
}
},
{
"name": "forced!: bool",
"ordinal": 7,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "forced"
}
}
},
{
"name": "sdh!: bool",
"ordinal": 8,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "sdh"
}
}
},
{
"name": "synced!: bool",
"ordinal": 9,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "synced"
}
}
},
{
"name": "sync_rejected!: bool",
"ordinal": 10,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "sync_rejected"
}
}
},
{
"name": "path",
"ordinal": 11,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
false,
true,
true,
true,
false,
false,
false,
false,
true
]
},
"hash": "62d625c322c8c64b48317cdbf466d74043708d8e12997e0efec23a18eaaaaace"
}
@@ -0,0 +1,98 @@
{
"db_name": "SQLite",
"query": "SELECT m.id AS \"movie_id!: i64\", m.tmdb_id AS \"tmdb_id!: i64\",\n m.title AS \"title!: String\", m.year, m.poster_path,\n mf.id AS \"media_file_id!: i64\", sf.language AS \"language!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id AND mf.owner_kind = 'movie'\n JOIN movies m ON m.id = mf.owner_id\n WHERE sf.sync_rejected = 1\n ORDER BY m.title, sf.language",
"describe": {
"columns": [
{
"name": "movie_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "id"
}
}
},
{
"name": "tmdb_id!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "tmdb_id"
}
}
},
{
"name": "title!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "movies",
"name": "title"
}
}
},
{
"name": "year",
"ordinal": 3,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "year"
}
}
},
{
"name": "poster_path",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "movies",
"name": "poster_path"
}
}
},
{
"name": "media_file_id!: i64",
"ordinal": 5,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
},
{
"name": "language!: String",
"ordinal": 6,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "language"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
false,
false,
true,
true,
false,
false
]
},
"hash": "69f34468a61979b85b7513056d58e34c743af3ae62d5327d4dea49fe7586a85b"
}
@@ -0,0 +1,158 @@
{
"db_name": "SQLite",
"query": "SELECT s.id AS \"series_id!: i64\", s.tmdb_id AS \"tmdb_id!: i64\",\n s.title AS \"title!: String\", s.year, s.poster_path,\n e.id AS \"episode_id!: i64\", se.number AS \"season_number!: i64\",\n e.number AS \"episode_number!: i64\", mf.id AS \"media_file_id!: i64\",\n sa.language AS \"language!: String\",\n sa.state AS \"state!: arr_db::SubtitleState\", sa.last_failure\n FROM subtitle_attempts sa\n JOIN media_files mf ON mf.id = sa.media_file_id AND mf.owner_kind = 'episode'\n JOIN episodes e ON e.id = mf.owner_id\n JOIN seasons se ON se.id = e.season_id\n JOIN series s ON s.id = se.series_id\n WHERE sa.state IN ('failed', 'capped', 'unavailable')\n ORDER BY s.title, se.number, e.number, sa.language",
"describe": {
"columns": [
{
"name": "series_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "id"
}
}
},
{
"name": "tmdb_id!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "tmdb_id"
}
}
},
{
"name": "title!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "series",
"name": "title"
}
}
},
{
"name": "year",
"ordinal": 3,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "year"
}
}
},
{
"name": "poster_path",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "series",
"name": "poster_path"
}
}
},
{
"name": "episode_id!: i64",
"ordinal": 5,
"type_info": "Integer",
"origin": {
"Table": {
"table": "episodes",
"name": "id"
}
}
},
{
"name": "season_number!: i64",
"ordinal": 6,
"type_info": "Integer",
"origin": {
"Table": {
"table": "seasons",
"name": "number"
}
}
},
{
"name": "episode_number!: i64",
"ordinal": 7,
"type_info": "Integer",
"origin": {
"Table": {
"table": "episodes",
"name": "number"
}
}
},
{
"name": "media_file_id!: i64",
"ordinal": 8,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
},
{
"name": "language!: String",
"ordinal": 9,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "language"
}
}
},
{
"name": "state!: arr_db::SubtitleState",
"ordinal": 10,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "state"
}
}
},
{
"name": "last_failure",
"ordinal": 11,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "last_failure"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
false,
false,
true,
true,
false,
false,
false,
false,
false,
false,
true
]
},
"hash": "6cdb9a9344bbab557433da35471968b6e5cf96b5b220ffb41f10b6d851884803"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_budget_spend (kind, name, day, spent)\n SELECT ?1, ?2, strftime('%Y-%m-%d', 'now'), ?3\n WHERE ?3 <= ?4\n ON CONFLICT (kind, name, day) DO UPDATE\n SET spent = subtitle_budget_spend.spent + excluded.spent\n WHERE subtitle_budget_spend.spent + excluded.spent <= ?4\n RETURNING spent AS \"spent!: i64\"",
"describe": {
"columns": [
{
"name": "spent!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_budget_spend",
"name": "spent"
}
}
}
],
"parameters": {
"Right": 4
},
"nullable": [
false
]
},
"hash": "6e27f3a4c642ac324ef5ae2017a54b93f1d95aa54c46ff7e169278a295d1a2cc"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "DELETE FROM subtitle_attempts WHERE language = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 1
},
"nullable": []
},
"hash": "74178ac3ec41aa8a5ce164103a8d8301cb555eea2fa5d655a7365fd02c557cce"
}
@@ -0,0 +1,74 @@
{
"db_name": "SQLite",
"query": "SELECT wanted_languages AS \"wanted_languages!: String\",\n providers_enabled AS \"providers_enabled!: String\",\n translation_engine AS \"translation_engine: String\",\n provider_daily_budgets AS \"provider_daily_budgets!: String\",\n translator_daily_budgets AS \"translator_daily_budgets!: String\"\n FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "wanted_languages!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "wanted_languages"
}
}
},
{
"name": "providers_enabled!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "providers_enabled"
}
}
},
{
"name": "translation_engine: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "translation_engine"
}
}
},
{
"name": "provider_daily_budgets!: String",
"ordinal": 3,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "provider_daily_budgets"
}
}
},
{
"name": "translator_daily_budgets!: String",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "translator_daily_budgets"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
false,
true,
false,
false
]
},
"hash": "77e55f2aa534468cf6e599d0c01679653553a68f2f1b9b5516f5d2fb42974b05"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT r.id AS \"id!: i64\", r.indexer_id AS \"indexer_id!: i64\", r.guid AS \"guid!: String\", r.name AS \"name!: String\", r.size AS \"size!: i64\", r.seeders, r.publish_date, r.download_url AS \"download_url!: String\", r.parsed AS \"parsed!: serde_json::Value\", r.score, r.verdict, r.rejected_rule FROM releases r JOIN episode_releases er ON er.release_id = r.id WHERE er.episode_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id",
"query": "SELECT r.id AS \"id!: i64\", r.indexer_id AS \"indexer_id!: i64\", r.guid AS \"guid!: String\", r.name AS \"name!: String\", r.size AS \"size!: i64\", r.seeders, r.publish_date, r.download_url AS \"download_url!: String\", r.parsed AS \"parsed!: serde_json::Value\", r.score, r.verdict, r.rejected_rule, NULL AS \"blacklist_reason?: String\" FROM releases r JOIN movie_releases mr ON mr.release_id = r.id WHERE mr.movie_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id",
"describe": {
"columns": [
{
@@ -134,6 +134,12 @@
"name": "rejected_rule"
}
}
},
{
"name": "blacklist_reason?: String",
"ordinal": 12,
"type_info": "Null",
"origin": "Expression"
}
],
"parameters": {
@@ -151,8 +157,9 @@
false,
true,
true,
true,
true
]
},
"hash": "f203b69afb44bfb5f906ff2e1ec13915645c35af5349032be9898a697d0d05ba"
"hash": "7aa154a1bd54ec84412a1be6d7db51bb65ce9702e3bb1d26b51b93c5fdefa79f"
}
@@ -0,0 +1,50 @@
{
"db_name": "SQLite",
"query": "SELECT remote_command_timeout_seconds AS \"remote_command_timeout_seconds!: i64\",\n openai_base_url AS \"openai_base_url: String\",\n openai_model AS \"openai_model: String\"\n FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "remote_command_timeout_seconds!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "remote_command_timeout_seconds"
}
}
},
{
"name": "openai_base_url: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "openai_base_url"
}
}
},
{
"name": "openai_model: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "openai_model"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
true,
true
]
},
"hash": "7c501e25106ea8d4ad9f28ca904c446551c6afd68d85e2231742bebd215a1a4b"
}
@@ -0,0 +1,110 @@
{
"db_name": "SQLite",
"query": "SELECT wanted_languages AS \"wanted_languages!: String\",\n providers_enabled AS \"providers_enabled!: String\",\n translation_engine AS \"translation_engine: String\",\n provider_daily_budgets AS \"provider_daily_budgets!: String\",\n translator_daily_budgets AS \"translator_daily_budgets!: String\",\n remote_command_timeout_seconds AS \"remote_command_timeout_seconds!: i64\",\n openai_base_url AS \"openai_base_url: String\",\n openai_model AS \"openai_model: String\"\n FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "wanted_languages!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "wanted_languages"
}
}
},
{
"name": "providers_enabled!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "providers_enabled"
}
}
},
{
"name": "translation_engine: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "translation_engine"
}
}
},
{
"name": "provider_daily_budgets!: String",
"ordinal": 3,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "provider_daily_budgets"
}
}
},
{
"name": "translator_daily_budgets!: String",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "translator_daily_budgets"
}
}
},
{
"name": "remote_command_timeout_seconds!: i64",
"ordinal": 5,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "remote_command_timeout_seconds"
}
}
},
{
"name": "openai_base_url: String",
"ordinal": 6,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "openai_base_url"
}
}
},
{
"name": "openai_model: String",
"ordinal": 7,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "openai_model"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
false,
true,
false,
false,
false,
true,
true
]
},
"hash": "82bb36910db14d5471117285ef2d3a9f8efefc5d400a8f94e1f4aed321284504"
}
@@ -0,0 +1,134 @@
{
"db_name": "SQLite",
"query": "SELECT s.id AS \"series_id!: i64\", s.tmdb_id AS \"tmdb_id!: i64\",\n s.title AS \"title!: String\", s.year, s.poster_path,\n e.id AS \"episode_id!: i64\", se.number AS \"season_number!: i64\",\n e.number AS \"episode_number!: i64\", mf.id AS \"media_file_id!: i64\",\n sf.language AS \"language!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id AND mf.owner_kind = 'episode'\n JOIN episodes e ON e.id = mf.owner_id\n JOIN seasons se ON se.id = e.season_id\n JOIN series s ON s.id = se.series_id\n WHERE sf.sync_rejected = 1\n ORDER BY s.title, se.number, e.number, sf.language",
"describe": {
"columns": [
{
"name": "series_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "id"
}
}
},
{
"name": "tmdb_id!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "tmdb_id"
}
}
},
{
"name": "title!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "series",
"name": "title"
}
}
},
{
"name": "year",
"ordinal": 3,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "year"
}
}
},
{
"name": "poster_path",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "series",
"name": "poster_path"
}
}
},
{
"name": "episode_id!: i64",
"ordinal": 5,
"type_info": "Integer",
"origin": {
"Table": {
"table": "episodes",
"name": "id"
}
}
},
{
"name": "season_number!: i64",
"ordinal": 6,
"type_info": "Integer",
"origin": {
"Table": {
"table": "seasons",
"name": "number"
}
}
},
{
"name": "episode_number!: i64",
"ordinal": 7,
"type_info": "Integer",
"origin": {
"Table": {
"table": "episodes",
"name": "number"
}
}
},
{
"name": "media_file_id!: i64",
"ordinal": 8,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
},
{
"name": "language!: String",
"ordinal": 9,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "language"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
false,
false,
true,
true,
false,
false,
false,
false,
false
]
},
"hash": "8543e4ad8c59e83911688ceace72f66d360240a45eee05aaf15dab233e22e7e6"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "UPDATE subtitle_files\n SET synced = ?, sync_rejected = ?,\n updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now')\n WHERE id = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 3
},
"nullable": []
},
"hash": "874079e2ed732468a1f249e6aa026753c78042e164d11eb7cc014c9500211b64"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT r.id AS \"id!: i64\", r.indexer_id AS \"indexer_id!: i64\", r.guid AS \"guid!: String\", r.name AS \"name!: String\", r.size AS \"size!: i64\", r.seeders, r.publish_date, r.download_url AS \"download_url!: String\", r.parsed AS \"parsed!: serde_json::Value\", r.score, r.verdict, r.rejected_rule FROM releases r JOIN movie_releases mr ON mr.release_id = r.id WHERE mr.movie_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id",
"query": "SELECT r.id AS \"id!: i64\", r.indexer_id AS \"indexer_id!: i64\", r.guid AS \"guid!: String\", r.name AS \"name!: String\", r.size AS \"size!: i64\", r.seeders, r.publish_date, r.download_url AS \"download_url!: String\", r.parsed AS \"parsed!: serde_json::Value\", r.score, r.verdict, r.rejected_rule, NULL AS \"blacklist_reason?: String\" FROM releases r JOIN episode_releases er ON er.release_id = r.id WHERE er.episode_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id",
"describe": {
"columns": [
{
@@ -134,6 +134,12 @@
"name": "rejected_rule"
}
}
},
{
"name": "blacklist_reason?: String",
"ordinal": 12,
"type_info": "Null",
"origin": "Expression"
}
],
"parameters": {
@@ -151,8 +157,9 @@
false,
true,
true,
true,
true
]
},
"hash": "aebbabd41e2086ac37dbd9d2151b6d54cbad525b1d1d63c2a1495c24af44db7f"
"hash": "8b2aa810e679ddde423a5dc5a1a89e0967206e8186b2157bb4b4d11e9a136759"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "DELETE FROM subtitle_files WHERE id = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 1
},
"nullable": []
},
"hash": "90480a6b6e724d7e4d05239799ad2f586c57deea100320fe433da007396f1e48"
}
@@ -0,0 +1,50 @@
{
"db_name": "SQLite",
"query": "SELECT media_file_id AS \"media_file_id!: i64\",\n language AS \"language!: String\",\n path\n FROM subtitle_files WHERE id = ?",
"describe": {
"columns": [
{
"name": "media_file_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "media_file_id"
}
}
},
{
"name": "language!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "language"
}
}
},
{
"name": "path",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
true
]
},
"hash": "9091de460ba1cb47c9faa24fe37fee03615e7b23b10ceef88ee12c8deb0dbaaf"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT normalised_name AS \"normalised_name!: String\", infohash FROM blacklist",
"query": "SELECT normalised_name AS \"normalised_name!: String\",\n infohash,\n reason AS \"reason!: String\"\n FROM blacklist\n ORDER BY id",
"describe": {
"columns": [
{
@@ -24,6 +24,17 @@
"name": "infohash"
}
}
},
{
"name": "reason!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "blacklist",
"name": "reason"
}
}
}
],
"parameters": {
@@ -31,8 +42,9 @@
},
"nullable": [
false,
true
true,
false
]
},
"hash": "071c14544e225001d31c5da60c90d3144fc5a071893c74b2c1eea51bfe00ac97"
"hash": "939252a81cdd0103b2aaa3cf19d5cae6e6e327709dc9e5a2073092b75137e13c"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT r.id AS \"id!: i64\", r.indexer_id AS \"indexer_id!: i64\", r.guid AS \"guid!: String\", r.name AS \"name!: String\", r.size AS \"size!: i64\", r.seeders, r.publish_date, r.download_url AS \"download_url!: String\", r.parsed AS \"parsed!: serde_json::Value\", r.score, r.verdict, r.rejected_rule FROM releases r JOIN season_releases sr ON sr.release_id = r.id WHERE sr.season_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id",
"query": "SELECT r.id AS \"id!: i64\", r.indexer_id AS \"indexer_id!: i64\", r.guid AS \"guid!: String\", r.name AS \"name!: String\", r.size AS \"size!: i64\", r.seeders, r.publish_date, r.download_url AS \"download_url!: String\", r.parsed AS \"parsed!: serde_json::Value\", r.score, r.verdict, r.rejected_rule, NULL AS \"blacklist_reason?: String\" FROM releases r JOIN season_releases sr ON sr.release_id = r.id WHERE sr.season_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id",
"describe": {
"columns": [
{
@@ -134,6 +134,12 @@
"name": "rejected_rule"
}
}
},
{
"name": "blacklist_reason?: String",
"ordinal": 12,
"type_info": "Null",
"origin": "Expression"
}
],
"parameters": {
@@ -151,8 +157,9 @@
false,
true,
true,
true,
true
]
},
"hash": "aaf6f4f7243bffa925fa17c9af3afb91c076ae3976b11cf18eb7583b8ce69a7e"
"hash": "9cb6a575aa3e0ff377551a324f839c02d0a070277a9ddf4d5749545315746389"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT spent AS \"spent!: i64\" FROM subtitle_budget_spend\n WHERE kind = ? AND name = ? AND day = strftime('%Y-%m-%d', 'now')",
"describe": {
"columns": [
{
"name": "spent!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_budget_spend",
"name": "spent"
}
}
}
],
"parameters": {
"Right": 2
},
"nullable": [
false
]
},
"hash": "9ee9ff74406e333df50076d1c456f0a9672173af252f36e5d75d2a9df99ab68d"
}
@@ -1,10 +1,10 @@
{
"db_name": "SQLite",
"query": "SELECT id FROM roots WHERE path = ? AND id <> ?",
"query": "SELECT id AS \"id!: i64\" FROM roots WHERE policy_id = ?",
"describe": {
"columns": [
{
"name": "id",
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
@@ -16,11 +16,11 @@
}
],
"parameters": {
"Right": 2
"Right": 1
},
"nullable": [
false
]
},
"hash": "fa3d1e4a6cae94780daf8fe20062a107963ba6ab2bcef2c8cfb9a1efbd905b59"
"hash": "a28caf97dfb6a7e6b69543df0693e15c746702c36a0d4068fa70d28b950e7562"
}
@@ -0,0 +1,86 @@
{
"db_name": "SQLite",
"query": "SELECT media_file_id AS \"media_file_id!: i64\",\n language AS \"language!: String\",\n state AS \"state!: SubtitleState\",\n attempts AS \"attempts!: i64\",\n last_attempt_at,\n last_failure\n FROM subtitle_attempts\n WHERE media_file_id = ?\n ORDER BY language",
"describe": {
"columns": [
{
"name": "media_file_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "media_file_id"
}
}
},
{
"name": "language!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "language"
}
}
},
{
"name": "state!: SubtitleState",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "state"
}
}
},
{
"name": "attempts!: i64",
"ordinal": 3,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "attempts"
}
}
},
{
"name": "last_attempt_at",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "last_attempt_at"
}
}
},
{
"name": "last_failure",
"ordinal": 5,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "last_failure"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
false,
true,
true
]
},
"hash": "adf45101bc3ba20d35b59794053029e89d22ada5c7ffecd47187ec34261bc223"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT probed FROM media_files WHERE id = ?",
"describe": {
"columns": [
{
"name": "probed",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "media_files",
"name": "probed"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
true
]
},
"hash": "af10ccd655f39235f525a531e38850b2e60ff713a038ee1e7c04c616b4613b52"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT sf.path AS \"path!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id\n JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id\n WHERE e.season_id = ? AND sf.path IS NOT NULL",
"describe": {
"columns": [
{
"name": "path!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
true
]
},
"hash": "b2d937e5b8381de44fc6c1915e3deb24236c2c8afa25ffec1c7bbe519aac8803"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT providers_enabled AS \"providers_enabled!: String\"\n FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "providers_enabled!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "providers_enabled"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false
]
},
"hash": "bc6dfe5a5cc639a64e3020c348a2dc74b4298101f8c29d9f8e6397cf79bb83d7"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\" FROM roots\n WHERE CASE WHEN rtrim(path, '/') = '' THEN '/' ELSE rtrim(path, '/') END = ?\n AND (? IS NULL OR id <> ?)",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "roots",
"name": "id"
}
}
}
],
"parameters": {
"Right": 3
},
"nullable": [
false
]
},
"hash": "bda8991590c009ca7084fed36fae088f1b6ae12ad95f972724a417711ad37fa1"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_attempts (media_file_id, language, state)\n VALUES (?, ?, 'satisfied')\n ON CONFLICT (media_file_id, language) DO UPDATE\n SET state = 'satisfied',\n last_failure = NULL,\n updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now')",
"describe": {
"columns": [],
"parameters": {
"Right": 2
},
"nullable": []
},
"hash": "c5a9e43cfeb315e70bbb603f22acc519d7df3cd0b4d606d0d9056414ac6c503b"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT count(*) AS \"failures!: i64\",\n max(grabbed_at) AS \"last_failed_at?: String\"\n FROM grabs\n WHERE target_kind = 'season' AND target_id = ? AND state = 'failed'",
"query": "SELECT count(*) AS \"failures!: i64\",\n max(coalesce(failed_at, grabbed_at)) AS \"last_failed_at?: String\"\n FROM grabs\n WHERE target_kind = 'season' AND target_id = ? AND state = 'failed'",
"describe": {
"columns": [
{
@@ -24,5 +24,5 @@
true
]
},
"hash": "0ca521a8dcb979cc90f5823eb3311d6a3ec1613976c52a8a7f8225e1dc06d162"
"hash": "c73422a1c9d28742f200d1744ba0862bba0dd2d708e948c32fd74f013cc4cda5"
}
@@ -0,0 +1,21 @@
{
"db_name": "SQLite",
"query": "SELECT EXISTS(SELECT 1 FROM movies WHERE id = ?) AS \"exists!: bool\"",
"describe": {
"columns": [
{
"name": "exists!: bool",
"ordinal": 0,
"type_info": "Integer",
"origin": "Expression"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "d81ec62e6c4879d08ce3551c0c71df012bed05ac2a37933adc2deb6552eeeac1"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT path AS \"path!: String\" FROM subtitle_files\n WHERE media_file_id = ? AND language = ? AND path IS NOT NULL",
"describe": {
"columns": [
{
"name": "path!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 2
},
"nullable": [
true
]
},
"hash": "e17b71f67d8b79c54d2d6f32d5e4197d77faac1763b360e8e609b0b9ba5820f1"
}
@@ -0,0 +1,21 @@
{
"db_name": "SQLite",
"query": "SELECT EXISTS(SELECT 1 FROM subtitle_files WHERE path = ?) AS \"exists!: bool\"",
"describe": {
"columns": [
{
"name": "exists!: bool",
"ordinal": 0,
"type_info": "Integer",
"origin": "Expression"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "e3b82dc7b2a3b28753c84864239699cb2007a8101cccc22d223d1f587a5533f8"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\" FROM subtitle_files WHERE path = ?",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "id"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "e5b075d0b7fe5410d3e55d5daf7837d0a9398b1954e713e80a1d82c0a7e172a6"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT translation_engine AS \"translation_engine: String\"\n FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "translation_engine: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "translation_engine"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
true
]
},
"hash": "e7a0291c2fb8cfe59a02ffef11d9657ea6a40e4079573f569573075a205c50e7"
}
@@ -0,0 +1,21 @@
{
"db_name": "SQLite",
"query": "SELECT EXISTS(SELECT 1 FROM media_files WHERE id = ?) AS \"exists!: bool\"",
"describe": {
"columns": [
{
"name": "exists!: bool",
"ordinal": 0,
"type_info": "Integer",
"origin": "Expression"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "e8066bed6fb53ff90d78f47bb1c9ce8160918d4b35764fd988e5e1dd6c101089"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "UPDATE subtitle_settings SET\n wanted_languages = ?, providers_enabled = ?, translation_engine = ?,\n provider_daily_budgets = ?, translator_daily_budgets = ?,\n remote_command_timeout_seconds = ?,\n openai_base_url = ?, openai_model = ?,\n updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now')\n WHERE id = 1",
"describe": {
"columns": [],
"parameters": {
"Right": 8
},
"nullable": []
},
"hash": "eab2c4e7c21cd5d22a583c36dd2d79590e9ce55801c4b87b8db1b90f7b317225"
}
@@ -0,0 +1,38 @@
{
"db_name": "SQLite",
"query": "SELECT mf.id AS \"media_file_id!: i64\", e.id AS \"episode_id!: i64\"\n FROM media_files mf\n JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id\n JOIN seasons se ON se.id = e.season_id\n WHERE se.series_id = ?\n ORDER BY mf.path",
"describe": {
"columns": [
{
"name": "media_file_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
},
{
"name": "episode_id!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "episodes",
"name": "id"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false
]
},
"hash": "f4c3eb506289667ace56da5d28c046163d5f12bd1dca53d3d6fc559f6e9ce393"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT sf.path AS \"path!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id\n WHERE mf.owner_kind = 'episode' AND mf.owner_id = ? AND sf.path IS NOT NULL",
"describe": {
"columns": [
{
"name": "path!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
true
]
},
"hash": "fa35de1c7e62c68dd1049d7e7e08c825d3ca31eb631744a57594aa2fc08fb945"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\" FROM media_files\n WHERE owner_kind = ? AND owner_id = ? ORDER BY path",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
}
],
"parameters": {
"Right": 2
},
"nullable": [
false
]
},
"hash": "fcba9e1f708e6bf2d8471d0c7d26be0b5ad8a30fa042434ffaaff8c69ccf31de"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\" FROM subtitle_files\n WHERE media_file_id = ? AND language = ? AND origin = 'embedded'\n AND forced = ? AND sdh = ?",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "id"
}
}
}
],
"parameters": {
"Right": 4
},
"nullable": [
false
]
},
"hash": "fe15c4a3774e663b03455300d0baec1dadc6cf995c20df803bc4bf49d1cc7c77"
}
+1
View File
@@ -18,6 +18,7 @@ crates/
├── arr-indexer/ Torznab via Prowlarr
├── arr-dl/ Transmission RPC
├── arr-probe/ ffprobe wrapper
├── arr-subs/ subtitle providers, translation, sync
├── arr-db/ sqlx + migrations
├── arr-api/ axum + OpenAPI
├── arr-compat/ Radarr/Sonarr v3 shim for Jellyseerr
Generated
+110
View File
@@ -2,6 +2,12 @@
# It is not intended for manual editing.
version = 4
[[package]]
name = "adler2"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
[[package]]
name = "aho-corasick"
version = "1.1.5"
@@ -35,6 +41,7 @@ dependencies = [
"arr-indexer",
"arr-meta",
"arr-parse",
"arr-subs",
"axum",
"chrono",
"reqwest",
@@ -88,6 +95,7 @@ dependencies = [
"arr-meta",
"arr-parse",
"arr-probe",
"arr-subs",
"axum",
"base64",
"chrono",
@@ -139,12 +147,14 @@ dependencies = [
name = "arr-e2e"
version = "0.1.0"
dependencies = [
"arr-db",
"arr-dl",
"arr-indexer",
"arr-meta",
"chrono",
"reqwest",
"serde_json",
"sqlx",
"tempfile",
"tokio",
"uuid",
@@ -194,11 +204,31 @@ dependencies = [
"arr-core",
"serde",
"serde_json",
"tempfile",
"thiserror",
"tokio",
"tracing",
]
[[package]]
name = "arr-subs"
version = "0.1.0"
dependencies = [
"arr-core",
"arr-parse",
"chardetng",
"encoding_rs",
"reqwest",
"serde",
"serde_json",
"tempfile",
"thiserror",
"tokio",
"tracing",
"wiremock",
"zip",
]
[[package]]
name = "assert-json-diff"
version = "2.0.2"
@@ -366,6 +396,17 @@ dependencies = [
"rand_core",
]
[[package]]
name = "chardetng"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13de944a44b5064ee5d3a5ceccc49a41bfec50f2580e66f82e87703acdb88b53"
dependencies = [
"cfg-if",
"encoding_rs",
"memchr",
]
[[package]]
name = "chrono"
version = "0.4.45"
@@ -423,6 +464,15 @@ version = "2.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "217698eaf96b4a3f0bc4f3662aaa55bdf913cd54d7204591faa790070c6d0853"
[[package]]
name = "crc32fast"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8498c871161e1742aaa9d52551b2d6ebdd4c3d45a3be423e3728f33b955be550"
dependencies = [
"cfg-if",
]
[[package]]
name = "crossbeam-queue"
version = "0.3.13"
@@ -531,6 +581,15 @@ dependencies = [
"serde",
]
[[package]]
name = "encoding_rs"
version = "0.8.35"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75030f3c4f45dafd7586dd6780965a8c7e8e285a5ecb86713e63a79c5b2766f3"
dependencies = [
"cfg-if",
]
[[package]]
name = "equivalent"
version = "1.0.2"
@@ -579,6 +638,16 @@ version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d45db016d36b838f563236e9193d0ee6ce38f3f68b6c94e914b4929c96bbb890"
[[package]]
name = "flate2"
version = "1.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
dependencies = [
"miniz_oxide",
"zlib-rs",
]
[[package]]
name = "flume"
version = "0.12.0"
@@ -1228,6 +1297,16 @@ dependencies = [
"unicase",
]
[[package]]
name = "miniz_oxide"
version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fa76a2c86f704bdb222d66965fb3d63269ce38518b83cb0575fca855ebb6316"
dependencies = [
"adler2",
"simd-adler32",
]
[[package]]
name = "mio"
version = "1.2.2"
@@ -1746,6 +1825,12 @@ dependencies = [
"libc",
]
[[package]]
name = "simd-adler32"
version = "0.3.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a219298ac11a56ea9a6d2120044824d6f01aeb034955e7af7bc16858527deea"
[[package]]
name = "slab"
version = "0.4.12"
@@ -2305,6 +2390,12 @@ version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b"
[[package]]
name = "typed-path"
version = "0.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e28f89b80c87b8fb0cf04ab448d5dd0dd0ade2f8891bae878de66a75a28600e"
[[package]]
name = "typenum"
version = "1.20.1"
@@ -2814,6 +2905,25 @@ dependencies = [
"syn 3.0.3",
]
[[package]]
name = "zip"
version = "8.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d04a6b5381502aa6087c94c669499eb1602eb9c5e8198e534de571f7154809b"
dependencies = [
"crc32fast",
"flate2",
"indexmap",
"memchr",
"typed-path",
]
[[package]]
name = "zlib-rs"
version = "0.6.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34b31d188d9d685a4f9c7b46d6e36631b07058d2cfe190267adce54dc230bf12"
[[package]]
name = "zmij"
version = "1.0.23"
+4
View File
@@ -20,6 +20,7 @@ arr-indexer = { path = "crates/arr-indexer" }
arr-meta = { path = "crates/arr-meta" }
arr-parse = { path = "crates/arr-parse" }
arr-probe = { path = "crates/arr-probe" }
arr-subs = { path = "crates/arr-subs" }
# Runtime and HTTP
axum = "0.8"
@@ -47,13 +48,16 @@ quick-xml = { version = "0.37", features = ["serialize"] }
toml = "0.8"
# Odds and ends
chardetng = "1"
chrono = { version = "0.4", default-features = false, features = ["clock", "serde"] }
clap = { version = "4.5", features = ["derive", "env"] }
encoding_rs = "0.8"
thiserror = "2"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
url = "2"
uuid = { version = "1", features = ["v4", "serde"] }
zip = { version = "8", default-features = false, features = ["deflate-flate2-zlib-rs"] }
# Test-only
tempfile = "3"
+151 -17
View File
@@ -201,14 +201,23 @@ both directions — `only_4k` tightens, `allow_english_audio` loosens.
**Stored verdicts follow the effective policy.** A release's verdict is
stamped by the search that found it, and both §9.3's deck and the daemon's
manual-grab gate read that stored column. So the three operator actions that
manual-grab gate read that stored column. So the four operator actions that
change a title's effective policy — editing its overrides, moving it to a
root with a different policy, and pointing a root at a different policy
re-derive the stored verdicts of everything they touch, inline in the same
request. The operator is never left reading a verdict computed under a
policy that no longer applies. Inline is affordable even root-wide:
re-evaluation is pure and in-memory, and a row whose verdict does not move
is not rewritten, so a library-sized repoint costs reads, not writes.
root with a different policy, pointing a root at a different policy, and
editing the contents of a policy some root points at — re-derive the stored
verdicts of everything they touch, inline in the same request. The operator
is never left reading a verdict computed under a policy that no longer
applies.
The fourth is the widest: a repoint moves one library, a policy edit moves
every library sharing the policy. Inline still holds there. Re-evaluation is
pure and in-memory, a row whose verdict does not move is not rewritten, and
the ceiling is the whole database rather than something that grows with the
number of roots — roots partition titles, and a title has exactly one root.
Measured over 2000 titles and 10 000 stored releases split across two roots
sharing one policy: 0.36 s when the edit moves no verdict, 2.7 s in the
pathological case where it moves all 10 000. A rename changes no rule and
re-derives nothing.
### 5.2 Language
@@ -413,13 +422,13 @@ guard retries at worst weekly (§6.2) — and stays queued for exactly as long a
it is genuinely broken. Nothing is dismissed by hand and no acknowledgement
state is stored, so there is no second thing to keep correct.
**Only a target still waiting for a file is queued.** The count and the window
are one rule seen twice — the queue is meant to be visited and emptied — and
this is its third face: a failure history queues a target only while that
target still has a gap to fill. A movie or an episode is queued while it is
`wanted` and not `available`. A season holds no intent of its own (§4.1), so it
is queued while at least one of its episodes is still wanted and still without
a file. A season pack that hard-failed twice, fell back to per-episode grabbing
**Only a target still waiting for a file is queued.** The count and the
window already express one rule — bounded attention — and this is another
face of it: a failure history queues a target only while that target still
has a gap to fill. A movie or an episode is queued while it is `wanted` and
not `available`. A season holds no intent of its own (§4.1), so it is queued
while at least one of its episodes is still `wanted` and not `available`. A
season pack that hard-failed twice, fell back to per-episode grabbing
exactly as §6.2 says it should, and was then fully acquired leaves at once
rather than waiting out the 30 days — that is the system working, not a
decision. A target that is still broken keeps producing failures and stays.
@@ -471,6 +480,22 @@ its own, the escape hatch is the season release deck, not a lane exception.
Targeted search backs off `1h → 6h → 1d → 3d`, capped at 7d, reset when the
title's metadata changes. It never gives up entirely, it goes quiet.
**The ladder runs from the failure, not the grab.** A failed season-pack grab
quiets the pack lane on that same curve, and the rung is measured from the
moment the grab entered `failed``grabs.failed_at`, the column §5.7's
attention window reads — not from when it was sent. The two are usually
minutes apart, but a torrent can stall on a dead swarm for five weeks before
`ffprobe` condemns it at import. Measured from the grab, the whole ladder has
already elapsed by the time the failure lands, so the lane retries the source
that just failed at once, which is the one thing the backoff exists to
prevent. The ladder's job is to stay off a source that has recently failed,
and "recently" can only mean recently failed.
One anchor covers both features. §5.7's window and this ladder ask the same
question of the same event and read the same column; a target still broken
keeps producing fresh failures, and each one both re-arms this backoff and
holds the target in the attention queue.
**Do not search before the release exists.** TMDB carries release dates; a
movie with no digital release date gets zero targeted searches. This is the
single largest source of wasted queries in Radarr and it is free to avoid.
@@ -539,7 +564,12 @@ Media kind first, hard audience boundary second, people nowhere.
- **Release group is deliberately absent.** It is not a selection criterion and
it makes filenames long enough to break a terminal.
Changing a title's root relocates its title folder into the new root; roots are assumed to share one filesystem, so the move is a rename, never a copy. Changing a root's path is the same move over every title under it, and it is all or nothing: one folder that cannot move puts back the ones that already did and leaves the root's path alone, so the stored path always describes the disk.
Changing a title's root relocates its title folder into the new root; roots
are assumed to share one filesystem, so the move is a rename, never a copy.
Changing a root's path is the same move over every title under it, and it
is all or nothing: one folder that cannot move puts back the ones that
already did and leaves the root's path alone, so the stored path always
describes the disk.
During transition, write into the existing roots so Jellyfin needs no
reconfiguration and new content appears immediately. Radarr will not touch a
@@ -649,7 +679,7 @@ notifying on everything and being muted within a week.
- **Imported** → to the title's owners. The only good-news notification.
- **Needs a decision** → to the operator alone. Entered the no-PT-source queue,
or hard-failed twice on different releases.
or the needs-a-decision queue (§5.7).
- **Broken** → to the operator alone. Prowlarr, Transmission or TMDB
unreachable, disk full.
@@ -739,6 +769,7 @@ arr-meta TMDB client
arr-indexer Torznab via Prowlarr
arr-dl Transmission RPC
arr-probe ffprobe wrapper
arr-subs subtitle providers, translation, sync
arr-db sqlx + migrations
arr-api axum + OpenAPI
arr-compat Radarr/Sonarr v3 shim for Jellyseerr
@@ -821,7 +852,7 @@ previous one.
grabbing, derived status.
7. **Owners and notifications** — tags, per-person ntfy topics, filtered views.
8. **Jellyseerr compat**`arr-compat`.
9. **Subtitles** — replaces Bazarr. Out of scope for this document.
9. **Subtitles** — replaces Bazarr. See §15.
Movies before TV because TV adds season packs, air-date calendars and
per-episode state on top of an otherwise identical pipeline. Doing it second
@@ -841,3 +872,106 @@ means that pipeline is already proven.
RSS lane obeys the same guard — it skips packs for any season with episodes
on disk. Whether a re-grab is ever wanted remains unresolved and deliberately
deferred.
## 15. Subtitles
Replaces Bazarr. Phase 9 in §13; the `arr-subs` crate in §11.
**Wanted set.** Global, not per root. Two languages are separately wanted for
every media file: Portuguese — pt-PT preferred, pt-BR accepted — and English.
A file is satisfied for a language when a subtitle in it exists, embedded or as
a sidecar. This is deliberately unlike §5.2's audio rules, which attach to a
root: subtitles carry no blacklist there and none here. pt-BR subtitles are
always fine.
**Embedded tracks.** An embedded subtitle track satisfies its language.
Text-format tracks (`subrip`, `ass`, `mov_text`) are additionally extracted to
a sidecar SRT, because an extracted track is a legal translation source.
Image-format tracks (PGS on BluRay, VobSub on DVD) carry bitmaps, not text:
they satisfy viewing but can never feed a translator, and arr does not OCR
them. `arr-probe` already reports subtitle tracks with resolved languages; the
format is the new fact it must carry.
**Providers.** OpenSubtitles.com and Podnapisi, behind one trait.
**Ranking.** A `moviehash` match wins outright. Then an exact release-name
match, then same release group or same source, then uploader rating and
download count as tiebreakers. Every rejected candidate names the rule that
killed it, so the manual view described in §9.3 works unchanged for subtitles.
**Forced and SDH.** A forced track covers only foreign-language lines and
on-screen signs; it never satisfies a want and arr never goes looking for one.
An SDH track is complete and satisfies, ranked below a plain subtitle.
Neither gets its own sidecar name. A language is satisfied by exactly one
sidecar, so `<video>.<lang>.srt` needs no segment distinguishing forced from
plain from SDH — where a plain subtitle exists for a language, the forced one
is ignored rather than kept beside it. The database says the same thing: one
sidecar row per (media file, language).
**Translation.** When no provider has a wanted language, arr translates
immediately — there is no waiting window. The source is an existing subtitle:
a downloaded one, or one extracted from a text-format embedded track. Being
able to translate from an embedded track is a deliberate improvement on
Bazarr, which cannot.
**Translation backends.** Pluggable, each behind its own cargo feature: an
OpenAI-compatible HTTP endpoint, DeepL, Google Translate, and a generic remote
command driven by a configured template (`ssh box claude -p` is one instance
of that template, not a backend of its own). The engine in use is a database
setting, so switching does not need a rebuild when the feature is compiled in.
Subtitles are sent in batches of cues; a reply whose cue count or numbering
does not match the batch is rejected. Timing data never leaves arr.
**No upgrade loop.** Once a language is satisfied — by a machine translation
too — arr stops working on it. A real subtitle appearing later does not
replace anything. Replacement is a manual action from the UI. This is §5.4's
rule applied to subtitles.
**On disk.** Sidecars live next to the video inside the §7.4 title folder,
named `<video basename>.<lang>.srt`, e.g.
`… - [2160p][WEB-DL][HDR10].pt-PT.srt`. A machine translation carries an extra
`.mt` segment: `… [HDR10].pt-PT.mt.srt`. That keeps §7.4's audit-by-`ls`
property — with the app stopped, the filename says which subtitles are
machine-made. Folder-level delete stays atomic because sidecars are inside the
folder.
`.mt` is the only optional segment. One language, one sidecar: a second
subtitle for a language arr already has is refused, and replacing one is the
manual delete-then-fetch §15's no-upgrade rule already describes.
**Sync.** `alass` runs on every fetched and every translated subtitle. It is a
single small binary invoked like `ffprobe`, so it costs nothing at rest. It
reports no confidence value, so its output is accepted unless it is
implausible — a shift beyond 60 seconds, or cues lost — in which case the
unsynced original is kept and the file is flagged.
**Configuration.** Provider credentials and translator API keys are bootstrap
config or environment, per §10 — a secret never becomes a database row. Wanted
languages, chosen engine, per-provider enable and the daily budgets are
database rows edited from `/settings` without a restart.
So is everything needed to point the OpenAI-compatible backend somewhere else:
its **base URL and its model name are database rows too**, not bootstrap
config. That backend is not "OpenAI" — it is any endpoint speaking that shape,
`llama.cpp` and a local gateway included, and which one is in use is a thing to
try and change, not a property of the deployment fixed at start-up. Only the
API key stays in the environment, and an endpoint that needs no key is a valid
configuration.
**Budgets.** A token bucket per provider and per translator, with a configured
daily allowance. The reconcile loop spends it newest-import-first, so enabling
this on an existing library drains the backlog over days instead of hitting
every rate limit at once. Being at the cap is a visible queue state, not an
error.
**Notifications.** No new event classes. §9.5 stands: subtitle fetches never
notify, and a provider or translator being unreachable folds into the existing
"Broken" message to the operator alone.
**Non-goals**, stated here so they do not creep back: OCR of image-based
tracks, transcribing audio when no subtitle exists anywhere, adopting subtitle
files already on disk that arr did not write (§2 already says the service
knows only what it put there — which does mean arr may fetch a second copy
alongside one Bazarr left), and any background loop that upgrades a subtitle
in place.
+8 -4
View File
@@ -15,17 +15,21 @@ fmt-check:
fmt:
cargo fmt --all
# Clippy, warnings denied.
# Clippy, warnings denied. `--all-features` matters: every subtitle
# translation backend sits behind a default-off cargo feature, so without it
# the gate never compiles a line of them (#191, #192).
lint:
cargo clippy --all-targets -- -D warnings
cargo clippy --all-targets --all-features -- -D warnings
# Unused workspace dependencies. Stable toolchain, unlike cargo-udeps.
deps:
cargo machete
# Unit and integration tests, excluding the e2e crate.
# Unit and integration tests, excluding the e2e crate. `--all-features` for
# the same reason `lint` needs it: the translation backends are feature-gated
# and would otherwise never run.
test:
cargo nextest run --workspace --exclude arr-e2e
cargo nextest run --workspace --exclude arr-e2e --all-features
# End-to-end tests. Needs a Transmission container (`just e2e-up`); not part
# of the push gate. Never point this at the production Transmission LXC.
+1
View File
@@ -12,6 +12,7 @@ arr-db = { workspace = true }
arr-indexer = { workspace = true }
arr-meta = { workspace = true }
arr-parse = { workspace = true }
arr-subs = { workspace = true }
axum = { workspace = true }
chrono = { workspace = true }
reqwest = { workspace = true }
+368 -6
View File
@@ -6,11 +6,15 @@
//! the body carries the verdict, so a degraded service can still explain
//! itself to the UI instead of looking like a fourth outage.
use std::ffi::OsStr;
use axum::extract::State;
use axum::Json;
use serde::Serialize;
use utoipa::ToSchema;
use arr_subs::{binary_present, translate as backend_error};
use crate::state::AppState;
/// Whether the service as a whole can do its job.
@@ -68,6 +72,44 @@ impl Check {
}
}
/// One enabled subtitle provider's verdict (#200).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, ToSchema)]
pub struct ProviderCheck {
/// The provider's id as settings name it.
pub id: String,
pub status: Status,
#[serde(skip_serializing_if = "Option::is_none")]
pub detail: Option<String>,
}
/// The subtitle lane's verdicts (DESIGN.md §15, #200). Only what is actually
/// in use appears here: providers nobody enabled and engines nobody selected
/// cannot be broken.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, ToSchema)]
pub struct SubtitleHealth {
/// One entry per provider the settings enable, in that order.
pub providers: Vec<ProviderCheck>,
/// The selected translation engine, or `None` when none is chosen —
/// which is a configuration state, not an outage.
#[serde(skip_serializing_if = "Option::is_none")]
pub translation: Option<Check>,
/// The `alass` sync binary, present at its configured path.
pub alass: Check,
/// The `ffmpeg` extraction binary, present at its configured path.
pub ffmpeg: Check,
}
impl SubtitleHealth {
/// Every verdict the lane carries, for the overall status.
fn statuses(&self) -> impl Iterator<Item = Status> + '_ {
self.providers
.iter()
.map(|provider| provider.status)
.chain(self.translation.iter().map(|check| check.status))
.chain([self.alass.status, self.ffmpeg.status])
}
}
/// The body of `GET /api/health`.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, ToSchema)]
pub struct HealthReport {
@@ -78,9 +120,11 @@ pub struct HealthReport {
pub prowlarr: Check,
pub transmission: Check,
pub tmdb: Check,
pub subtitles: SubtitleHealth,
}
/// Report reachability of Prowlarr, Transmission and TMDB.
/// Report reachability of Prowlarr, Transmission and TMDB, plus the
/// subtitle upstreams (#200).
#[utoipa::path(
get,
path = "/api/health",
@@ -90,17 +134,19 @@ pub struct HealthReport {
),
)]
pub async fn health(State(state): State<AppState>) -> Json<HealthReport> {
// Three independent network probes; serialising them would make the
// endpoint as slow as the sum of the timeouts.
let (prowlarr, transmission, tmdb) = tokio::join!(
// Independent network probes; serialising them would make the endpoint
// as slow as the sum of the timeouts.
let (prowlarr, transmission, tmdb, subtitles) = tokio::join!(
probe_prowlarr(&state),
probe_transmission(&state),
probe_tmdb(&state),
probe_subtitles(&state)
);
let status = if [prowlarr.status, transmission.status, tmdb.status]
.iter()
.all(|s| *s == Status::Ok)
.into_iter()
.chain(subtitles.statuses())
.all(|check| check == Status::Ok)
{
Health::Ok
} else {
@@ -113,6 +159,7 @@ pub async fn health(State(state): State<AppState>) -> Json<HealthReport> {
prowlarr,
transmission,
tmdb,
subtitles,
})
}
@@ -181,3 +228,318 @@ async fn probe_tmdb(state: &AppState) -> Check {
fn describe(err: reqwest::Error) -> String {
err.without_url().to_string()
}
// ---- the subtitle lane (DESIGN.md §15, #200) -----------------------------
/// Probe every subtitle lamp: enabled providers, the selected engine, and
/// the two binaries. The settings row decides what is *in use* — a provider
/// nobody enabled or an engine nobody selected is not probed at all, so it
/// cannot fail a lamp.
async fn probe_subtitles(state: &AppState) -> SubtitleHealth {
let settings = match state.database() {
Some(_) => crate::subtitle_settings::load(state).await.ok(),
None => None,
};
// No readable row means nothing is known to be in use; the lamps stay
// quiet rather than failing on data the operator has not entered yet.
let enabled = settings
.as_ref()
.map(|settings| settings.providers_enabled.as_slice())
.unwrap_or_default();
let engine = settings
.as_ref()
.and_then(|settings| settings.translation_engine.as_deref());
let mut probes = Vec::with_capacity(enabled.len());
for id in enabled {
probes.push(probe_provider(state, id).await);
}
let translation = match engine {
Some(engine) => Some(probe_engine(state, engine).await),
None => None,
};
SubtitleHealth {
providers: probes,
translation,
alass: binary_check("alass", state.syncer().binary_path()),
ffmpeg: binary_check("ffmpeg", state.ffmpeg_binary()),
}
}
/// Reachable and credentials accepted, per enabled provider (#200).
///
/// A provider that is enabled but was never attached is unconfigured, not
/// unreachable: its credentials are bootstrap config that this deployment
/// simply does not have.
async fn probe_provider(state: &AppState, id: &str) -> ProviderCheck {
let check = match state.subtitle_provider(id) {
Some(provider) => check_from(provider.probe().await),
// Only reachable for a provider that needs credentials: Podnapisi is
// anonymous (#215) and is always attached, so it never lands here.
None => Check::unconfigured(format!(
"{id} is enabled but has no credentials — they are bootstrap config, not a setting"
)),
};
ProviderCheck {
id: id.to_owned(),
status: check.status,
detail: check.detail,
}
}
/// Reachable, credentials accepted — for the remote-command backend,
/// "reachable" means the command ran and exited cleanly (#200).
async fn probe_engine(state: &AppState, engine: &str) -> Check {
match state.translation_backend(engine) {
Some(backend) => backend_check(backend.probe().await),
None => Check::unconfigured(format!(
"'{engine}' is selected but unavailable in this build"
)),
}
}
/// One probe verdict, whatever kind of upstream produced it. A refused key
/// gets its own wording because it never resolves by retrying. The two error
/// enums — providers' and backends' — carry the same shape for this purpose.
fn check_from(result: Result<(), arr_subs::Error>) -> Check {
match result {
Ok(()) => Check::ok(),
Err(arr_subs::Error::Unauthorized { .. }) => Check::unreachable("credentials refused"),
Err(error) => Check::unreachable(error.to_string()),
}
}
fn backend_check(result: Result<(), backend_error::Error>) -> Check {
match result {
Ok(()) => Check::ok(),
Err(backend_error::Error::Unauthorized { .. }) => Check::unreachable("credentials refused"),
Err(error) => Check::unreachable(error.to_string()),
}
}
/// A binary lamp (#200): present and executable at its configured path.
/// Nothing is spawned — the endpoint is polled, and starting `ffmpeg` per
/// poll would be neither cheap nor side-effect-free.
fn binary_check(name: &str, binary: &OsStr) -> Check {
if binary_present(binary) {
Check::ok()
} else {
Check::unreachable(format!("{name} not found at {}", binary.to_string_lossy()))
}
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use arr_db::Db;
use arr_subs::translate as backend;
use arr_subs::{
Backend, CandidateId, DownloadFuture, Provider, ProviderId, SearchFuture, SearchRequest,
Syncer,
};
use crate::{router, AppState, Upstreams};
/// A provider whose lamp is what `lamp` says — the probe under test is
/// ours, so the stub never touches the network.
#[derive(Debug)]
struct StubProvider {
id: &'static str,
up: bool,
}
impl Provider for StubProvider {
fn id(&self) -> ProviderId {
ProviderId::new(self.id)
}
fn search<'a>(&'a self, _request: &'a SearchRequest) -> SearchFuture<'a> {
Box::pin(async move { Ok(Vec::new()) })
}
fn download<'a>(&'a self, _id: &'a CandidateId) -> DownloadFuture<'a> {
unreachable!("health probes never download");
}
fn probe(&self) -> arr_subs::ProbeFuture<'_> {
let result = if self.up {
Ok(())
} else {
Err(arr_subs::Error::Unauthorized {
provider: ProviderId::new(self.id),
})
};
Box::pin(async move { result })
}
}
/// Same idea for the selected engine; the closure rebuilds its error per
/// call because probing borrows.
#[derive(Debug)]
struct StubBackend {
#[allow(dead_code)]
detail: &'static str,
}
impl Backend for StubBackend {
fn id(&self) -> backend::BackendId {
backend::BackendId::new("openai")
}
fn supports(&self, _target: &arr_core::Language) -> bool {
true
}
fn translate<'a>(&'a self, _batch: &'a backend::Batch) -> backend::TranslateFuture<'a> {
unreachable!("health probes never translate");
}
fn probe(&self) -> backend::ProbeFuture<'_> {
Box::pin(async move {
Err(backend::Error::Transport {
backend: backend::BackendId::new("openai"),
source: "connection refused".into(),
})
})
}
}
/// Serve the app on an ephemeral port and return its base URL.
async fn serve(state: AppState) -> String {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind");
let address = listener.local_addr().expect("address");
tokio::spawn(async move { axum::serve(listener, router(state)).await.expect("serve") });
format!("http://{address}")
}
/// An app with a migrated database and both binaries somewhere findable.
/// The three classic upstreams point at a dead port; these tests read the
/// subtitle lamps, not theirs.
async fn application() -> (tempfile::TempDir, AppState) {
let dir = tempfile::tempdir().expect("tempdir");
let database = Db::connect(dir.path().join("arr.db"))
.await
.expect("connect");
database.migrate().await.expect("migrate");
let state = AppState::new(Upstreams::new(
"http://127.0.0.1:1".into(),
"http://127.0.0.1:1".into(),
))
.expect("state")
.with_database(database)
.with_syncer(Syncer::new().with_binary("sh"))
.with_ffmpeg_binary("sh");
(dir, state)
}
async fn report(state: AppState) -> serde_json::Value {
let base = serve(state).await;
reqwest::get(format!("{base}/api/health"))
.await
.expect("request health")
.json()
.await
.expect("health body")
}
/// Replace the seeded enabled set. Runtime-checked rather than going
/// through the settings API, whose validation refuses engines this test
/// binary has not compiled in.
async fn set_enabled(state: &AppState, providers: &str, engine: Option<&str>) {
sqlx::query("UPDATE subtitle_settings SET providers_enabled = ?, translation_engine = ?")
.bind(providers)
.bind(engine)
.execute(state.database().expect("database").pool())
.await
.expect("settings update");
}
#[tokio::test]
async fn without_a_settings_row_the_lane_is_quiet_and_binaries_still_checked() {
// No database attached: nothing is known to be in use, so nothing
// may fail a lamp — but a missing binary is a fact about the host.
let state = AppState::new(Upstreams::new(
"http://127.0.0.1:1".into(),
"http://127.0.0.1:1".into(),
))
.expect("state")
.with_syncer(Syncer::new().with_binary("sh"))
.with_ffmpeg_binary("/nowhere/ffmpeg");
let body = report(state).await;
assert_eq!(body["subtitles"]["providers"], serde_json::json!([]));
assert!(body["subtitles"]["translation"].is_null());
assert_eq!(body["subtitles"]["alass"]["status"], "ok");
assert_eq!(body["subtitles"]["ffmpeg"]["status"], "unreachable");
assert_eq!(body["status"], "degraded");
}
#[tokio::test]
async fn an_enabled_provider_without_credentials_is_unconfigured() {
let (_dir, state) = application().await;
// The seed row enables opensubtitles and podnapisi; none are attached.
let body = report(state).await;
assert_eq!(
body["subtitles"]["providers"],
serde_json::json!([
{ "id": "opensubtitles", "status": "unconfigured",
"detail": "opensubtitles is enabled but has no credentials — they are bootstrap config, not a setting" },
{ "id": "podnapisi", "status": "unconfigured",
"detail": "podnapisi is enabled but has no credentials — they are bootstrap config, not a setting" },
])
);
assert_eq!(body["status"], "degraded");
}
#[tokio::test]
async fn an_attached_provider_lamp_follows_its_probe() {
let (_dir, state) = application().await;
set_enabled(&state, r#"["ok","bad"]"#, None).await;
let state = state.with_subtitle_providers(vec![
Arc::new(StubProvider { id: "ok", up: true }),
Arc::new(StubProvider {
id: "bad",
up: false,
}),
]);
let body = report(state).await;
assert_eq!(body["subtitles"]["providers"][0]["id"], "ok");
assert_eq!(body["subtitles"]["providers"][0]["status"], "ok");
assert_eq!(body["subtitles"]["providers"][1]["id"], "bad");
assert_eq!(body["subtitles"]["providers"][1]["status"], "unreachable");
assert_eq!(
body["subtitles"]["providers"][1]["detail"],
"credentials refused"
);
}
#[tokio::test]
async fn a_selected_engine_is_probed_only_when_selected() {
let (_dir, state) = application().await;
let body = report(state.clone()).await;
assert!(body["subtitles"]["translation"].is_null(), "{body}");
set_enabled(&state, "[]", Some("openai")).await;
// Selected but never attached: unavailable in this deployment.
let body = report(state).await;
assert_eq!(body["subtitles"]["translation"]["status"], "unconfigured");
}
#[tokio::test]
async fn an_unreachable_engine_fails_the_whole_report() {
let (_dir, state) = application().await;
set_enabled(&state, "[]", Some("openai")).await;
let state = state.with_translation_backends(vec![Arc::new(StubBackend {
detail: "connection refused",
})]);
let body = report(state).await;
assert_eq!(body["subtitles"]["translation"]["status"], "unreachable");
assert_eq!(body["status"], "degraded");
}
}
+50 -2
View File
@@ -18,6 +18,8 @@ mod roots;
mod search;
mod series;
mod state;
mod subtitle_settings;
mod subtitles;
mod trailer;
use axum::routing::get;
@@ -46,6 +48,13 @@ pub use state::{
AppState, EpisodeCommand, MetadataCommand, MovieCommand, SeasonCommand, Upstreams,
DEFAULT_TMDB_URL,
};
pub use subtitle_settings::{SubtitleSettings, SubtitleSettingsInput};
pub use subtitles::{
EpisodeSubtitleGaps, EpisodeSubtitleStatus, MissingSubtitle, MovieSubtitleGaps,
SeasonSubtitleGaps, SeriesSubtitleGaps, Subtitle, SubtitleCandidate, SubtitleGap,
SubtitleGrabInput, SubtitleProviderError, SubtitleQueue, SubtitleSearchInput,
SubtitleSearchResults, SubtitleStatus, SubtitleTranslateInput,
};
pub use trailer::{Trailer, TrailerKind};
/// Where the generated document is served, and where `just gen-client` reads
@@ -71,7 +80,8 @@ pub const DOCS_PATH: &str = "/api/docs";
(name = "owners", description = "Owner tags and filtered views (DESIGN.md §4.3)"),
(name = "policies", description = "Quality policies (DESIGN.md §5)"),
(name = "search", description = "Unified title and release search"),
(name = "roots", description = "Root folders and their policies")
(name = "roots", description = "Root folders and their policies"),
(name = "subtitles", description = "Subtitles and their configuration (DESIGN.md §15)")
),
)]
struct ApiDoc;
@@ -118,6 +128,18 @@ fn api_router() -> OpenApiRouter<AppState> {
.routes(routes!(roots::get, roots::update, roots::delete))
.routes(routes!(policies::list, policies::create))
.routes(routes!(policies::get, policies::update, policies::delete))
.routes(routes!(subtitle_settings::get, subtitle_settings::update))
.routes(routes!(subtitles::list_for_media_file))
.routes(routes!(subtitles::list_for_movie))
.routes(routes!(subtitles::list_for_episode))
.routes(routes!(subtitles::status_for_movie))
.routes(routes!(subtitles::status_for_episode))
.routes(routes!(subtitles::status_for_series))
.routes(routes!(subtitles::search))
.routes(routes!(subtitles::grab))
.routes(routes!(subtitles::translate))
.routes(routes!(subtitles::delete))
.routes(routes!(subtitles::queue))
}
/// The generated `OpenAPI` document.
@@ -213,7 +235,11 @@ mod tests {
.with_tmdb_url(tmdb.uri())
.with_tmdb_api_key(Some("key".into())),
)
.expect("state");
.expect("state")
// The subtitle binaries default to `PATH`; pin them to something
// every machine has so this test stays about the classic upstreams.
.with_syncer(arr_subs::Syncer::new().with_binary("sh"))
.with_ffmpeg_binary("sh");
let body = report(state).await;
assert_eq!(body["status"], "ok");
@@ -359,6 +385,22 @@ mod tests {
("/api/policies/{policy_id}", "put"),
("/api/policies/{policy_id}", "delete"),
("/api/roots", "post"),
("/api/settings/subtitles", "get"),
("/api/settings/subtitles", "put"),
("/api/media-files/{media_file_id}/subtitles", "get"),
("/api/movies/{movie_id}/subtitles", "get"),
("/api/episodes/{episode_id}/subtitles", "get"),
("/api/movies/{movie_id}/subtitles/status", "get"),
("/api/episodes/{episode_id}/subtitles/status", "get"),
("/api/series/{series_id}/subtitles/status", "get"),
("/api/media-files/{media_file_id}/subtitles/search", "post"),
("/api/media-files/{media_file_id}/subtitles/grab", "post"),
(
"/api/media-files/{media_file_id}/subtitles/translate",
"post",
),
("/api/subtitles/{subtitle_id}", "delete"),
("/api/queues/subtitles", "get"),
] {
assert!(
json["paths"][path][method].is_object(),
@@ -373,6 +415,12 @@ mod tests {
"AttentionQueues",
"SeriesAttention",
"Series",
"Subtitle",
"SubtitleCandidate",
"SubtitleSearchResults",
"SubtitleStatus",
"MissingSubtitle",
"EpisodeSubtitleStatus",
] {
assert!(
json["components"]["schemas"][schema].is_object(),
+188 -23
View File
@@ -76,7 +76,42 @@ pub struct Release {
pub parsed: serde_json::Value,
pub score: Option<f64>,
pub verdict: Option<String>,
// The rule behind the verdict: the one that killed a `rejected` row, or
// the one a `waived` row relaxed (#211). Null on an `eligible` row, and
// on a `waived` row stored before migration 0032, which could not record
// it. A plain comment, not a doc comment: doc comments here become
// OpenAPI descriptions and would put the generated client in web/ out of
// date, which #227 and #232 own.
pub rejected_rule: Option<String>,
// #227: what the blacklist recorded this release as failing on, when
// `rejected_rule` is `blacklisted`. Null on every other row, and on a
// blacklisted row whose blacklist entry has since gone. A size rejection
// is a policy opinion the operator can relax; a corrupt or mismatched
// release is not, and a bare `blacklisted` reads the same for both.
// Plain comment for the same reason as the field above.
pub blacklist_reason: Option<String>,
}
/// Fill in [`Release::blacklist_reason`] for every deck row the blacklist
/// holds (#227, §6.3).
///
/// The blacklist is keyed on the *normalised* name, which SQL cannot compute,
/// so the match happens here over the whole table — a handful of rows, the
/// same reasoning as [`arr_db::blacklist::Blacklist`] itself.
pub(crate) async fn attach_blacklist_reasons(
pool: &sqlx::SqlitePool,
releases: &mut [Release],
) -> Result<(), ApiError> {
if releases.is_empty() {
return Ok(());
}
let blacklist = arr_db::blacklist::Blacklist::load(pool).await?;
for release in releases.iter_mut() {
release.blacklist_reason = blacklist
.reason_for_candidate(&release.name, &release.download_url)
.map(str::to_owned);
}
Ok(())
}
/// A library file and what it cost to accept it (`DESIGN.md` §5.7).
@@ -145,6 +180,11 @@ pub struct Accepted {
#[derive(Debug, Clone, Serialize, ToSchema)]
pub struct ErrorBody {
pub error: String,
/// A machine-readable discriminant, set only where a client needs to
/// branch on the failure rather than display it (issue #221). `None`
/// everywhere else — the message is for the operator, not the client.
#[serde(skip_serializing_if = "Option::is_none")]
pub code: Option<String>,
}
#[derive(Debug)]
@@ -156,6 +196,22 @@ pub enum ApiError {
OwnerNotFound,
PolicyNotFound,
RootNotFound,
/// A `media_files` row that is not there. Named apart from
/// [`Self::NotFound`] because the subtitle surface (§15) is keyed on
/// files, not on titles, and "movie not found" would misdirect.
MediaFileNotFound,
SubtitleNotFound,
/// A subtitle provider or translation backend could not do what a manual
/// action asked (§15). Carries the message because two providers and
/// four engines are configurable at once and an unnamed failure is
/// unactionable.
SubtitleUpstream(String),
/// A grab named a `candidate_id` the provider no longer recognises
/// (issue #221): a search's results outlive the search itself only in
/// the client's memory, and the provider can expire one at will. Kept
/// apart from [`Self::SubtitleUpstream`] so the panel can offer "search
/// again" from a `code`, not from matching the message text.
SubtitleCandidateExpired,
/// The §9.6 chip outcome: the title exists upstream but has no trailer.
/// Ordinary, so it must stay distinguishable from an upstream failure.
NoTrailer,
@@ -176,38 +232,60 @@ pub enum ApiError {
impl IntoResponse for ApiError {
fn into_response(self) -> Response {
let (status, error) = match self {
Self::NotFound => (StatusCode::NOT_FOUND, "movie not found".to_string()),
Self::SeriesNotFound => (StatusCode::NOT_FOUND, "series not found".to_string()),
Self::SeasonNotFound => (StatusCode::NOT_FOUND, "season not found".to_string()),
Self::EpisodeNotFound => (StatusCode::NOT_FOUND, "episode not found".to_string()),
Self::OwnerNotFound => (StatusCode::NOT_FOUND, "owner not found".to_string()),
Self::PolicyNotFound => (StatusCode::NOT_FOUND, "policy not found".to_string()),
Self::RootNotFound => (StatusCode::NOT_FOUND, "root not found".to_string()),
Self::NoTrailer => (StatusCode::NOT_FOUND, "no trailer".to_string()),
Self::Conflict(error) => (StatusCode::CONFLICT, error),
Self::Invalid(error) => (StatusCode::UNPROCESSABLE_ENTITY, error),
let (status, error, code) = match self {
Self::NotFound => (StatusCode::NOT_FOUND, "movie not found".to_string(), None),
Self::SeriesNotFound => (StatusCode::NOT_FOUND, "series not found".to_string(), None),
Self::SeasonNotFound => (StatusCode::NOT_FOUND, "season not found".to_string(), None),
Self::EpisodeNotFound => (StatusCode::NOT_FOUND, "episode not found".to_string(), None),
Self::OwnerNotFound => (StatusCode::NOT_FOUND, "owner not found".to_string(), None),
Self::PolicyNotFound => (StatusCode::NOT_FOUND, "policy not found".to_string(), None),
Self::RootNotFound => (StatusCode::NOT_FOUND, "root not found".to_string(), None),
Self::MediaFileNotFound => (
StatusCode::NOT_FOUND,
"media file not found".to_string(),
None,
),
Self::SubtitleNotFound => (
StatusCode::NOT_FOUND,
"subtitle not found".to_string(),
None,
),
Self::SubtitleUpstream(error) => (StatusCode::SERVICE_UNAVAILABLE, error, None),
Self::SubtitleCandidateExpired => (
StatusCode::NOT_FOUND,
"candidate no longer exists — search again".to_string(),
Some("candidate_expired".to_string()),
),
Self::NoTrailer => (StatusCode::NOT_FOUND, "no trailer".to_string(), None),
Self::Conflict(error) => (StatusCode::CONFLICT, error, None),
Self::Invalid(error) => (StatusCode::UNPROCESSABLE_ENTITY, error, None),
Self::Unavailable => (
StatusCode::SERVICE_UNAVAILABLE,
"database unavailable".into(),
None,
),
Self::Upstream(name) => (
StatusCode::SERVICE_UNAVAILABLE,
format!("{name} unavailable"),
None,
),
Self::Database(error) => {
tracing::error!(%error, "API database error");
(StatusCode::INTERNAL_SERVER_ERROR, "database error".into())
(
StatusCode::INTERNAL_SERVER_ERROR,
"database error".into(),
None,
)
}
// The message is the caller's: this variant is returned by the
// delete lane and by the relocate lane, and "files not removed"
// is a lie about a move that failed.
Self::Filesystem(error) => {
tracing::error!(%error, "API filesystem error");
(StatusCode::INTERNAL_SERVER_ERROR, error.clone())
(StatusCode::INTERNAL_SERVER_ERROR, error.clone(), None)
}
};
(status, Json(ErrorBody { error })).into_response()
(status, Json(ErrorBody { error, code })).into_response()
}
}
@@ -644,7 +722,7 @@ pub async fn releases(
Path(id): Path<i64>,
) -> Result<Json<Vec<Release>>, ApiError> {
load_movie(&state, id).await?;
let mut releases = sqlx::query_as!(Release, r#"SELECT r.id AS "id!: i64", r.indexer_id AS "indexer_id!: i64", r.guid AS "guid!: String", r.name AS "name!: String", r.size AS "size!: i64", r.seeders, r.publish_date, r.download_url AS "download_url!: String", r.parsed AS "parsed!: serde_json::Value", r.score, r.verdict, r.rejected_rule FROM releases r JOIN movie_releases mr ON mr.release_id = r.id WHERE mr.movie_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id"#, id)
let mut releases = sqlx::query_as!(Release, r#"SELECT r.id AS "id!: i64", r.indexer_id AS "indexer_id!: i64", r.guid AS "guid!: String", r.name AS "name!: String", r.size AS "size!: i64", r.seeders, r.publish_date, r.download_url AS "download_url!: String", r.parsed AS "parsed!: serde_json::Value", r.score, r.verdict, r.rejected_rule, NULL AS "blacklist_reason?: String" FROM releases r JOIN movie_releases mr ON mr.release_id = r.id WHERE mr.movie_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id"#, id)
.fetch_all(pool(&state)?)
.await?;
let policy = state
@@ -656,6 +734,7 @@ pub async fn releases(
.ok_or(ApiError::NotFound)?
.policy;
rescore(&mut releases, &policy, None, 0)?;
attach_blacklist_reasons(pool(&state)?, &mut releases).await?;
Ok(Json(releases))
}
@@ -1303,6 +1382,10 @@ mod tests {
.expect("releases json");
assert_eq!(releases.len(), 1);
assert_eq!(releases[0]["verdict"], "waived");
// #211: and it says which rule the waiver relaxed, rather than
// sitting in the deck as a bare `waived` beside rejections that each
// name their own.
assert_eq!(releases[0]["rejected_rule"], "size");
}
/// #241: moving a title to a root with a different policy re-derives its
@@ -1413,6 +1496,26 @@ mod tests {
.expect("releases json");
assert_eq!(releases[0]["verdict"], "rejected");
assert_eq!(releases[0]["rejected_rule"], "blacklisted");
// #227: a row the blacklist has no entry for keeps rendering — the
// rule is all the record holds, and no reason is invented for it.
assert_eq!(releases[0]["blacklist_reason"], serde_json::Value::Null);
// With the entry, the row says what it was blacklisted for: a
// corrupt file is not the same decision as a size rejection.
arr_db::blacklist::add(pool, None, name, "no original-language audio")
.await
.expect("blacklist");
let releases: Vec<serde_json::Value> =
reqwest::get(format!("{base}/api/movies/{movie_id}/releases"))
.await
.expect("releases")
.json()
.await
.expect("releases json");
assert_eq!(
releases[0]["blacklist_reason"],
"no original-language audio"
);
}
#[tokio::test]
@@ -1644,6 +1747,62 @@ mod tests {
);
}
/// Reported in production: moving a title into a root whose directory has
/// never been written to failed with
/// `could not move '...The Batman...': No such file or directory`, naming
/// a folder that was sitting exactly where the operator left it. A root is
/// a database row and nothing makes its directory exist; `rename` reports
/// a missing destination parent as the same `NotFound` as a missing
/// source. Every earlier test pointed the destination at a `tempdir`,
/// which is why the assumption was never exercised.
#[tokio::test]
async fn a_root_whose_directory_does_not_exist_yet_still_receives_the_title() {
let (_dir, state, base) = application().await;
let movie = add_movie(&base, 693_134, 1).await;
let id = movie["id"].as_i64().expect("id");
let source = tempfile::tempdir().expect("source root");
let parent = tempfile::tempdir().expect("media tree");
let folder = library_on_disk(&state, id, source.path()).await;
// Configured, but never written to: the row exists, the folder does not.
let destination = parent.path().join("movies").join("kids");
assert!(!destination.exists());
point_root_at(&state, 2, &destination).await;
let response = reqwest::Client::new()
.patch(format!("{base}/api/movies/{id}"))
.json(&serde_json::json!({"root_id": 2}))
.send()
.await
.expect("move root");
assert_eq!(
response.status(),
StatusCode::OK,
"a root that has no folder yet is not a reason to refuse the move"
);
assert!(!folder.exists(), "the folder left the old root");
assert!(
destination
.join("Dune Part Two (2024) [tmdbid-693134]")
.join("Dune Part Two (2024) [tmdbid-693134] - [2160p].mkv")
.exists(),
"the feature arrived in a root that had to be created for it"
);
let path: String = sqlx::query_scalar(
"SELECT path FROM media_files WHERE owner_kind = 'movie' AND owner_id = ?",
)
.bind(id)
.fetch_one(state.database().expect("database").pool())
.await
.expect("media file row");
assert!(
std::path::Path::new(&path).starts_with(&destination),
"the row follows the file: {path}"
);
}
/// Issue #228: a title with nothing on disk changes root with no
/// filesystem work at all — the seeded root paths do not even exist.
#[tokio::test]
@@ -1720,22 +1879,21 @@ mod tests {
/// Issue #228: if the rename fails, the row must not change — the
/// operator sees the title where its files actually are and can retry.
#[cfg(unix)]
#[tokio::test]
async fn a_failed_rename_leaves_the_row_alone() {
use std::os::unix::fs::PermissionsExt;
let (_dir, state, base) = application().await;
let movie = add_movie(&base, 693_134, 1).await;
let id = movie["id"].as_i64().expect("id");
let source = tempfile::tempdir().expect("source root");
let destination = tempfile::tempdir().expect("destination root");
let folder = library_on_disk(&state, id, source.path()).await;
// A destination whose parent does not exist makes the rename itself
// fail while the collision pre-check still passes.
point_root_at(
&state,
2,
&destination.path().join("missing").join("library"),
)
.await;
point_root_at(&state, 2, destination.path()).await;
tokio::fs::set_permissions(&folder, std::fs::Permissions::from_mode(0o555))
.await
.expect("freeze the title folder");
let response = reqwest::Client::new()
.patch(format!("{base}/api/movies/{id}"))
@@ -1744,7 +1902,14 @@ mod tests {
.await
.expect("move root");
assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR);
let body: serde_json::Value = response.json().await.expect("error body");
let error = body["error"].as_str().expect("error text");
assert!(error.contains(folder.to_str().expect("utf-8 source")));
assert!(error.contains(destination.path().to_str().expect("utf-8 destination")));
tokio::fs::set_permissions(&folder, std::fs::Permissions::from_mode(0o755))
.await
.expect("thaw the title folder");
assert!(folder.exists(), "the folder never left the old root");
let (root_id, path): (i64, String) = sqlx::query_as(
"SELECT m.root_id, f.path FROM movies m
+332 -9
View File
@@ -173,6 +173,33 @@ struct PolicyColumns {
score_weights: String,
}
/// Everything in a policy row that a verdict depends on — the name is the
/// one column that does not.
#[derive(PartialEq, Eq)]
struct PolicyRules {
required_audio: String,
dub_blacklist: String,
hdr_rules: String,
size_bands: String,
resolution_pref: String,
source_weights: String,
score_weights: String,
}
impl PolicyColumns {
fn rules(&self) -> PolicyRules {
PolicyRules {
required_audio: self.required_audio.clone(),
dub_blacklist: self.dub_blacklist.clone(),
hdr_rules: self.hdr_rules.clone(),
size_bands: self.size_bands.clone(),
resolution_pref: self.resolution_pref.clone(),
source_weights: self.source_weights.clone(),
score_weights: self.score_weights.clone(),
}
}
}
fn column<T: serde::de::DeserializeOwned>(
column: &'static str,
value: &str,
@@ -376,6 +403,8 @@ pub async fn update(
input.validate().map_err(ApiError::Invalid)?;
let name = input.name.trim().to_owned();
let columns = input.into_columns(name)?;
let after = columns.rules();
let before = stored_rules(&state, id).await?;
let result = sqlx::query!(
r#"UPDATE policies SET
name = ?, required_audio = ?, dub_blacklist = ?, hdr_rules = ?,
@@ -405,9 +434,44 @@ pub async fn update(
if result.rows_affected() == 0 {
return Err(ApiError::PolicyNotFound);
}
// Every verdict stored under every root pointing here was reached under
// the rules this write just replaced; §9.3's deck and the grab gate both
// read them (`reclassify`). A rename leaves the rules alone, so it walks
// nothing.
if before.is_none_or(|before| before != after) {
crate::reclassify::policy(&state, id).await?;
}
Ok(Json(load_policy(&state, id).await?))
}
/// The rule columns of a policy as they stand, or `None` when there is no
/// such row. Compared against what the write is about to store, so an edit
/// that only moves the name does not re-derive a library.
async fn stored_rules(state: &AppState, id: i64) -> Result<Option<PolicyRules>, ApiError> {
let row = sqlx::query!(
r#"SELECT required_audio AS "required_audio!: String",
dub_blacklist AS "dub_blacklist!: String",
hdr_rules AS "hdr_rules!: String",
size_bands AS "size_bands!: String",
resolution_pref AS "resolution_pref!: String",
source_weights AS "source_weights!: String",
score_weights AS "score_weights!: String"
FROM policies WHERE id = ?"#,
id
)
.fetch_optional(pool(state)?)
.await?;
Ok(row.map(|row| PolicyRules {
required_audio: row.required_audio,
dub_blacklist: row.dub_blacklist,
hdr_rules: row.hdr_rules,
size_bands: row.size_bands,
resolution_pref: row.resolution_pref,
source_weights: row.source_weights,
score_weights: row.score_weights,
}))
}
#[utoipa::path(
delete, path = "/api/policies/{policy_id}", tag = "policies",
params(("policy_id" = i64, Path, description = "Policy row id")),
@@ -450,7 +514,7 @@ mod tests {
use crate::{router, Upstreams};
use axum::http::StatusCode;
async fn application() -> (tempfile::TempDir, String) {
async fn application() -> (tempfile::TempDir, AppState, String) {
let dir = tempfile::tempdir().expect("tempdir");
let database = arr_db::Db::connect(dir.path().join("arr.db"))
.await
@@ -466,9 +530,9 @@ mod tests {
.await
.expect("bind");
let address = listener.local_addr().expect("address");
let app = router(state);
let app = router(state.clone());
tokio::spawn(async move { axum::serve(listener, app).await.expect("serve") });
(dir, format!("http://{address}"))
(dir, state, format!("http://{address}"))
}
fn valid_input(name: &str) -> serde_json::Value {
@@ -501,7 +565,7 @@ mod tests {
#[tokio::test]
async fn crud_round_trips_a_policy() {
let (_dir, base) = application().await;
let (_dir, _state, base) = application().await;
let created: serde_json::Value = create(&base, valid_input("test policy"))
.await
@@ -557,7 +621,7 @@ mod tests {
#[tokio::test]
async fn a_referenced_policy_refuses_to_die() {
let (_dir, base) = application().await;
let (_dir, _state, base) = application().await;
let roots: Vec<serde_json::Value> = reqwest::get(format!("{base}/api/roots"))
.await
.expect("roots")
@@ -582,7 +646,7 @@ mod tests {
#[tokio::test]
async fn an_unknown_resolution_is_a_422_naming_the_field() {
let (_dir, base) = application().await;
let (_dir, _state, base) = application().await;
let mut payload = valid_input("bad bands");
payload["size_bands"]["1440p"] = serde_json::json!({ "floor_gib": 2, "target_gib": 6, "penalty_points_per_gib_over": 60 });
@@ -596,7 +660,7 @@ mod tests {
#[tokio::test]
async fn every_field_validates_by_name() {
let (_dir, base) = application().await;
let (_dir, _state, base) = application().await;
let with = |patch: &dyn Fn(&mut serde_json::Value)| {
let mut payload = valid_input("validation probe");
patch(&mut payload);
@@ -645,7 +709,7 @@ mod tests {
#[tokio::test]
async fn malformed_json_is_422_not_400_or_500() {
let (_dir, base) = application().await;
let (_dir, _state, base) = application().await;
let client = reqwest::Client::new();
let response = client
.post(format!("{base}/api/policies"))
@@ -667,8 +731,267 @@ mod tests {
#[tokio::test]
async fn a_duplicate_name_conflicts() {
let (_dir, base) = application().await;
let (_dir, _state, base) = application().await;
let response = create(&base, valid_input("Movies — main")).await;
assert_eq!(response.status(), StatusCode::CONFLICT);
}
async fn policy_id_named(base: &str, name: &str) -> i64 {
let policies: Vec<serde_json::Value> = reqwest::get(format!("{base}/api/policies"))
.await
.expect("policies")
.json()
.await
.expect("policies json");
policies
.iter()
.find(|policy| policy["name"] == name)
.and_then(|policy| policy["id"].as_i64())
.expect("policy id")
}
/// A movie under `root_id`, with one English 1080p release stamped
/// `verdict` as a search would have stamped it.
async fn movie_with_release(
state: &AppState,
base: &str,
root_id: i64,
tmdb_id: i64,
verdict: &str,
) -> i64 {
let movie: serde_json::Value = reqwest::Client::new()
.post(format!("{base}/api/movies"))
.json(&serde_json::json!({
"tmdb_id": tmdb_id, "title": format!("Title {tmdb_id}"), "year": 2024,
"original_language": "en", "root_id": root_id, "overrides": {}
}))
.send()
.await
.expect("create movie")
.json()
.await
.expect("movie json");
let movie_id = movie["id"].as_i64().expect("movie id");
let name = format!("Title {tmdb_id} 2024 1080p WEB-DL ENGLISH x264-GROUP");
let release_id = stamped_release(state, &name, 6_i64 * (1 << 30), verdict).await;
sqlx::query("INSERT INTO movie_releases (movie_id, release_id) VALUES (?, ?)")
.bind(movie_id)
.bind(release_id)
.execute(state.database().expect("database").pool())
.await
.expect("movie association");
movie_id
}
/// One release row, verdict stamped by hand. `waived` carries no rule,
/// which the `releases` CHECK allows — only `rejected` needs one.
async fn stamped_release(state: &AppState, name: &str, size: i64, verdict: &str) -> i64 {
let parsed = arr_parse::parse(name);
sqlx::query_scalar::<_, i64>(
"INSERT INTO releases (indexer_id, guid, name, size, seeders, download_url, parsed, score, verdict)
VALUES (7, ?, ?, ?, 40, 'url', ?, 0, ?) RETURNING id",
)
.bind(name)
.bind(name)
.bind(size)
.bind(serde_json::to_string(&parsed).expect("parsed json"))
.bind(verdict)
.fetch_one(state.database().expect("database").pool())
.await
.expect("release")
}
/// A series under `root_id` with a ten-episode season 9 pack stamped
/// `waived`. Under its own policy the pack is eligible, so the stamp
/// only survives if nothing re-derived it.
async fn series_with_waived_pack(state: &AppState, base: &str, root_id: i64) -> i64 {
let client = reqwest::Client::new();
let series: serde_json::Value = client
.post(format!("{base}/api/series"))
.json(&serde_json::json!({
"tmdb_id": 82_728, "title": "Bluey", "year": 2018,
"original_language": "en", "root_id": root_id,
"auto_track": false
}))
.send()
.await
.expect("create series")
.json()
.await
.expect("series json");
let series_id = series["id"].as_i64().expect("series id");
let episodes: Vec<serde_json::Value> = (1..=10)
.map(|number| {
serde_json::json!({
"number": number, "title": format!("Episode {number}"),
"air_date": "2025-01-01"
})
})
.collect();
let season: serde_json::Value = client
.post(format!("{base}/api/series/{series_id}/seasons"))
.json(&serde_json::json!({"number": 9, "episodes": episodes}))
.send()
.await
.expect("create season")
.json()
.await
.expect("season json");
let season_id = season["id"].as_i64().expect("season id");
// Ten episodes in 15 GiB: 1.5 GiB each, inside the 1080p band.
let release_id = stamped_release(
state,
"Bluey S09 1080p WEB-DL ENGLISH x264-GROUP",
15_i64 * (1 << 30),
"waived",
)
.await;
sqlx::query("INSERT INTO season_releases (season_id, release_id) VALUES (?, ?)")
.bind(season_id)
.bind(release_id)
.execute(state.database().expect("database").pool())
.await
.expect("season association");
series_id
}
async fn verdict_at(url: String) -> serde_json::Value {
let releases: Vec<serde_json::Value> = reqwest::get(url)
.await
.expect("releases")
.json()
.await
.expect("releases json");
releases[0]["verdict"].clone()
}
async fn movie_verdict(base: &str, movie_id: i64) -> serde_json::Value {
verdict_at(format!("{base}/api/movies/{movie_id}/releases")).await
}
async fn root_id_of(state: &AppState, kind: &str, audience: &str) -> i64 {
sqlx::query_scalar("SELECT id FROM roots WHERE kind = ? AND audience = ?")
.bind(kind)
.bind(audience)
.fetch_one(state.database().expect("database").pool())
.await
.expect("root id")
}
/// The same policy document with a different required-audio rule, so an
/// English release that was eligible becomes a waiver.
fn requires_portuguese(name: &str) -> serde_json::Value {
let mut payload = valid_input(name);
payload["required_audio"] = serde_json::json!({ "require": "any_of", "langs": ["pt-PT"] });
payload
}
/// #246: editing a policy re-derives the stored verdicts of every title
/// under every root pointing at it (§5.1) — both libraries when two
/// roots share it, and nothing under a root that does not.
#[tokio::test]
async fn editing_a_policy_rederives_every_root_that_points_at_it() {
let (_dir, state, base) = application().await;
let client = reqwest::Client::new();
let shared = policy_id_named(&base, "Movies — main").await;
let main_root = root_id_of(&state, "movie", "main").await;
let kids_root = root_id_of(&state, "movie", "kids").await;
// Two roots on one policy: the edit has two libraries to reach
// rather than one. The path stays, so nothing on disk moves.
let response = client
.put(format!("{base}/api/roots/{kids_root}"))
.json(&serde_json::json!({
"kind": "movie", "audience": "kids",
"path": "/mnt/media/movies/kids", "policy_id": shared,
}))
.send()
.await
.expect("repoint kids root");
assert_eq!(
response.status(),
StatusCode::OK,
"{:?}",
response.text().await
);
let here = movie_with_release(&state, &base, main_root, 693_134, "eligible").await;
let there = movie_with_release(&state, &base, kids_root, 27_205, "eligible").await;
// On the TV main root, a different policy, so the edit must not
// reach it. Stamped against its own policy's answer, so a walk that
// did reach it would show.
let tv_root = root_id_of(&state, "tv", "main").await;
let series_id = series_with_waived_pack(&state, &base, tv_root).await;
let response = client
.put(format!("{base}/api/policies/{shared}"))
.json(&requires_portuguese("Movies — main"))
.send()
.await
.expect("edit policy");
assert_eq!(
response.status(),
StatusCode::OK,
"{:?}",
response.text().await
);
assert_eq!(
movie_verdict(&base, here).await,
"waived",
"the first root's library re-derives"
);
assert_eq!(
movie_verdict(&base, there).await,
"waived",
"and so does the second root's, sharing the policy"
);
assert_eq!(
verdict_at(format!("{base}/api/series/{series_id}/seasons/9/releases")).await,
"waived",
"a root on another policy keeps the verdict it was stamped with"
);
}
/// Renaming a policy changes no rule, so it re-derives nothing — the
/// stamped verdict survives even though the rules would not produce it.
#[tokio::test]
async fn renaming_a_policy_leaves_verdicts_alone() {
let (_dir, state, base) = application().await;
let client = reqwest::Client::new();
let kids = policy_id_named(&base, "Movies — kids").await;
let kids_root = root_id_of(&state, "movie", "kids").await;
// Write the rules through the API once, so the rename that follows
// stores byte-identical rule columns and the only change is the name.
let response = client
.put(format!("{base}/api/policies/{kids}"))
.json(&valid_input("Movies — kids"))
.send()
.await
.expect("normalise policy");
assert_eq!(response.status(), StatusCode::OK);
// Deliberately the wrong answer: these rules make an English release
// a waiver, so a re-derivation would move this row.
let movie = movie_with_release(&state, &base, kids_root, 157_336, "eligible").await;
let response = client
.put(format!("{base}/api/policies/{kids}"))
.json(&valid_input("Movies — children"))
.send()
.await
.expect("rename policy");
assert_eq!(
response.status(),
StatusCode::OK,
"{:?}",
response.text().await
);
assert_eq!(
movie_verdict(&base, movie).await,
"eligible",
"a rename touches no rule, so it walks nothing"
);
}
}
+33 -7
View File
@@ -1,6 +1,7 @@
//! Stored verdicts, re-derived when a title's effective policy changes:
//! an overrides edit (§9.3), a move to a root with a different policy, or a
//! root pointed at a different policy (§5.1).
//! an overrides edit (§9.3), a move to a root with a different policy, a
//! root pointed at a different policy, or an edit to the contents of a
//! policy some root points at (§5.1).
//!
//! A release's verdict is stamped once, by the search that found it. Both the
//! deck and the daemon's manual-grab gate read that stored column, so an
@@ -9,7 +10,8 @@
//! reading `rejected` and the grab would be refused. A root move and a
//! policy repoint invalidate the column the same way, just wider: nothing
//! else re-reads it, so the write that changed the effective policy is the
//! only place the correction can happen.
//! only place the correction can happen. A policy edit invalidates it wider
//! still: every root pointing at that policy, not just one.
//!
//! So the rules run again here, over the releases already attached to the
//! title. This is the same correction the daemon makes when a grab turns out
@@ -176,6 +178,30 @@ pub(crate) async fn root(state: &AppState, root_id: i64) -> Result<(), ApiError>
Ok(())
}
/// Re-evaluate every release of every title under every root that points at
/// one policy, for a `PUT /api/policies/{id}` that changed its rules (§5.1).
///
/// Root by root through [`root`], which is title by title through [`movie`]
/// and [`series`]: one walker, one set of skip rules, one place that decides
/// a row does not need rewriting.
///
/// This is the widest of the four re-derivations — a root repoint moves one
/// library, a policy edit moves every library sharing the policy — but the
/// ceiling is the whole database rather than something that grows with it,
/// since roots partition titles and a title has exactly one root.
pub(crate) async fn policy(state: &AppState, policy_id: i64) -> Result<(), ApiError> {
let root_ids = sqlx::query_scalar!(
r#"SELECT id AS "id!: i64" FROM roots WHERE policy_id = ?"#,
policy_id
)
.fetch_all(pool(state)?)
.await?;
for root_id in root_ids {
root(state, root_id).await?;
}
Ok(())
}
#[allow(clippy::too_many_arguments)]
async fn apply(
state: &AppState,
@@ -208,11 +234,11 @@ async fn apply(
episodes,
runtime_minutes,
);
// `releases` allows a rule name only on a rejected row
// (`CHECK ((verdict = 'rejected') = (rejected_rule IS NOT NULL))`),
// which is also how the daemon writes a waiver.
// A waiver records the rule it relaxed, the same as a rejection
// (#211). Migration 0032 relaxed `releases` to
// `CHECK (verdict != 'rejected' OR rejected_rule IS NOT NULL)` so it
// can, and the daemon writes waivers the same way.
let (verdict, rule) = verdict_columns(&evaluation.verdict);
let rule = if verdict == "rejected" { rule } else { None };
if release.verdict.as_deref() == Some(verdict) && release.rejected_rule == rule {
continue;
}
+58 -40
View File
@@ -50,20 +50,14 @@ struct PlannedRename {
pub(crate) struct Relocation {
performed: Vec<PlannedRename>,
rewrites: Vec<(i64, String)>,
/// The new root directory, set only when this request created it (it did
/// not already exist). [`Self::undo`] removes it, so it is cleaned up
/// exactly when the move it was created for does not complete; a move
/// that commits never calls `undo` and the directory stays.
created_root: Option<PathBuf>,
}
/// Whether the destination root is a directory that must already be there.
/// A title moves into another configured root, which exists; a root moving to
/// a new path is moving somewhere that need not exist yet.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Destination {
Existing,
Create,
/// Every directory level this request materialised for the new root,
/// deepest first. `create_dir_all` can make more than one — moving a
/// root to `/mnt/media-v2/tv/kids` when `/mnt/media-v2` is all that
/// exists creates both `tv` and `kids` — and [`Self::undo`] removes all
/// of them, so a failed move leaves nothing behind (#244). Levels that
/// were already on disk are never in this list and are never touched. A
/// move that commits never calls `undo` and the directories stay.
created_dirs: Vec<PathBuf>,
}
/// Rename the title's folders into the new root. Called before the row is
@@ -88,7 +82,7 @@ pub(crate) async fn relocate_title(
let old_root = root_path(state, old_root_id).await?;
let new_root = root_path(state, new_root_id).await?;
let files = title_files(state, kind, title_id).await?;
relocate_files(&files, &old_root, &new_root, Destination::Existing).await
relocate_files(&files, &old_root, &new_root).await
}
/// Rename every title folder under a root into the root's new path, for a
@@ -111,7 +105,7 @@ pub(crate) async fn relocate_root(
new_path: &str,
) -> Result<Relocation, ApiError> {
let files = root_files(state, root_id).await?;
relocate_files(&files, old_path, new_path, Destination::Create).await
relocate_files(&files, old_path, new_path).await
}
/// The one mover both callers share: plan every rename, refuse every
@@ -121,7 +115,6 @@ async fn relocate_files(
files: &[(i64, String)],
old_root: &str,
new_root: &str,
destination: Destination,
) -> Result<Relocation, ApiError> {
let mut renames: Vec<PlannedRename> = Vec::new();
let mut rewrites: Vec<(i64, String)> = Vec::new();
@@ -179,20 +172,21 @@ async fn relocate_files(
}
}
// Tracked only when this call is the one that created the directory, so
// a failed move can remove it again without ever touching a root path
// that already existed on disk.
let mut created_root: Option<PathBuf> = None;
if destination == Destination::Create && !renames.is_empty() {
let already_there = tokio::fs::symlink_metadata(new_root).await.is_ok();
// Tracked only for the levels this call is the one to create, so a
// failed move can remove them again without ever touching a directory
// that already existed on disk. Recorded before `create_dir_all`, since
// afterwards there is no way to tell which levels it made.
let mut created_dirs: Vec<PathBuf> = Vec::new();
// A configured root need not exist on disk. Without creating it here,
// `rename` reports its missing parent as the same `NotFound` as a missing
// source and sends the operator looking at the wrong end of the move.
if !renames.is_empty() {
created_dirs = missing_levels(std::path::Path::new(new_root)).await;
if let Err(error) = tokio::fs::create_dir_all(new_root).await {
return Err(ApiError::Filesystem(format!(
"could not create '{new_root}': {error}"
)));
}
if !already_there {
created_root = Some(PathBuf::from(new_root));
}
}
let mut performed: Vec<PlannedRename> = Vec::new();
@@ -209,7 +203,7 @@ async fn relocate_files(
);
continue;
}
Err(error) => return Err(failed(&rename, &error, performed, created_root).await),
Err(error) => return Err(failed(&rename, &error, performed, created_dirs).await),
}
match tokio::fs::rename(&rename.source, &rename.destination).await {
Ok(()) => {
@@ -220,37 +214,57 @@ async fn relocate_files(
);
performed.push(rename);
}
Err(error) => return Err(failed(&rename, &error, performed, created_root).await),
Err(error) => return Err(failed(&rename, &error, performed, created_dirs).await),
}
}
Ok(Relocation {
performed,
rewrites,
created_root,
created_dirs,
})
}
/// The levels of `path` that are not on disk, deepest first — exactly what a
/// following `create_dir_all` will materialise. The walk stops at the first
/// ancestor that exists, so nothing already there is ever listed.
async fn missing_levels(path: &std::path::Path) -> Vec<PathBuf> {
let mut missing = Vec::new();
for ancestor in path.ancestors() {
// `ancestors` ends in an empty path for a relative input; there is
// no level above that to create.
if ancestor.as_os_str().is_empty() || tokio::fs::symlink_metadata(ancestor).await.is_ok() {
break;
}
missing.push(ancestor.to_path_buf());
}
missing
}
/// One rename failed: move back everything that had already moved, remove
/// the new root if this request is the one that created it, and name the
/// folder that stopped the move so the operator knows which title to look at
/// before retrying.
/// every directory level this request created, and name the folder that
/// stopped the move so the operator knows which title to look at before
/// retrying.
async fn failed(
rename: &PlannedRename,
error: &std::io::Error,
performed: Vec<PlannedRename>,
created_root: Option<PathBuf>,
created_dirs: Vec<PathBuf>,
) -> ApiError {
Relocation {
performed,
rewrites: Vec::new(),
created_root,
created_dirs,
}
.undo()
.await;
// Name both ends. `rename` returns NotFound for a missing destination
// parent as readily as for a missing source, and naming only the source
// sent an operator looking for a file that was sitting where they left it.
ApiError::Filesystem(format!(
"could not move '{}': {error}",
rename.source.display()
"could not move '{}' to '{}': {error}",
rename.source.display(),
rename.destination.display()
))
}
@@ -297,13 +311,17 @@ impl Relocation {
// `remove_dir` rather than `remove_dir_all`: it only succeeds on an
// empty directory, so anything unexpected left inside it — this
// request's own undo failing, say — is a reason to leave it alone.
if let Some(root) = &self.created_root {
if let Err(error) = tokio::fs::remove_dir(root).await {
// Deepest first, since a parent cannot go while its child is there;
// the first level that will not go stops the walk, because every
// level above it now has content and refusing is the right answer.
for directory in &self.created_dirs {
if let Err(error) = tokio::fs::remove_dir(directory).await {
tracing::warn!(
path = %root.display(),
path = %directory.display(),
%error,
"could not remove the directory created for a move that did not complete"
"could not remove a directory created for a move that did not complete"
);
break;
}
}
}
+228 -9
View File
@@ -152,6 +152,7 @@ pub async fn create(
input.validate().map_err(ApiError::Invalid)?;
input.policy_exists(&state).await?;
let path = normalize_path(&input.path);
path_is_free(&state, None, &path).await?;
let result = sqlx::query!(
"INSERT INTO roots (kind, audience, path, policy_id) VALUES (?, ?, ?, ?)",
input.kind,
@@ -188,17 +189,25 @@ pub async fn update(
input.policy_exists(&state).await?;
let current = load_root(&state, id).await?;
let path = normalize_path(&input.path);
// #244: rows written before #243 can hold a trailing separator, so the
// stored value is normalised too. Comparing a normalised payload against
// a raw stored value means the row can never compare equal: every edit,
// policy changes included, takes the relocation branch, and there each
// planned destination is its own source. The normalised value is what
// `relocate_root` gets as well, so no planned path carries a doubled
// separator.
let current_path = normalize_path(&current.path);
// A path change moves every §7.4 title folder under this root with the
// row (issue #236), the same way changing a title's root moves one
// (#228). Disk first, row second: a failed rename leaves the root row
// alone, so the operator sees the library where its files actually are
// and can retry. A path already taken is refused before any of it, since
// the write would fail afterwards anyway.
let relocation = if path == current.path {
let relocation = if path == current_path {
None
} else {
path_is_free(&state, id, &path).await?;
Some(crate::relocate::relocate_root(&state, id, &current.path, &path).await?)
path_is_free(&state, Some(id), &path).await?;
Some(crate::relocate::relocate_root(&state, id, &current_path, &path).await?)
};
let mut transaction = pool(&state)?.begin().await?;
let written: Result<(), sqlx::Error> = async {
@@ -245,12 +254,25 @@ pub async fn update(
}
/// The unique index on `path` would catch this after the move; catching it
/// first keeps a doomed write from touching the disk at all.
async fn path_is_free(state: &AppState, id: i64, path: &str) -> Result<(), ApiError> {
let taken: Option<i64> =
sqlx::query_scalar!("SELECT id FROM roots WHERE path = ? AND id <> ?", path, id)
.fetch_optional(pool(state)?)
.await?;
/// first keeps a doomed write from touching the disk at all. `except` is the
/// row being updated, or `None` when creating.
///
/// The stored side is normalised in SQL, mirroring [`normalize_path`], so
/// `/mnt/media/x` and `/mnt/media/x/` cannot be two roots for one directory
/// (#244). The unique index cannot see that — it compares the raw strings —
/// and migration 0031 leaves any pair that already collides alone rather
/// than failing to apply, so such a row can still be on disk.
async fn path_is_free(state: &AppState, except: Option<i64>, path: &str) -> Result<(), ApiError> {
let taken: Option<i64> = sqlx::query_scalar!(
r#"SELECT id AS "id!: i64" FROM roots
WHERE CASE WHEN rtrim(path, '/') = '' THEN '/' ELSE rtrim(path, '/') END = ?
AND (? IS NULL OR id <> ?)"#,
path,
except,
except
)
.fetch_optional(pool(state)?)
.await?;
if taken.is_some() {
return Err(ApiError::Conflict(
"a root with this path already exists".into(),
@@ -749,6 +771,17 @@ mod tests {
assert_eq!(response.status(), StatusCode::NO_CONTENT);
}
/// A path written straight into the row, no normalisation — how a root
/// created before #243 could end up holding a trailing separator.
async fn point_root_at_raw(state: &AppState, root_id: i64, path: &str) {
sqlx::query("UPDATE roots SET path = ? WHERE id = ?")
.bind(path)
.bind(root_id)
.execute(state.database().expect("database").pool())
.await
.expect("store the raw path");
}
async fn point_root_at(state: &AppState, root_id: i64, path: &std::path::Path) {
sqlx::query("UPDATE roots SET path = ? WHERE id = ?")
.bind(path.to_str().expect("utf-8 root"))
@@ -1257,6 +1290,192 @@ mod tests {
);
}
/// Issue #244: a root *stored* with a trailing separator — creatable
/// through the API at any point before #243 — could not be edited at
/// all. The payload was normalised and the stored value was not, so no
/// payload compared equal: every edit took the relocation branch, where
/// every planned destination is its own source and the pre-check 409s.
#[tokio::test]
async fn a_root_stored_with_a_trailing_separator_can_still_be_edited() {
let (_dir, state, base) = application().await;
let old = tempfile::tempdir().expect("old root");
let unslashed = old.path().to_str().expect("utf-8").to_owned();
point_root_at_raw(&state, 1, &format!("{unslashed}/")).await;
let id = add_movie(&base, 100, "Dune", 1).await;
let folder = library_folder(&state, id, old.path(), "Dune").await;
// What the settings view sends back: the path exactly as stored,
// separator included, with only the policy changed.
let kids_policy = policy_id_named(&base, "Movies — kids").await;
let mut payload = root_payload(&base, 1, &format!("{unslashed}/")).await;
payload["policy_id"] = kids_policy.into();
let response = reqwest::Client::new()
.put(format!("{base}/api/roots/1"))
.json(&payload)
.send()
.await
.expect("edit the root");
assert_eq!(
response.status(),
StatusCode::OK,
"{:?}",
response.text().await
);
assert!(folder.join("feature.mkv").exists(), "nothing on disk moved");
assert_eq!(
stored_path(&base, 1).await,
unslashed,
"the row is left normalised, so the next edit compares equal too"
);
// And the same payload without the separator is not a relocation
// either.
let response = reqwest::Client::new()
.put(format!("{base}/api/roots/1"))
.json(&root_payload(&base, 1, &unslashed).await)
.send()
.await
.expect("edit the root again");
assert_eq!(response.status(), StatusCode::OK);
assert!(folder.join("feature.mkv").exists(), "still nothing moved");
}
/// Issue #244: a real path change from a root stored with a trailing
/// separator plans from the normalised value, so neither a destination
/// nor a rewritten row carries a doubled separator.
#[tokio::test]
async fn relocating_a_slash_stored_root_plans_no_doubled_separator() {
let (_dir, state, base) = application().await;
let old = tempfile::tempdir().expect("old root");
let home = tempfile::tempdir().expect("home of the new path");
let new = home.path().join("relocated-main");
let unslashed = old.path().to_str().expect("utf-8").to_owned();
point_root_at_raw(&state, 1, &format!("{unslashed}/")).await;
let id = add_movie(&base, 100, "Dune", 1).await;
let folder = library_folder(&state, id, old.path(), "Dune").await;
let response = reqwest::Client::new()
.put(format!("{base}/api/roots/1"))
.json(&root_payload(&base, 1, new.to_str().expect("utf-8")).await)
.send()
.await
.expect("move the root");
assert_eq!(
response.status(),
StatusCode::OK,
"{:?}",
response.text().await
);
assert!(!folder.exists(), "the folder left the old path");
assert!(new.join("Dune").join("feature.mkv").exists());
let paths = file_paths(&state).await;
assert!(
!paths[0].contains("//"),
"no doubled separator in the rewritten row: {}",
paths[0]
);
assert!(
std::path::Path::new(&paths[0]).exists(),
"the rewritten path describes the disk: {}",
paths[0]
);
assert_eq!(stored_path(&base, 1).await, new.to_str().expect("utf-8"));
}
/// Issue #244: `create_dir_all` can materialise more than one level for
/// a root path pointed somewhere fresh. A failed move removes every
/// level it created, not only the leaf — and still nothing that was
/// already on disk.
#[cfg(unix)]
#[tokio::test]
async fn a_failed_move_removes_every_level_it_created() {
use std::os::unix::fs::PermissionsExt;
let (_dir, state, base) = application().await;
let old = tempfile::tempdir().expect("old root");
let home = tempfile::tempdir().expect("home of the new path");
// Three levels below a directory that is already there.
let top = home.path().join("media-v2");
let new = top.join("tv").join("kids");
point_root_at(&state, 1, old.path()).await;
let id = add_movie(&base, 100, "Dune", 1).await;
let stuck = library_folder(&state, id, old.path(), "Dune").await;
// Moving a directory to another parent rewrites its `..`, which
// needs write permission on the directory itself.
tokio::fs::set_permissions(&stuck, std::fs::Permissions::from_mode(0o555))
.await
.expect("freeze the title folder");
let response = reqwest::Client::new()
.put(format!("{base}/api/roots/1"))
.json(&root_payload(&base, 1, new.to_str().expect("utf-8")).await)
.send()
.await
.expect("move the root");
assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR);
tokio::fs::set_permissions(&stuck, std::fs::Permissions::from_mode(0o755))
.await
.expect("thaw the title folder");
assert!(
!top.exists(),
"every level the failed move created is gone, not only the leaf"
);
assert!(
home.path().exists(),
"the level that was already there is left alone"
);
assert!(
stuck.join("feature.mkv").exists(),
"the folder is still where the row says it is"
);
assert_eq!(
stored_path(&base, 1).await,
old.path().to_str().expect("utf-8")
);
}
/// Issue #244: two roots naming one directory, differing only by a
/// trailing separator, are not two roots. The unique index compares raw
/// strings and cannot see it, so the check normalises both sides.
#[tokio::test]
async fn a_slash_stored_path_is_not_free_for_another_root() {
let (_dir, state, base) = application().await;
point_root_at_raw(&state, 2, "/mnt/media/movies/archive/").await;
// An update onto the stripped form of a path another root holds.
let response = reqwest::Client::new()
.put(format!("{base}/api/roots/1"))
.json(&root_payload(&base, 1, "/mnt/media/movies/archive").await)
.send()
.await
.expect("update onto the other root's path");
assert_eq!(response.status(), StatusCode::CONFLICT);
// And a create, which the unique index would have let through.
let policy_ids = first_policy_ids(&base).await;
let mut payload = root_input(policy_ids[0]);
payload["kind"] = serde_json::json!("tv");
payload["audience"] = serde_json::json!("main");
payload["path"] = serde_json::json!("/mnt/media/movies/archive");
let deleted = reqwest::Client::new()
.delete(format!("{base}/api/roots/3"))
.send()
.await
.expect("free the (tv, main) pair");
assert_eq!(deleted.status(), StatusCode::NO_CONTENT);
let response = reqwest::Client::new()
.post(format!("{base}/api/roots"))
.json(&payload)
.send()
.await
.expect("create onto the other root's path");
assert_eq!(response.status(), StatusCode::CONFLICT);
}
/// Issue #236: a path another root already holds is refused before the
/// disk is touched at all.
#[tokio::test]
+10
View File
@@ -156,6 +156,10 @@ pub struct ClassifiedRelease {
pub score_terms: ScoreTerms,
pub verdict: String,
pub rule: Option<String>,
/// What the blacklist recorded this release as failing on (#227, §6.3),
/// when `rule` is `blacklisted`. `None` on every other row. A policy
/// rejection and a bad release both read as `blacklisted` without it.
pub blacklist_reason: Option<String>,
}
#[utoipa::path(
@@ -730,6 +734,11 @@ fn classify(
episodes,
runtime_minutes,
);
// #227: the reason the blacklist holds is what tells a policy rejection
// the operator can relax from a release that should never be retried.
let blacklist_reason = blacklist
.reason_for_candidate(&release.name, &release.download_url)
.map(str::to_owned);
let (verdict, rule) = if blacklist.blocks_candidate(&release.name, &release.download_url) {
("rejected", Some(blacklist::RULE.to_owned()))
} else {
@@ -781,6 +790,7 @@ fn classify(
score_terms: terms,
verdict: verdict.to_owned(),
rule,
blacklist_reason,
})
}
+466 -13
View File
@@ -29,7 +29,9 @@ use chrono::{DateTime, NaiveDate, NaiveTime, Utc};
use serde::{Deserialize, Serialize};
use utoipa::{IntoParams, ToSchema};
use crate::movies::{pool, rescore, Accepted, ApiError, ErrorBody, Release};
use crate::movies::{
attach_blacklist_reasons, pool, rescore, Accepted, ApiError, ErrorBody, Release,
};
use crate::owners::Owner;
use crate::search::tmdb_client;
use crate::state::{AppState, EpisodeCommand, MetadataCommand, SeasonCommand};
@@ -110,6 +112,10 @@ pub struct Season {
/// which is why the row still exists. A conflict for the operator to
/// resolve; nothing was deleted from disk.
pub vanished: bool,
/// #227. The last season pack that downloaded in full and was condemned
/// at import, while the season is still waiting for a file. `None` when
/// no pack was abandoned, or when the gap has since been filled.
pub import_failure: Option<ImportFailure>,
pub episodes: Vec<Episode>,
}
@@ -702,8 +708,9 @@ enum FileScope {
/// subfolders, sidecar subtitles and artwork go with it. A season or a single
/// episode resolves to the recorded file and nothing else — the title folder
/// holds the siblings this call must not touch, and a season subfolder would
/// have to be re-derived to be named, which §2 forbids. Sidecars beside a
/// removed episode therefore stay; they are not rows this service wrote.
/// have to be re-derived to be named, which §2 forbids. Subtitle sidecars are
/// rows this service wrote too (#186), so they are resolved and unlinked the
/// same way as the video they sit beside (#218).
///
/// The torrent is untouched (§7.3). It keeps seeding under its own rule and
/// the reaper deletes it; a hardlinked file loses only its library name.
@@ -712,7 +719,8 @@ enum FileScope {
/// recorded and can retry rather than losing the record of what is on disk.
async fn remove_library_files(state: &AppState, scope: FileScope) -> Result<(), ApiError> {
let root = scope_root(state, scope).await?;
let paths = scope_paths(state, scope).await?;
let mut paths = scope_paths(state, scope).await?;
paths.extend(subtitle_paths(state, scope).await?);
let mut targets: Vec<std::path::PathBuf> = Vec::new();
for path in &paths {
@@ -832,6 +840,50 @@ async fn scope_paths(state: &AppState, scope: FileScope) -> Result<Vec<String>,
})
}
/// Every subtitle sidecar (#186) this service wrote for the scope. Mirrors
/// [`scope_paths`]: same owner filter, same guard against widening. An
/// embedded track has no `path` — it never touched disk — and is excluded.
async fn subtitle_paths(state: &AppState, scope: FileScope) -> Result<Vec<String>, ApiError> {
Ok(match scope {
FileScope::Series(id) => {
sqlx::query_scalar!(
r#"SELECT sf.path AS "path!: String"
FROM subtitle_files sf
JOIN media_files mf ON mf.id = sf.media_file_id
JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id
JOIN seasons se ON se.id = e.season_id
WHERE se.series_id = ? AND sf.path IS NOT NULL"#,
id
)
.fetch_all(pool(state)?)
.await?
}
FileScope::Season(id) => {
sqlx::query_scalar!(
r#"SELECT sf.path AS "path!: String"
FROM subtitle_files sf
JOIN media_files mf ON mf.id = sf.media_file_id
JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id
WHERE e.season_id = ? AND sf.path IS NOT NULL"#,
id
)
.fetch_all(pool(state)?)
.await?
}
FileScope::Episode(id) => {
sqlx::query_scalar!(
r#"SELECT sf.path AS "path!: String"
FROM subtitle_files sf
JOIN media_files mf ON mf.id = sf.media_file_id
WHERE mf.owner_kind = 'episode' AND mf.owner_id = ? AND sf.path IS NOT NULL"#,
id
)
.fetch_all(pool(state)?)
.await?
}
})
}
/// The one recorded file, when it really sits inside the root. `None` when it
/// does not, which is the guard that keeps a sub-series delete inside the
/// library it belongs to.
@@ -1039,6 +1091,8 @@ async fn load_seasons(state: &AppState, series_id: i64) -> Result<Vec<Season>, A
.fetch_all(pool(state)?)
.await?;
let mut failures = season_import_failures(state, series_id).await?;
Ok(seasons
.into_iter()
.map(|season| Season {
@@ -1047,6 +1101,7 @@ async fn load_seasons(state: &AppState, series_id: i64) -> Result<Vec<Season>, A
number: season.number,
tracked: season.tracked,
vanished: season.vanished,
import_failure: failures.remove(&season.id),
episodes: episodes
.iter()
.filter(|episode| episode.season_id == season.id)
@@ -1416,7 +1471,7 @@ pub async fn episode_releases(
Path(id): Path<i64>,
) -> Result<Json<Vec<Release>>, ApiError> {
let episode = load_episode(&state, id).await?;
let mut releases = sqlx::query_as!(Release, r#"SELECT r.id AS "id!: i64", r.indexer_id AS "indexer_id!: i64", r.guid AS "guid!: String", r.name AS "name!: String", r.size AS "size!: i64", r.seeders, r.publish_date, r.download_url AS "download_url!: String", r.parsed AS "parsed!: serde_json::Value", r.score, r.verdict, r.rejected_rule FROM releases r JOIN episode_releases er ON er.release_id = r.id WHERE er.episode_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id"#, id)
let mut releases = sqlx::query_as!(Release, r#"SELECT r.id AS "id!: i64", r.indexer_id AS "indexer_id!: i64", r.guid AS "guid!: String", r.name AS "name!: String", r.size AS "size!: i64", r.seeders, r.publish_date, r.download_url AS "download_url!: String", r.parsed AS "parsed!: serde_json::Value", r.score, r.verdict, r.rejected_rule, NULL AS "blacklist_reason?: String" FROM releases r JOIN episode_releases er ON er.release_id = r.id WHERE er.episode_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id"#, id)
.fetch_all(pool(&state)?)
.await?;
let policy = state
@@ -1430,6 +1485,7 @@ pub async fn episode_releases(
let lengths = season_lengths(&state, episode.series_id).await?;
let runtime = series_runtime(&state, episode.series_id).await?;
rescore(&mut releases, &policy, Some(&lengths), runtime)?;
attach_blacklist_reasons(pool(&state)?, &mut releases).await?;
Ok(Json(releases))
}
@@ -1571,7 +1627,7 @@ pub async fn season_releases(
) -> Result<Json<Vec<Release>>, ApiError> {
load_series_row(&state, series_id).await?;
let season_id = load_season_id(&state, series_id, number).await?;
let mut releases = sqlx::query_as!(Release, r#"SELECT r.id AS "id!: i64", r.indexer_id AS "indexer_id!: i64", r.guid AS "guid!: String", r.name AS "name!: String", r.size AS "size!: i64", r.seeders, r.publish_date, r.download_url AS "download_url!: String", r.parsed AS "parsed!: serde_json::Value", r.score, r.verdict, r.rejected_rule FROM releases r JOIN season_releases sr ON sr.release_id = r.id WHERE sr.season_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id"#, season_id)
let mut releases = sqlx::query_as!(Release, r#"SELECT r.id AS "id!: i64", r.indexer_id AS "indexer_id!: i64", r.guid AS "guid!: String", r.name AS "name!: String", r.size AS "size!: i64", r.seeders, r.publish_date, r.download_url AS "download_url!: String", r.parsed AS "parsed!: serde_json::Value", r.score, r.verdict, r.rejected_rule, NULL AS "blacklist_reason?: String" FROM releases r JOIN season_releases sr ON sr.release_id = r.id WHERE sr.season_id = ? ORDER BY CASE r.verdict WHEN 'eligible' THEN 0 WHEN 'waived' THEN 1 ELSE 2 END, r.score DESC, r.id"#, season_id)
.fetch_all(pool(&state)?)
.await?;
let policy = state
@@ -1585,9 +1641,92 @@ pub async fn season_releases(
let lengths = season_lengths(&state, series_id).await?;
let runtime = series_runtime(&state, series_id).await?;
rescore(&mut releases, &policy, Some(&lengths), runtime)?;
attach_blacklist_reasons(pool(&state)?, &mut releases).await?;
Ok(Json(releases))
}
/// A grab for this season that downloaded in full and was then condemned at
/// import (#227, §5.7).
///
/// The torrent stays at 100% in Transmission — §7.3 leaves that lifecycle to
/// the reaper — the release is blacklisted and every episode it was covering
/// reopens as a gap. Nothing on screen connected the two, so the season read
/// `0/10` as though no grab had ever been tried. Every fact is already
/// recorded; this is the row that carries them out.
#[derive(Debug, Clone, Serialize, ToSchema)]
pub struct ImportFailure {
/// The blacklisted release name, as the indexer spelled it.
pub release: String,
/// What the blacklist recorded it as failing on — a policy rule name the
/// operator can relax, or a sentence about the release itself. `None`
/// when no blacklist row answers to either key, which is possible after a
/// blacklist row is cleared by hand; the failure is still true.
pub reason: Option<String>,
/// When the grab entered `failed`, RFC3339. `None` on a row that failed
/// before migration 0030 gave the column a value.
pub failed_at: Option<String>,
}
/// The most recent abandoned pack per season of one series (#227).
///
/// Only seasons still waiting for a file are answered: once the gap is
/// filled, the failure is history and the season has nothing to explain. The
/// blacklist reason is matched in memory because its name key is normalised,
/// which SQL cannot compute.
async fn season_import_failures(
state: &AppState,
series_id: i64,
) -> Result<HashMap<i64, ImportFailure>, ApiError> {
let pool = pool(state)?;
let rows = sqlx::query!(
r#"SELECT g.target_id AS "season_id!: i64",
r.name AS "name!: String",
g.infohash AS "infohash!: String",
g.failed_at,
g.grabbed_at AS "grabbed_at!: String"
FROM grabs g
JOIN releases r ON r.id = g.release_id
JOIN seasons s ON s.id = g.target_id
WHERE g.target_kind = 'season'
AND g.state = 'failed'
AND s.series_id = ?
AND EXISTS (
SELECT 1 FROM episodes e
WHERE e.season_id = s.id AND e.wanted
AND NOT EXISTS (
SELECT 1 FROM media_files f
WHERE f.owner_kind = 'episode' AND f.owner_id = e.id
)
)
ORDER BY coalesce(g.failed_at, g.grabbed_at), g.id"#,
series_id
)
.fetch_all(pool)
.await?;
if rows.is_empty() {
return Ok(HashMap::new());
}
let blacklist = arr_db::blacklist::Blacklist::load(pool).await?;
// Ascending order, so the last row written for a season wins.
Ok(rows
.into_iter()
.map(|row| {
let reason = blacklist
.reason_for_infohash(&row.infohash)
.or_else(|| blacklist.reason_for_name(&row.name))
.map(str::to_owned);
(
row.season_id,
ImportFailure {
release: row.name,
reason,
failed_at: row.failed_at,
},
)
})
.collect())
}
/// Which lane a season's missing episodes take (#182, §6.2).
#[derive(Debug, Clone, Copy, Serialize, ToSchema)]
#[serde(rename_all = "snake_case")]
@@ -1629,6 +1768,11 @@ pub struct SeasonPackState {
/// When a pack sweep for this season last completed. `None` means no
/// pack search has ever run, so an empty deck is pending, not settled.
pub last_pack_search_at: Option<String>,
/// #227. The last pack that downloaded in full and was then condemned at
/// import, while the season is still waiting for a file. A deck that
/// cannot say this leaves the season reading as though nothing was ever
/// tried, with the torrent still sitting at 100% in Transmission.
pub import_failure: Option<ImportFailure>,
}
#[utoipa::path(
@@ -1672,9 +1816,12 @@ pub async fn season_pack_state(
.await?;
// The same failed-pack tally the grab lane backs off on (#181), read
// here so the deck can name the window instead of guessing at one.
// `last_failed_at` really is the failure time (#245): `reopens_at` and
// `pack_retry_at` below hand it to the deck, so an alias holding a grab
// time would put a grab under a name §5.7 gave to something else.
let failed = sqlx::query!(
r#"SELECT count(*) AS "failures!: i64",
max(grabbed_at) AS "last_failed_at?: String"
r#"SELECT count(*) AS "failures!: i64",
max(coalesce(failed_at, grabbed_at)) AS "last_failed_at?: String"
FROM grabs
WHERE target_kind = 'season' AND target_id = ? AND state = 'failed'"#,
season_id
@@ -1720,6 +1867,9 @@ pub async fn season_pack_state(
pack_failures: failed.failures,
pack_retry_at: pack_retry_at.map(|at| at.to_rfc3339()),
last_pack_search_at,
import_failure: season_import_failures(&state, series_id)
.await?
.remove(&season_id),
}))
}
@@ -1939,6 +2089,133 @@ mod tests {
response.json().await.expect("season json")
}
async fn seasons_json(base: &str, series_id: i64) -> Vec<serde_json::Value> {
reqwest::get(format!("{base}/api/series/{series_id}/seasons"))
.await
.expect("seasons")
.json()
.await
.expect("seasons json")
}
/// #227, the operator's own report: a pack downloaded in full, §5.7
/// condemned it at import, every episode went back to `missing`, and the
/// season read `0/10` with nothing anywhere saying a grab had been tried.
/// The season row, its deck and the deck's blacklisted release now each
/// carry the failure and the reason it failed on.
#[tokio::test]
async fn an_abandoned_pack_is_visible_on_the_season_and_its_deck() {
let (_dir, state, base) = application().await;
let root_id = tv_root(&state, "main").await;
let series = add_series(&base, root_id, false).await;
let series_id = series["id"].as_i64().expect("id");
let episodes: Vec<serde_json::Value> = (1..=10)
.map(|number| {
serde_json::json!({
"number": number, "title": format!("Episode {number}"),
"air_date": "2025-01-01"
})
})
.collect();
let season = add_season(&base, series_id, 8, serde_json::json!(episodes)).await;
let season_id = season["id"].as_i64().expect("season id");
assert!(
season["import_failure"].is_null(),
"nothing has been grabbed yet"
);
// §4.1: tracking the season is what makes its episodes wanted, and a
// season with no intent has no gap to explain.
let tracked = reqwest::Client::new()
.patch(format!("{base}/api/series/{series_id}/seasons/8"))
.json(&serde_json::json!({"tracked": true}))
.send()
.await
.expect("track the season");
assert_eq!(tracked.status(), StatusCode::OK);
let pool = state.database().expect("database").pool();
let name = "Rick.And.Morty.S08.1080p.WEB-DL.x264-GROUP";
let parsed = arr_parse::parse(name);
let release_id = sqlx::query_scalar::<_, i64>(
"INSERT INTO releases (indexer_id, guid, name, size, seeders, download_url, parsed, score, verdict)
VALUES (7, 'pack', ?, 1000, 50, 'url', ?, 0, 'eligible') RETURNING id",
)
.bind(name)
.bind(serde_json::to_string(&parsed).expect("parsed json"))
.fetch_one(pool)
.await
.expect("release");
sqlx::query("INSERT INTO season_releases (season_id, release_id) VALUES (?, ?)")
.bind(season_id)
.bind(release_id)
.execute(pool)
.await
.expect("association");
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash, state, grabbed_at, failed_at)
VALUES (?, 'season', ?, 'abc123', 'failed', '2026-01-01T00:00:00.000Z', '2026-01-02T00:00:00.000Z')",
)
.bind(release_id)
.bind(season_id)
.execute(pool)
.await
.expect("failed pack grab");
// The failure with no blacklist row yet: still a failure, and the
// reason is simply not known. Rows written before the blacklist
// carried one read this way and must keep rendering.
let seasons = seasons_json(&base, series_id).await;
let row = &seasons[0]["import_failure"];
assert_eq!(row["release"], name);
assert_eq!(row["reason"], serde_json::Value::Null);
assert_eq!(row["failed_at"], "2026-01-02T00:00:00.000Z");
arr_db::blacklist::add(pool, Some("ABC123"), name, "size")
.await
.expect("blacklist");
let seasons = seasons_json(&base, series_id).await;
assert_eq!(seasons[0]["import_failure"]["reason"], "size");
// The deck the season row leads to says the same thing.
let pack_state: serde_json::Value = reqwest::get(format!(
"{base}/api/series/{series_id}/seasons/8/pack-state"
))
.await
.expect("pack state")
.json()
.await
.expect("pack state json");
assert_eq!(pack_state["import_failure"]["release"], name);
assert_eq!(pack_state["import_failure"]["reason"], "size");
// §6.3: the release itself is rejected in the deck, and now names
// what it was blacklisted for — a size rejection the operator can
// relax, not a corrupt file they should leave alone.
let releases: Vec<serde_json::Value> =
reqwest::get(format!("{base}/api/series/{series_id}/seasons/8/releases"))
.await
.expect("releases")
.json()
.await
.expect("releases json");
assert_eq!(releases.len(), 1);
assert_eq!(releases[0]["blacklist_reason"], "size");
// Once the gap is filled the failure is history: the season has
// nothing left to explain and stops saying it.
for episode in seasons[0]["episodes"].as_array().expect("episodes") {
sqlx::query("INSERT INTO media_files (owner_kind, owner_id, path, size) VALUES ('episode', ?, ?, 1)")
.bind(episode["id"].as_i64().expect("episode id"))
.bind(format!("/library/e{}.mkv", episode["number"]))
.execute(pool)
.await
.expect("file on disk");
}
let seasons = seasons_json(&base, series_id).await;
assert!(seasons[0]["import_failure"].is_null());
}
/// The production case behind #210: every pack of a season is under
/// §5.5's per-episode floor, so the deck holds three candidates and
/// nothing is grabbable. Writing `allow_below_floor` turns the
@@ -2014,9 +2291,10 @@ mod tests {
releases[0]["verdict"], "waived",
"a waived grab stays a waiver; nothing here makes it eligible"
);
// The row's rule name goes with the rejection; what survives is the
// dashed `waived` verdict the deck reads (§9.3).
assert!(releases[0]["rejected_rule"].is_null());
// #211: the waiver keeps the rule it relaxed, so the deck names it
// the way it names a rejection (§9.3) instead of showing a bare
// `waived`.
assert_eq!(releases[0]["rejected_rule"], "size");
// The grab the deck's one click sends is now accepted.
let response = reqwest::Client::new()
@@ -2851,8 +3129,11 @@ mod tests {
.await
.expect("release");
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash, state, grabbed_at)
VALUES (?, 'season', ?, 'hash', 'failed', strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))",
"INSERT INTO grabs (release_id, target_kind, target_id, infohash, state,
grabbed_at, failed_at)
VALUES (?, 'season', ?, 'hash', 'failed',
strftime('%Y-%m-%dT%H:%M:%fZ', 'now'),
strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))",
)
.bind(release_id)
.bind(season_id)
@@ -2868,6 +3149,26 @@ mod tests {
"the deck offers a date, not just a closed door"
);
// §6.2/#245: push the grab five weeks back and leave the failure
// where it is. The deck reads the failure, so the window it names is
// unmoved — anchored on the grab it would have expired long ago and
// the deck would claim the lane was open.
sqlx::query(
"UPDATE grabs SET grabbed_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now', '-35 days')
WHERE target_kind = 'season' AND target_id = ?",
)
.bind(season_id)
.execute(pool)
.await
.expect("age the grab");
let stalled = state_of(url.clone()).await;
assert_eq!(stalled["lane"], "per_episode");
assert_eq!(stalled["reason"], "pack_backoff");
assert_eq!(
stalled["pack_retry_at"], quiet["pack_retry_at"],
"the window is anchored on the failure, so aging the grab moves nothing"
);
// §14 outranks it: a pack would re-import what is on disk, so
// clearing the failure would not earn a pack anyway.
sqlx::query("INSERT INTO media_files (owner_kind, owner_id, path, size) VALUES ('episode', ?, '/library/e01.mkv', 1)")
@@ -3420,6 +3721,42 @@ mod tests {
file
}
/// Same as [`episode_file_on_disk`], plus a §15 sidecar next to it with a
/// `subtitle_files` row pointing at the video's `media_files` row (#218).
async fn episode_file_with_sidecar_on_disk(
state: &AppState,
episode_id: i64,
root: &std::path::Path,
season: i64,
name: &str,
) -> (std::path::PathBuf, std::path::PathBuf) {
let file = episode_file_on_disk(state, episode_id, root, season, name).await;
let pool = state.database().expect("database").pool();
let media_file_id: i64 = sqlx::query_scalar(
"SELECT id FROM media_files WHERE owner_kind = 'episode' AND owner_id = ? AND path = ?",
)
.bind(episode_id)
.bind(file.to_str().expect("utf-8 path"))
.fetch_one(pool)
.await
.expect("media file id");
let sidecar = file.with_extension("pt-PT.srt");
tokio::fs::write(&sidecar, b"subs")
.await
.expect("write sidecar");
sqlx::query(
"INSERT INTO subtitle_files (media_file_id, language, origin, provider, path)
VALUES (?, 'pt-PT', 'provider', 'opensubtitles', ?)",
)
.bind(media_file_id)
.bind(sidecar.to_str().expect("utf-8 path"))
.execute(pool)
.await
.expect("subtitle file");
(file, sidecar)
}
async fn point_root_at(state: &AppState, root_id: i64, path: &std::path::Path) {
sqlx::query("UPDATE roots SET path = ? WHERE id = ?")
.bind(path.to_str().expect("utf-8 root"))
@@ -3622,6 +3959,122 @@ mod tests {
assert_eq!(remaining, vec![second], "only the episode's row goes");
}
/// #218: a season-scoped delete unlinks its subtitle sidecars too — the
/// `subtitle_files` rows go with the video's `media_files` row, but the
/// files on disk do not follow without this.
#[tokio::test]
async fn removing_a_season_takes_its_subtitle_sidecars() {
let (_dir, state, base) = application().await;
let root_id = tv_root(&state, "main").await;
let series = add_series(&base, root_id, true).await;
let series_id = series["id"].as_i64().expect("id");
let root = tempfile::tempdir().expect("root");
point_root_at(&state, root_id, root.path()).await;
let first = add_season(
&base,
series_id,
1,
serde_json::json!([{"number": 1, "title": "The Magic Xylophone"}]),
)
.await;
let second = add_season(
&base,
series_id,
2,
serde_json::json!([{"number": 1, "title": "Dance Mode"}]),
)
.await;
let s01e01 = first["episodes"][0]["id"].as_i64().expect("id");
let s02e01 = second["episodes"][0]["id"].as_i64().expect("id");
let (_, in_scope_sidecar) =
episode_file_with_sidecar_on_disk(&state, s01e01, root.path(), 1, "Bluey - S01E01.mkv")
.await;
let (_, other_sidecar) =
episode_file_with_sidecar_on_disk(&state, s02e01, root.path(), 2, "Bluey - S02E01.mkv")
.await;
let response = reqwest::Client::new()
.delete(format!("{base}/api/series/{series_id}/seasons/1/files"))
.send()
.await
.expect("delete season files");
assert_eq!(response.status(), StatusCode::NO_CONTENT);
assert!(!in_scope_sidecar.exists(), "its sidecar is gone with it");
assert!(
other_sidecar.exists(),
"another season's sidecar is not in scope"
);
let pool = state.database().expect("database").pool();
let orphans: i64 = sqlx::query_scalar("SELECT count(*) FROM subtitle_files")
.fetch_one(pool)
.await
.expect("count subtitle rows");
assert_eq!(
orphans, 1,
"only the deleted season's subtitle row cascades"
);
}
/// #218: the same fix at the narrower episode scope, leaving a sibling
/// episode's sidecar untouched.
#[tokio::test]
async fn removing_one_episode_takes_its_subtitle_sidecar() {
let (_dir, state, base) = application().await;
let root_id = tv_root(&state, "main").await;
let series = add_series(&base, root_id, true).await;
let series_id = series["id"].as_i64().expect("id");
let root = tempfile::tempdir().expect("root");
point_root_at(&state, root_id, root.path()).await;
let season = add_season(
&base,
series_id,
1,
serde_json::json!([
{"number": 1, "title": "The Magic Xylophone"},
{"number": 2, "title": "Hospital"}
]),
)
.await;
let episodes = season["episodes"].as_array().expect("episodes");
let first = episodes
.iter()
.find(|episode| episode["number"] == 1)
.expect("s01e01")["id"]
.as_i64()
.expect("id");
let second = episodes
.iter()
.find(|episode| episode["number"] == 2)
.expect("s01e02")["id"]
.as_i64()
.expect("id");
let (_, first_sidecar) =
episode_file_with_sidecar_on_disk(&state, first, root.path(), 1, "Bluey - S01E01.mkv")
.await;
let (_, second_sidecar) =
episode_file_with_sidecar_on_disk(&state, second, root.path(), 1, "Bluey - S01E02.mkv")
.await;
let response = reqwest::Client::new()
.delete(format!("{base}/api/episodes/{first}/files"))
.send()
.await
.expect("delete episode files");
assert_eq!(response.status(), StatusCode::NO_CONTENT);
assert!(!first_sidecar.exists(), "the episode's sidecar is gone");
assert!(
second_sidecar.exists(),
"its sibling's sidecar is not in scope"
);
}
/// #174: removal is not conditional on there being anything to remove.
/// A scope with no files still clears intent, and still answers 204.
#[tokio::test]
+118 -1
View File
@@ -1,10 +1,12 @@
//! What the API needs to answer a request: one HTTP client and the addresses
//! of the three upstreams the service cannot work without (DESIGN.md §3).
use std::sync::Arc;
use std::ffi::{OsStr, OsString};
use std::sync::{atomic::AtomicU64, Arc};
use std::time::Duration;
use arr_db::Db;
use arr_subs::{Backend, OpenAiEndpoint, Provider, Syncer};
use tokio::sync::mpsc;
use crate::jellyfin::JellyfinClient;
@@ -13,6 +15,10 @@ use crate::jellyfin::JellyfinClient;
/// is configurable, so this is a constant that tests point elsewhere.
pub const DEFAULT_TMDB_URL: &str = "https://api.themoviedb.org/3";
/// The `ffmpeg` invoked when nothing else is configured — same default as
/// `arr-probe`'s extractor, which is where it is actually run.
pub const DEFAULT_FFMPEG_BINARY: &str = "ffmpeg";
/// How long an upstream has to answer a probe before it counts as
/// unreachable. Health is polled by a human waiting on a page.
const PROBE_TIMEOUT: Duration = Duration::from_secs(3);
@@ -78,7 +84,21 @@ pub struct AppState {
pending_season_commands: Arc<tokio::sync::Mutex<mpsc::Receiver<SeasonCommand>>>,
metadata_commands: mpsc::Sender<MetadataCommand>,
pending_metadata_commands: Arc<tokio::sync::Mutex<mpsc::Receiver<MetadataCommand>>>,
subtitle_providers: Arc<Vec<Arc<dyn Provider>>>,
translation_backends: Arc<Vec<Arc<dyn Backend>>>,
/// The remote-command backend's live timeout, in milliseconds (#219).
/// `None` unless the daemon compiled and configured that backend; the
/// settings API writes it on every edit of the row.
command_timeout: Option<Arc<AtomicU64>>,
/// Where the OpenAI-compatible backend points and which model it names
/// (#220). `None` unless that backend is compiled in and wired up; the
/// settings API repoints it on every edit, and the health lamp probes
/// whatever it currently holds.
openai_endpoint: Option<OpenAiEndpoint>,
/// The configured `ffmpeg` binary, for the health lamps (#200).
ffmpeg_binary: OsString,
jellyfin: Option<JellyfinClient>,
syncer: Syncer,
}
/// Work explicitly requested through the movie API.
@@ -149,7 +169,13 @@ impl AppState {
pending_season_commands: Arc::new(tokio::sync::Mutex::new(pending_season_commands)),
metadata_commands,
pending_metadata_commands: Arc::new(tokio::sync::Mutex::new(pending_metadata_commands)),
subtitle_providers: Arc::new(Vec::new()),
translation_backends: Arc::new(Vec::new()),
command_timeout: None,
openai_endpoint: None,
ffmpeg_binary: DEFAULT_FFMPEG_BINARY.into(),
jellyfin: None,
syncer: Syncer::default(),
})
}
@@ -160,6 +186,48 @@ impl AppState {
self
}
/// Attach the subtitle providers this deployment has credentials for
/// (`DESIGN.md` §15).
///
/// Which of them a search actually runs is the `providers_enabled`
/// setting, read per request; this is the narrower fact of which ones
/// exist at all, because credentials are bootstrap config and never
/// reach the database (§10).
#[must_use]
pub fn with_subtitle_providers(mut self, providers: Vec<Arc<dyn Provider>>) -> Self {
self.subtitle_providers = Arc::new(providers);
self
}
/// Attach the translation backends this binary compiled in (§15).
///
/// Empty when no `translate-*` cargo feature is on, which is the default
/// — a manual translation then fails with a message saying so rather
/// than silently doing nothing.
#[must_use]
pub fn with_translation_backends(mut self, backends: Vec<Arc<dyn Backend>>) -> Self {
self.translation_backends = Arc::new(backends);
self
}
/// Attach the cell the remote-command backend re-reads per batch (#219),
/// so an edit of `remote_command_timeout_seconds` reaches it without a
/// restart. Absent when that backend is not configured.
#[must_use]
pub fn with_command_timeout(mut self, timeout: Arc<AtomicU64>) -> Self {
self.command_timeout = Some(timeout);
self
}
/// Attach the cell the OpenAI-compatible backend re-reads per request
/// (#220), so an edit of `openai_base_url` or `openai_model` reaches it
/// — and the health lamp probes it — without a restart.
#[must_use]
pub fn with_openai_endpoint(mut self, endpoint: OpenAiEndpoint) -> Self {
self.openai_endpoint = Some(endpoint);
self
}
/// Attach the Jellyfin client, so a manual subtitle grab or translation
/// can trigger the same library refresh import does (§7.5, §15).
#[must_use]
@@ -172,6 +240,55 @@ impl AppState {
self.jellyfin.as_ref()
}
pub(crate) fn subtitle_provider(&self, id: &str) -> Option<&Arc<dyn Provider>> {
self.subtitle_providers
.iter()
.find(|provider| provider.id().as_str() == id)
}
pub(crate) fn subtitle_providers(&self) -> &[Arc<dyn Provider>] {
&self.subtitle_providers
}
pub(crate) fn translation_backend(&self, id: &str) -> Option<&Arc<dyn Backend>> {
self.translation_backends
.iter()
.find(|backend| backend.id().as_str() == id)
}
pub(crate) fn command_timeout(&self) -> Option<&Arc<AtomicU64>> {
self.command_timeout.as_ref()
}
pub(crate) fn openai_endpoint(&self) -> Option<&OpenAiEndpoint> {
self.openai_endpoint.as_ref()
}
/// Attach the `alass` binary this deployment runs (§15). Defaults to
/// resolving `alass` from `PATH`.
#[must_use]
pub fn with_syncer(mut self, syncer: Syncer) -> Self {
self.syncer = syncer;
self
}
/// Attach the configured `ffmpeg` binary, for the health lamps (#200).
/// Defaults to resolving `ffmpeg` from `PATH`.
#[must_use]
pub fn with_ffmpeg_binary(mut self, binary: impl Into<OsString>) -> Self {
self.ffmpeg_binary = binary.into();
self
}
/// The configured `ffmpeg` binary.
pub(crate) fn ffmpeg_binary(&self) -> &OsStr {
&self.ffmpeg_binary
}
pub(crate) fn syncer(&self) -> &Syncer {
&self.syncer
}
/// Wait for the next manual movie action in the daemon's reconcile loop.
///
/// # Errors
+761
View File
@@ -0,0 +1,761 @@
//! The runtime-editable half of subtitle configuration (`DESIGN.md` §15,
//! §10, issue #198). Provider credentials, translator keys, the
//! remote-command template and the `alass`/`ffmpeg` paths never reach here —
//! those are config/env, per §10, and this surface would leak them into a
//! `sqlite3 .backup` on a timer if it did.
//!
//! A single row rather than a CRUD collection: the wanted set, enabled
//! providers and translation engine are global, not per root (§15).
use std::collections::BTreeMap;
use axum::extract::rejection::JsonRejection;
use axum::extract::State;
use axum::Json;
use serde::{Deserialize, Serialize};
use utoipa::ToSchema;
use crate::movies::{pool, ApiError, ErrorBody};
use crate::policies::parsed;
use crate::state::AppState;
/// The subtitle settings row, plus which translation engines this binary
/// actually has compiled in.
#[derive(Debug, Clone, Serialize, ToSchema)]
pub struct SubtitleSettings {
pub wanted_languages: Vec<String>,
pub providers_enabled: Vec<String>,
pub translation_engine: Option<String>,
pub provider_daily_budgets: BTreeMap<String, u32>,
pub translator_daily_budgets: BTreeMap<String, u32>,
pub remote_command_timeout_seconds: u32,
/// Where the OpenAI-compatible backend points (#220). `null` means its
/// own default, `https://api.openai.com/v1/` — that backend is anything
/// speaking the shape, so a `llama.cpp` address belongs here. Not a
/// secret: the API key stays in the environment (§10).
pub openai_base_url: Option<String>,
/// The model that backend names. `null` means its own default.
pub openai_model: Option<String>,
/// Engines DESIGN.md §15 knows about that this binary compiled in.
/// `translation_engine` is always a member of this list or `null` — a
/// backend whose cargo feature is missing is never selectable.
pub available_engines: Vec<String>,
}
/// The payload for replacing the settings row. Same shape as
/// [`SubtitleSettings`] minus `available_engines`, which is a fact about the
/// binary, not something an operator sets.
#[derive(Debug, Clone, Deserialize, ToSchema)]
pub struct SubtitleSettingsInput {
pub wanted_languages: Vec<String>,
pub providers_enabled: Vec<String>,
pub translation_engine: Option<String>,
#[serde(default)]
pub provider_daily_budgets: BTreeMap<String, u32>,
#[serde(default)]
pub translator_daily_budgets: BTreeMap<String, u32>,
pub remote_command_timeout_seconds: u32,
#[serde(default)]
pub openai_base_url: Option<String>,
#[serde(default)]
pub openai_model: Option<String>,
}
impl SubtitleSettingsInput {
/// Validate against the vocabulary the running binary actually knows.
/// Every failure names its field so a rejected edit is fixable without
/// reading the schema.
fn validate(&self) -> Result<(), String> {
if self.wanted_languages.is_empty() {
return Err("wanted_languages: must not be empty".into());
}
if self.wanted_languages.iter().any(String::is_empty) {
return Err("wanted_languages: language tags must not be empty".into());
}
let mut seen = std::collections::BTreeSet::new();
for lang in &self.wanted_languages {
if !seen.insert(lang.as_str()) {
return Err(format!("wanted_languages: '{lang}' appears twice"));
}
}
if self.providers_enabled.iter().any(String::is_empty) {
return Err("providers_enabled: provider ids must not be empty".into());
}
let mut seen = std::collections::BTreeSet::new();
for provider in &self.providers_enabled {
if !seen.insert(provider.as_str()) {
return Err(format!("providers_enabled: '{provider}' appears twice"));
}
}
if let Some(engine) = &self.translation_engine {
if !arr_subs::ENGINES.contains(&engine.as_str()) {
return Err(format!(
"translation_engine: '{engine}' is not a known engine"
));
}
if !arr_subs::compiled_engines().contains(&engine.as_str()) {
return Err(format!(
"translation_engine: '{engine}' is not compiled into this binary"
));
}
}
if self.remote_command_timeout_seconds == 0 {
return Err("remote_command_timeout_seconds: must be greater than zero".into());
}
// #220: an unparseable base URL is rejected here rather than at the
// next translation, where it would surface as an engine that has
// quietly stopped working. Validated whether or not this build
// compiled the backend in — the column exists either way.
if let Some(base_url) = blank_to_none(self.openai_base_url.as_deref()) {
arr_subs::OpenAiEndpoint::new(Some(base_url), None).map_err(|error| {
// The backend's own Display talks about a reply that came
// back wrong; here nothing was ever sent, so only the
// reason belongs in the message.
let reason = match &error {
arr_subs::translate::Error::Malformed { detail, .. } => detail.clone(),
other => other.to_string(),
};
format!("openai_base_url: {reason}")
})?;
}
Ok(())
}
fn into_columns(self) -> Result<SettingsColumns, ApiError> {
fn json(value: impl serde::Serialize) -> Result<String, ApiError> {
serde_json::to_string(&value).map_err(|error| {
tracing::error!(%error, "subtitle settings serialisation failed");
ApiError::Database("serialisation failed".into())
})
}
Ok(SettingsColumns {
wanted_languages: json(&self.wanted_languages)?,
providers_enabled: json(&self.providers_enabled)?,
translation_engine: self.translation_engine,
provider_daily_budgets: json(&self.provider_daily_budgets)?,
translator_daily_budgets: json(&self.translator_daily_budgets)?,
remote_command_timeout_seconds: i64::from(self.remote_command_timeout_seconds),
// An empty field means "use the backend's default", which is the
// NULL the migration describes — not an endpoint named "".
openai_base_url: blank_to_none(self.openai_base_url.as_deref()).map(str::to_owned),
openai_model: blank_to_none(self.openai_model.as_deref()).map(str::to_owned),
})
}
}
/// The row as the table stores it, before JSON parsing.
struct SettingsColumns {
wanted_languages: String,
providers_enabled: String,
translation_engine: Option<String>,
provider_daily_budgets: String,
translator_daily_budgets: String,
remote_command_timeout_seconds: i64,
openai_base_url: Option<String>,
openai_model: Option<String>,
}
/// A field the operator left empty is absent, not an empty setting.
fn blank_to_none(value: Option<&str>) -> Option<&str> {
value.map(str::trim).filter(|value| !value.is_empty())
}
fn column<T: serde::de::DeserializeOwned>(
column: &'static str,
value: &str,
) -> Result<T, ApiError> {
serde_json::from_str(value).map_err(|error| {
tracing::error!(column, %error, "subtitle settings column holds unexpected JSON");
ApiError::Database(format!("subtitle settings column {column} is not valid"))
})
}
impl SettingsColumns {
fn into_settings(self) -> Result<SubtitleSettings, ApiError> {
Ok(SubtitleSettings {
wanted_languages: column("wanted_languages", &self.wanted_languages)?,
providers_enabled: column("providers_enabled", &self.providers_enabled)?,
translation_engine: self.translation_engine,
provider_daily_budgets: column("provider_daily_budgets", &self.provider_daily_budgets)?,
translator_daily_budgets: column(
"translator_daily_budgets",
&self.translator_daily_budgets,
)?,
remote_command_timeout_seconds: u32::try_from(self.remote_command_timeout_seconds)
.unwrap_or(0),
openai_base_url: self.openai_base_url,
openai_model: self.openai_model,
available_engines: arr_subs::compiled_engines()
.into_iter()
.map(str::to_owned)
.collect(),
})
}
}
/// Read the row and parse it. Shared with the health lamps (#200), which
/// need the enabled set and the chosen engine but not the budgets.
pub(crate) async fn load(state: &AppState) -> Result<SubtitleSettings, ApiError> {
let row = sqlx::query_as!(
SettingsColumns,
r#"SELECT wanted_languages AS "wanted_languages!: String",
providers_enabled AS "providers_enabled!: String",
translation_engine AS "translation_engine: String",
provider_daily_budgets AS "provider_daily_budgets!: String",
translator_daily_budgets AS "translator_daily_budgets!: String",
remote_command_timeout_seconds AS "remote_command_timeout_seconds!: i64",
openai_base_url AS "openai_base_url: String",
openai_model AS "openai_model: String"
FROM subtitle_settings WHERE id = 1"#
)
.fetch_one(pool(state)?)
.await?;
row.into_settings()
}
#[utoipa::path(
get, path = "/api/settings/subtitles", tag = "subtitles",
responses(
(status = 200, body = SubtitleSettings),
(status = 500, body = ErrorBody),
(status = 503, body = ErrorBody)
)
)]
pub async fn get(State(state): State<AppState>) -> Result<Json<SubtitleSettings>, ApiError> {
Ok(Json(load(&state).await?))
}
#[utoipa::path(
put, path = "/api/settings/subtitles", tag = "subtitles", request_body = SubtitleSettingsInput,
responses(
(status = 200, body = SubtitleSettings),
(status = 422, body = ErrorBody),
(status = 500, body = ErrorBody),
(status = 503, body = ErrorBody)
)
)]
pub async fn update(
State(state): State<AppState>,
body: Result<Json<SubtitleSettingsInput>, JsonRejection>,
) -> Result<Json<SubtitleSettings>, ApiError> {
let input = parsed(body)?;
input.validate().map_err(ApiError::Invalid)?;
let new_wanted = input.wanted_languages.clone();
let columns = input.into_columns()?;
let timeout_seconds = columns.remote_command_timeout_seconds;
let openai_base_url = columns.openai_base_url.clone();
let openai_model = columns.openai_model.clone();
let previous_wanted: String =
sqlx::query_scalar!("SELECT wanted_languages FROM subtitle_settings WHERE id = 1")
.fetch_one(pool(&state)?)
.await?;
let dropped: Vec<String> = column::<Vec<String>>("wanted_languages", &previous_wanted)?
.into_iter()
.filter(|language| !new_wanted.contains(language))
.collect();
let mut transaction = pool(&state)?.begin().await?;
sqlx::query!(
r#"UPDATE subtitle_settings SET
wanted_languages = ?, providers_enabled = ?, translation_engine = ?,
provider_daily_budgets = ?, translator_daily_budgets = ?,
remote_command_timeout_seconds = ?,
openai_base_url = ?, openai_model = ?,
updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now')
WHERE id = 1"#,
columns.wanted_languages,
columns.providers_enabled,
columns.translation_engine,
columns.provider_daily_budgets,
columns.translator_daily_budgets,
columns.remote_command_timeout_seconds,
columns.openai_base_url,
columns.openai_model,
)
.execute(&mut *transaction)
.await?;
// #224: a dropped language's attempt bookkeeping (backoff counter,
// `last_failure`) must not resurrect if the language is re-added later.
// Subtitle files stay — only the attempt rows are wanted-set-scoped.
for language in &dropped {
sqlx::query!("DELETE FROM subtitle_attempts WHERE language = ?", language)
.execute(&mut *transaction)
.await?;
}
transaction.commit().await?;
// Issue #219: the row alone never reaches the running backend. Push it
// into the cell the remote-command translator re-reads per batch; the
// cell counts milliseconds, the row counts seconds.
if let Some(timeout) = state.command_timeout() {
timeout.store(
u64::try_from(timeout_seconds)
.unwrap_or(u64::MAX)
.saturating_mul(1_000),
std::sync::atomic::Ordering::Relaxed,
);
}
// #220: the same path for the OpenAI-compatible backend. `validate` has
// already parsed the base URL, so this cannot fail for a reason the
// operator has not been told about.
if let Some(endpoint) = state.openai_endpoint() {
if let Err(error) = endpoint.set(openai_base_url.as_deref(), openai_model.as_deref()) {
tracing::error!(%error, "validated openai endpoint failed to apply");
}
}
Ok(Json(load(&state).await?))
}
#[cfg(test)]
mod tests {
use axum::http::StatusCode;
use crate::{router, AppState, Upstreams};
async fn application() -> (tempfile::TempDir, String) {
let (dir, base, _timeout) = application_with_timeout().await;
(dir, base)
}
/// The same app, with the remote-command backend's live timeout cell
/// attached — what the daemon wires up when that backend is configured.
async fn application_with_timeout() -> (
tempfile::TempDir,
String,
std::sync::Arc<std::sync::atomic::AtomicU64>,
) {
use std::sync::{atomic::AtomicU64, Arc};
let dir = tempfile::tempdir().expect("tempdir");
let database = arr_db::Db::connect(dir.path().join("arr.db"))
.await
.expect("connect database");
database.migrate().await.expect("migrate database");
let timeout = Arc::new(AtomicU64::new(30_000));
let state = AppState::new(Upstreams::new(
"http://127.0.0.1:1".into(),
"http://127.0.0.1:1".into(),
))
.expect("state")
.with_database(database)
.with_command_timeout(timeout.clone());
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind");
let address = listener.local_addr().expect("address");
let app = router(state);
tokio::spawn(async move { axum::serve(listener, app).await.expect("serve") });
(dir, format!("http://{address}"), timeout)
}
/// The same app, with the OpenAI-compatible backend's live endpoint
/// attached — what the daemon wires up when that backend is compiled in.
async fn application_with_openai() -> (tempfile::TempDir, String, arr_subs::OpenAiEndpoint) {
let dir = tempfile::tempdir().expect("tempdir");
let database = arr_db::Db::connect(dir.path().join("arr.db"))
.await
.expect("connect database");
database.migrate().await.expect("migrate database");
let endpoint = arr_subs::OpenAiEndpoint::new(None, None).expect("defaults resolve");
let state = AppState::new(Upstreams::new(
"http://127.0.0.1:1".into(),
"http://127.0.0.1:1".into(),
))
.expect("state")
.with_database(database)
.with_openai_endpoint(endpoint.clone());
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind");
let address = listener.local_addr().expect("address");
let app = router(state);
tokio::spawn(async move { axum::serve(listener, app).await.expect("serve") });
(dir, format!("http://{address}"), endpoint)
}
/// The same app, with the underlying pool exposed so a test can seed or
/// inspect rows the API surface does not read back directly.
async fn application_with_pool() -> (tempfile::TempDir, String, sqlx::SqlitePool) {
let dir = tempfile::tempdir().expect("tempdir");
let database = arr_db::Db::connect(dir.path().join("arr.db"))
.await
.expect("connect database");
database.migrate().await.expect("migrate database");
let pool = database.pool().clone();
let state = AppState::new(Upstreams::new(
"http://127.0.0.1:1".into(),
"http://127.0.0.1:1".into(),
))
.expect("state")
.with_database(database);
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind");
let address = listener.local_addr().expect("address");
let app = router(state);
tokio::spawn(async move { axum::serve(listener, app).await.expect("serve") });
(dir, format!("http://{address}"), pool)
}
fn valid_input() -> serde_json::Value {
serde_json::json!({
"wanted_languages": ["pt-PT", "en"],
"providers_enabled": ["opensubtitles", "podnapisi"],
"translation_engine": null,
"provider_daily_budgets": { "opensubtitles": 100 },
"translator_daily_budgets": {},
"remote_command_timeout_seconds": 45
})
}
#[tokio::test]
async fn the_seeded_row_reads_back_with_the_engines_this_build_compiled() {
let (_dir, base) = application().await;
let settings: serde_json::Value = reqwest::get(format!("{base}/api/settings/subtitles"))
.await
.expect("get settings")
.json()
.await
.expect("settings json");
assert_eq!(
settings["wanted_languages"],
serde_json::json!(["pt-PT", "en"])
);
assert_eq!(
settings["providers_enabled"],
serde_json::json!(["opensubtitles", "podnapisi"])
);
assert!(settings["translation_engine"].is_null());
// `available_engines` is a fact about the build, not about the seed
// row, so it tracks the `translate-*` features rather than a literal.
assert_eq!(
settings["available_engines"],
serde_json::json!(arr_subs::compiled_engines())
);
}
#[tokio::test]
async fn the_settings_round_trip_through_a_put() {
let (_dir, base) = application().await;
let updated: serde_json::Value = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&valid_input())
.send()
.await
.expect("put settings")
.json()
.await
.expect("updated json");
assert_eq!(
updated["provider_daily_budgets"],
serde_json::json!({ "opensubtitles": 100 })
);
assert_eq!(updated["remote_command_timeout_seconds"], 45);
let refetched: serde_json::Value = reqwest::get(format!("{base}/api/settings/subtitles"))
.await
.expect("get settings")
.json()
.await
.expect("settings json");
assert_eq!(refetched, updated);
}
/// #224: dropping a language from `wanted_languages` must clear its
/// `subtitle_attempts` rows — otherwise re-adding it later resurrects a
/// stale backoff counter as though the attempts had just happened.
#[tokio::test]
async fn dropping_a_language_clears_its_attempt_rows() {
let (_dir, base, pool) = application_with_pool().await;
sqlx::query(
"INSERT INTO media_files (id, owner_kind, owner_id, path, size)
VALUES (1, 'movie', 1, 'x.mkv', 1)",
)
.execute(&pool)
.await
.expect("media file");
sqlx::query(
"INSERT INTO subtitle_attempts
(media_file_id, language, state, attempts, last_attempt_at, last_failure)
VALUES (1, 'en', 'failed', 3, '2024-01-01T00:00:00Z', 'no provider match')",
)
.execute(&pool)
.await
.expect("dropped-language attempt");
sqlx::query(
"INSERT INTO subtitle_attempts (media_file_id, language, state) VALUES (1, 'pt-PT', 'wanted')",
)
.execute(&pool)
.await
.expect("kept-language attempt");
let mut payload = valid_input();
payload["wanted_languages"] = serde_json::json!(["pt-PT"]);
reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings")
.error_for_status()
.expect("valid input accepted");
let remaining: Vec<String> =
sqlx::query_scalar("SELECT language FROM subtitle_attempts ORDER BY language")
.fetch_all(&pool)
.await
.expect("attempts");
assert_eq!(remaining, vec!["pt-PT".to_string()]);
}
/// Issue #219: the row alone never reaches the running backend, so a PUT
/// must push its value into the cell the command translator re-reads.
#[tokio::test]
async fn a_put_updates_the_live_command_timeout() {
let (_dir, base, timeout) = application_with_timeout().await;
assert_eq!(
timeout.load(std::sync::atomic::Ordering::Relaxed),
30_000,
"seeded from the row at startup"
);
reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&valid_input())
.send()
.await
.expect("put settings")
.error_for_status()
.expect("valid input accepted");
assert_eq!(
timeout.load(std::sync::atomic::Ordering::Relaxed),
45_000,
"the edited value reaches the running backend"
);
}
/// #220: the two OpenAI-compatible endpoint fields are ordinary settings
/// — they round-trip, and an omitted or empty one reads back as `null`,
/// which the backend takes as "use your own default".
#[tokio::test]
async fn the_openai_endpoint_round_trips_and_blanks_read_back_null() {
let (_dir, base) = application().await;
let client = reqwest::Client::new();
let seeded: serde_json::Value = reqwest::get(format!("{base}/api/settings/subtitles"))
.await
.expect("get settings")
.json()
.await
.expect("settings json");
assert!(seeded["openai_base_url"].is_null());
assert!(seeded["openai_model"].is_null());
let mut payload = valid_input();
payload["openai_base_url"] = serde_json::json!("http://127.0.0.1:8080/v1");
payload["openai_model"] = serde_json::json!("qwen2.5:7b");
let updated: serde_json::Value = client
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings")
.json()
.await
.expect("updated json");
assert_eq!(updated["openai_base_url"], "http://127.0.0.1:8080/v1");
assert_eq!(updated["openai_model"], "qwen2.5:7b");
// An emptied field means "back to the default", not an endpoint
// named "" — the settings form sends an empty input, not a null.
payload["openai_base_url"] = serde_json::json!("");
payload["openai_model"] = serde_json::json!(" ");
let cleared: serde_json::Value = client
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings")
.json()
.await
.expect("updated json");
assert!(cleared["openai_base_url"].is_null());
assert!(cleared["openai_model"].is_null());
}
/// #220: a base URL that does not parse is a 422 naming the field, the
/// same shape `translation_engine` already rejects with.
#[tokio::test]
async fn a_base_url_that_does_not_parse_is_a_422_naming_the_field() {
let (_dir, base) = application().await;
let mut payload = valid_input();
payload["openai_base_url"] = serde_json::json!("not a url");
let response = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings");
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
let body: serde_json::Value = response.json().await.expect("error json");
assert_eq!(
body["error"], "openai_base_url: bad base URL: relative URL without a base",
"the message names the field and the reason, not a reply that never came"
);
}
/// #220, the point of the issue: the row alone never reaches the running
/// backend, so a PUT must repoint the cell it re-reads per request —
/// which is also what the health lamp probes.
#[tokio::test]
async fn a_put_repoints_the_live_openai_endpoint() {
let (_dir, base, endpoint) = application_with_openai().await;
assert_eq!(endpoint.base_url(), arr_subs::OPENAI_DEFAULT_BASE_URL);
assert_eq!(endpoint.model(), arr_subs::OPENAI_DEFAULT_MODEL);
let mut payload = valid_input();
payload["openai_base_url"] = serde_json::json!("http://127.0.0.1:8080/v1");
payload["openai_model"] = serde_json::json!("qwen2.5:7b");
reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings")
.error_for_status()
.expect("valid input accepted");
assert_eq!(endpoint.base_url(), "http://127.0.0.1:8080/v1/");
assert_eq!(endpoint.model(), "qwen2.5:7b");
}
/// Whether a known engine is selectable depends on which `translate-*`
/// features this binary was built with, so the test asks the build rather
/// than assuming. With no feature on, every engine is uncompiled and must
/// be refused; with all of them on there is nothing to refuse, and the
/// complementary truth — a compiled engine is accepted — is what holds.
#[tokio::test]
async fn an_uncompiled_engine_is_a_422_naming_the_field() {
let compiled = arr_subs::compiled_engines();
let Some(uncompiled) = arr_subs::ENGINES
.iter()
.find(|engine| !compiled.contains(*engine))
else {
return a_compiled_engine_is_accepted().await;
};
let (_dir, base) = application().await;
let mut payload = valid_input();
payload["translation_engine"] = serde_json::json!(uncompiled);
let response = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings");
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
let body: serde_json::Value = response.json().await.expect("error body");
assert!(
body["error"]
.as_str()
.expect("error text")
.contains("translation_engine"),
"{body}"
);
}
/// The other side of the feature gate: an engine this binary *did*
/// compile in is selectable. Called directly when no engine is uncompiled.
async fn a_compiled_engine_is_accepted() {
let compiled = arr_subs::compiled_engines();
let Some(engine) = compiled.first() else {
return;
};
let (_dir, base) = application().await;
let mut payload = valid_input();
payload["translation_engine"] = serde_json::json!(engine);
let response = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings");
assert_eq!(response.status(), StatusCode::OK, "{engine} is compiled in");
}
#[tokio::test]
async fn an_unknown_engine_name_is_a_422() {
let (_dir, base) = application().await;
let mut payload = valid_input();
payload["translation_engine"] = serde_json::json!("bing-translate");
let response = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings");
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
}
#[tokio::test]
async fn every_field_validates_by_name() {
let (_dir, base) = application().await;
let with = |patch: &dyn Fn(&mut serde_json::Value)| {
let mut payload = valid_input();
patch(&mut payload);
payload
};
let cases: Vec<(serde_json::Value, &str)> = vec![
(
with(&|payload| payload["wanted_languages"] = serde_json::json!([])),
"wanted_languages",
),
(
with(&|payload| {
payload["wanted_languages"] = serde_json::json!(["pt-PT", "pt-PT"]);
}),
"wanted_languages",
),
(
with(&|payload| {
payload["providers_enabled"] = serde_json::json!(["opensubtitles", ""]);
}),
"providers_enabled",
),
(
with(&|payload| payload["remote_command_timeout_seconds"] = serde_json::json!(0)),
"remote_command_timeout_seconds",
),
];
for (payload, field) in cases {
let response = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings");
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
let body: serde_json::Value = response.json().await.expect("error body");
let error = body["error"].as_str().expect("error text");
assert!(error.contains(field), "{field}: {error}");
}
}
#[tokio::test]
async fn malformed_json_is_422_not_400_or_500() {
let (_dir, base) = application().await;
let response = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.header("content-type", "application/json")
.body("{not json")
.send()
.await
.expect("malformed put");
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
}
}
File diff suppressed because it is too large Load Diff
+29
View File
@@ -97,6 +97,21 @@ pub fn episode_file_name(
}
}
/// A subtitle sidecar's filename (§15): the video's name, the language, and
/// `srt` — `… [2160p][WEB-DL][HDR10].pt-PT.srt`. A machine translation
/// carries an extra `.mt` segment so `ls` says which subtitles arr made.
#[must_use]
pub fn subtitle_name(video_name: &str, language: &Language, machine_translated: bool) -> String {
let stem = video_name
.rsplit_once('.')
.map_or(video_name, |(stem, _)| stem);
if machine_translated {
format!("{stem}.{language}.mt.srt")
} else {
format!("{stem}.{language}.srt")
}
}
/// The §7.4 attribute tags, in a fixed order: resolution, source, HDR,
/// Portuguese audio.
///
@@ -215,6 +230,20 @@ mod tests {
);
}
/// The §15 sidecar names: plain for a real subtitle, `.mt` for arr's own.
#[test]
fn subtitle_sidecars_carry_the_language_and_the_mt_marker() {
let video = "Dune Part Two (2024) [tmdbid-693134] - [2160p][WEB-DL][HDR10].mkv";
assert_eq!(
subtitle_name(video, &Language::PortuguesePortugal, false),
"Dune Part Two (2024) [tmdbid-693134] - [2160p][WEB-DL][HDR10].pt-PT.srt"
);
assert_eq!(
subtitle_name(video, &Language::PortugueseBrazil, true),
"Dune Part Two (2024) [tmdbid-693134] - [2160p][WEB-DL][HDR10].pt-BR.mt.srt"
);
}
/// The kids audit surface: a pt-PT track is tagged, SDR is not.
#[test]
fn portuguese_audio_is_tagged_and_sdr_is_not() {
+53
View File
@@ -11,6 +11,7 @@ pub mod matching;
pub mod policy;
pub mod score;
pub mod status;
pub mod subs;
pub mod tracking;
pub use arr_parse::NameClaims as ParsedRelease;
@@ -24,6 +25,10 @@ pub use matching::{
};
pub use score::{Score, ScoreWeights};
pub use status::{derive_series_status, SeriesStatus};
pub use subs::{
rank as rank_subtitles, RankedSubtitle, SubtitleCandidate, SubtitleRule, SubtitleTarget,
SubtitleVerdict,
};
macro_rules! id_type {
($name:ident) => {
@@ -318,9 +323,57 @@ pub struct AudioTrack {
pub handler_name: Option<String>,
}
/// How an embedded subtitle track is encoded (DESIGN.md §15).
///
/// The split that matters is text versus bitmap: a text track extracts to a
/// sidecar SRT and can feed a translator, a bitmap one satisfies its language
/// for viewing and nothing more. There is no OCR.
#[derive(Clone, Copy, Debug, Eq, Hash, Ord, PartialEq, PartialOrd)]
pub enum SubtitleCodec {
/// `subrip`, the format sidecars are written in.
SubRip,
/// Advanced `SubStation` Alpha, and SSA with it.
Ass,
/// MP4's timed text.
MovText,
/// Presentation graphics — the bitmap track on `BluRay`.
Pgs,
/// `VobSub` — the bitmap track on DVD.
VobSub,
/// Anything else `ffprobe` names that is not one of the above.
Other,
}
impl fmt::Display for SubtitleCodec {
/// `ffprobe`'s codec name, so the probe column spells what the file said.
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str(match self {
Self::SubRip => "subrip",
Self::Ass => "ass",
Self::MovText => "mov_text",
Self::Pgs => "hdmv_pgs_subtitle",
Self::VobSub => "dvd_subtitle",
Self::Other => "other",
})
}
}
impl SubtitleCodec {
/// §15: only these extract to SRT and may become a translation source.
#[must_use]
pub const fn is_text(self) -> bool {
matches!(self, Self::SubRip | Self::Ass | Self::MovText)
}
}
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct SubtitleTrack {
pub language: Language,
pub codec: SubtitleCodec,
/// Foreign lines and on-screen signs only. Never satisfies a want (§15).
pub forced: bool,
/// Complete, with sound descriptions. Satisfies, ranked below plain (§15).
pub sdh: bool,
}
#[derive(Clone, Debug, Eq, PartialEq)]
+3
View File
@@ -1241,6 +1241,9 @@ mod tests {
let mut media = probed_audio(vec![track(Language::PortuguesePortugal)]);
media.subtitle_tracks = vec![SubtitleTrack {
language: Language::PortugueseBrazil,
codec: crate::SubtitleCodec::SubRip,
forced: false,
sdh: false,
}];
assert_eq!(
verdict_for(&kids_policy(), &en(), Candidate::PostDownload(&media)),
+345
View File
@@ -0,0 +1,345 @@
//! Ranking subtitle candidates against a media file (`DESIGN.md` §15).
//!
//! Pure and IO-free, alongside the release scoring in §5.5: the input is the
//! facts already known about the file plus a list of candidates as reported
//! by a provider, the output is those candidates ordered best first, each
//! carrying the verdict that put it there.
//!
//! The verdict vocabulary mirrors the release path — [`SubtitleVerdict`]
//! carries `Eligible` or `Rejected` a named [`SubtitleRule`], not a string
//! built for a log line — so §9.3's manual view needs no second concept for
//! subtitles. There is no `waived` bucket: nothing about a subtitle is worth
//! overriding by hand.
use crate::Source;
/// What is known about the file a subtitle candidate is ranked against.
#[derive(Clone, Copy, Debug, Default)]
pub struct SubtitleTarget<'a> {
/// The file's own `moviehash`, when computed.
pub moviehash: Option<&'a str>,
/// The exact name of the release that produced the file.
pub release_name: Option<&'a str>,
/// The release group claimed by that release name.
pub release_group: Option<&'a str>,
/// The source tier claimed by that release name.
pub source: Option<Source>,
}
/// One subtitle candidate, as reported by a provider.
#[derive(Clone, Copy, Debug, Default)]
pub struct SubtitleCandidate<'a> {
/// Covers foreign-language lines and on-screen signs only; never
/// satisfies a want (§15's "Forced and SDH").
pub forced: bool,
/// Complete and satisfies a want, but ranks below a plain subtitle for
/// the same language.
pub hearing_impaired: bool,
pub moviehash: Option<&'a str>,
pub release_name: Option<&'a str>,
pub release_group: Option<&'a str>,
pub source: Option<Source>,
pub uploader_rating: f64,
pub download_count: u64,
}
/// Why a subtitle candidate was rejected.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum SubtitleRule {
/// A forced track covers foreign-language lines only and cannot satisfy
/// a want (§15).
Forced,
}
impl SubtitleRule {
/// The stable name shared by the API and UI, so one rule reads the same
/// everywhere (mirrors [`crate::Rule::name`]).
#[must_use]
pub const fn name(&self) -> &'static str {
match self {
Self::Forced => "forced",
}
}
}
/// The verdict for one subtitle candidate.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum SubtitleVerdict {
Eligible,
Rejected(SubtitleRule),
}
/// One candidate paired with the verdict that placed it in the ranking.
#[derive(Clone, Copy, Debug)]
pub struct RankedSubtitle<'a> {
/// Position of this candidate in the slice handed to [`rank`]. Ranking
/// reorders, and a candidate carries no identity of its own, so this is
/// how a caller maps a result back to the provider candidate it came
/// from — and therefore to the id it must ask the provider to download.
pub index: usize,
pub candidate: SubtitleCandidate<'a>,
pub verdict: SubtitleVerdict,
}
/// The ordering key for an eligible candidate, most significant field first.
/// Every field is a boolean win or an integer tiebreaker, so `Ord` alone
/// gives the ranking — no float, no partial order.
type TierKey = (bool, bool, bool, bool, i64, u64);
fn tier_key(candidate: &SubtitleCandidate<'_>, target: &SubtitleTarget<'_>) -> TierKey {
let plain = !candidate.hearing_impaired;
let moviehash_match = matches_ci(candidate.moviehash, target.moviehash);
let exact_release_match = matches_ci(candidate.release_name, target.release_name);
let group_or_source_match = matches_ci(candidate.release_group, target.release_group)
|| matches_source(candidate.source, target.source);
// Scaled to hundredths so the tiebreaker is an integer, not a float.
// Ratings sit in 0..10, nowhere near i64's range.
#[allow(clippy::cast_possible_truncation)]
let rating = (candidate.uploader_rating * 100.0).round() as i64;
(
plain,
moviehash_match,
exact_release_match,
group_or_source_match,
rating,
candidate.download_count,
)
}
fn matches_ci(candidate: Option<&str>, target: Option<&str>) -> bool {
match (candidate, target) {
(Some(candidate), Some(target)) => candidate.eq_ignore_ascii_case(target),
_ => false,
}
}
fn matches_source(candidate: Option<Source>, target: Option<Source>) -> bool {
matches!((candidate, target), (Some(candidate), Some(target)) if candidate == target)
}
/// Rank candidates against a target, best first.
///
/// A forced candidate is rejected outright and sorts after every eligible
/// one, in the order it was given (§15: it can never satisfy a want). Among
/// eligible candidates, a plain subtitle always outranks a hearing-impaired
/// one for the same language; within that split, a `moviehash` match wins
/// outright, then an exact release-name match, then a shared release group
/// or source, then uploader rating and download count as tiebreakers.
#[must_use]
pub fn rank<'a>(
target: &SubtitleTarget<'_>,
candidates: &[SubtitleCandidate<'a>],
) -> Vec<RankedSubtitle<'a>> {
let mut ranked: Vec<RankedSubtitle<'a>> = candidates
.iter()
.enumerate()
.map(|(index, candidate)| RankedSubtitle {
index,
candidate: *candidate,
verdict: if candidate.forced {
SubtitleVerdict::Rejected(SubtitleRule::Forced)
} else {
SubtitleVerdict::Eligible
},
})
.collect();
ranked.sort_by(|a, b| match (a.verdict, b.verdict) {
(SubtitleVerdict::Rejected(_), SubtitleVerdict::Rejected(_)) => std::cmp::Ordering::Equal,
(SubtitleVerdict::Rejected(_), SubtitleVerdict::Eligible) => std::cmp::Ordering::Greater,
(SubtitleVerdict::Eligible, SubtitleVerdict::Rejected(_)) => std::cmp::Ordering::Less,
(SubtitleVerdict::Eligible, SubtitleVerdict::Eligible) => {
tier_key(&b.candidate, target).cmp(&tier_key(&a.candidate, target))
}
});
ranked
}
#[cfg(test)]
mod tests {
use super::*;
fn target() -> SubtitleTarget<'static> {
SubtitleTarget {
moviehash: Some("abc123"),
release_name: Some("Movie.2024.1080p.WEB-DL-GROUP"),
release_group: Some("GROUP"),
source: Some(Source::WebDl),
}
}
fn plain() -> SubtitleCandidate<'static> {
SubtitleCandidate {
uploader_rating: 5.0,
download_count: 100,
..Default::default()
}
}
#[test]
fn ranking_reports_where_each_candidate_came_from() {
// Ranking reorders, and a candidate carries no id of its own, so the
// index is the only way back to the provider candidate — and so to
// the id the provider is asked to download.
let weak = plain();
let strong = SubtitleCandidate {
moviehash: Some("abc123"),
..plain()
};
let ranked = rank(&target(), &[weak, strong]);
assert_eq!(ranked[0].index, 1);
assert_eq!(ranked[1].index, 0);
}
#[test]
fn a_rejected_candidate_still_reports_its_index() {
let forced = SubtitleCandidate {
forced: true,
..plain()
};
let ranked = rank(&target(), &[forced, plain()]);
assert_eq!(ranked[0].index, 1);
assert_eq!(ranked[1].index, 0);
assert_eq!(
ranked[1].verdict,
SubtitleVerdict::Rejected(SubtitleRule::Forced)
);
}
#[test]
fn a_moviehash_match_wins_outright_over_every_other_tier() {
let hash_match = SubtitleCandidate {
moviehash: Some("abc123"),
uploader_rating: 0.0,
download_count: 0,
..plain()
};
let everything_else = SubtitleCandidate {
release_name: Some("Movie.2024.1080p.WEB-DL-GROUP"),
release_group: Some("GROUP"),
source: Some(Source::WebDl),
uploader_rating: 10.0,
download_count: 1_000_000,
..plain()
};
let ranked = rank(&target(), &[everything_else, hash_match]);
assert!(matches!(ranked[0].candidate.moviehash, Some("abc123")));
assert_eq!(ranked[0].verdict, SubtitleVerdict::Eligible);
}
#[test]
fn an_exact_release_name_match_outranks_group_or_source_alone() {
let exact_name = SubtitleCandidate {
release_name: Some("Movie.2024.1080p.WEB-DL-GROUP"),
..plain()
};
let group_only = SubtitleCandidate {
release_group: Some("GROUP"),
uploader_rating: 10.0,
download_count: 1_000_000,
..plain()
};
let ranked = rank(&target(), &[group_only, exact_name]);
assert_eq!(
ranked[0].candidate.release_name,
Some("Movie.2024.1080p.WEB-DL-GROUP")
);
}
#[test]
fn same_source_alone_outranks_no_match_at_all() {
let same_source = SubtitleCandidate {
source: Some(Source::WebDl),
..plain()
};
let no_match = SubtitleCandidate {
uploader_rating: 10.0,
download_count: 1_000_000,
..plain()
};
let ranked = rank(&target(), &[no_match, same_source]);
assert_eq!(ranked[0].candidate.source, Some(Source::WebDl));
}
#[test]
#[allow(clippy::float_cmp)]
fn rating_then_downloads_break_ties() {
let low = SubtitleCandidate {
uploader_rating: 3.0,
download_count: 50,
..plain()
};
let high_rating = SubtitleCandidate {
uploader_rating: 8.0,
download_count: 10,
..plain()
};
let ranked = rank(&target(), &[low, high_rating]);
assert_eq!(ranked[0].candidate.uploader_rating, 8.0);
let same_rating_more_downloads = SubtitleCandidate {
uploader_rating: 8.0,
download_count: 9_000,
..plain()
};
let ranked = rank(&target(), &[high_rating, same_rating_more_downloads]);
assert_eq!(ranked[0].candidate.download_count, 9_000);
}
#[test]
fn a_forced_candidate_is_never_eligible() {
let forced = SubtitleCandidate {
forced: true,
moviehash: Some("abc123"),
..plain()
};
let ranked = rank(&target(), &[forced]);
assert_eq!(
ranked[0].verdict,
SubtitleVerdict::Rejected(SubtitleRule::Forced)
);
assert_eq!(SubtitleRule::Forced.name(), "forced");
}
#[test]
fn an_sdh_candidate_ranks_below_any_plain_candidate() {
let sdh_with_hash_match = SubtitleCandidate {
hearing_impaired: true,
moviehash: Some("abc123"),
uploader_rating: 10.0,
download_count: 1_000_000,
..plain()
};
let plain_with_nothing = SubtitleCandidate {
uploader_rating: 0.0,
download_count: 0,
..plain()
};
let ranked = rank(&target(), &[sdh_with_hash_match, plain_with_nothing]);
assert!(!ranked[0].candidate.hearing_impaired);
assert!(ranked[1].candidate.hearing_impaired);
}
#[test]
fn rejected_candidates_sort_after_every_eligible_one_in_input_order() {
let forced_a = SubtitleCandidate {
forced: true,
download_count: 1,
..plain()
};
let forced_b = SubtitleCandidate {
forced: true,
download_count: 2,
..plain()
};
let eligible = plain();
let ranked = rank(&target(), &[forced_a, eligible, forced_b]);
assert_eq!(ranked[0].verdict, SubtitleVerdict::Eligible);
assert_eq!(ranked[1].candidate.download_count, 1);
assert_eq!(ranked[2].candidate.download_count, 2);
}
}
+11
View File
@@ -10,6 +10,16 @@ publish = false
name = "arr"
path = "src/main.rs"
# Forwards straight to `arr-subs`' own features (DESIGN.md §15): which
# translators a build ships is a compile-time choice, off by default, same as
# the crate that implements them.
[features]
default = []
translate-openai = ["arr-subs/translate-openai"]
translate-deepl = ["arr-subs/translate-deepl"]
translate-google = ["arr-subs/translate-google"]
translate-command = ["arr-subs/translate-command"]
[dependencies]
arr-api = { workspace = true }
arr-compat = { workspace = true }
@@ -20,6 +30,7 @@ arr-indexer = { workspace = true }
arr-meta = { workspace = true }
arr-parse = { workspace = true }
arr-probe = { workspace = true }
arr-subs = { workspace = true }
axum = { workspace = true }
chrono = { workspace = true }
include_dir = { workspace = true }
+204 -13
View File
@@ -1,5 +1,5 @@
//! §9.5 *broken* → the operator alone: Prowlarr, Transmission or TMDB
//! unreachable.
//! unreachable, or a subtitle lamp failing (#200).
//!
//! Edge-triggered: notifies once when an upstream stops answering, and
//! silently re-arms once it answers again. There is no "fixed" notification —
@@ -29,52 +29,121 @@ pub struct Upstreams {
pub tmdb_api_key: Option<String>,
}
/// The subtitle upstreams (#200): the providers and engines this deployment
/// has credentials for, and the two binaries with their configured paths.
///
/// What is *in use* is read from the settings row per tick; this only holds
/// what could ever answer.
#[derive(Debug, Clone)]
pub struct SubtitleUpstreams {
pub providers: Vec<Arc<dyn arr_subs::Provider>>,
pub backends: Vec<Arc<dyn arr_subs::Backend>>,
pub alass_path: std::path::PathBuf,
pub ffmpeg_path: std::path::PathBuf,
}
impl SubtitleUpstreams {
/// The lamps as `(name, reachable)` pairs. Only what the settings row
/// has in use is probed — a provider nobody enabled cannot be broken.
async fn probe(&self, database: &Db) -> Vec<(String, bool)> {
let (providers_enabled, engine) = match crate::subtitles::load_settings(database).await {
Ok(settings) => (settings.providers_enabled, settings.translation_engine),
// An unreadable row says nothing about any upstream; skipping the
// whole lane beats notifying on our own database.
Err(error) => {
tracing::warn!(%error, "subtitle settings unreadable; subtitle lamps skipped");
return Vec::new();
}
};
let mut lamps = Vec::new();
for id in &providers_enabled {
let reachable = match self
.providers
.iter()
.find(|p| p.id().as_str() == id.as_str())
{
Some(provider) => provider.probe().await.is_ok(),
None => false,
};
lamps.push((id.clone(), reachable));
}
if let Some(engine) = engine {
let reachable = match self.backends.iter().find(|b| b.id().as_str() == engine) {
Some(backend) => backend.probe().await.is_ok(),
None => false,
};
lamps.push((engine, reachable));
}
lamps.push((
"alass".to_owned(),
arr_subs::binary_present(std::ffi::OsStr::new(&self.alass_path)),
));
lamps.push((
"ffmpeg".to_owned(),
arr_subs::binary_present(std::ffi::OsStr::new(&self.ffmpeg_path)),
));
lamps
}
}
#[derive(Debug)]
pub struct BrokenAction {
http: Client,
upstreams: Upstreams,
subtitles: SubtitleUpstreams,
notifier: Notifier,
operator_topic: String,
/// Which upstreams are currently notified as broken. Transient — a
/// restart re-probes and re-notifies whatever is still down.
broken: Arc<Mutex<HashSet<&'static str>>>,
broken: Arc<Mutex<HashSet<String>>>,
}
impl BrokenAction {
#[must_use]
pub fn new(upstreams: Upstreams, notifier: Notifier, operator_topic: String) -> Self {
pub fn new(
upstreams: Upstreams,
subtitles: SubtitleUpstreams,
notifier: Notifier,
operator_topic: String,
) -> Self {
Self {
http: Client::new(),
upstreams,
subtitles,
notifier,
operator_topic,
broken: Arc::new(Mutex::new(HashSet::new())),
}
}
async fn tick(&self) -> Vec<Outcome> {
let (prowlarr, transmission, tmdb) = tokio::join!(
async fn tick(&self, database: &Db) -> Vec<Outcome> {
let (prowlarr, transmission, tmdb, subtitles) = tokio::join!(
self.probe_prowlarr(),
self.probe_transmission(),
self.probe_tmdb(),
self.subtitles.probe(database),
);
let mut outcomes = Vec::new();
outcomes.extend(self.notify_transition("prowlarr", prowlarr).await);
outcomes.extend(self.notify_transition("transmission", transmission).await);
outcomes.extend(self.notify_transition("tmdb", tmdb).await);
for (name, reachable) in subtitles {
outcomes.extend(self.notify_transition(&name, reachable).await);
}
outcomes
}
/// `reachable` is `true` when the upstream answered, or when it needs no
/// key and none is configured (not an outage — see `probe_tmdb`).
async fn notify_transition(&self, name: &'static str, reachable: bool) -> Option<Outcome> {
async fn notify_transition(&self, name: &str, reachable: bool) -> Option<Outcome> {
let mut broken = self.broken.lock().await;
if reachable {
broken.remove(name);
return None;
}
if !broken.insert(name) {
if !broken.insert(name.to_owned()) {
return None;
}
match self
@@ -153,8 +222,8 @@ impl Action for BrokenAction {
"broken"
}
fn run<'a>(&'a self, _database: &'a Db) -> ActionFuture<'a> {
Box::pin(async move { Ok(self.tick().await) })
fn run<'a>(&'a self, database: &'a Db) -> ActionFuture<'a> {
Box::pin(async move { Ok(self.tick(database).await) })
}
}
@@ -176,6 +245,32 @@ mod tests {
}
}
/// A migrated database whose settings enable nothing — the classic
/// upstreams under test here must not share the tick with subtitle
/// lamps the seed row would otherwise put in use.
async fn database() -> (tempfile::TempDir, Db) {
let dir = tempfile::tempdir().unwrap();
let db = Db::connect(dir.path().join("broken-test.db"))
.await
.unwrap();
db.migrate().await.unwrap();
sqlx::query("UPDATE subtitle_settings SET providers_enabled = '[]'")
.execute(db.pool())
.await
.unwrap();
(dir, db)
}
fn subtitles() -> SubtitleUpstreams {
SubtitleUpstreams {
providers: Vec::new(),
backends: Vec::new(),
// Present on every machine that runs these tests.
alass_path: "sh".into(),
ffmpeg_path: "sh".into(),
}
}
#[tokio::test]
async fn an_unreachable_upstream_notifies_once_then_re_arms() {
let prowlarr = MockServer::start().await;
@@ -192,14 +287,16 @@ mod tests {
.mount(&ntfy)
.await;
let (_dir, db) = database().await;
let action = BrokenAction::new(
upstreams(prowlarr.uri(), transmission.uri()),
subtitles(),
Notifier::new(ntfy.uri()).unwrap(),
"operator-topic".to_string(),
);
let first = action.tick().await;
let second = action.tick().await;
let first = action.tick(&db).await;
let second = action.tick(&db).await;
assert_eq!(first.len(), 1, "notifies on the tick it goes unreachable");
assert_eq!(second.len(), 0, "does not repeat while still broken");
@@ -209,12 +306,106 @@ mod tests {
.respond_with(ResponseTemplate::new(200))
.mount(&prowlarr)
.await;
let recovered = action.tick().await;
let recovered = action.tick(&db).await;
assert_eq!(recovered.len(), 0, "recovery is silent, no fourth event");
// Take it down again: a fresh outage re-arms and notifies again.
prowlarr.reset().await;
let broken_again = action.tick().await;
let broken_again = action.tick(&db).await;
assert_eq!(broken_again.len(), 1, "re-arms after recovering");
}
/// #200: an enabled provider nobody configured is a broken lamp like any
/// other, and it notifies once — then stays quiet while it stays broken.
#[tokio::test]
async fn an_enabled_but_missing_subtitle_provider_notifies_once() {
let ntfy = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200))
.mount(&ntfy)
.await;
let dir = tempfile::tempdir().unwrap();
let db = Db::connect(dir.path().join("broken-subs.db"))
.await
.unwrap();
db.migrate().await.unwrap();
// The seed row already enables opensubtitles and podnapisi; none are
// attached to this action.
let prowlarr = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/ping"))
.respond_with(ResponseTemplate::new(200))
.mount(&prowlarr)
.await;
let transmission = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(409))
.mount(&transmission)
.await;
let action = BrokenAction::new(
upstreams(prowlarr.uri(), transmission.uri()),
subtitles(),
Notifier::new(ntfy.uri()).unwrap(),
"operator-topic".to_string(),
);
let first = action.tick(&db).await;
let second = action.tick(&db).await;
assert_eq!(first.len(), 2, "one lamp per missing provider");
assert_eq!(second.len(), 0, "does not repeat while still broken");
// Attaching nothing but disabling them silences the lamps.
sqlx::query("UPDATE subtitle_settings SET providers_enabled = '[]'")
.execute(db.pool())
.await
.unwrap();
let third = action.tick(&db).await;
assert_eq!(third.len(), 0, "a disabled provider cannot be broken");
}
/// #200: binaries are judged at their configured paths.
#[tokio::test]
async fn a_missing_binary_is_a_broken_lamp() {
let ntfy = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200))
.mount(&ntfy)
.await;
let (_dir, db) = database().await;
let prowlarr = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/ping"))
.respond_with(ResponseTemplate::new(200))
.mount(&prowlarr)
.await;
let transmission = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(409))
.mount(&transmission)
.await;
let subs = SubtitleUpstreams {
alass_path: "/nowhere/alass".into(),
..subtitles()
};
let action = BrokenAction::new(
upstreams(prowlarr.uri(), transmission.uri()),
subs,
Notifier::new(ntfy.uri()).unwrap(),
"operator-topic".to_string(),
);
let first = action.tick(&db).await;
assert_eq!(first.len(), 1, "only the missing binary notifies");
assert!(
first
.iter()
.any(|outcome| format!("{outcome:?}").contains("alass")),
"{first:?}"
);
}
}
+287 -1
View File
@@ -30,6 +30,32 @@ pub const ENV_JELLYFIN_URL: &str = "ARR_JELLYFIN_URL";
pub const ENV_JELLYFIN_API_KEY: &str = "ARR_JELLYFIN_API_KEY";
pub const ENV_NTFY_URL: &str = "ARR_NTFY_URL";
pub const ENV_NTFY_OPERATOR_TOPIC: &str = "ARR_NTFY_OPERATOR_TOPIC";
// DESIGN.md §15 subtitle bootstrap. Credentials are secrets (env-only,
// below); base URLs, the remote-command template and the binary paths may
// also sit in the config file.
pub const ENV_OPENSUBTITLES_API_KEY: &str = "ARR_OPENSUBTITLES_API_KEY";
pub const ENV_OPENSUBTITLES_USERNAME: &str = "ARR_OPENSUBTITLES_USERNAME";
pub const ENV_OPENSUBTITLES_PASSWORD: &str = "ARR_OPENSUBTITLES_PASSWORD";
// Podnapisi takes no credentials (#215): its search and download are
// unauthenticated, so there is nothing here for an operator to set.
// The OpenAI-compatible backend's base URL and model are `subtitle_settings`
// rows, not bootstrap keys (#220): that backend is anything speaking the
// shape, and which endpoint is in use is something the operator changes from
// `/settings`. Only the key is here, because §10 keeps secrets out of the
// database — and an endpoint needing no key at all is valid.
pub const ENV_TRANSLATE_OPENAI_API_KEY: &str = "ARR_TRANSLATE_OPENAI_API_KEY";
pub const ENV_TRANSLATE_DEEPL_API_KEY: &str = "ARR_TRANSLATE_DEEPL_API_KEY";
pub const ENV_TRANSLATE_DEEPL_BASE_URL: &str = "ARR_TRANSLATE_DEEPL_BASE_URL";
pub const ENV_TRANSLATE_GOOGLE_API_KEY: &str = "ARR_TRANSLATE_GOOGLE_API_KEY";
pub const ENV_TRANSLATE_GOOGLE_BASE_URL: &str = "ARR_TRANSLATE_GOOGLE_BASE_URL";
pub const ENV_TRANSLATE_COMMAND_TEMPLATE: &str = "ARR_TRANSLATE_COMMAND_TEMPLATE";
pub const ENV_ALASS_PATH: &str = "ARR_ALASS_PATH";
pub const ENV_FFMPEG_PATH: &str = "ARR_FFMPEG_PATH";
/// E2E seam only, env-only, same shape as [`ENV_TMDB_URL`]: `None` means
/// Podnapisi's real address. Kept out of the config file for the same
/// reason `tmdb_url` is — DESIGN.md §10 has no business exposing a seam that
/// only a test harness uses.
pub const ENV_PODNAPISI_URL: &str = "ARR_PODNAPISI_URL";
pub const DEFAULT_BIND_ADDR: &str = "0.0.0.0:7878";
pub const DEFAULT_DATABASE_PATH: &str = "arr.db";
@@ -45,7 +71,8 @@ pub const DEFAULT_SEED_RATIO_LIMIT: f64 = 1.0;
pub const DEFAULT_SEED_IDLE_LIMIT_MINUTES: u64 = 4320;
pub const DEFAULT_JELLYFIN_URL: &str = "http://localhost:8096";
pub const DEFAULT_NTFY_URL: &str = "http://localhost";
pub const DEFAULT_ALASS_PATH: &str = "alass";
pub const DEFAULT_FFMPEG_PATH: &str = "ffmpeg";
#[derive(Debug, thiserror::Error)]
pub enum ConfigError {
#[error("io: {0}")]
@@ -92,6 +119,16 @@ struct ConfigFile {
ntfy_url: Option<String>,
#[serde(default)]
ntfy_operator_topic: Option<String>,
#[serde(default)]
translate_deepl_base_url: Option<String>,
#[serde(default)]
translate_google_base_url: Option<String>,
#[serde(default)]
translate_command_template: Option<String>,
#[serde(default)]
alass_path: Option<PathBuf>,
#[serde(default)]
ffmpeg_path: Option<PathBuf>,
}
#[derive(Debug, Clone, Copy, PartialEq, Deserialize)]
@@ -129,8 +166,20 @@ pub struct EnvOverrides {
pub tmdb_url: Option<String>,
pub jellyfin_url: Option<String>,
pub jellyfin_api_key: Option<String>,
pub podnapisi_url: Option<String>,
pub ntfy_url: Option<String>,
pub ntfy_operator_topic: Option<String>,
pub opensubtitles_api_key: Option<String>,
pub opensubtitles_username: Option<String>,
pub opensubtitles_password: Option<String>,
pub translate_openai_api_key: Option<String>,
pub translate_deepl_api_key: Option<String>,
pub translate_deepl_base_url: Option<String>,
pub translate_google_api_key: Option<String>,
pub translate_google_base_url: Option<String>,
pub translate_command_template: Option<String>,
pub alass_path: Option<String>,
pub ffmpeg_path: Option<String>,
}
impl EnvOverrides {
@@ -150,8 +199,20 @@ impl EnvOverrides {
tmdb_url: std::env::var(ENV_TMDB_URL).ok(),
jellyfin_url: std::env::var(ENV_JELLYFIN_URL).ok(),
jellyfin_api_key: std::env::var(ENV_JELLYFIN_API_KEY).ok(),
podnapisi_url: std::env::var(ENV_PODNAPISI_URL).ok(),
ntfy_url: std::env::var(ENV_NTFY_URL).ok(),
ntfy_operator_topic: std::env::var(ENV_NTFY_OPERATOR_TOPIC).ok(),
opensubtitles_api_key: std::env::var(ENV_OPENSUBTITLES_API_KEY).ok(),
opensubtitles_username: std::env::var(ENV_OPENSUBTITLES_USERNAME).ok(),
opensubtitles_password: std::env::var(ENV_OPENSUBTITLES_PASSWORD).ok(),
translate_openai_api_key: std::env::var(ENV_TRANSLATE_OPENAI_API_KEY).ok(),
translate_deepl_api_key: std::env::var(ENV_TRANSLATE_DEEPL_API_KEY).ok(),
translate_deepl_base_url: std::env::var(ENV_TRANSLATE_DEEPL_BASE_URL).ok(),
translate_google_api_key: std::env::var(ENV_TRANSLATE_GOOGLE_API_KEY).ok(),
translate_google_base_url: std::env::var(ENV_TRANSLATE_GOOGLE_BASE_URL).ok(),
translate_command_template: std::env::var(ENV_TRANSLATE_COMMAND_TEMPLATE).ok(),
alass_path: std::env::var(ENV_ALASS_PATH).ok(),
ffmpeg_path: std::env::var(ENV_FFMPEG_PATH).ok(),
}
}
}
@@ -177,11 +238,92 @@ pub struct Config {
pub tmdb_url: Option<String>,
pub jellyfin_url: String,
pub jellyfin_api_key: Option<String>,
/// E2E seam only, env-only. `None` means Podnapisi's built-in default
/// address; see [`ENV_PODNAPISI_URL`].
pub podnapisi_url: Option<String>,
pub ntfy_url: String,
/// The operator's ntfy topic (DESIGN.md §9.5) for *needs a decision* and
/// *broken*. `None` means those two notifications are skipped — there is
/// no sensible default topic name to fall back to.
pub ntfy_operator_topic: Option<String>,
/// §15. `None` means OpenSubtitles.com search runs unauthenticated,
/// which its API allows at a lower rate.
pub opensubtitles_api_key: Option<String>,
pub opensubtitles_username: Option<String>,
pub opensubtitles_password: Option<String>,
/// §15. `None` is valid: `llama.cpp` and a local gateway serve without
/// authentication. Where that backend points and which model it names
/// are `subtitle_settings` rows, read at start-up and re-read on every
/// edit (#220), not bootstrap config.
pub translate_openai_api_key: Option<String>,
pub translate_deepl_api_key: Option<String>,
/// `None` means the backend's own built-in default when it lands (#192).
pub translate_deepl_base_url: Option<String>,
pub translate_google_api_key: Option<String>,
/// `None` means the backend's own built-in default when it lands (#193).
pub translate_google_base_url: Option<String>,
/// §15. The generic remote-command backend's invocation template, e.g.
/// `ssh box claude -p`. `None` means that backend is unconfigured.
pub translate_command_template: Option<String>,
/// §15. `alass` runs on every fetched and every translated subtitle; a
/// bare name resolves through `PATH`, matching `arr-probe`'s `ffprobe`.
pub alass_path: PathBuf,
/// §15. Extracts text-format embedded tracks to sidecar SRTs.
pub ffmpeg_path: PathBuf,
}
/// §15/§10 subtitle bootstrap: provider and translator credentials, base
/// URLs, the remote-command template, and binary paths. Resolved separately
/// from [`Config::resolve`] for the same reason `arr-daemon::api_state` is
/// split out of `run` — one field per provider or backend, and `resolve` is
/// already at the too-many-lines limit.
struct SubtitleBootstrap {
opensubtitles_api_key: Option<String>,
opensubtitles_username: Option<String>,
opensubtitles_password: Option<String>,
translate_openai_api_key: Option<String>,
translate_deepl_api_key: Option<String>,
translate_deepl_base_url: Option<String>,
translate_google_api_key: Option<String>,
translate_google_base_url: Option<String>,
translate_command_template: Option<String>,
alass_path: PathBuf,
ffmpeg_path: PathBuf,
}
fn resolve_subtitle_bootstrap(env: &EnvOverrides, file: &ConfigFile) -> SubtitleBootstrap {
SubtitleBootstrap {
opensubtitles_api_key: env.opensubtitles_api_key.clone(),
opensubtitles_username: env.opensubtitles_username.clone(),
opensubtitles_password: env.opensubtitles_password.clone(),
translate_openai_api_key: env.translate_openai_api_key.clone(),
translate_deepl_api_key: env.translate_deepl_api_key.clone(),
translate_deepl_base_url: env
.translate_deepl_base_url
.clone()
.or_else(|| file.translate_deepl_base_url.clone()),
translate_google_api_key: env.translate_google_api_key.clone(),
translate_google_base_url: env
.translate_google_base_url
.clone()
.or_else(|| file.translate_google_base_url.clone()),
translate_command_template: env
.translate_command_template
.clone()
.or_else(|| file.translate_command_template.clone()),
alass_path: env
.alass_path
.clone()
.map(PathBuf::from)
.or_else(|| file.alass_path.clone())
.unwrap_or_else(|| PathBuf::from(DEFAULT_ALASS_PATH)),
ffmpeg_path: env
.ffmpeg_path
.clone()
.map(PathBuf::from)
.or_else(|| file.ffmpeg_path.clone())
.unwrap_or_else(|| PathBuf::from(DEFAULT_FFMPEG_PATH)),
}
}
impl Config {
@@ -197,6 +339,7 @@ impl Config {
Some(path) => ConfigFile::load(Path::new(path))?,
None => ConfigFile::default(),
};
let subtitles = resolve_subtitle_bootstrap(&env, &file);
let bind_addr = match &env.bind_addr {
Some(raw) => parse_bind_addr(raw, ENV_BIND_ADDR)?,
@@ -258,11 +401,23 @@ impl Config {
.or(file.jellyfin_url)
.unwrap_or_else(|| DEFAULT_JELLYFIN_URL.to_string()),
jellyfin_api_key: env.jellyfin_api_key,
podnapisi_url: env.podnapisi_url,
ntfy_url: env
.ntfy_url
.or(file.ntfy_url)
.unwrap_or_else(|| DEFAULT_NTFY_URL.to_string()),
ntfy_operator_topic: env.ntfy_operator_topic.or(file.ntfy_operator_topic),
opensubtitles_api_key: subtitles.opensubtitles_api_key,
opensubtitles_username: subtitles.opensubtitles_username,
opensubtitles_password: subtitles.opensubtitles_password,
translate_openai_api_key: subtitles.translate_openai_api_key,
translate_deepl_api_key: subtitles.translate_deepl_api_key,
translate_deepl_base_url: subtitles.translate_deepl_base_url,
translate_google_api_key: subtitles.translate_google_api_key,
translate_google_base_url: subtitles.translate_google_base_url,
translate_command_template: subtitles.translate_command_template,
alass_path: subtitles.alass_path,
ffmpeg_path: subtitles.ffmpeg_path,
})
}
}
@@ -302,12 +457,17 @@ mod tests {
DEFAULT_SEED_IDLE_LIMIT_MINUTES
);
assert_eq!(config.jellyfin_url, DEFAULT_JELLYFIN_URL);
assert_eq!(config.podnapisi_url, None);
assert!(config.tracker_seeding.is_empty());
assert_eq!(config.ntfy_url, DEFAULT_NTFY_URL);
assert_eq!(config.ntfy_operator_topic, None);
assert_eq!(config.prowlarr_api_key, None);
assert_eq!(config.tmdb_api_key, None);
assert_eq!(config.jellyfin_api_key, None);
assert_eq!(config.opensubtitles_api_key, None);
assert_eq!(config.translate_command_template, None);
assert_eq!(config.alass_path, PathBuf::from(DEFAULT_ALASS_PATH));
assert_eq!(config.ffmpeg_path, PathBuf::from(DEFAULT_FFMPEG_PATH));
}
#[test]
@@ -376,6 +536,101 @@ prowlarr_url = "http://prowlarr.internal:9696"
assert_eq!(config.jellyfin_api_key.as_deref(), Some("secret-3"));
}
/// §15: `OpenSubtitles` and every translator's credentials are secrets,
/// so this mirrors [`secrets_come_only_from_env`] for them. Podnapisi has
/// no entry: it takes no credentials at all (#215).
#[test]
fn subtitle_secrets_come_only_from_env() {
let env = EnvOverrides {
opensubtitles_api_key: Some("os-key".into()),
opensubtitles_username: Some("os-user".into()),
opensubtitles_password: Some("os-pass".into()),
translate_openai_api_key: Some("oa-key".into()),
translate_deepl_api_key: Some("dl-key".into()),
translate_google_api_key: Some("gg-key".into()),
..EnvOverrides::default()
};
let config = Config::resolve(env).unwrap();
assert_eq!(config.opensubtitles_api_key.as_deref(), Some("os-key"));
assert_eq!(config.opensubtitles_username.as_deref(), Some("os-user"));
assert_eq!(config.opensubtitles_password.as_deref(), Some("os-pass"));
assert_eq!(config.translate_openai_api_key.as_deref(), Some("oa-key"));
assert_eq!(config.translate_deepl_api_key.as_deref(), Some("dl-key"));
assert_eq!(config.translate_google_api_key.as_deref(), Some("gg-key"));
}
#[test]
fn a_subtitle_secret_in_the_config_file_is_a_parse_error() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("arr.toml");
std::fs::write(&path, "opensubtitles_api_key = \"leaked\"\n").unwrap();
let env = EnvOverrides {
config_file: Some(path.to_string_lossy().into_owned()),
..EnvOverrides::default()
};
assert!(matches!(
Config::resolve(env),
Err(ConfigError::TomlDecode(_))
));
}
/// §15: base URLs, the remote-command template and the binary paths are
/// not secrets, so the file, the environment and their precedence all
/// apply the same way they do for `jellyfin_url` and friends.
#[test]
fn subtitle_non_secret_bootstrap_comes_from_file_or_env() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("arr.toml");
std::fs::write(
&path,
r#"
translate_command_template = "ssh box claude -p"
alass_path = "/usr/local/bin/alass"
"#,
)
.unwrap();
let env = EnvOverrides {
config_file: Some(path.to_string_lossy().into_owned()),
..EnvOverrides::default()
};
let config = Config::resolve(env.clone()).unwrap();
assert_eq!(
config.translate_command_template.as_deref(),
Some("ssh box claude -p")
);
assert_eq!(config.alass_path, PathBuf::from("/usr/local/bin/alass"));
// ffmpeg_path was not set anywhere, so it still defaults.
assert_eq!(config.ffmpeg_path, PathBuf::from(DEFAULT_FFMPEG_PATH));
let config = Config::resolve(EnvOverrides {
ffmpeg_path: Some("/opt/bin/ffmpeg".into()),
..env
})
.unwrap();
assert_eq!(config.ffmpeg_path, PathBuf::from("/opt/bin/ffmpeg"));
}
/// #220 retired `translate_openai_base_url` and `translate_openai_model`
/// — both are `subtitle_settings` rows now. `deny_unknown_fields` turns
/// a config file still carrying them into a parse error, so an operator
/// who upgrades sees the move rather than a setting silently ignored.
#[test]
fn the_retired_openai_endpoint_keys_are_a_parse_error() {
for key in ["translate_openai_base_url", "translate_openai_model"] {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("arr.toml");
std::fs::write(&path, format!("{key} = \"x\"\n")).unwrap();
let env = EnvOverrides {
config_file: Some(path.to_string_lossy().into_owned()),
..EnvOverrides::default()
};
assert!(
matches!(Config::resolve(env), Err(ConfigError::TomlDecode(_))),
"{key} must no longer be accepted"
);
}
}
#[test]
fn tmdb_url_is_an_env_only_seam() {
let config = Config::resolve(EnvOverrides::default()).unwrap();
@@ -401,6 +656,37 @@ prowlarr_url = "http://prowlarr.internal:9696"
));
}
/// Same env-only seam as `tmdb_url`, for the same reason: `arr-e2e` needs
/// to point Podnapisi at a `wiremock` fake without a live-tracker risk
/// creeping into the config file (DESIGN.md §15, §10).
#[test]
fn podnapisi_url_is_an_env_only_seam() {
let config = Config::resolve(EnvOverrides::default()).unwrap();
assert_eq!(config.podnapisi_url, None);
let env = EnvOverrides {
podnapisi_url: Some("http://127.0.0.1:9/subtitles".into()),
..EnvOverrides::default()
};
let config = Config::resolve(env).unwrap();
assert_eq!(
config.podnapisi_url.as_deref(),
Some("http://127.0.0.1:9/subtitles")
);
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("arr.toml");
std::fs::write(&path, "podnapisi_url = \"http://127.0.0.1:9/subtitles\"\n").unwrap();
let env = EnvOverrides {
config_file: Some(path.to_string_lossy().into_owned()),
..EnvOverrides::default()
};
assert!(matches!(
Config::resolve(env),
Err(ConfigError::TomlDecode(_))
));
}
#[test]
fn a_secret_in_the_config_file_is_a_parse_error() {
let dir = tempfile::tempdir().unwrap();
+24 -1
View File
@@ -1534,10 +1534,17 @@ fn search_query(movie: &PendingMovie) -> String {
)
}
/// The `verdict` and `rejected_rule` columns for a verdict.
///
/// A waiver names the rule it relaxed (#211). §5.7 calls a soft fail
/// "watchable but not what was asked", and which rule was relaxed is the
/// whole content of that sentence, so §9.3's deck can name it the way it
/// names a rejection. Rows written before 0032 hold `NULL` there and stay
/// readable.
fn verdict_columns(verdict: &Verdict) -> (&'static str, Option<String>) {
match verdict {
Verdict::Eligible => ("eligible", None),
Verdict::Waived(_) => ("waived", None),
Verdict::Waived(rule) => ("waived", Some(rule.name())),
Verdict::Rejected(rule) => ("rejected", Some(rule.name())),
}
}
@@ -2029,6 +2036,22 @@ mod tests {
);
}
/// #211: a waiver names the rule it relaxed, the same as a rejection, so
/// §9.3's deck shows what was given up instead of a bare `waived`.
/// Migration 0032 relaxed the constraint that forbade it.
#[test]
fn a_waiver_records_the_rule_it_relaxed() {
assert_eq!(
verdict_columns(&Verdict::Waived(arr_core::Rule::Size)),
("waived", Some("size".to_owned()))
);
assert_eq!(verdict_columns(&Verdict::Eligible), ("eligible", None));
assert_eq!(
verdict_columns(&Verdict::Rejected(arr_core::Rule::RequiredAudio)),
("rejected", Some("required_audio".to_owned()))
);
}
/// Every candidate is cached with its verdict, which is what the manual
/// search view and the attention queues read (§9.3).
#[tokio::test]
+8 -1
View File
@@ -1219,7 +1219,14 @@ fn probed_json(media: &ProbedMedia) -> serde_json::Value {
"sub_tracks": media
.subtitle_tracks
.iter()
.map(|track| serde_json::json!({ "language": track.language.to_string() }))
.map(|track| {
serde_json::json!({
"language": track.language.to_string(),
"codec": track.codec.to_string(),
"forced": track.forced,
"sdh": track.sdh,
})
})
.collect::<Vec<_>>(),
})
}
+338 -25
View File
@@ -13,11 +13,12 @@ mod reaper;
pub mod reconcile;
mod rss;
mod series_refresh;
mod subtitles;
mod tv_grab;
mod web;
use std::process::ExitCode;
use std::sync::Arc;
use std::sync::{atomic::AtomicU64, Arc};
use arr_api::{AppState, Upstreams};
use arr_compat::CompatState;
@@ -33,6 +34,7 @@ use reaper::ReaperAction;
use reconcile::{ReconcileLoop, Tick};
use rss::RssAction;
use series_refresh::SeriesRefreshAction;
use subtitles::SubtitleAction;
use tower_http::trace::TraceLayer;
use tv_grab::TvGrabAction;
@@ -107,6 +109,44 @@ enum Error {
BackgroundTask(#[from] tokio::task::JoinError),
}
/// Everything the HTTP layer needs, assembled from config.
///
/// Split out of [`run`] because it grows a line per upstream the API learns
/// to talk to, and `run` is already at the too-many-lines limit.
fn api_state(
config: &Config,
database: &Db,
jellyfin: arr_api::jellyfin::JellyfinClient,
translators: &Translators,
) -> Result<AppState, Error> {
let mut upstreams =
Upstreams::new(config.prowlarr_url.clone(), config.transmission_url.clone())
.with_prowlarr_api_key(config.prowlarr_api_key.clone())
.with_tmdb_api_key(config.tmdb_api_key.clone());
if let Some(tmdb_url) = config.tmdb_url.clone() {
upstreams = upstreams.with_tmdb_url(tmdb_url);
}
let mut state = AppState::new(upstreams)?
.with_database(database.clone())
.with_subtitle_providers(subtitle_providers(
config.opensubtitles_api_key.clone(),
config.opensubtitles_username.clone(),
config.opensubtitles_password.clone(),
config.podnapisi_url.clone(),
))
.with_translation_backends(translators.backends.clone())
.with_jellyfin(jellyfin)
.with_syncer(arr_subs::Syncer::new().with_binary(config.alass_path.clone()));
if let Some(timeout) = &translators.command_timeout {
state = state.with_command_timeout(Arc::clone(timeout));
}
if let Some(endpoint) = &translators.openai_endpoint {
state = state.with_openai_endpoint(endpoint.clone());
}
Ok(state)
}
async fn run() -> Result<(), Error> {
let config = Config::load()?;
let database = Db::connect(&config.database_path).await?;
@@ -124,8 +164,19 @@ async fn run() -> Result<(), Error> {
};
let notifier = Notifier::new(config.ntfy_url.clone())?;
let api_jellyfin = jellyfin_client(&config)?;
let (reconcile, manual_grab, manual_tv) =
reconcile_loop(&database, &config, &transmission, tmdb.as_ref(), &notifier)?;
// Built once and shared: the API's translate handler and the reconcile
// lane must see the same backends, or the command translator's live
// timeout cell (#219) would fork.
let translators = translation_backends(&config);
seed_translator_settings(&database, &translators).await?;
let (reconcile, manual_grab, manual_tv) = reconcile_loop(
&database,
&config,
&transmission,
tmdb.as_ref(),
&notifier,
&translators,
)?;
// Issue #176: the on-demand half of the metadata lane needs its own
// handle — the sweep's `SeriesRefreshAction` is owned by `ReconcileLoop`,
// and the compat shim takes the other clone below.
@@ -138,15 +189,7 @@ async fn run() -> Result<(), Error> {
compat = compat.with_tmdb(tmdb);
}
let mut upstreams = Upstreams::new(config.prowlarr_url, config.transmission_url)
.with_prowlarr_api_key(config.prowlarr_api_key)
.with_tmdb_api_key(config.tmdb_api_key);
if let Some(tmdb_url) = config.tmdb_url {
upstreams = upstreams.with_tmdb_url(tmdb_url);
}
let state = AppState::new(upstreams)?
.with_database(database.clone())
.with_jellyfin(api_jellyfin);
let state = api_state(&config, &database, api_jellyfin, &translators)?;
let app = arr_api::router(state.clone())
.merge(arr_compat::router(compat))
@@ -224,6 +267,48 @@ async fn run() -> Result<(), Error> {
}
}
/// Seed the translators' live settings from the row, so a restart does not
/// fall back to compiled-in defaults until the next settings edit (issues
/// #219 and #220): the command translator's timeout, and where the
/// OpenAI-compatible backend points plus which model it names.
///
/// The row is guaranteed to exist — migration `0025` seeds it — and the
/// timeout is guaranteed positive by its column CHECK. The two `OpenAI`
/// columns are nullable, and NULL means the backend's own default.
async fn seed_translator_settings(database: &Db, translators: &Translators) -> Result<(), Error> {
if translators.command_timeout.is_none() && translators.openai_endpoint.is_none() {
return Ok(());
}
let row = sqlx::query!(
r#"SELECT remote_command_timeout_seconds AS "remote_command_timeout_seconds!: i64",
openai_base_url AS "openai_base_url: String",
openai_model AS "openai_model: String"
FROM subtitle_settings WHERE id = 1"#
)
.fetch_one(database.pool())
.await?;
if let Some(cell) = &translators.command_timeout {
cell.store(
u64::try_from(row.remote_command_timeout_seconds)
.unwrap_or(u64::MAX)
.saturating_mul(1_000),
std::sync::atomic::Ordering::Relaxed,
);
}
if let Some(endpoint) = &translators.openai_endpoint {
// A row that does not parse must not stop the daemon booting: the
// API validates on write, so this only fires for a hand-edited
// database. The backend stays at its default and the health lamp
// says so.
if let Err(error) =
endpoint.set(row.openai_base_url.as_deref(), row.openai_model.as_deref())
{
tracing::warn!(%error, "stored OpenAI endpoint is unusable; keeping the default");
}
}
Ok(())
}
/// Wire the reconcile lanes (DESIGN.md §8). Grab and RSS both need a
/// Prowlarr key and grab needs TMDB as well; a lane whose upstream is not
/// configured stays unregistered rather than failing every tick.
@@ -239,6 +324,7 @@ fn reconcile_loop(
transmission: &arr_dl::TransmissionClient,
tmdb: Option<&Arc<TmdbClient>>,
notifier: &Notifier,
translators: &Translators,
) -> Result<(ReconcileLoop, Option<GrabAction>, Option<TvGrabAction>), Error> {
let reconcile = ReconcileLoop::new(database.clone());
let seeding = SeedingRules::new(
@@ -310,6 +396,12 @@ fn reconcile_loop(
),
);
// §15: subtitle gaps are reconciled from the same rows the API writes.
reconcile = reconcile.register(
Tick::Reconcile,
subtitle_action(config, notifier, translators)?,
);
// §9.5 *needs a decision* and *broken* both go to the operator alone;
// without a topic configured there is nowhere to send them.
if let Some(operator_topic) = &config.ntfy_operator_topic {
@@ -317,19 +409,12 @@ fn reconcile_loop(
Tick::Reconcile,
AttentionAction::new(notifier.clone(), operator_topic.clone()),
);
let broken_upstreams = broken::Upstreams {
prowlarr_url: config.prowlarr_url.clone(),
prowlarr_api_key: config.prowlarr_api_key.clone(),
transmission_url: config.transmission_url.clone(),
tmdb_url: config
.tmdb_url
.clone()
.unwrap_or_else(|| arr_api::DEFAULT_TMDB_URL.to_string()),
tmdb_api_key: config.tmdb_api_key.clone(),
};
reconcile = reconcile.register(
Tick::Reconcile,
BrokenAction::new(broken_upstreams, notifier.clone(), operator_topic.clone()),
reconcile = register_broken(
reconcile,
config,
translators,
notifier,
operator_topic.clone(),
);
} else {
tracing::warn!(
@@ -341,6 +426,48 @@ fn reconcile_loop(
Ok((reconcile, manual_grab, manual_tv))
}
/// The *broken* lane (#200 included): Prowlarr, Transmission and TMDB, plus
/// the subtitle lamps — an enabled provider, the selected engine, or a
/// missing `alass`/`ffmpeg` all fold into the same operator message.
fn register_broken(
reconcile: ReconcileLoop,
config: &Config,
translators: &Translators,
notifier: &Notifier,
operator_topic: String,
) -> ReconcileLoop {
let broken_upstreams = broken::Upstreams {
prowlarr_url: config.prowlarr_url.clone(),
prowlarr_api_key: config.prowlarr_api_key.clone(),
transmission_url: config.transmission_url.clone(),
tmdb_url: config
.tmdb_url
.clone()
.unwrap_or_else(|| arr_api::DEFAULT_TMDB_URL.to_string()),
tmdb_api_key: config.tmdb_api_key.clone(),
};
let broken_subtitles = broken::SubtitleUpstreams {
providers: subtitle_providers(
config.opensubtitles_api_key.clone(),
config.opensubtitles_username.clone(),
config.opensubtitles_password.clone(),
config.podnapisi_url.clone(),
),
backends: translators.backends.clone(),
alass_path: config.alass_path.clone(),
ffmpeg_path: config.ffmpeg_path.clone(),
};
reconcile.register(
Tick::Reconcile,
BrokenAction::new(
broken_upstreams,
broken_subtitles,
notifier.clone(),
operator_topic,
),
)
}
/// Register the TV grab lane on `reconcile` and hand back a second,
/// independent instance for `manual::run` (issue #132). `None` when Prowlarr
/// is not configured. TV grabbing needs no TMDB at grab time: air dates are
@@ -452,3 +579,189 @@ fn jellyfin_client(config: &Config) -> Result<arr_api::jellyfin::JellyfinClient,
config.jellyfin_api_key.clone(),
)?)
}
/// The §15 reconcile lane: closes subtitle gaps from the attempt rows.
///
/// Takes the same [`Translators`] the API is given, built once at startup, so
/// a translation behaves identically whether the reconcile sweep or the manual
/// endpoint asked for it. Which of the offered backends actually runs is the
/// `translation_engine` database setting, read per translation; with none
/// compiled in the translate step records "not compiled" and backs off rather
/// than failing obscurely.
fn subtitle_action(
config: &Config,
notifier: &Notifier,
translators: &Translators,
) -> Result<SubtitleAction, Error> {
let action = SubtitleAction::new(
subtitle_providers(
config.opensubtitles_api_key.clone(),
config.opensubtitles_username.clone(),
config.opensubtitles_password.clone(),
config.podnapisi_url.clone(),
),
translators.backends.clone(),
arr_subs::Syncer::new().with_binary(config.alass_path.clone()),
arr_probe::Extractor::new().with_binary(config.ffmpeg_path.clone()),
jellyfin_client(config)?,
);
Ok(match &config.ntfy_operator_topic {
Some(topic) => action.with_notifier(notifier.clone(), topic.clone()),
None => action,
})
}
/// The subtitle providers this deployment can reach (DESIGN.md §15).
///
/// Credentials are bootstrap config and never reach the database (§10), so
/// which providers *exist* is decided here, once, at startup; which of them a
/// search *runs* is the `providers_enabled` setting the API reads per
/// request. OpenSubtitles.com needs a registered API key to be called at all,
/// so without one it is not offered; Podnapisi is anonymous and always is.
fn subtitle_providers(
opensubtitles_api_key: Option<String>,
username: Option<String>,
password: Option<String>,
podnapisi_url: Option<String>,
) -> Vec<std::sync::Arc<dyn arr_subs::Provider>> {
let mut providers: Vec<std::sync::Arc<dyn arr_subs::Provider>> = Vec::new();
// `podnapisi_url` is the e2e-only seam (`ARR_PODNAPISI_URL`); absent, this
// is the real Podnapisi.net.
let mut builder = arr_subs::Podnapisi::builder();
if let Some(url) = podnapisi_url {
builder = builder.base_url(url);
}
match builder.build() {
Ok(podnapisi) => providers.push(std::sync::Arc::new(podnapisi)),
Err(error) => tracing::warn!(%error, "Podnapisi not available"),
}
let Some(api_key) = opensubtitles_api_key else {
tracing::info!("no OpenSubtitles.com API key configured; that provider is off");
return providers;
};
match arr_subs::OpenSubtitles::new(arr_subs::OpenSubtitlesConfig {
api_key,
username,
password,
}) {
Ok(opensubtitles) => providers.push(std::sync::Arc::new(opensubtitles)),
Err(error) => tracing::warn!(%error, "OpenSubtitles.com not available"),
}
providers
}
/// The translation backends this deployment can offer, built once at startup
/// and shared by the API and the reconcile lane (DESIGN.md §15, issue #216).
///
/// Which cargo features this binary was built with decides what could ever
/// be here (`compiled_engines`); credentials decide what actually is, same
/// split `subtitle_providers` makes for search. Which one of these a
/// translation *uses* is the `translation_engine` database setting, read per
/// request (#198) — this only decides which ids exist to be picked.
///
/// When the remote-command backend is one of them, its live timeout cell
/// rides along (#219): the API writes it on every settings edit, so the row
/// reaches the running process without a restart.
struct Translators {
backends: Vec<std::sync::Arc<dyn arr_subs::Backend>>,
command_timeout: Option<Arc<AtomicU64>>,
/// The OpenAI-compatible backend's live endpoint (#220), when that
/// backend is compiled in. Its base URL and model are database rows, so
/// this rides along the same way the command timeout does.
openai_endpoint: Option<arr_subs::OpenAiEndpoint>,
}
#[cfg_attr(
not(any(
feature = "translate-openai",
feature = "translate-deepl",
feature = "translate-google",
feature = "translate-command"
)),
allow(unused_variables, unused_mut)
)]
fn translation_backends(config: &Config) -> Translators {
let mut backends: Vec<std::sync::Arc<dyn arr_subs::Backend>> = Vec::new();
let mut command_timeout: Option<Arc<AtomicU64>> = None;
let mut openai_endpoint: Option<arr_subs::OpenAiEndpoint> = None;
#[cfg(feature = "translate-openai")]
{
// Built at the backend's own defaults and repointed from the
// settings row a moment later (#220). Never gated on the API key:
// `llama.cpp` serves without authentication, so a base URL and no
// key is a valid configuration (DESIGN.md §15).
let openai_config = arr_subs::OpenAiConfig {
api_key: config.translate_openai_api_key.clone(),
};
match arr_subs::OpenAi::new(openai_config) {
Ok(backend) => {
openai_endpoint = Some(backend.endpoint());
backends.push(std::sync::Arc::new(backend));
}
Err(error) => tracing::warn!(%error, "OpenAI-compatible translator not available"),
}
}
#[cfg(feature = "translate-deepl")]
{
if let Some(auth_key) = config.translate_deepl_api_key.clone() {
let deepl_config = arr_subs::DeepLConfig { auth_key };
let backend = match &config.translate_deepl_base_url {
Some(base_url) => arr_subs::DeepL::with_base_url(deepl_config, base_url),
None => arr_subs::DeepL::new(deepl_config),
};
match backend {
Ok(backend) => backends.push(std::sync::Arc::new(backend)),
Err(error) => tracing::warn!(%error, "DeepL not available"),
}
} else {
tracing::info!("no DeepL auth key configured; that translator is off");
}
}
#[cfg(feature = "translate-google")]
{
if let Some(api_key) = config.translate_google_api_key.clone() {
let google_config = arr_subs::GoogleConfig { api_key };
let backend = match &config.translate_google_base_url {
Some(base_url) => arr_subs::Google::with_base_url(google_config, base_url),
None => arr_subs::Google::new(google_config),
};
match backend {
Ok(backend) => backends.push(std::sync::Arc::new(backend)),
Err(error) => tracing::warn!(%error, "Google Translate not available"),
}
} else {
tracing::info!("no Google Translate API key configured; that translator is off");
}
}
#[cfg(feature = "translate-command")]
{
if let Some(template) = config.translate_command_template.clone() {
let command_config = arr_subs::CommandConfig {
template,
timeout: arr_subs::COMMAND_DEFAULT_TIMEOUT,
};
match arr_subs::Command::new(command_config) {
Ok(backend) => {
command_timeout = Some(backend.timeout_cell());
backends.push(std::sync::Arc::new(backend));
}
Err(error) => tracing::warn!(%error, "remote-command translator not available"),
}
} else {
tracing::info!("no remote-command template configured; that translator is off");
}
}
Translators {
backends,
command_timeout,
openai_endpoint,
}
}
+58 -2
View File
@@ -625,9 +625,11 @@ async fn pack_allowed(database: &Db, season_id: i64) -> Result<bool, GrabError>
)
.fetch_all(database.pool())
.await?;
// §6.2's ladder runs from the failure, not the grab (#245), with
// `grabbed_at` as the fallback for rows older than #239's column.
let failed_packs = sqlx::query!(
r#"SELECT count(*) AS "failures!: i64",
max(grabbed_at) AS "last_failed_at?: String"
r#"SELECT count(*) AS "failures!: i64",
max(coalesce(failed_at, grabbed_at)) AS "last_failed_at?: String"
FROM grabs
WHERE target_kind = 'season' AND target_id = ? AND state = 'failed'"#,
season_id
@@ -1164,6 +1166,60 @@ mod tests {
);
}
/// §6.2, issue #245: the RSS lane reads the same ladder, anchored on the
/// failure. A pack sent five weeks ago and condemned at import ten
/// minutes ago holds the lane shut, where anchoring on the grab would
/// have handed it the very release class that just failed.
#[tokio::test]
async fn a_pack_that_stalled_for_weeks_stays_backed_off_on_rss() {
let (_dir, database) = wanted(&[]).await;
let (season_id, episodes) =
wanted_series(&database, &["2024-04-11", "2024-04-18", "2024-04-25"]).await;
let release_id: i64 = sqlx::query_scalar(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, verdict)
VALUES (7, 'oldpack', 'Fallout.S01.2160p.WEB-DL.OLD', 85899345920,
'https://tracker/oldpack.torrent', '{}', 'eligible')
RETURNING id",
)
.fetch_one(database.pool())
.await
.unwrap();
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash, state,
grabbed_at, failed_at)
VALUES (?, 'season', ?, 'dead', 'failed',
strftime('%Y-%m-%dT%H:%M:%fZ', 'now', '-35 days'),
strftime('%Y-%m-%dT%H:%M:%fZ', 'now', '-10 minutes'))",
)
.bind(release_id)
.bind(season_id)
.execute(database.pool())
.await
.unwrap();
let indexer = prowlarr(TV_FEED).await;
let (downloader, _fake) = transmission().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
let sent: Vec<(String, i64, String)> = tv_grabs(&database)
.await
.into_iter()
.filter(|(_, _, state)| state == "sent")
.collect();
assert!(
!sent.iter().any(|(kind, _, _)| kind == "season"),
"the failure is ten minutes old, so the pack lane is shut: {sent:?}"
);
assert_eq!(
sent,
vec![
("episode".to_owned(), episodes[0], "sent".to_owned()),
("episode".to_owned(), episodes[1], "sent".to_owned()),
("episode".to_owned(), episodes[2], "sent".to_owned()),
]
);
}
/// §6.2 with #117's guard: an episode already on disk keeps the season
/// per-episode here too — the pack is skipped and the open gaps take
/// their singles.
File diff suppressed because it is too large Load Diff
+51 -8
View File
@@ -1139,12 +1139,15 @@ async fn record_pack_search(database: &Db, season_id: i64) -> Result<(), GrabErr
/// Whether failed season-pack grabs still hold this season off the pack
/// lane. §6.2: a failure quiets the pack search on the shared backoff curve
/// (each failed grab is one attempt), it never disables it. Anchored on the
/// latest failed grab's `grabbed_at`, not `failed_at` — moving §6.2's retry
/// cadence to failure time is its own decision, not #239's.
/// latest `failed_at` (#245), the same anchor §5.7's window uses: a torrent
/// can stall for weeks before `ffprobe` condemns it, and measured from the
/// grab the whole ladder would already have elapsed when the failure lands.
/// `grabbed_at` is the fallback for rows written before #239 added the
/// column.
async fn pack_backoff_active(database: &Db, season_id: i64) -> Result<bool, GrabError> {
let row = sqlx::query!(
r#"SELECT count(*) AS "failures!: i64",
max(grabbed_at) AS "last_failed_at?: String"
r#"SELECT count(*) AS "failures!: i64",
max(coalesce(failed_at, grabbed_at)) AS "last_failed_at?: String"
FROM grabs
WHERE target_kind = 'season' AND target_id = ? AND state = 'failed'"#,
season_id
@@ -1497,8 +1500,17 @@ mod tests {
/// release on the blacklist and the season falls back to per-episode —
/// the pack is not tried again and the episodes are not written off.
/// Seed what the import tick leaves behind after a pack fails: one
/// `failed` season grab per (infohash, age) pair.
/// `failed` season grab per (infohash, age) pair, grabbed and failed at
/// the same age, which is the usual case — the two are minutes apart.
async fn failed_packs(database: &Db, season_id: i64, ages: &[&str]) {
let pairs: Vec<(&str, &str)> = ages.iter().map(|age| (*age, *age)).collect();
stalled_failed_packs(database, season_id, &pairs).await;
}
/// The same seed, but with the grab and the failure at different ages —
/// the #245 case, where a torrent stalls for weeks before `ffprobe`
/// condemns it at import.
async fn stalled_failed_packs(database: &Db, season_id: i64, ages: &[(&str, &str)]) {
let release_id: i64 = sqlx::query_scalar(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, verdict)
VALUES (7, 'oldpack', 'Fallout.S01.2160p.WEB-DL.OLD', 85899345920,
@@ -1508,16 +1520,19 @@ mod tests {
.fetch_one(database.pool())
.await
.unwrap();
for (index, age) in ages.iter().enumerate() {
for (index, (grabbed_age, failed_age)) in ages.iter().enumerate() {
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash, state, grabbed_at)
"INSERT INTO grabs (release_id, target_kind, target_id, infohash, state,
grabbed_at, failed_at)
VALUES (?, 'season', ?, ?, 'failed',
strftime('%Y-%m-%dT%H:%M:%fZ', 'now', ?),
strftime('%Y-%m-%dT%H:%M:%fZ', 'now', ?))",
)
.bind(release_id)
.bind(season_id)
.bind(format!("dead{index}"))
.bind(age)
.bind(grabbed_age)
.bind(failed_age)
.execute(database.pool())
.await
.unwrap();
@@ -1576,6 +1591,34 @@ mod tests {
);
}
/// §6.2, issue #245: the ladder runs from the failure, not the grab. A
/// pack sent five weeks ago and condemned by `ffprobe` ten minutes ago
/// is one minute into a 1h window, not five weeks past it — the lane
/// stays quiet and the episodes carry the season instead.
#[tokio::test]
async fn a_pack_that_stalled_for_weeks_backs_off_from_the_failure() {
let (_dir, database, season_id) =
wanted_season(&["2024-04-11", "2024-04-11", "2024-04-11"]).await;
stalled_failed_packs(&database, season_id, &[("-35 days", "-10 minutes")]).await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
let sources: Vec<String> = fake
.torrents()
.into_iter()
.map(|torrent| torrent.source)
.collect();
assert!(
sources
.iter()
.all(|source| !source.ends_with("pack.torrent")),
"the grab is five weeks old but the failure is ten minutes old: {sources:?}"
);
assert_eq!(sources.len(), 3, "{sources:?}");
}
/// Repeated failures ride the capped curve: five failed packs mean a 7d
/// window — still closed at 6d, open at 8d. Quiet, never off.
#[tokio::test]
@@ -0,0 +1,81 @@
-- #186. What subtitles exist for a media file, and what arr has tried per
-- wanted language (DESIGN.md §15). Settings rows (#198) and the budget
-- buckets (#197) are deliberately absent.
-- One row per subtitle arr knows about. An embedded track that was never
-- extracted has no `path`: it is recorded because it satisfies a language,
-- not because there is a file. Everything else is a sidecar next to the
-- video (§15, §7.4), so the path is the row's identity on disk.
CREATE TABLE subtitle_files (
id INTEGER PRIMARY KEY,
media_file_id INTEGER NOT NULL REFERENCES media_files (id) ON DELETE CASCADE,
-- As `arr_core::Language` spells it: `pt-PT`, `pt-BR`, `en`.
language TEXT NOT NULL,
origin TEXT NOT NULL
CHECK (origin IN ('embedded', 'extracted', 'provider', 'translated')),
-- §15. The named provider a fetch came from, and the candidate id it was
-- ranked as, so a re-fetch of the same candidate is recognisable.
provider TEXT,
candidate_id TEXT,
-- §15. The translation backend that produced it, when machine made.
engine TEXT,
-- §15. A forced track never satisfies a want; an SDH one does, ranked
-- below a plain subtitle. Both are facts about the file, so both are
-- stored and the ranking rule stays in `arr-core`.
forced INTEGER NOT NULL DEFAULT 0 CHECK (forced IN (0, 1)),
sdh INTEGER NOT NULL DEFAULT 0 CHECK (sdh IN (0, 1)),
-- §15. `alass` reports no confidence, so its output is accepted unless
-- implausible. `synced` is an accepted shift; `sync_rejected` means the
-- unsynced original was kept and this file is flagged. Never both.
synced INTEGER NOT NULL DEFAULT 0 CHECK (synced IN (0, 1)),
sync_rejected INTEGER NOT NULL DEFAULT 0 CHECK (sync_rejected IN (0, 1)),
path TEXT UNIQUE,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
CHECK ((path IS NULL) = (origin = 'embedded')),
CHECK ((provider IS NOT NULL) = (origin = 'provider')),
CHECK (candidate_id IS NULL OR provider IS NOT NULL),
CHECK ((engine IS NOT NULL) = (origin = 'translated')),
CHECK (NOT (synced = 1 AND sync_rejected = 1))
) STRICT;
-- §15's satisfaction check is "is there a subtitle in this language", asked
-- per file per tick.
CREATE INDEX subtitle_files_media_file ON subtitle_files (media_file_id, language);
-- An embedded track is identified by what it satisfies, not by a file, so a
-- second probe of the same video re-records it rather than duplicating it.
CREATE UNIQUE INDEX subtitle_files_embedded
ON subtitle_files (media_file_id, language, forced, sdh)
WHERE origin = 'embedded';
-- One row per (media file, wanted language). What the reconcile loop reads to
-- decide whether a language is a gap, what backs a failing provider off, and
-- what the missing-subtitles queue renders.
--
-- `capped` is a state and not an error (§15): being at a daily allowance is
-- something the queue shows. The allowance itself is #197 and not here.
-- `unavailable` is the end of the line — no provider has it and there is no
-- text source to translate from, so retrying costs budget for nothing.
CREATE TABLE subtitle_attempts (
media_file_id INTEGER NOT NULL REFERENCES media_files (id) ON DELETE CASCADE,
language TEXT NOT NULL,
state TEXT NOT NULL DEFAULT 'wanted'
CHECK (state IN ('wanted', 'satisfied', 'failed', 'capped', 'unavailable')),
attempts INTEGER NOT NULL DEFAULT 0 CHECK (attempts >= 0),
last_attempt_at TEXT,
-- Why the last attempt failed, for the queue and for backoff. Cleared
-- when an attempt succeeds.
last_failure TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
PRIMARY KEY (media_file_id, language),
CHECK ((attempts = 0) = (last_attempt_at IS NULL))
) STRICT;
-- §8, §15. The work list is every language that is not satisfied yet, least
-- recently attempted first within it; the ordering across files is
-- newest-import-first and comes from the join to `media_files`.
CREATE INDEX subtitle_attempts_pending
ON subtitle_attempts (state, last_attempt_at)
WHERE state <> 'satisfied';
@@ -0,0 +1,37 @@
-- #198. The runtime-editable half of subtitle configuration (DESIGN.md §15,
-- §10). The other half — provider credentials, translator keys, the
-- remote-command template, the alass and ffmpeg binary paths — is
-- config/env only and never reaches this table.
--
-- One row: the wanted set, enabled providers and translation engine are
-- global, not per root (§15), so there is nothing to key this on.
CREATE TABLE subtitle_settings (
id INTEGER PRIMARY KEY CHECK (id = 1),
-- Language tags as `arr_core::Language` spells them, e.g. `pt-PT`, `en`.
wanted_languages TEXT NOT NULL DEFAULT '["pt-PT","en"]'
CHECK (json_valid(wanted_languages)),
-- Provider ids in search order (DESIGN.md §15's ranking runs per
-- provider before it runs per candidate). Absent from the array means
-- disabled.
providers_enabled TEXT NOT NULL DEFAULT '["opensubtitles","podnapisi"]'
CHECK (json_valid(providers_enabled)),
-- The compiled-in backend in use, or NULL when translation is off. Not
-- checked against the running binary's features here — the API does
-- that (#198) — so a rebuild with a different feature set never leaves
-- an unreadable row.
translation_engine TEXT
CHECK (translation_engine IN ('openai', 'deepl', 'google', 'command')),
-- Provider id / engine name -> a daily allowance (§15's token buckets,
-- #197). A missing key means no allowance configured yet, not zero.
provider_daily_budgets TEXT NOT NULL DEFAULT '{}' CHECK (json_valid(provider_daily_budgets)),
translator_daily_budgets TEXT NOT NULL DEFAULT '{}' CHECK (json_valid(translator_daily_budgets)),
-- Only the remote-command backend needs one of these: the HTTP backends
-- carry their own client timeout.
remote_command_timeout_seconds INTEGER NOT NULL DEFAULT 30
CHECK (remote_command_timeout_seconds > 0),
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
) STRICT;
INSERT INTO subtitle_settings (id) VALUES (1);
@@ -0,0 +1,19 @@
-- #197. Token bucket state for §15's provider and translator budgets. The
-- allowance itself is `subtitle_settings.provider_daily_budgets` /
-- `translator_daily_budgets` (#198); this table only tracks what has already
-- been spent today, so a bucket with no allowance configured reads as
-- unlimited rather than zero (0025's own comment on that column).
--
-- One row per (kind, name, day); a new day is a fresh row rather than a
-- reset column, so the refill is "today has no row yet" and needs no cron.
CREATE TABLE subtitle_budget_spend (
kind TEXT NOT NULL CHECK (kind IN ('provider', 'translator')),
name TEXT NOT NULL,
-- UTC calendar day, `YYYY-MM-DD`.
day TEXT NOT NULL,
-- A provider unit is one download; a translator unit is one character of
-- source text sent (DESIGN.md §15: translators bill per character).
spent INTEGER NOT NULL DEFAULT 0 CHECK (spent >= 0),
PRIMARY KEY (kind, name, day)
) STRICT;
@@ -0,0 +1,17 @@
-- #220. Where the OpenAI-compatible translation backend points, and which
-- model it names (DESIGN.md §15). That backend is not "OpenAI" — it is
-- anything speaking that request shape, `llama.cpp` and a local gateway
-- included — so which endpoint and which model are in use is something the
-- operator tries and changes, not a property of the deployment fixed at
-- start-up. Both replace bootstrap keys #216 added as an explicit stopgap.
--
-- Only the API key stays in the environment (§10): it is a secret, and an
-- endpoint that needs no key at all is a valid configuration.
ALTER TABLE subtitle_settings
-- NULL means the backend's own default, `https://api.openai.com/v1/`.
ADD COLUMN openai_base_url TEXT;
ALTER TABLE subtitle_settings
-- NULL means the backend's own default model.
ADD COLUMN openai_model TEXT;
@@ -0,0 +1,41 @@
-- #222. DESIGN.md §15, as amended: a language is satisfied by exactly one
-- sidecar. No filename segment distinguishes forced from plain from SDH, so
-- two sidecar rows for one (media file, language) describe two files
-- competing for one name. The schema says so now rather than leaving it to
-- the API's path check.
--
-- Embedded rows are deliberately untouched. They describe tracks inside the
-- video, not files on disk, and several can legitimately coexist for one
-- language — a plain track and a forced one, say. Their key stays
-- `(media_file_id, language, forced, sdh)`.
-- A database that predates the constraint may already hold a duplicate from
-- a manual grab that beat the path check: a fetched `.pt-PT.srt` beside a
-- translated `.pt-PT.mt.srt`, for instance. Resolve rather than fail. Of the
-- rows for one (media file, language): keep a real subtitle over a machine
-- translation, and of two of the same kind the newest.
--
-- The files stay on disk. Deleting a viewer's subtitle during a migration is
-- worse than leaving an orphan, and the manual delete (#218, #223) cleans one
-- up on request.
DELETE FROM subtitle_files
WHERE path IS NOT NULL
AND id NOT IN (
SELECT id
FROM (
SELECT id,
ROW_NUMBER() OVER (
PARTITION BY media_file_id, language
ORDER BY (origin = 'translated') ASC,
created_at DESC,
id DESC
) AS place
FROM subtitle_files
WHERE path IS NOT NULL
)
WHERE place = 1
);
CREATE UNIQUE INDEX subtitle_files_one_sidecar
ON subtitle_files (media_file_id, language)
WHERE path IS NOT NULL;
@@ -0,0 +1,34 @@
-- Issue #244. A root's path could be stored with a trailing separator until
-- #243 normalised the incoming value. `roots::update` normalises the payload
-- and compares it against the stored value, so such a row never compares
-- equal: every edit, a policy change included, takes the relocation branch,
-- and there every planned destination is its own source. Normalising the
-- payload alone fixed the half that cannot bite; this is the other half.
--
-- `rtrim` strips every trailing separator at once, so '/mnt/x//' normalises
-- in one pass. A bare '/' rtrims to the empty string and is put back, which
-- is what `normalize_path` in arr-api does.
--
-- Guarded, because `roots.path` is UNIQUE and a migration that cannot apply
-- stops the daemon booting -- worse than the bug it fixes. A row is
-- normalised only when no other row shares its normalised path: neither a
-- row already holding the stripped value, nor another trailing-separator row
-- that would strip to the same thing. Every row in such a group is left
-- exactly as it is. That leaves two roots naming one directory, which is a
-- settings mistake for the operator to resolve by hand, not a reason to
-- refuse to boot.
--
-- This cannot introduce a collision either. An updated row's new value is a
-- normalised path no other row normalises to, and a row left alone whose raw
-- path equalled that value would have had the same normalised path, which is
-- the case the guard excludes.
UPDATE roots
SET path = CASE WHEN rtrim(path, '/') = '' THEN '/' ELSE rtrim(path, '/') END
WHERE path <> CASE WHEN rtrim(path, '/') = '' THEN '/' ELSE rtrim(path, '/') END
AND NOT EXISTS (
SELECT 1
FROM roots AS other
WHERE other.id <> roots.id
AND CASE WHEN rtrim(other.path, '/') = '' THEN '/' ELSE rtrim(other.path, '/') END
= CASE WHEN rtrim(roots.path, '/') = '' THEN '/' ELSE rtrim(roots.path, '/') END
);
@@ -0,0 +1,68 @@
-- no-transaction
-- #211: a waived release could not record which rule it relaxed. The old
-- constraint made the rule name an exact synonym for rejection:
--
-- CHECK ((verdict = 'rejected') = (rejected_rule IS NOT NULL))
--
-- so §9.3's deck showed a bare `waived` beside rejected rows that each named
-- their own rule, and §5.7's "watchable but not what was asked" lost the half
-- that says what was not asked for. The relaxed form still demands a rule on
-- a rejection and stops demanding its absence elsewhere.
--
-- SQLite cannot alter a CHECK, so the table is rebuilt (see 0014). Unlike the
-- rebuilds there, `releases` is a parent: `grabs`, `movie_releases`,
-- `episode_releases` and `season_releases` all point at it, three of them
-- ON DELETE CASCADE. Dropping the old table with foreign keys enforced would
-- delete those children (or, for `grabs`, refuse outright), so this follows
-- SQLite's own procedure — foreign keys off, the rebuild in one transaction,
-- foreign keys back on. `PRAGMA foreign_keys` is a no-op inside a
-- transaction, which is why the file opens `-- no-transaction` and manages
-- its own; the migration is still all-or-nothing.
--
-- Rows are copied verbatim. Every existing `waived` row keeps its NULL and
-- goes on reading as it does today; only rows written after this migration
-- carry a waived rule.
PRAGMA foreign_keys = OFF;
BEGIN;
CREATE TABLE releases_new (
id INTEGER PRIMARY KEY,
-- Prowlarr's indexer id. Not a foreign key: indexers live in Prowlarr.
indexer_id INTEGER NOT NULL,
guid TEXT NOT NULL,
name TEXT NOT NULL,
size INTEGER NOT NULL,
seeders INTEGER,
publish_date TEXT,
download_url TEXT NOT NULL,
-- §5.6. What the release name claims, before anything is downloaded.
parsed TEXT NOT NULL CHECK (json_valid(parsed)),
score REAL,
-- §9.3. Three buckets. `rejected_rule` names the rule that killed it so
-- an over-strict filter is visible without reading release names, and
-- §5.7's waiver names the rule it relaxed for the same reason.
verdict TEXT CHECK (verdict IN ('eligible', 'waived', 'rejected')),
rejected_rule TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
UNIQUE (indexer_id, guid),
CHECK (verdict != 'rejected' OR rejected_rule IS NOT NULL)
) STRICT;
INSERT INTO releases_new (
id, indexer_id, guid, name, size, seeders, publish_date, download_url,
parsed, score, verdict, rejected_rule, created_at
)
SELECT
id, indexer_id, guid, name, size, seeders, publish_date, download_url,
parsed, score, verdict, rejected_rule, created_at
FROM releases;
DROP TABLE releases;
ALTER TABLE releases_new RENAME TO releases;
CREATE INDEX releases_verdict ON releases (verdict, score);
COMMIT;
PRAGMA foreign_keys = ON;
+106 -9
View File
@@ -12,7 +12,7 @@
//! infohash, and
//! - **infohash**, because the same torrent is re-listed under a new name.
use std::collections::HashSet;
use std::collections::HashMap;
use sqlx::SqlitePool;
@@ -21,15 +21,21 @@ use sqlx::SqlitePool;
/// rejected row names the rule that killed it" reads the same everywhere.
pub const RULE: &str = "blacklisted";
/// Every blacklist key, loaded once per tick or request.
/// Every blacklist key and the reason it was written under, loaded once per
/// tick or request.
///
/// Household scale: a handful of rows. Loading it whole and matching in
/// memory keeps the check identical for a release that has a database row and
/// one that has only just arrived from an indexer.
///
/// The reason travels with the key because §9.3's deck has to say what a row
/// was blacklisted for (#227): a release the policy rejected on size is one
/// the operator can relax and try again, and a corrupt or mismatched one is
/// not. A bare `blacklisted` makes those two read the same.
#[derive(Debug, Clone, Default)]
pub struct Blacklist {
names: HashSet<String>,
infohashes: HashSet<String>,
names: HashMap<String, String>,
infohashes: HashMap<String, String>,
}
impl Blacklist {
@@ -39,17 +45,30 @@ impl Blacklist {
///
/// If the query fails.
pub async fn load(pool: &SqlitePool) -> Result<Self, sqlx::Error> {
// Oldest first, so a key that hard-failed twice under different
// reasons keeps the first one — the same rule [`add`] applies when it
// refuses to write the second row.
let rows = sqlx::query!(
r#"SELECT normalised_name AS "normalised_name!: String", infohash FROM blacklist"#
r#"SELECT normalised_name AS "normalised_name!: String",
infohash,
reason AS "reason!: String"
FROM blacklist
ORDER BY id"#
)
.fetch_all(pool)
.await?;
let mut blacklist = Self::default();
for row in rows {
blacklist.names.insert(row.normalised_name);
blacklist
.names
.entry(row.normalised_name)
.or_insert_with(|| row.reason.clone());
if let Some(infohash) = row.infohash {
blacklist.infohashes.insert(infohash.to_ascii_lowercase());
blacklist
.infohashes
.entry(infohash.to_ascii_lowercase())
.or_insert(row.reason);
}
}
Ok(blacklist)
@@ -58,14 +77,44 @@ impl Blacklist {
/// Whether this release name has been blacklisted, under any spelling.
#[must_use]
pub fn blocks_name(&self, release_name: &str) -> bool {
self.names.contains(&arr_parse::normalise(release_name))
self.names.contains_key(&arr_parse::normalise(release_name))
}
/// Whether this infohash has been blacklisted. Case-insensitive:
/// Transmission and Torznab disagree on the hex casing.
#[must_use]
pub fn blocks_infohash(&self, infohash: &str) -> bool {
self.infohashes.contains(&infohash.to_ascii_lowercase())
self.infohashes.contains_key(&infohash.to_ascii_lowercase())
}
/// What this release name was blacklisted for, or `None` if it was not.
#[must_use]
pub fn reason_for_name(&self, release_name: &str) -> Option<&str> {
self.names
.get(&arr_parse::normalise(release_name))
.map(String::as_str)
}
/// What this infohash was blacklisted for, or `None` if it was not.
#[must_use]
pub fn reason_for_infohash(&self, infohash: &str) -> Option<&str> {
self.infohashes
.get(&infohash.to_ascii_lowercase())
.map(String::as_str)
}
/// What a candidate was blacklisted for, under either key (#227).
///
/// The name is asked first: it is the key every candidate has, and a
/// `.torrent` URL hides its infohash until the download client fetches
/// it, exactly as [`Blacklist::blocks_candidate`] describes.
#[must_use]
pub fn reason_for_candidate(&self, release_name: &str, download_url: &str) -> Option<&str> {
self.reason_for_name(release_name).or_else(|| {
magnet_infohash(download_url)
.and_then(|hash| self.infohashes.get(&hash))
.map(String::as_str)
})
}
/// Whether a candidate is blacklisted before anything is sent to the
@@ -175,6 +224,54 @@ mod tests {
assert!(!blacklist.blocks_name("Dune Part Two 2024 1080p WEB-DL"));
}
#[tokio::test]
async fn a_key_carries_the_reason_it_was_blacklisted_for() {
let (database, _dir) = database().await;
add(
database.pool(),
Some(HASH),
"Rick.And.Morty.S08.1080p",
"size",
)
.await
.unwrap();
add(
database.pool(),
None,
"Some.Other.Pack.S01",
"no file matches a wanted episode",
)
.await
.unwrap();
let blacklist = Blacklist::load(database.pool()).await.unwrap();
// #227: the deck has to tell a policy rejection from a bad release,
// and the reason is the only thing that says which.
assert_eq!(
blacklist.reason_for_name("Rick And Morty S08 1080p"),
Some("size")
);
assert_eq!(
blacklist.reason_for_infohash(&HASH.to_ascii_uppercase()),
Some("size")
);
assert_eq!(
blacklist.reason_for_name("Some.Other.Pack.S01"),
Some("no file matches a wanted episode")
);
assert_eq!(blacklist.reason_for_name("Never.Failed.S01"), None);
let magnet = format!("magnet:?xt=urn:btih:{HASH}&dn=Renamed.Pack");
assert_eq!(
blacklist.reason_for_candidate("Renamed.Pack", &magnet),
Some("size")
);
assert_eq!(
blacklist.reason_for_candidate("Renamed.Pack", "https://tracker/x.torrent"),
None
);
}
#[tokio::test]
async fn a_second_hard_fail_of_the_same_torrent_adds_no_row() {
let (database, _dir) = database().await;
+265
View File
@@ -7,9 +7,16 @@ use std::path::Path;
pub mod blacklist;
pub mod policy;
pub mod subtitle_budget;
pub mod subtitles;
pub use blacklist::Blacklist;
pub use policy::{MoviePolicy, PolicyColumns, PolicyError, TitlePolicy};
pub use subtitle_budget::BudgetKind;
pub use subtitles::{
NewSubtitleFile, PendingSubtitle, SubtitleAttempt, SubtitleFile, SubtitleOrigin, SubtitleState,
SubtitleSync,
};
use sqlx::sqlite::{SqliteConnectOptions, SqliteJournalMode, SqlitePoolOptions, SqliteSynchronous};
use sqlx::{migrate::MigrateError, SqlitePool};
@@ -347,6 +354,93 @@ mod tests {
assert_eq!(renamed_from_imported, "available");
}
/// The four seeded roots, up to but not including migration 0031, with
/// the given legacy paths written straight into the rows.
async fn roots_before_normalisation(paths: &[(i64, &str)]) -> (tempfile::TempDir, Db) {
let dir = tempfile::tempdir().expect("tempdir");
let db = Db::connect(dir.path().join("arr.db"))
.await
.expect("connect");
MIGRATOR
.run_to(30, db.pool())
.await
.expect("migrations before #244");
for (id, path) in paths {
sqlx::query("UPDATE roots SET path = ? WHERE id = ?")
.bind(path)
.bind(id)
.execute(db.pool())
.await
.expect("legacy path, valid under the unique index");
}
(dir, db)
}
async fn root_path(db: &Db, id: i64) -> String {
sqlx::query_scalar("SELECT path FROM roots WHERE id = ?")
.bind(id)
.fetch_one(db.pool())
.await
.expect("root path")
}
/// #244: a root's path could be stored with a trailing separator until
/// #243 normalised the incoming value, and `roots::update` compares a
/// normalised payload against the stored value — so such a row could
/// never be edited again. Migration 0031 strips the separator, and skips
/// a row whose stripped form another row already holds rather than
/// tripping the unique index and refusing to apply.
#[tokio::test]
async fn root_paths_are_normalised_but_never_onto_a_path_in_use() {
let (_dir, db) = roots_before_normalisation(&[
(1, "/mnt/media/movies/main/"),
(2, "/mnt/collide"),
(3, "/mnt/collide/"),
(4, "/mnt/media/tv/kids///"),
])
.await;
db.migrate()
.await
.expect("0031 applies with a collision present");
assert_eq!(root_path(&db, 1).await, "/mnt/media/movies/main");
assert_eq!(
root_path(&db, 4).await,
"/mnt/media/tv/kids",
"every trailing separator goes in one pass"
);
assert_eq!(root_path(&db, 2).await, "/mnt/collide");
assert_eq!(
root_path(&db, 3).await,
"/mnt/collide/",
"left exactly as it is: normalising it would collide with root 2"
);
}
/// #244: the harder half of the same guard. Two rows that strip to the
/// same path, *neither* of which already holds the stripped value, still
/// have to be left alone — normalising them would collide with each
/// other, and a migration that cannot apply stops the daemon booting.
#[tokio::test]
async fn two_rows_that_would_collide_with_each_other_stop_nothing() {
let (_dir, db) =
roots_before_normalisation(&[(1, "/mnt/one/"), (2, "/mnt/dup/"), (3, "/mnt/dup//")])
.await;
db.migrate()
.await
.expect("0031 applies with a mutually colliding pair present");
assert_eq!(root_path(&db, 2).await, "/mnt/dup/");
assert_eq!(root_path(&db, 3).await, "/mnt/dup//");
assert_eq!(
root_path(&db, 1).await,
"/mnt/one",
"the rest of the table is still normalised"
);
}
/// #155: the table-rebuild migrations (0007, 0014, 0021) run outside a
/// transaction so `PRAGMA foreign_keys = OFF` holds and dropping the old
/// tables does not cascade-delete `movie_releases`/`episode_releases`.
@@ -430,6 +524,177 @@ mod tests {
);
}
/// #211: 0032 rebuilds `releases`, which is a parent — `grabs`,
/// `movie_releases`, `episode_releases` and `season_releases` all point
/// at it, three of them ON DELETE CASCADE. The rebuild runs with foreign
/// keys off so dropping the old table neither cascades those children
/// away nor is refused by `grabs`.
#[tokio::test]
async fn the_releases_rebuild_preserves_children_and_rows() {
let (_dir, db) = a_deck_with_every_child_row().await;
db.migrate().await.expect("remaining migrations");
for (label, query, expected) in [
("releases", "SELECT count(*) FROM releases", 3),
("movie_releases", "SELECT count(*) FROM movie_releases", 1),
(
"episode_releases",
"SELECT count(*) FROM episode_releases",
1,
),
("season_releases", "SELECT count(*) FROM season_releases", 1),
("grabs", "SELECT count(*) FROM grabs", 1),
] {
let rows: i64 = sqlx::query_scalar(query)
.fetch_one(db.pool())
.await
.expect("count");
assert_eq!(rows, expected, "{label} survives the releases rebuild");
}
// An existing waived row keeps its NULL and reads as it did before.
let waived: Option<String> =
sqlx::query_scalar("SELECT rejected_rule FROM releases WHERE guid = 'waived-guid'")
.fetch_one(db.pool())
.await
.expect("waived row");
assert_eq!(waived, None);
let rejected: Option<String> =
sqlx::query_scalar("SELECT rejected_rule FROM releases WHERE guid = 'rejected-guid'")
.fetch_one(db.pool())
.await
.expect("rejected row");
assert_eq!(rejected.as_deref(), Some("size"));
// Foreign keys are back on for the connection the migration used.
let foreign_keys: i64 = sqlx::query_scalar("PRAGMA foreign_keys")
.fetch_one(db.pool())
.await
.expect("foreign_keys");
assert_eq!(foreign_keys, 1);
}
/// A database migrated to just before 0032, holding one release per
/// verdict and one row in every table that points at `releases`.
async fn a_deck_with_every_child_row() -> (tempfile::TempDir, Db) {
let dir = tempfile::tempdir().expect("tempdir");
let db = Db::connect(dir.path().join("arr.db"))
.await
.expect("connect");
MIGRATOR
.run_to(31, db.pool())
.await
.expect("migrations before the releases rebuild");
let movie_id = sqlx::query(
"INSERT INTO movies (tmdb_id, title, root_id)
SELECT 693134, 'Dune Part Two', id FROM roots WHERE kind = 'movie' LIMIT 1",
)
.execute(db.pool())
.await
.expect("movie")
.last_insert_rowid();
let series_id = sqlx::query(
"INSERT INTO series (tmdb_id, title, root_id)
SELECT 82728, 'Bluey', id FROM roots WHERE kind = 'tv' LIMIT 1",
)
.execute(db.pool())
.await
.expect("series")
.last_insert_rowid();
let season_id = sqlx::query("INSERT INTO seasons (series_id, number) VALUES (?, 1)")
.bind(series_id)
.execute(db.pool())
.await
.expect("season")
.last_insert_rowid();
let episode_id =
sqlx::query("INSERT INTO episodes (season_id, number, title) VALUES (?, 1, 'x')")
.bind(season_id)
.execute(db.pool())
.await
.expect("episode")
.last_insert_rowid();
// One row per verdict, including the waived row this issue is about,
// which under the old constraint could only hold NULL.
for (guid, verdict, rule) in [
("eligible-guid", "eligible", None),
("waived-guid", "waived", None),
("rejected-guid", "rejected", Some("size")),
] {
sqlx::query(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, verdict, rejected_rule)
VALUES (1, ?, ?, 1024, 'http://x', '{}', ?, ?)",
)
.bind(guid)
.bind(guid)
.bind(verdict)
.bind(rule)
.execute(db.pool())
.await
.expect("release");
}
let release_id: i64 =
sqlx::query_scalar("SELECT id FROM releases WHERE guid = 'eligible-guid'")
.fetch_one(db.pool())
.await
.expect("release id");
sqlx::query("INSERT INTO movie_releases (movie_id, release_id) VALUES (?, ?)")
.bind(movie_id)
.bind(release_id)
.execute(db.pool())
.await
.expect("movie link");
sqlx::query("INSERT INTO episode_releases (episode_id, release_id) VALUES (?, ?)")
.bind(episode_id)
.bind(release_id)
.execute(db.pool())
.await
.expect("episode link");
sqlx::query("INSERT INTO season_releases (season_id, release_id) VALUES (?, ?)")
.bind(season_id)
.bind(release_id)
.execute(db.pool())
.await
.expect("season link");
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash)
VALUES (?, 'movie', ?, 'infohash-1')",
)
.bind(release_id)
.bind(movie_id)
.execute(db.pool())
.await
.expect("grab");
(dir, db)
}
/// The relaxed constraint (#211) admits a rule on a waiver and still
/// refuses a rejection without one.
#[tokio::test]
async fn a_waiver_may_name_the_rule_it_relaxed() {
let (_dir, db) = fresh().await;
sqlx::query(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, verdict, rejected_rule)
VALUES (1, 'waived', 'Some.Release', 1024, 'http://x', '{}', 'waived', 'size')",
)
.execute(db.pool())
.await
.expect("a waiver names its rule");
sqlx::query(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, verdict)
VALUES (1, 'rejected', 'Some.Release', 1024, 'http://x', '{}', 'rejected')",
)
.execute(db.pool())
.await
.expect_err("a rejection still has to name its rule");
}
#[tokio::test]
async fn seeds_two_tv_roots_with_distinct_policies() {
let (_dir, db) = fresh().await;
+222
View File
@@ -0,0 +1,222 @@
//! Token buckets for §15's provider and translator daily budgets (#197).
//!
//! One row per (kind, name, today). The allowance lives in
//! `subtitle_settings` (#198) and is passed in by the caller rather than
//! read here, because the reconcile loop already loads settings once per
//! tick; this module only tracks and gates spend against it.
//!
//! [`try_spend`] is the single gate, and it is the only one: check and
//! increment happen in one statement, so concurrent closes racing the same
//! bucket (§8 runs up to four at once) cannot both slip a spend past the cap
//! between a read and a write. [`spent_today`] is read-only, kept apart for
//! callers that only need to report what a bucket holds.
use sqlx::SqlitePool;
/// Which bucket a spend counts against.
#[derive(Debug, Clone, Copy, PartialEq, Eq, sqlx::Type)]
#[sqlx(rename_all = "lowercase")]
pub enum BudgetKind {
/// One unit per download (DESIGN.md §15's example: a single-digit daily
/// download cap on `OpenSubtitles`' free tier).
Provider,
/// One unit per character of source text sent, not per call (DESIGN.md
/// §15: translators bill per character).
Translator,
}
/// What has already been spent from `kind`/`name`'s bucket today.
///
/// # Errors
///
/// If the query fails.
pub async fn spent_today(
pool: &SqlitePool,
kind: BudgetKind,
name: &str,
) -> Result<i64, sqlx::Error> {
Ok(sqlx::query_scalar!(
r#"SELECT spent AS "spent!: i64" FROM subtitle_budget_spend
WHERE kind = ? AND name = ? AND day = strftime('%Y-%m-%d', 'now')"#,
kind,
name
)
.fetch_optional(pool)
.await?
.unwrap_or(0))
}
/// Try to spend `amount` from `kind`/`name`'s bucket today, atomically.
///
/// `None` allowance means unlimited (§10): always spends, and nothing is
/// recorded — there is no cap to account spend against. With an allowance,
/// the insert-or-increment and the cap check are one statement, so a spend
/// that would cross `allowance` neither applies nor is recorded.
///
/// # Errors
///
/// If the query fails.
pub async fn try_spend(
pool: &SqlitePool,
kind: BudgetKind,
name: &str,
amount: i64,
allowance: Option<i64>,
) -> Result<bool, sqlx::Error> {
let Some(allowance) = allowance else {
return Ok(true);
};
let spent = sqlx::query_scalar!(
r#"INSERT INTO subtitle_budget_spend (kind, name, day, spent)
SELECT ?1, ?2, strftime('%Y-%m-%d', 'now'), ?3
WHERE ?3 <= ?4
ON CONFLICT (kind, name, day) DO UPDATE
SET spent = subtitle_budget_spend.spent + excluded.spent
WHERE subtitle_budget_spend.spent + excluded.spent <= ?4
RETURNING spent AS "spent!: i64""#,
kind,
name,
amount,
allowance
)
.fetch_optional(pool)
.await?;
Ok(spent.is_some())
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::{spent_today, try_spend, BudgetKind};
use crate::Db;
async fn database() -> (Db, tempfile::TempDir) {
let dir = tempfile::tempdir().unwrap();
let database = Db::connect(dir.path().join("test.db")).await.unwrap();
database.migrate().await.unwrap();
(database, dir)
}
#[tokio::test]
async fn an_unconfigured_allowance_is_unlimited_and_untracked() {
let (db, _dir) = database().await;
let spent = try_spend(db.pool(), BudgetKind::Provider, "opensubtitles", 1, None)
.await
.unwrap();
assert!(spent);
assert_eq!(
spent_today(db.pool(), BudgetKind::Provider, "opensubtitles")
.await
.unwrap(),
0,
"nothing to cap means nothing to account"
);
}
#[tokio::test]
async fn spend_accumulates_until_the_allowance_is_reached() {
let (db, _dir) = database().await;
let allowance = Some(5);
for _ in 0..5 {
assert!(try_spend(
db.pool(),
BudgetKind::Provider,
"opensubtitles",
1,
allowance
)
.await
.unwrap());
}
assert_eq!(
spent_today(db.pool(), BudgetKind::Provider, "opensubtitles")
.await
.unwrap(),
5
);
let refused = try_spend(
db.pool(),
BudgetKind::Provider,
"opensubtitles",
1,
allowance,
)
.await
.unwrap();
assert!(!refused, "the sixth call is over the cap");
assert_eq!(
spent_today(db.pool(), BudgetKind::Provider, "opensubtitles")
.await
.unwrap(),
5,
"a refused spend is not recorded"
);
}
#[tokio::test]
async fn a_spend_that_would_cross_the_cap_is_refused_whole() {
let (db, _dir) = database().await;
let allowance = Some(100);
// A translator bucket counts characters, not calls, so one spend can
// be far larger than what remains.
assert!(
try_spend(db.pool(), BudgetKind::Translator, "deepl", 90, allowance)
.await
.unwrap()
);
let refused = try_spend(db.pool(), BudgetKind::Translator, "deepl", 20, allowance)
.await
.unwrap();
assert!(!refused, "90 + 20 would cross 100");
assert_eq!(
spent_today(db.pool(), BudgetKind::Translator, "deepl")
.await
.unwrap(),
90,
"the refused spend left the bucket untouched"
);
assert!(
try_spend(db.pool(), BudgetKind::Translator, "deepl", 10, allowance)
.await
.unwrap()
);
}
#[tokio::test]
async fn buckets_are_independent_per_kind_and_name() {
let (db, _dir) = database().await;
let allowance = Some(1);
assert!(try_spend(
db.pool(),
BudgetKind::Provider,
"opensubtitles",
1,
allowance
)
.await
.unwrap());
// Same name, other kind: untouched.
assert_eq!(
spent_today(db.pool(), BudgetKind::Translator, "opensubtitles")
.await
.unwrap(),
0
);
// Same kind, other name: untouched.
assert_eq!(
spent_today(db.pool(), BudgetKind::Provider, "podnapisi")
.await
.unwrap(),
0
);
}
}
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More