Compare commits

...

179 Commits

Author SHA1 Message Date
Miguel Palhas d97cdfd557 fix(subs): ask OpenSubtitles for SRT at download
ci / web (push) Successful in 35s
ci / rust (push) Successful in 1m44s
e2e / e2e (push) Failing after 1m51s
ci / image (push) Successful in 2m23s
A search entry often carries no `format`, so `format_of(None)` produced
`Other("")`, that was stashed at search time and handed back as the
download's format, and conversion had no parser to pick:

    4073669 could not be converted from  to SRT: no parser for ""

`POST /download` takes `sub_format`, and the API converts on its side, so
asking for srt makes the answer srt whatever the uploader posted. The
format guessed from the search entry is dead weight and goes with it.

The download mock matched only `file_id`, so it never exercised what the
real API returns.

Closes #272

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 10:10:06 +01:00
Miguel Palhas 7bdbecd68a fix(subs): log in to OpenSubtitles with a JSON body
ci / web (push) Successful in 56s
e2e / e2e (push) Failing after 2m1s
ci / rust (push) Successful in 2m4s
ci / image (push) Successful in 4m8s
`login` posted the credentials as HTTP basic auth with no request body.
The API takes them as JSON, answers basic auth with a 401, and the lane
reported that back as "opensubtitles rejected the configured
credentials" — for correct credentials.

Search was unaffected and hid this: it authenticates with the Api-Key
header alone and never logs in, so candidates were found and only the
download failed.

`mount_login` matched on method and path, so the mock answered a token
to any request shape. It now matches the body.

Closes #271

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 09:57:58 +01:00
Miguel Palhas 3a5e768029 fix(subs): search OpenSubtitles by the real id params
ci / web (push) Successful in 55s
e2e / e2e (push) Failing after 1m41s
ci / rust (push) Successful in 1m48s
ci / image (push) Successful in 2m22s
The id lane sent `tmdb_movie_id` and `tmdb_series_id`. Neither is a
parameter of this API, and unknown ones are ignored rather than refused,
so every search silently degraded to a `moviehash` lookup: one result for
a release someone had already hashed, none at all for anything else.

Two further requirements the API documents and answers a 301 to when
missed: parameters sorted by name, and language codes lowercase and
sorted.

The integration tests asserted the old names against a mock, which
answers whatever it is asked for. They pinned the bug rather than
catching it.

Closes #270

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 09:46:43 +01:00
Miguel Palhas 6cd2902341 feat(subs): translate from an image-only release
ci / web (push) Successful in 1m0s
e2e / e2e (push) Failing after 3m17s
ci / rust (push) Successful in 4m0s
ci / image (push) Successful in 3m56s
A release whose only subtitle is a PGS or VobSub track was stuck both
ways: the track satisfied English so no English SRT was ever fetched,
and bitmaps can never feed a translator. §15 is amended to separate
satisfying viewing from providing a translation source, and to carve a
fetch made to obtain a source out of the no-upgrade rule.

Timings come from the disc rather than from alass guessing at the audio.
At import, each non-forced image track's packet timestamps are paired
show-to-clear into a cue skeleton and stored; the fetched source is then
aligned against that skeleton, and the translation made from it skips
the post-translation pass, which could only move disc-exact timings off.

Pairing is validated before it is trusted — even packet count, plausible
durations, sane density for the runtime — because PGS allows several
composition segments per subtitle and a slipped pairing is quietly half
a second out. A track that fails validation gets no skeleton and falls
back to aligning against the video, as does any file imported before
this: there is no backfill.

Closes #268

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-30 11:53:26 +01:00
Miguel Palhas 015b90a458 fix(api): a deleted title takes its grabs with it
ci / web (push) Successful in 1m3s
e2e / e2e (push) Failing after 1m59s
ci / rust (push) Successful in 2m36s
ci / image (push) Successful in 2m20s
`grabs.infohash` is unique and grabs point at their target
polymorphically, so nothing cascaded them off a deleted movie or series.
An orphan holding an infohash then blocked the row a later grab of the
same release needed: the upsert only reclaims a `vanished` row, so the
insert did nothing while the new title was still flipped to
`downloading`. It never imported, and `/api/downloads` attributed the
torrent to a title that no longer existed.

Deletes now clear the grabs, and the upsert also reclaims a row whose
target is gone whatever its state — which heals the databases that
already have orphans. A row belonging to a live title stays off limits,
so the restart case still cannot let one title steal another's torrent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 20:34:58 +01:00
Miguel Palhas 9ecbfde656 fix(dl): only log in when qBittorrent asks
ci / web (push) Successful in 32s
e2e / e2e (push) Failing after 1m56s
ci / rust (push) Successful in 2m0s
ci / image (push) Successful in 2m27s
Logging in before the first call turned a stale or wrong password into a
hard failure on calls that would have succeeded: an instance that bypasses
authentication for arr's address answers everything, and never needed the
credentials at all. Authentication is now established on the first 403.

The health probe was reading a 403 from `app/version` as healthy, on the
reasoning that a live daemon is all the lamp claims. That is exactly the
state arr cannot use, and it left the lamp green while every grab and every
reaper tick failed on rejected credentials. It now lists torrents through
the configured client, so the lamp fails when arr's own calls do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 18:42:43 +01:00
Miguel Palhas ab4f95b166 feat(dl): replace Transmission with qBittorrent
ci / web (push) Successful in 32s
ci / rust (push) Successful in 3m26s
e2e / e2e (push) Failing after 5m2s
ci / image (push) Successful in 4m1s
qBittorrent's add call answers `Ok.` and nothing else — no hash, no name,
no duplicate signal — so arr derives the v1 infohash from the magnet or the
`.torrent` bytes before the call and looks the torrent up by it. That also
drops Transmission's numeric torrent id: the hash is the only identity now.

The reaper needs "stopped because a share limit was reached", and
qBittorrent's state field cannot tell that apart from a hand-paused torrent.
So the ratio and idle counters are checked against the limits arr set, and a
torrent stopped by a global limit reads as still seeding rather than being
deleted.

The WebUI needs a login, so `ARR_QBITTORRENT_USERNAME` and
`ARR_QBITTORRENT_PASSWORD` join the env-only secrets; leaving both unset is
valid for an instance that whitelists arr's subnet.

Verified against qBittorrent 5 (WebAPI 2.15.1) in a container: it rejects
`setShareLimits` without `shareLimitAction`, which 4.x ignores, so it is
always sent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 18:13:50 +01:00
Miguel Palhas afe167ca7b ci: enable the deploy trigger
ci / web (push) Successful in 59s
ci / rust (push) Successful in 1m47s
e2e / e2e (push) Failing after 2m7s
ci / image (push) Successful in 2m29s
Reverts the temporary docker-branch gating used to probe the runner, and
restores the deploy step now that the image build is verified.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 15:48:35 +01:00
Miguel Palhas a9848fddce ci: build the production image in CI
ci / web (push) Successful in 33s
ci / rust (push) Successful in 1m9s
ci / image (push) Successful in 4m59s
Dokploy built from source on the host, so every deploy paid a full cold
cargo build and the layer cache died with the builder. CI now builds and
pushes to the Gitea registry, and the deploy is a pull.

The Gitea push webhook fires before Actions starts, so autoDeploy on the
Dokploy application is off and this job triggers the deploy itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 15:37:50 +01:00
Miguel Palhas 4d347cd285 fix(db): restore migration 0025 to what was applied
ci / web (push) Successful in 1m8s
ci / rust (push) Successful in 2m17s
e2e / e2e (push) Failing after 2m41s
4cec2b6 edited an already-applied migration to drop podnapisi from the
subtitle_settings default. sqlx checksums migrations, so every boot since
has failed with "migration 25 was previously applied but has been
modified" — the daemon exits 1, swarm keeps the old task, and three
commits never reached production.

0033 already strips podnapisi from existing rows, and it runs after 0025
seeds row 1, so a fresh database still ends up with opensubtitles alone.
The edit changed nothing except the checksum.
2026-08-29 14:24:57 +01:00
Miguel Palhas dbce23f570 fix(web): count on-disk episodes, not just wanted
ci / web (push) Successful in 36s
ci / rust (push) Successful in 2m28s
e2e / e2e (push) Failing after 2m56s
Untracking a season clears `wanted` on every episode (§4.1) and leaves
the files where they were, so a season grabbed once and then untracked
rendered `0/0 eps` beside a column of green check glyphs. The same
counter could also read `10/1`, since the numerator was drawn from the
wanted set the denominator had already shrunk.

An episode now counts once it is wanted, on disk, or both, and the chip
is hidden rather than shown as `0/0` when a season or series accounts
for nothing. `Series.wanted_episodes` becomes `total_episodes`.
2026-08-29 12:38:26 +01:00
Miguel Palhas 706a0ec978 docs(design): refresh design.json sidecar
ci / web (push) Successful in 57s
ci / rust (push) Successful in 2m47s
e2e / e2e (push) Failing after 2m58s
status-channel didn't match the real --status-airing token, and the
score heat scale (issue 111), the available check-glyph exemption,
and the score chip component were undocumented. DESIGN.md untouched.
2026-08-26 23:23:08 +01:00
Miguel Palhas a2240debfd fix(web): drop redundant missing chip on episode rows
ci / web (push) Successful in 1m22s
e2e / e2e (push) Failing after 3m7s
ci / rust (push) Successful in 3m10s
The want button already only shows for state=="missing", so the
chip repeated the same signal. Other non-available states keep it.

Also fixes a pre-existing doc-markdown lint failure blocking CI.
2026-08-26 23:17:29 +01:00
Miguel Palhas 4cec2b6ed5 refactor(subs): remove podnapisi
ci / web (push) Successful in 30s
ci / rust (push) Failing after 1m58s
e2e / e2e (push) Failing after 2m15s
2026-08-26 22:58:01 +01:00
Miguel Palhas 3f4c1ade3b fix(web): tighten library poster grid
ci / web (push) Successful in 1m17s
ci / rust (push) Successful in 2m2s
e2e / e2e (push) Failing after 3m20s
2026-08-26 22:38:53 +01:00
Miguel Palhas 9d2b818c52 fix(web): drop the subtitles panel below the episode row
ci / web (push) Successful in 28s
ci / rust (push) Successful in 2m3s
e2e / e2e (push) Failing after 3m39s
The row and its disclosed panel were siblings in one wrapping flex line,
which only stacked them by accident of flex-basis: 100%. At phone width the
row is told not to wrap so the title truncates instead of pushing the
controls down — and the panel then had nowhere to go but beside the episode.

The row is now its own strip, .episode-line, with the panel a block under
it. Same pairing the movie file row already uses.
2026-08-26 20:12:21 +01:00
Miguel Palhas 5610e5976d fix(web): one language chip per file, not per track
ci / web (push) Successful in 35s
ci / rust (push) Successful in 1m45s
e2e / e2e (push) Failing after 3m25s
Commentary tracks carry the same language as the feature audio, so a file
row rendered `en` two or three times. The chips answer what a file can be
watched in; repeats say nothing the first said.
2026-08-26 18:42:40 +01:00
Miguel Palhas 58ad90951c fix(core): stop the size floor condemning a download
ci / web (push) Successful in 1m1s
ci / rust (push) Successful in 2m23s
e2e / e2e (push) Failing after 3m20s
The floor is a selection filter (§5.5): it keeps the ranking from picking
mud while better candidates are still on the table. Once a release is
grabbed there is nothing left to choose between, so a hard fail at import
condemns a file already on disk and blacklists the release under §5.7 —
over a number the indexer gave before the grab. Size is not post-download
evidence either; §5.6 gives ffprobe audio, HDR, codec and duration.

It bit packs hardest. Pre-grab measures a pack by its total over the
season length; import measures each file on its own, so one short episode
condemned the whole release and reopened the season. And §6.3 outlasts the
fix: a blacklisted release stays rejected however the operator later sets
allow_below_floor, with no delete path.

Post-download the floor now waives instead, whatever the overrides say.
The file imports and carries a waiver naming the floor it missed.
2026-08-26 18:34:57 +01:00
Miguel Palhas d3613c6081 Merge the Download visibility milestone
ci / web (push) Successful in 1m16s
ci / rust (push) Successful in 2m13s
e2e / e2e (push) Failing after 3m14s
Live download state, inline on the row that owns the item: DESIGN.md §9.8,
GET /api/downloads over arr's own grabs, chips on the movie, season and
episode rows, and a stale season import failure that now yields to a newer
attempt. Closes #262 #263 #264 #265 #266 #267.
2026-08-26 13:12:10 +01:00
Miguel Palhas 61d84ff51b Merge remote-tracking branch 'origin/issue/266-drop-status-chip' into blitz/download-visibility 2026-08-26 13:07:25 +01:00
Miguel Palhas 640bda3128 fix(web): show download chip on movie detail (#267) 2026-08-26 13:05:30 +01:00
Miguel Palhas 120416aabe fix(web): drop redundant status chip beside counts (#266) 2026-08-26 13:05:07 +01:00
Miguel Palhas 46554cfc75 fix(api): a vanished grab takes no headline
Superseding an import failure with any non-failed grab let a 'vanished'
one take the headline: the torrent has left Transmission, so the season
would show neither the failure nor any progress while the gap is still
there.
2026-08-26 12:55:52 +01:00
Miguel Palhas 42c1cd4dce Merge remote-tracking branch 'origin/issue/265-supersede-failure' into blitz/download-visibility 2026-08-26 12:55:20 +01:00
Miguel Palhas 77bf409c5d fix(arr): supersede stale import failure 2026-08-26 12:54:47 +01:00
Miguel Palhas eb21897c3d Merge remote-tracking branch 'origin/issue/264-download-chips' into blitz/download-visibility 2026-08-26 12:48:13 +01:00
Miguel Palhas a5b2131d4c feat(web): show live download state on rows (#264) 2026-08-26 12:47:19 +01:00
Miguel Palhas 5ba49edfd4 fix(dl): take stalled from Transmission
A zero download rate is not a stalled torrent: one between peers reads
zero for a poll or two and finishes fine, and at §9.8's 15s cadence that
flicker would raise the one chip reserved for a download that never
finishes. Transmission already decides this with its own stalled window;
carry isStalled and use it.
2026-08-26 12:27:42 +01:00
Miguel Palhas ad36a0aa83 Merge remote-tracking branch 'origin/issue/263-downloads-api' into blitz/download-visibility 2026-08-26 12:25:54 +01:00
Miguel Palhas 49d23e071a feat(arr): expose live downloads 2026-08-26 12:25:15 +01:00
Miguel Palhas 54438bb6ad docs(design): add the download surface as §9.8 2026-08-26 12:12:37 +01:00
naps62-yolo adb61dda3b feat(api): extract an embedded track on demand (#261)
ci / web (push) Successful in 40s
ci / rust (push) Successful in 3m22s
e2e / e2e (push) Failing after 3m31s
Closes #260.
2026-08-26 12:08:42 +01:00
naps62-yolo dfa4656509 fix(infra): ship alass and the translation backends (#259)
ci / web (push) Successful in 1m29s
ci / rust (push) Successful in 2m34s
e2e / e2e (push) Failing after 3m57s
2026-08-26 09:51:09 +01:00
naps62-yolo bd5131f424 fix(web): collapse subtitles into one row chip (#258)
ci / web (push) Successful in 32s
ci / rust (push) Successful in 2m26s
e2e / e2e (push) Failing after 3m58s
2026-08-26 08:58:14 +01:00
Miguel Palhas 656ca6c259 fix(api): skip mode-bit tests when they cannot bind
ci / web (push) Successful in 1m5s
ci / rust (push) Successful in 1m56s
e2e / e2e (push) Failing after 7m5s
Four tests force a filesystem failure by freezing a directory to 0o555.
The CI container runs as root, mode bits do not constrain root, and the
rename those tests expect to fail succeeds — main has been red on
a_failed_rename_leaves_the_row_alone since the move-on-root-change work
landed, with nextest's fail-fast hiding the other three.

The guard probes the filesystem rather than the uid: what the tests
depend on is the refusal, and a container can hold CAP_DAC_OVERRIDE
without being uid 0.
2026-08-26 08:35:26 +01:00
Miguel Palhas 425d154d54 Merge the Subtitles milestone
ci / web (push) Successful in 42s
ci / rust (push) Failing after 2m18s
e2e / e2e (push) Failing after 3m36s
DESIGN.md §15 end to end: provider search and download, embedded-track
extraction, translation behind four feature-gated backends, `alass` sync,
sidecar naming and the `.mt` machine-made segment, per-provider and
per-translator daily budgets, and the subtitle lane in /settings.

One sidecar per language, enforced in the schema (#222), so a forced track
never consumes the language's only slot. The OpenAI-compatible backend's
base URL and model are database rows, not bootstrap config (#220) — that
backend is any endpoint speaking the shape, `llama.cpp` included, and an
endpoint that needs no key is a valid configuration.

41 issues, plus #255 landing the provider reorder control on the shared
icon and control vocabulary.

`just ci` green: 778 tests. Migrations 0024, 0025 and 0027-0029 apply out
of order against a database that already has 0030-0032 — verified against
a database built from main's migration set, which is the shape production
is in.

CI's `rust` job is red on main independently of this merge; see #256.
2026-08-25 22:40:29 +01:00
Miguel Palhas 0eec97a7af Merge remote-tracking branch 'origin/main' into fix/255-provider-reorder
# Conflicts:
#	web/src/icons.ts
2026-08-25 22:34:55 +01:00
Miguel Palhas 7798cbc3e6 fix(arr): make the provider reorder honest
`visibility: hidden` on the disabled reorder button left a control in the
DOM that rendered as nothing. `disabled` already blocks the click and drops
it out of the tab order, so the hiding was decoration; the shared
`.control:disabled` colour says the same thing on screen, and the row does
not reflow because the hidden button reserved its box anyway.

Exposing it surfaced a second problem: `.control-quiet:hover` sits after
`.control:disabled` at the same specificity, so a disabled quiet control
lit accent-bright under the cursor and invited a click that does nothing.
Guarded with `:not(:disabled)` in the shared rule rather than on this one
control.

`refreshProviderEdges` found the buttons by `button:nth-of-type(1|2)`.
Position in the row is not a contract — a third button would have quietly
moved the edge logic onto the wrong control. They carry `data-move` now.

Closes #255
2026-08-25 22:33:40 +01:00
Miguel Palhas f3266a712b fix(web): keep episode rows on one line
ci / web (push) Successful in 45s
e2e / e2e (push) Successful in 1m44s
ci / rust (push) Failing after 2m1s
2026-08-25 22:04:34 +01:00
Miguel Palhas eeada0ef65 fix(web): keep season controls on one line
ci / web (push) Successful in 55s
e2e / e2e (push) Successful in 1m30s
ci / rust (push) Failing after 1m38s
2026-08-25 21:14:29 +01:00
Miguel Palhas 20a3b2847b Merge #251: adopt the shared icon and control vocabulary
Closes #251
2026-08-25 18:14:27 +01:00
Miguel Palhas 86af84be6d Merge #237: rewrite subtitle paths on relocate
Closes #237
2026-08-25 18:14:27 +01:00
Miguel Palhas a3186b2305 refactor(arr): chevron icons for provider order 2026-08-25 18:11:11 +01:00
Miguel Palhas 1afd8c1998 refactor(arr): subtitle chip takes shared delete
Drops the hand-rolled arm, disarm, timeout and label swap, and the local
12-grid close glyph that came with it.
2026-08-25 18:11:11 +01:00
Miguel Palhas 1e778afd18 feat(arr): compact armed delete and new glyphs
The subtitle chip needs the settings arm-then-confirm at chip scale: a
chip fits neither the control plate nor the word "confirm". The shared
control grows a compact variant that swaps the glyph instead, rather than
each view keeping a private copy of the idiom.
2026-08-25 18:11:06 +01:00
Miguel Palhas c9369d1c4f fix(arr): rewrite subtitle_files paths on relocate
relocate.rs gathered only media_files rows for a title/root move,
leaving subtitle_files rows pointing at the old folder. Sidecars
already ride along in the folder rename; only their rows were stale.

Tags each rewrite with its owning table (media_files or
subtitle_files) and updates both in the same transaction.
2026-08-25 18:10:52 +01:00
Miguel Palhas 34e5663920 Merge main into blitz/subtitles
Feedback pass 2 and the size-band work landed on main while this branch
was finishing. Brings them in ahead of the merge back.

# Conflicts:
#	crates/arr-api/src/movies.rs
#	crates/arr-api/src/state.rs
#	crates/arr-daemon/src/main.rs
#	web/src/main.ts
2026-08-25 17:53:18 +01:00
Miguel Palhas d7e9f710e0 Merge #233: stop offering a forced subtitle grab
Closes #233
2026-08-25 08:47:00 +01:00
Miguel Palhas 097e081f4d fix(arr): drop the forced flag from subtitle grab
§15 gives forced tracks no sidecar name and never lets them
satisfy a want; ranking already rejects every forced candidate
(#222). A grab endpoint accepting `forced: true` had nowhere
coherent to put the result, so the flag and its stale test are
gone (#233).
2026-08-25 08:42:56 +01:00
Miguel Palhas 1126a52bbb Merge #222: one sidecar per language, enforced in the schema
Closes #222
2026-08-25 08:35:58 +01:00
Miguel Palhas 2c5cd85499 Merge #220: configure the OpenAI-compatible endpoint and model
Closes #220
2026-08-25 08:35:54 +01:00
Miguel Palhas a45d8ff86a style(arr): give the OpenAI endpoint its own row
Five fields wrapped the model onto a line of its own and clipped the base
URL placeholder. Paired under one label, the way the translator budgets
already are, and the base URL gets the width it needs.
2026-08-25 08:33:47 +01:00
Miguel Palhas abad9b9cfd fix(arr): name the field, not a reply, in the 422
The backend's Malformed Display describes an answer that came back wrong.
Nothing is sent while validating a settings write, so only the reason
belongs in the message.
2026-08-25 08:33:47 +01:00
Miguel Palhas e467bd47fb feat(arr): add the OpenAI endpoint fields to /settings
Shown only when openai is among available_engines. Placeholders say the
base URL is optional and what a local one looks like.
2026-08-25 08:29:59 +01:00
Miguel Palhas 5aa914f81c refactor(arr): retire the OpenAI bootstrap keys
#216 added translate_openai_model as an explicit stopgap; the database row
replaces it, along with translate_openai_base_url. Only the API key stays
in the environment. deny_unknown_fields makes a config file still carrying
either one a parse error, so the move is visible rather than ignored.
2026-08-25 08:29:59 +01:00
Miguel Palhas e8bc766d4b feat(arr): expose the OpenAI endpoint on /settings
Two more fields on the subtitle settings row, validated on write — a base
URL that does not parse is a 422 naming the field — and pushed into the
cell the running backend and the health lamp both read.
2026-08-25 08:29:59 +01:00
Miguel Palhas c6906bffae feat(arr): make the OpenAI endpoint a live setting
The base URL and model move into a cell the backend re-reads per request,
so an operator can repoint it without a restart. Migration 0028 adds the
two columns; DESIGN.md §15 calls both database rows. Construction never
depends on the API key — llama.cpp serves without one.
2026-08-25 08:29:45 +01:00
Miguel Palhas d358c58844 test(arr): a forced candidate writes no sidecar
Ranking already rejects one (#185); this holds the loop to it at the end
of the pipeline, where the row would appear.
2026-08-25 08:28:56 +01:00
Miguel Palhas 3122d5b0a0 feat(arr): refuse a grab for a satisfied language
The 409 on a second subtitle for one language now comes from §15's
invariant rather than from the sidecar filename, so a provider fetch is
refused even when the language is held by a `.mt.srt` the path check
cannot see. The message names the manual delete as the way to replace
it. `claim_path` stays: two `media_files` rows for one video still
derive the same name from different ids.
2026-08-25 08:28:56 +01:00
Miguel Palhas 29c31beceb feat(arr): one sidecar per language, in the schema
DESIGN.md §15 as amended: a language is satisfied by exactly one
sidecar, and no filename segment distinguishes forced from plain from
SDH. A unique index over sidecar rows says so; embedded rows keep their
own key, since several tracks for one language can legitimately coexist
inside a video.

Existing databases may hold a duplicate from a manual grab that beat the
API's path check, so the migration resolves them rather than failing: a
real subtitle beats a machine translation, and of two of the same kind
the newest wins. The files stay on disk for the manual delete to clean
up.

`record_file` no longer swallows every conflict — only the two that mean
"arr already knows this file".
2026-08-25 08:28:51 +01:00
Miguel Palhas ab001b512f docs(arr): settle the two open §15 questions
Forced and SDH get no sidecar name of their own: one language, one
sidecar, and a forced track is ignored where a plain one exists. That
makes the database constraint fall out rather than needing a scheme.

The OpenAI-compatible backend's base URL and model become database rows.
It is not "OpenAI" — it is any endpoint speaking that shape, llama.cpp
included — so which one is in use is something to try and change, not a
property of the deployment. Only the API key stays in the environment.
2026-08-25 08:17:36 +01:00
Miguel Palhas 84dc5ba27b docs(arr): sharpen the unconfigured provider lamp
Said 'not configured'; the branch is specifically about missing bootstrap
credentials, and after #215 only a provider that needs them can reach it.
2026-08-25 06:51:11 +01:00
Miguel Palhas 60257f3177 Merge #223: offer subtitle delete in the UI
Closes #223
2026-08-25 06:49:39 +01:00
Miguel Palhas b73a58d1a4 Merge #225: cache the remote-command probe
Closes #225
2026-08-25 06:49:39 +01:00
Miguel Palhas 80101e789b Merge #215: drop the unused Podnapisi credentials
Closes #215
2026-08-25 06:49:39 +01:00
Miguel Palhas 73d3f04398 fix(arr): cache the remote-command probe verdict
Reused for 5 minutes (#225) instead of running the configured
command on every /api/health poll.
2026-08-25 06:40:50 +01:00
Miguel Palhas 5f1fffdd97 feat(arr): drop unused podnapisi credentials
Podnapisi's search and download are unauthenticated (#188), so the
ARR_PODNAPISI_USERNAME/PASSWORD fields were config nothing read. §10
keeps bootstrap config an honest list.
2026-08-25 06:40:38 +01:00
Miguel Palhas 687c0c106f feat(arr): delete affordance on subtitle chips
Adds a two-click delete icon to each present-subtitle chip, naming
the sidecar in its title/aria-label. Wires the existing DELETE
/api/subtitles/{id} endpoint and reuses the panel's refresh() so
chips and the manual panel repaint after a delete (#223).
2026-08-25 06:40:26 +01:00
Miguel Palhas 5bc9022046 fix(arr): pass the podnapisi seam to the lamp probes
#200 branched before #205's ARR_PODNAPISI_URL seam was pushed, so its new
broken::SubtitleUpstreams call site was written against the three-argument
subtitle_providers. The merge was textually clean and did not build.
2026-08-25 06:31:27 +01:00
Miguel Palhas 8043ef614a Merge #200: lamp the subtitle upstreams
Closes #200
2026-08-25 06:30:34 +01:00
Miguel Palhas 228db06d83 Merge #221: give an expired candidate a typed error
Closes #221
2026-08-25 06:30:34 +01:00
Miguel Palhas 783a6ba760 Merge #224: clear subtitle attempts on language drop
Closes #224
2026-08-25 06:30:34 +01:00
Miguel Palhas a4422e26e5 style(arr): formatting 2026-08-25 06:28:20 +01:00
Miguel Palhas a836967e32 feat(web): subtitle lane under the signal chain 2026-08-25 06:24:32 +01:00
Miguel Palhas 8c6d4ca577 feat(daemon): fold subtitle lamps into broken notifications 2026-08-25 06:12:22 +01:00
Miguel Palhas be7fa87e74 feat(api): subtitle lamps in the health report 2026-08-25 06:04:30 +01:00
Miguel Palhas 8c5613b247 feat(arr): probe methods for subtitle providers and engines 2026-08-25 05:54:19 +01:00
Miguel Palhas 2357e72113 fix(api): give an expired subtitle candidate a typed error
A grab naming a stale candidate_id now fails as ApiError::SubtitleCandidateExpired (404, code candidate_expired) instead of the generic upstream 503 string the panel had to pattern-match for 'not found'.
2026-08-25 05:35:18 +01:00
Miguel Palhas 7b4def4516 style(arr): cargo fmt 2026-08-25 05:32:25 +01:00
Miguel Palhas 3cc9ab4aff fix(arr): clear subtitle attempts on language drop
Removing a language from wanted_languages left its subtitle_attempts
rows behind, resurrecting stale backoff on re-add (#224).
2026-08-25 05:32:08 +01:00
Miguel Palhas baea1dddbb Merge #204: add the subtitle section to settings
Closes #204
2026-08-25 05:26:19 +01:00
Miguel Palhas fe00220c5b Merge #205: cover the subtitle path in arr-e2e
Closes #205
2026-08-25 05:26:19 +01:00
Miguel Palhas bc43084f74 Merge #217: sync translated subtitles with alass
Closes #217
2026-08-25 05:26:19 +01:00
Miguel Palhas 71c161eff7 ci(arr): install alass for the e2e workflow
No distro package for it; cargo install alass-cli, symlinked to the
name the wrapper actually invokes. Also builds the daemon with
translate-command and triggers on arr-subs changes.
2026-08-25 05:25:37 +01:00
Miguel Palhas 6ee79519ca test(arr-e2e): cover the subtitle path end to end
Cross-process against the real arr binary, real ffmpeg and real alass:
a provider fetch synced and named under DESIGN.md §15, never
re-searched once satisfied; an embedded English track extracted and
translated into a .mt sidecar; an implausible sync kept unsynced and
surfaced in the missing-subtitles queue. Podnapisi and the translation
backend are stubbed at the HTTP/process boundary, never a live tracker
or a live translation API.
2026-08-25 05:25:33 +01:00
Miguel Palhas e47d2c920b feat(arr-e2e): extend the harness for subtitle scenarios
Daemon::spawn_with_env for extra child environment, database_path()
and media_root() accessors so a test can seed a media file directly,
the daemon built with translate-command, and the plausible/farfetched
Podnapisi zip fixtures the subtitle tests download from.
2026-08-25 05:25:28 +01:00
Miguel Palhas 5fc6854705 feat(arr): add an env-only Podnapisi URL seam
ARR_PODNAPISI_URL, same shape as the existing tmdb_url seam: a test
harness can point the provider at a wiremock fake without touching
DESIGN.md §10's config surface.
2026-08-25 05:25:25 +01:00
Miguel Palhas da1a932459 fix(arr): correct alass reference/subtitle argument order
Syncer::run passed [subtitle, video] where alass expects
<reference-file> <incorrect-sub-file>; every real invocation failed
before comparing timings, silently degrading to SyncState::NotRun.
2026-08-25 05:25:19 +01:00
Miguel Palhas 6d665e7c2f feat(arr): render subtitle settings in /settings
Adds wanted languages, translation engine (restricted to
available_engines), remote-command timeout, and an ordered
enable/disable list per provider with daily budgets, wired to
the existing GET/PUT /api/settings/subtitles from #198.
2026-08-25 05:08:58 +01:00
Miguel Palhas 9f0a7de37a fix(arr): sync translated subtitles too 2026-08-25 04:56:35 +01:00
Miguel Palhas e0e7ddf6de docs(arr): correct the subtitle lane's comment
Said translation backends are passed empty; #216 wired them and #219 made
both callers share one set.
2026-08-25 04:55:06 +01:00
Miguel Palhas 78aabe99e5 Merge #202: add the missing-subtitles queue
Closes #202
2026-08-25 04:54:12 +01:00
Miguel Palhas a62a8de8fc Merge #219: apply the remote-command timeout setting
Closes #219
2026-08-25 04:54:12 +01:00
Miguel Palhas b3cca4f692 fix(api): drop subtitle queue gaps for unwanted languages
An attempt row survives after a language leaves wanted_languages —
missing_for (#201) already bounds by the current wanted set, so the
queue reads the same way instead of showing a stale gap forever.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-25 03:16:54 +01:00
Miguel Palhas e338a76c66 fix(arr): wire the command timeout to the settings row 2026-08-25 03:15:33 +01:00
Miguel Palhas 47c7ef6682 fix(arr): re-read the command timeout every batch 2026-08-25 03:15:33 +01:00
Miguel Palhas a780b49ab3 feat(web): missing-subtitles queue alongside no-pt-source
A third lane in the attention queues view: one row per movie or
series with a gap chip per language and why (#186's reasons, plus a
sync alass flagged). Reuses the existing deck-group markup and
missingChipLabel wording so it reads like its two siblings; each
row opens the title's own page, where the #203 fetch/translate
panel already carries the resolving actions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-25 03:11:34 +01:00
Miguel Palhas fb40b35156 feat(api): add the missing-subtitles queue endpoint
GET /api/queues/subtitles, grouped by title with why each language
is a gap (#186's attempt states, plus a sync alass rejected). Series
episodes collapse into one season row when the gap is uniform, the
same restraint §9.5 gives the TV attention queues.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-25 03:03:17 +01:00
Miguel Palhas 4003c3a5a0 fix(arr): gate the shared LLM prompt helpers
#193 moved them out of openai.rs so the command backend could share them,
but left them ungated: with neither LLM backend compiled in they are dead
code. just ci missed it because it lints the workspace with --all-features;
building arr-api alone does not enable arr-subs' features.
2026-08-25 02:50:55 +01:00
Miguel Palhas 9ca3c3ce43 Merge #203: manual subtitle search and translation
Closes #203
2026-08-25 02:49:45 +01:00
Miguel Palhas 100a869c89 Merge #218: delete subtitle sidecars with their file
Closes #218
2026-08-25 02:49:45 +01:00
Miguel Palhas d6fdd3850f Merge #197: budget provider and translator calls
Closes #197
2026-08-25 02:49:45 +01:00
Miguel Palhas 22faad66ca fix(arr): allow the candidate DTO its five bools 2026-08-25 02:49:02 +01:00
Miguel Palhas e850ddff81 feat(arr): manual subtitle fetch and translate
The manual surface §9.3 describes, applied to subtitles (§15). It opens
inline under the file's own row and borrows the release deck's layout
language rather than inventing a second one: fixed-width chips lead, the
release name is secondary, eligible shows and rejected collapses to a
count naming the rule.

The translate lane takes any subtitle already on the file as a source,
including an extracted embedded track — the thing Bazarr cannot do — and
lists only the engines this binary was compiled with.
2026-08-25 02:45:32 +01:00
Miguel Palhas 218c83589e chore(arr): refresh sqlx offline query cache
For #197's budget queries and the settings query's new columns.
2026-08-25 02:40:12 +01:00
Miguel Palhas 3a7d80c295 feat(arr): spend the subtitle budget in the reconcile loop
Providers are charged one unit per download, claimed atomically
right before the call; a provider at its cap is skipped in favour of
the next-ranked candidate rather than failing the whole gap.
Translators are charged the source character count before
translating. Either cap is a queue state, same as a provider's own
429.
2026-08-25 02:40:09 +01:00
Miguel Palhas 4d84625332 feat(arr): token bucket table for subtitle budgets
One row per provider/translator per day; try_spend is a single
atomic upsert so concurrent reconcile closes can't both slip a spend
past the daily cap. No allowance configured reads as unlimited.
2026-08-25 02:40:06 +01:00
Miguel Palhas f49507b7d1 refactor(arr): compute translator billing per call
Backend::characters_billed() polled a cumulative counter that races
under concurrent closes and can't attribute cost to one call. Drop it
in favour of the caller counting source characters before it sends
anything, which #197's budget needs anyway.
2026-08-25 02:40:02 +01:00
Miguel Palhas 9a21649afa feat(arr): report release-name match per candidate
The manual subtitle view shows the facts that decided a row (§9.3), and
release-name match is §15's second ranking tier. The file's own release
name is not otherwise on the wire, so the UI cannot derive it.
2026-08-25 02:29:50 +01:00
Miguel Palhas 2fa74137f9 fix(arr): delete subtitle sidecars with their file
remove_library_files only resolved video paths from media_files, so a
season/episode-scoped delete dropped subtitle_files rows via cascade
but left the .srt sidecars on disk (#218).
2026-08-25 02:29:13 +01:00
Miguel Palhas 7b4cff1874 fix(arr): give the reconcile loop its translators
#196 built SubtitleAction with an empty backend list, correctly: no
translation backend existed when it was written. #216 then built
translation_backends() and wired it into AppState, so the API can translate.
Merged, the reconcile loop still got Vec::new() and its translate step
reported "no engine" in every real deployment.

Neither branch was wrong alone; the gap only exists once both are in.
2026-08-25 02:21:21 +01:00
Miguel Palhas 4110555183 Merge #196: close the subtitle gap in the reconcile loop
Closes #196
2026-08-25 02:20:11 +01:00
Miguel Palhas 987bfa2864 Merge #216: wire translation backends into the daemon
Closes #216
2026-08-25 02:20:07 +01:00
Miguel Palhas 5ee5c56b88 Merge #201: show subtitle state on title detail
Closes #201
2026-08-25 02:20:07 +01:00
Miguel Palhas e684813d0c feat(arr): show subtitle chips on title detail
Per media file: a chip per present language (origin, forced/SDH, sync
flag, dashed for machine translation) and a chip per still-missing
wanted language naming why (§15, #201). Movie and series pages both
wire it in through the new status endpoints.
2026-08-25 02:16:25 +01:00
Miguel Palhas e5092034d3 feat(arr): serve subtitle status for title detail
Exposes per-media-file subtitles and missing wanted languages, with the
attempt reason, so #201's UI has one call per title (movies, episodes)
and one bulk call per series instead of one per episode.
2026-08-25 02:16:20 +01:00
Miguel Palhas c2ca25895f feat(arr): reconcile subtitle gaps in the daemon
Closes each unsatisfied wanted language per DESIGN.md §8/§15: embedded
tracks satisfy for free (recording them as #189 left to this issue),
then provider search + ranked fetch + alass sync + sidecar write, then
immediate machine translation — extracting a text-format embedded track
when that is the only source — and otherwise the reason lands on the
attempt row for the missing-subtitles queue.

Closes run as detached tasks because alass and translation outlive the
25 s reconcile action budget; every outcome is recorded in domain rows
first, so a crash converges on the next tick. Failures back off on the
same §6.2 curve as movie searches; a rate-limited provider is the
'capped' queue state; unreachable providers and translators fold into
the existing §9.5 broken notification, edge-triggered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 02:10:49 +01:00
Miguel Palhas e49bc2736d feat(arr): wire translation backends into daemon
Forward each translate-* feature from arr-daemon to arr-subs, add the
missing OpenAI model bootstrap key, and construct the compiled and
credentialed backends at startup so the translate endpoint stops
answering 503 unconditionally.
2026-08-25 02:01:55 +01:00
Miguel Palhas f1a58c0810 refactor(arr): split api_state out of run
Both #195 and #214 added a builder call to the AppState chain, which pushed
run past the too-many-lines limit. The chain grows a line per upstream the
API learns to talk to, so it gets its own function.
2026-08-25 01:50:17 +01:00
Miguel Palhas 999cb8b08c Merge #214: sync manually grabbed subtitles with alass
Closes #214

# Conflicts:
#	crates/arr-api/src/state.rs
#	crates/arr-daemon/src/main.rs
2026-08-25 01:49:20 +01:00
Miguel Palhas 0d91beff14 Merge #195: write subtitle sidecars, refresh Jellyfin
Closes #195
2026-08-25 01:48:30 +01:00
Miguel Palhas ae604e53d5 Merge #193: remote-command translation backend
Closes #193
2026-08-25 01:48:30 +01:00
Miguel Palhas 07921c9e1e test(arr): drive the command backend via stub scripts 2026-08-25 01:44:52 +01:00
Miguel Palhas 8b92a68edd feat(arr): remote-command translation backend 2026-08-25 01:44:52 +01:00
Miguel Palhas 4af6e0a083 refactor(arr): share the LLM prompt helpers across backends 2026-08-25 01:44:52 +01:00
Miguel Palhas 8baef11c0e feat(arr): refresh jellyfin after a subtitle write
DESIGN.md §7.5's watcher gap applies to a sidecar dropped next to a
file Jellyfin already knows about, same as an imported file. A grab
or translation now makes the same refresh call import does; a
refresh failure logs and never fails the write that already landed.
2026-08-25 01:31:59 +01:00
Miguel Palhas 910d28f639 refactor(arr): give arr-api its own jellyfin client
arr-daemon depends on arr-api, so a handler in arr-api can never
reach the daemon's private JellyfinClient. Move it into arr-api and
attach an instance to AppState, so a manual subtitle write can ask
for the same refresh import already does (#195).
2026-08-25 01:31:54 +01:00
Miguel Palhas f06e0e94bc feat(arr): sync manually grabbed subtitles with alass
The grab handler runs alass before recording the row, replacing the
sidecar with the synced text on acceptance and flagging it as
rejected otherwise (§15). Wires a Syncer into AppState, defaulting
to alass on PATH; the daemon binary points it at config.alass_path.
2026-08-25 01:24:09 +01:00
Miguel Palhas 635a651bee feat(arr): add Syncer::settle as the alass seam
Folds an implausible result and an unusable alass binary into one
SyncState both the grab handler (#199) and reconcile loop (#196)
can record without re-deriving the same match arms.
2026-08-25 01:24:02 +01:00
Miguel Palhas c6cfdff2c9 test(arr): read available_engines from the build
Second test that assumed the empty default feature set. It asserted
available_engines was literally [], which --all-features makes false.
2026-08-25 01:11:08 +01:00
Miguel Palhas f773489fd7 test(arr): make the engine gate test ask the build
Turning on --all-features compiled every translation backend, so the test
asserting that a known-but-uncompiled engine is refused had nothing left to
refuse and failed. It hardcoded "deepl" and a comment that no feature was
on, which stopped being true in the same commit that made CI see it.

It now picks whichever engine this build did not compile, and when all of
them are compiled asserts the complementary truth instead: a compiled
engine is accepted. Meaningful under either feature set.
2026-08-25 01:08:15 +01:00
Miguel Palhas db9ae271cb fix(arr): make CI compile the translation backends
Every translation backend sits behind a default-off cargo feature, and the
gate ran with the default set, so clippy and the test run never saw a line
of arr-subs' openai, deepl or google modules. #191 and #192 each reported
it after verifying their own work by hand. lint and test now pass
--all-features.

Also drops a duplicated wiremock suppression the #188 merge left behind and
puts the module list back in order.
2026-08-25 01:05:50 +01:00
Miguel Palhas 0f0d56ef0e Merge #192: DeepL and Google Translate backends
Closes #192
2026-08-25 01:04:51 +01:00
Miguel Palhas dd1e5d02a3 Merge #191: OpenAI-compatible translation backend
Closes #191
2026-08-25 01:04:47 +01:00
Miguel Palhas f84b04536a Merge #194: sync subtitles with alass
Closes #194
2026-08-25 01:04:43 +01:00
Miguel Palhas 3e5fdc6908 feat(arr): translate through Google Translate 2026-08-25 01:01:17 +01:00
Miguel Palhas 0c07fbaed5 feat(arr): translate through the DeepL API 2026-08-25 01:01:01 +01:00
Miguel Palhas ef0fac4933 feat(arr): let backends report billed characters 2026-08-25 01:00:41 +01:00
Miguel Palhas 82620e940c feat(arr): wrap alass for subtitle sync 2026-08-25 00:50:25 +01:00
Miguel Palhas 8efb9cbf23 feat(arr): OpenAI-compatible translation backend
Behind the existing translate-openai feature. One JSON-in/JSON-out
chat/completions request per batch; refusal, content-filter and
truncated replies fail loudly instead of validating as a short batch.
2026-08-25 00:50:06 +01:00
Miguel Palhas 26bd25e2f6 feat(arr): convert non-SRT grabs instead of refusing
#199 shipped the manual grab before #213 existed, so srt_text answered 422
for any provider not serving SRT. Fetched::to_srt already decodes and
converts, so this is that call. A format with no parser still fails rather
than reaching the disk.

Leaves the alass half of #214 open; that waits on #194.
2026-08-25 00:46:33 +01:00
Miguel Palhas 63472d99cf Merge #199: serve the subtitle API
Closes #199
2026-08-25 00:41:02 +01:00
Miguel Palhas 182285c356 Merge #213: convert non-SRT downloads to SRT
Closes #213
2026-08-25 00:41:02 +01:00
Miguel Palhas faa7a0c056 feat(arr): offer subtitle providers to the API
Provider credentials are bootstrap config and never reach the database
(DESIGN.md §10), so which providers exist is settled once at startup;
which of them a search runs is the `providers_enabled` row the API reads
per request. OpenSubtitles.com cannot be called without a registered API
key, so without one it is not offered at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-24 23:27:08 +01:00
Miguel Palhas 449750e426 feat(arr): serve the subtitle API
Issue #199, DESIGN.md §15 and §9.1. Lists what exists per media file and
per title, runs the enabled providers for one language and returns every
candidate with §9.3's verdict vocabulary — including the rejected ones
naming the rule that killed each — then grabs, translates and deletes.

Inline rather than 202-and-poll like the release deck: a subtitle search
is one or two HTTP calls and nothing persists its candidates, so there
is nothing to come back for. The cost is that a grab repeats the
`forced` and `sdh` facts the search reported, since the server does not
remember them.

Every write ends by marking the language satisfied, whether or not it is
in the wanted set. That is §15's "manual actions bypass the wanted-set
logic": the operator asking for Spanish gets Spanish, and the loop does
not then read it as a gap. A forced track is the exception §15 names — it
covers signs only — so it is recorded and satisfies nothing.

`alass` (#194) does not run yet: a fetched sidecar is recorded unsynced.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-24 23:27:03 +01:00
Miguel Palhas 0c9b9fb796 feat(arr): reopen a language on subtitle delete
DESIGN.md §15 reads satisfaction off the files, so a `satisfied` attempt
row whose sidecar was just deleted by hand is a stale claim that hides
the gap from the reconcile loop's work list. `unsatisfy` withdraws only
that claim: the attempt count and timestamp stay, because the backoff is
a fact about what providers were already asked and a delete does not
un-ask them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-24 23:26:55 +01:00
Miguel Palhas 8450b06e76 feat(arr): convert VTT and ASS downloads to SRT 2026-08-24 23:13:25 +01:00
Miguel Palhas b9f4ee98d5 fix(arr): recover lost fixtures and align provider names
The Podnapisi zip fixtures never reached the branch: a global gitignore
excludes *.zip, so git add skipped them silently and the worker's CI passed
against untracked files. Recovered, with a fixtures .gitignore that keeps
the next binary fixture from vanishing the same way.

Two providers landing in parallel also disagreed on names and on which
crate dependencies each integration-test target uses. PodnapisiProvider is
now Podnapisi, matching OpenSubtitles, and both test targets declare the
dependencies they do not use so the per-target lint stays quiet.
2026-08-24 23:07:46 +01:00
Miguel Palhas 529d7a4ee4 Merge #188: Podnapisi provider
Closes #188

# Conflicts:
#	Cargo.lock
#	crates/arr-subs/Cargo.toml
#	crates/arr-subs/src/error.rs
#	crates/arr-subs/src/lib.rs
2026-08-24 23:06:34 +01:00
Miguel Palhas ee3e2ba4b4 Merge #187: OpenSubtitles.com provider
Closes #187

# Conflicts:
#	Cargo.lock
#	crates/arr-subs/Cargo.toml
2026-08-24 23:04:46 +01:00
Miguel Palhas f617403912 Merge #212: decode fetched subtitles to UTF-8
Closes #212
2026-08-24 23:04:13 +01:00
Miguel Palhas da0c30c606 feat(arr): add opensubtitles.com subtitle provider
Searches by moviehash computed from the media file and by TMDB id
with season/episode for TV, ranks candidates through
arr_core::subs::rank, and downloads under a lazily-fetched user
token. The daily download cap (429/406) surfaces as Error::RateLimited
so the loop can show a queue state. Credentials come from config or
environment only; tests run against wiremock fixtures.
2026-08-24 22:58:48 +01:00
Miguel Palhas 6ec4703edb test(arr): cover the Podnapisi provider 2026-08-24 22:54:13 +01:00
Miguel Palhas 8311307453 feat(arr): fetch subtitles from Podnapisi 2026-08-24 22:54:13 +01:00
Miguel Palhas d8acf30fb7 feat(arr): add zip dep and provider config error 2026-08-24 22:54:13 +01:00
Miguel Palhas 81dd606414 feat(arr): decode fetched subtitles to UTF-8 2026-08-24 22:41:17 +01:00
Miguel Palhas 5d7d881ab9 feat(arr): add Error::Decode for bad subtitle bytes 2026-08-24 22:41:17 +01:00
Miguel Palhas 94d9fc8b89 Merge #190: pluggable subtitle translation
Closes #190
2026-08-24 22:36:13 +01:00
Miguel Palhas 2cee177940 Merge #198: configure subtitles via env and database
Closes #198
2026-08-24 22:36:12 +01:00
Miguel Palhas 340c113007 Merge #189: extract text subtitle streams to SRT
Closes #189
2026-08-24 22:36:12 +01:00
Miguel Palhas 0bdf0103bd feat(arr): extract text subtitle tracks to srt
ffmpeg, spawned and left to die like ffprobe, maps one subtitle stream
and converts it to SRT under §15's sidecar name. Text formats become
legal translation sources; bitmap tracks never extract.
2026-08-24 22:32:26 +01:00
Miguel Palhas e852f42a9c feat(arr): name subtitle codecs and dispositions
SubtitleTrack now carries the codec, split text formats from bitmap
ones per DESIGN.md §15, plus the forced and SDH dispositions ffprobe
reports. Without the forced flag a file carrying only a forced track
read as satisfied for that language.
2026-08-24 22:32:01 +01:00
Miguel Palhas 17d40f3e71 feat(arr): add subtitle bootstrap config 2026-08-24 22:29:18 +01:00
Miguel Palhas 802d2a1208 feat(arr): expose subtitle settings API 2026-08-24 22:28:31 +01:00
Miguel Palhas ed1a74e03f feat(arr): report compiled subtitle engines 2026-08-24 22:28:17 +01:00
Miguel Palhas 1900a82d55 feat(arr): add subtitle_settings table 2026-08-24 22:28:14 +01:00
Miguel Palhas 39ddf65afa style(arr): rustfmt arr-subs 2026-08-24 22:21:40 +01:00
Miguel Palhas e55ce05880 feat(arr): add pluggable subtitle translation layer
The Backend trait plus everything the backends (#191-#193) share, so no
backend can skip it: chunking into character-budgeted batches, rejection
of replies whose cue count or numbering drifted, and reassembly onto the
original timings. Timing data never leaves arr; pt-PT and pt-BR are
distinct targets a backend must refuse rather than conflate.
2026-08-24 22:21:09 +01:00
Miguel Palhas cd304d552c feat(arr): parse and render SRT cues in arr-subs
Translation reassembles translated text onto original timings (§15), so
cues need a structured form. Parsing is tolerant of real files (CRLF,
BOM, missing indices, dot milliseconds); rendering is strict and
renumbers from 1.
2026-08-24 22:21:09 +01:00
Miguel Palhas cdd6133bce fix(arr): close the subtitle ranking seam
#184 and #185 were built in parallel and their candidate types did not
meet. Ranking returned no identity for a candidate, so the winner of a
rank() could not be handed back to Provider::download -- rank() reorders,
so the caller could not recover it by position either.

RankedSubtitle now carries the index of the candidate in the slice it was
given, and Candidate::to_core is the one place the two shapes are mapped:
hash_match against a compared hash, sdh against hearing_impaired, group
against release_group, and the two optional tiebreakers defaulted to sort
last rather than block a candidate.
2026-08-24 22:14:01 +01:00
Miguel Palhas 44c7013e49 Merge #186: persist subtitle files and attempts
Closes #186
2026-08-24 22:06:01 +01:00
Miguel Palhas b0e3905486 Merge #185: rank subtitle candidates in arr-core
Closes #185
2026-08-24 22:06:01 +01:00
Miguel Palhas 3d2b816f9f Merge #184: add arr-subs with the provider trait
Closes #184
2026-08-24 22:06:01 +01:00
Miguel Palhas c3bd009442 feat(arr): add subtitle queries
Recording a subtitle is idempotent on both keys the schema carries: the
sidecar path, and the language an embedded track satisfies, so a second
probe or a re-import converges instead of duplicating. Attempts are
upserted per (media file, language); the work list is every language that
is not satisfied, newest import first, which is the order §15 wants the
daily allowance spent in.
2026-08-24 21:46:13 +01:00
Miguel Palhas 151135b545 feat(arr): add subtitle tables
DESIGN.md §15 needs two shapes: what subtitles exist for a media file,
and what arr has tried per wanted language. An embedded track carries no
path — it is recorded because it satisfies a language, not because there
is a file — and the CHECK constraints tie provider, engine and path to
the origin so an impossible row cannot be written.
2026-08-24 21:46:04 +01:00
Miguel Palhas 89154614e1 feat(arr): rank subtitle candidates in arr-core
Implements DESIGN.md §15 ranking: moviehash > exact release-name >
group/source > rating/downloads, forced always rejected, SDH always
below plain. Verdict mirrors the release path (eligible / rejected(rule)).
2026-08-24 21:43:46 +01:00
Miguel Palhas 8b83ac9511 feat(arr): add arr-subs with the provider trait
The skeleton the subtitles milestone hangs off: provider domain types, an
object-safe Provider trait, the crate's own error type, and the cargo
features the translation backends will sit behind. No provider, no
translation, no ranking — DESIGN.md §15 keeps ranking pure in arr-core.
2026-08-24 21:42:40 +01:00
Miguel Palhas 695cb69f2f Merge #183: DESIGN.md §15 subtitles contract
Closes #183
2026-08-24 21:38:01 +01:00
Miguel Palhas 1401e9e00f docs(arr): add DESIGN.md §15 subtitles contract
Records the subtitle decisions settled during Subtitles milestone
planning: global wanted set (pt + en), embedded-track extraction,
providers, ranking, translation backends, sidecar naming, alass sync,
budgets, and stated non-goals. Points §13 item 9 at §15 and adds
arr-subs to the crate lists.

Closes-Issue: #183

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 21:37:37 +01:00
172 changed files with 27839 additions and 1496 deletions
+66
View File
@@ -119,3 +119,69 @@ jobs:
- name: design tokens
if: steps.probe.outputs.present == 'true'
run: pnpm -C web run check-tokens
# Build the production image here rather than on the Dokploy host: the gate
# above has to pass first, the layer cache lives in the registry instead of
# being thrown away every deploy, and Dokploy's job shrinks to a pull.
#
# This job is also the deploy trigger. The Gitea push webhook fires the
# instant the commit lands, long before the image exists, so autoDeploy on
# the Dokploy application must stay off — the deploy is the last step here.
image:
needs: [rust, web]
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
env:
IMAGE: git.naps.pt/yolo/arr
DOKPLOY_APP_ID: uEaWG5JDpVIrJloG5rMAP
steps:
- uses: actions/checkout@v4
- name: Docker CLI and buildx
run: |
apt-get update
apt-get install -y --no-install-recommends ca-certificates curl gnupg
. /etc/os-release
install -m 0755 -d /etc/apt/keyrings
curl -fsSL "https://download.docker.com/linux/$ID/gpg" \
-o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc]" \
"https://download.docker.com/linux/$ID $VERSION_CODENAME stable" \
> /etc/apt/sources.list.d/docker.list
apt-get update
apt-get install -y --no-install-recommends docker-ce-cli docker-buildx-plugin
docker version
- name: Registry login
run: |
printf '%s' "${{ secrets.REGISTRY_TOKEN }}" \
| docker login git.naps.pt -u "${{ secrets.REGISTRY_USER }}" --password-stdin
- name: Buildx builder
# The default docker driver cannot export a cache. docker-container can.
run: docker buildx create --name arr --driver docker-container --use
- name: Build and push
run: |
docker buildx build \
--tag "$IMAGE:sha-$GITHUB_SHA" \
--tag "$IMAGE:main" \
--cache-from "type=registry,ref=$IMAGE:buildcache" \
--cache-to "type=registry,ref=$IMAGE:buildcache,mode=max" \
--push .
- name: Deploy
# Pinned to the commit sha, not :main — swarm does not reliably re-pull
# an unchanged tag name, and an older sha is the rollback.
run: |
# The package is public (org yolo is public), so Dokploy pulls with
# no credentials and the registry fields stay null.
curl -fsS -X POST https://dokploy.n62.casa/api/application.update \
-H "x-api-key: ${{ secrets.DOKPLOY_API_KEY }}" \
-H 'content-type: application/json' \
-d "{\"applicationId\":\"$DOKPLOY_APP_ID\",\"dockerImage\":\"$IMAGE:sha-$GITHUB_SHA\"}"
curl -fsS -X POST https://dokploy.n62.casa/api/application.deploy \
-H "x-api-key: ${{ secrets.DOKPLOY_API_KEY }}" \
-H 'content-type: application/json' \
-d "{\"applicationId\":\"$DOKPLOY_APP_ID\",\"title\":\"ci ${GITHUB_SHA:0:8}\"}"
+40 -12
View File
@@ -13,6 +13,7 @@ on:
- 'crates/arr-indexer/**'
- 'crates/arr-meta/**'
- 'crates/arr-probe/**'
- 'crates/arr-subs/**'
- 'crates/arr-api/**'
- 'crates/arr-daemon/**'
- '.gitea/workflows/e2e.yml'
@@ -25,15 +26,6 @@ jobs:
e2e:
runs-on: ubuntu-latest
services:
transmission:
image: linuxserver/transmission:latest
env:
PUID: "1000"
PGID: "1000"
ports:
- 9091:9091
steps:
- uses: actions/checkout@v4
@@ -70,12 +62,48 @@ jobs:
curl -LsSf https://get.nexte.st/latest/linux \
| tar zxf - -C "$HOME/.cargo/bin"
- name: alass
# No distro package; the subtitle sync path (DESIGN.md §15) shells
# out to a binary literally named `alass`, which the crate is not.
run: |
export PATH="$HOME/.cargo/bin:$PATH"
command -v alass >/dev/null || {
cargo install alass-cli --locked
ln -sf "$HOME/.cargo/bin/alass-cli" "$HOME/.cargo/bin/alass"
}
- name: qBittorrent
# Not a `services:` container: qBittorrent has no environment
# variable for the WebUI password and prints a random one per boot,
# so the only scriptable login is a config seeded before first start,
# which a service container cannot be given.
run: |
mkdir -p /tmp/qbt/config/qBittorrent
cat > /tmp/qbt/config/qBittorrent/qBittorrent.conf <<'CONF'
[Preferences]
WebUI\Port=8080
WebUI\AuthSubnetWhitelistEnabled=true
WebUI\AuthSubnetWhitelist=0.0.0.0/0
WebUI\CSRFProtection=false
WebUI\HostHeaderValidation=false
Downloads\SavePath=/downloads
CONF
docker run -d --name arr-e2e-qbittorrent --network host \
-e PUID=1000 -e PGID=1000 -e WEBUI_PORT=8080 \
-v /tmp/qbt/config:/config \
linuxserver/qbittorrent:latest
for _ in $(seq 1 60); do
curl -sf http://127.0.0.1:8080/api/v2/app/version && break
sleep 2
done
- name: build daemon
# The harness spawns this binary; building it here keeps the compile
# out of the first test's boot window.
run: cargo build -p arr-daemon --bin arr
# out of the first test's boot window. `translate-command` is the
# subtitle e2e scenarios' stub-script translation backend.
run: cargo build -p arr-daemon --bin arr --features translate-command
- name: e2e
env:
TRANSMISSION_RPC_URL: http://transmission:9091/transmission/rpc
QBITTORRENT_URL: http://127.0.0.1:8080
run: cargo nextest run -p arr-e2e
+45 -4
View File
@@ -1,6 +1,6 @@
{
"schemaVersion": 2,
"generatedAt": "2026-08-22T00:00:00.000Z",
"generatedAt": "2026-08-26T00:00:00.000Z",
"title": "Design System: arr — Master Control",
"extensions": {
"colorMeta": {
@@ -96,7 +96,7 @@
"oklch(0.8 0.015 250)"
]
},
"status-channel": {
"status-airing": {
"role": "tertiary",
"displayName": "Channel Violet",
"canonical": "oklch(0.78 0.14 305)",
@@ -106,6 +106,20 @@
"oklch(0.78 0.14 305)",
"oklch(0.86 0.09 305)"
]
},
"score-heat": {
"role": "tertiary",
"displayName": "Score Heat",
"canonical": "oklch(0.8 0.17 150)",
"tonalRamp": [
"oklch(0.68 0.19 25)",
"oklch(0.7 0.185 46)",
"oklch(0.72 0.175 67)",
"oklch(0.75 0.165 88)",
"oklch(0.77 0.16 108)",
"oklch(0.79 0.165 129)",
"oklch(0.8 0.17 150)"
]
}
},
"typographyMeta": {
@@ -202,11 +216,27 @@
{
"name": "Status chip",
"kind": "custom",
"refersTo": "status-channel",
"refersTo": "status-airing",
"description": "Library status (§4.2): airing / incomplete / waiting / complete / ended. Informational, not severity — colour marks activity in one channel-violet family (airing brightest, incomplete, waiting dimmest), satisfaction is neutral (complete bright ink, ended faint). The mono word is always present.",
"html": "<span class=\"chip readout\" data-status=\"airing\">airing</span>",
"css": ".chip[data-status=\"airing\"] { color: var(--status-airing); border-color: oklch(from var(--status-airing) l c h / 45%); } .chip[data-status=\"incomplete\"] { color: var(--signal-warn); border-color: oklch(from var(--signal-warn) l c h / 55%); } .chip[data-status=\"complete\"], .chip[data-status=\"ended\"] { color: var(--signal-ok); border-color: oklch(from var(--signal-ok) l c h / 45%); } .chip[data-counts=\"complete\"] { color: var(--signal-ok); } .chip[data-counts=\"partial\"] { color: var(--signal-warn); }"
},
{
"name": "Media state chip",
"kind": "custom",
"refersTo": "signal-ok",
"description": "§4.2's per-title state chip. Every state keeps the word, with one drawn exemption: `available` renders as a check glyph instead of the word `available` — a shape, not only a hue, so it survives a greyscale read the rule otherwise protects. The word moves to the chip's aria-label (exposed via role=\"img\", since a bare span's aria-label is not reliably read).",
"html": "<span class=\"chip readout\" role=\"img\" aria-label=\"available\" data-movie-state=\"available\"><svg viewBox=\"0 0 16 16\" width=\"12\" height=\"12\"><path d=\"M3.2 8.4l3 3 6.6-6.8\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"1.6\" stroke-linecap=\"round\" stroke-linejoin=\"round\"/></svg></span>",
"css": ".chip[data-movie-state=\"available\"] { color: var(--signal-ok); border-color: oklch(from var(--signal-ok) l c h / 45%); } .chip[data-movie-state=\"missing\"][data-wanted=\"true\"] { color: var(--signal-warn); border-color: oklch(from var(--signal-warn) l c h / 55%); } .chip[data-movie-state=\"downloading\"] { color: var(--status-airing); border-color: oklch(from var(--status-airing) l c h / 45%); }"
},
{
"name": "Score chip",
"kind": "custom",
"refersTo": "score-heat",
"description": "Issue 111: the release deck's score column (§9.3) reads on a list-relative heat scale — seven stops from the lowest-scoring candidate on screen (red) to the highest (green) — rather than a flat eligible-green fill. Every other column chip stays the plain neutral Chip.",
"html": "<span class=\"chip readout cw cw-score\" data-score=\"6\" aria-label=\"score 128\">128</span>",
"css": ".cw-score[data-score=\"0\"] { color: var(--score-0); border-color: oklch(from var(--score-0) l c h / 45%); } .cw-score[data-score=\"6\"] { color: var(--score-6); border-color: oklch(from var(--score-6) l c h / 45%); }"
},
{
"name": "Module (channel strip)",
"kind": "card",
@@ -290,6 +320,16 @@
"name": "The Dead-Air Rule",
"body": "When nothing is flowing, nothing animates. Pulse means probing; data-state=\"off\" is dark and still. Motion that misreports activity is a truth bug.",
"section": "motion"
},
{
"name": "The Check-Glyph Exemption",
"body": "The one state chip drawn instead of worded: `available` is a check glyph, because a drawn shape survives the greyscale read the Word-Beside-The-Lamp doctrine otherwise protects with text alone. The word is not deleted — it moves to the chip's aria-label. No other state gets this treatment; `incomplete` and `waiting` have no unambiguous glyph, so they keep the word on the chip.",
"section": "colors"
},
{
"name": "The List-Relative Heat Rule",
"body": "The release deck's score chip (issue 111) is not an absolute quality scale — its seven stops are relative to the lowest and highest score on screen right now. Colour there answers \"which of these\", not \"how good is this release in general\".",
"section": "colors"
}
],
"dos": [
@@ -297,7 +337,8 @@
"Do keep every control ≥44px, focus rings cyan and visible, and reduced-motion honored.",
"Do keep muted text at or above 4.5:1 on panel ground (--ink-muted floor).",
"Do self-host every asset; the SPA is embedded in the binary and must work with no network.",
"Do take every colour, spacing and font-size from the :root tokens; the token check (web/scripts/check-tokens.mjs) fails CI on literals."
"Do take every colour, spacing and font-size from the :root tokens; the token check (web/scripts/check-tokens.mjs) fails CI on literals.",
"Do suppress a redundant status chip where another control on the same row already implies the exact same state one-to-one — the missing chip disappears from episode rows because the want button only ever renders for that state."
],
"donts": [
"Colour state chips on the ramp — `complete` and `ended` green, `incomplete` and a partial count amber, `airing` and `downloading` violet, `waiting` and `untracked` neutral. The word stays in every chip, so state survives with colour removed.",
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\", path AS \"path!: String\"\n FROM media_files WHERE owner_kind = 'movie' AND owner_id = ?",
"query": "SELECT id AS \"id!: i64\", path AS \"path!: String\"\n FROM media_files WHERE owner_kind = 'movie' AND owner_id = ?",
"describe": {
"columns": [
{
@@ -34,5 +34,5 @@
false
]
},
"hash": "3be9c350ef24a69490540379225d95d2949fe2a0f1f59d55d6272cca1bd431aa"
"hash": "028ba37575a1081f7678fd8649227f39c29afc1883e3a3607260455d1a088143"
}
@@ -0,0 +1,38 @@
{
"db_name": "SQLite",
"query": "SELECT r.name AS \"name!: String\",\n r.parsed AS \"parsed!: serde_json::Value\"\n FROM episodes e\n JOIN grabs g ON g.target_kind = 'season' AND g.target_id = e.season_id\n JOIN releases r ON r.id = g.release_id\n WHERE e.id = ?\n ORDER BY g.imported_at DESC, g.id DESC LIMIT 1",
"describe": {
"columns": [
{
"name": "name!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "releases",
"name": "name"
}
}
},
{
"name": "parsed!: serde_json::Value",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "releases",
"name": "parsed"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false
]
},
"hash": "05b5757f0fb7e6ab1176387854a14fc8db793871193b6f16e3719d73040e8ce8"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_attempts\n (media_file_id, language, state, attempts, last_attempt_at, last_failure)\n VALUES (?, ?, ?, 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now'), ?)\n ON CONFLICT (media_file_id, language) DO UPDATE\n SET state = excluded.state,\n attempts = subtitle_attempts.attempts + 1,\n last_attempt_at = excluded.last_attempt_at,\n last_failure = excluded.last_failure,\n updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now')",
"describe": {
"columns": [],
"parameters": {
"Right": 4
},
"nullable": []
},
"hash": "08c356bfcd7f7cd7a4bac0bf0ce44ce3276577cb18f32b0c01839150e7827bca"
}
@@ -0,0 +1,21 @@
{
"db_name": "SQLite",
"query": "SELECT EXISTS(SELECT 1 FROM episodes WHERE id = ?) AS \"exists!: bool\"",
"describe": {
"columns": [
{
"name": "exists!: bool",
"ordinal": 0,
"type_info": "Integer",
"origin": "Expression"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "0b40dbc80628244531a044e872238e356ab6bc66567c72008cff506e595932b1"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT g.target_id AS \"season_id!: i64\",\n r.name AS \"name!: String\",\n g.infohash AS \"infohash!: String\",\n g.failed_at,\n g.grabbed_at AS \"grabbed_at!: String\"\n FROM grabs g\n JOIN releases r ON r.id = g.release_id\n JOIN seasons s ON s.id = g.target_id\n WHERE g.target_kind = 'season'\n AND g.state = 'failed'\n AND s.series_id = ?\n AND EXISTS (\n SELECT 1 FROM episodes e\n WHERE e.season_id = s.id AND e.wanted\n AND NOT EXISTS (\n SELECT 1 FROM media_files f\n WHERE f.owner_kind = 'episode' AND f.owner_id = e.id\n )\n )\n ORDER BY coalesce(g.failed_at, g.grabbed_at), g.id",
"query": "SELECT g.target_id AS \"season_id!: i64\",\n r.name AS \"name!: String\",\n g.infohash AS \"infohash!: String\",\n g.failed_at,\n g.grabbed_at AS \"grabbed_at!: String\"\n FROM grabs g\n JOIN releases r ON r.id = g.release_id\n JOIN seasons s ON s.id = g.target_id\n WHERE g.target_kind = 'season'\n AND g.state = 'failed'\n AND s.series_id = ?\n AND EXISTS (\n SELECT 1 FROM episodes e\n WHERE e.season_id = s.id AND e.wanted\n AND NOT EXISTS (\n SELECT 1 FROM media_files f\n WHERE f.owner_kind = 'episode' AND f.owner_id = e.id\n )\n )\n -- A later attempt owns the headline while it is still in\n -- flight. Keep the failed grab recorded; it is simply no\n -- longer the current explanation for the gap.\n AND NOT EXISTS (\n SELECT 1 FROM grabs newer\n WHERE newer.target_kind = 'season'\n AND newer.target_id = g.target_id\n -- 'vanished' is not a live attempt either: the\n -- torrent left qBittorrent, so letting it take the\n -- headline would leave the season saying nothing at\n -- all about a gap that still exists.\n AND newer.state NOT IN ('failed', 'vanished')\n AND (newer.grabbed_at > coalesce(g.failed_at, g.grabbed_at)\n OR (newer.grabbed_at = coalesce(g.failed_at, g.grabbed_at)\n AND newer.id > g.id))\n )\n ORDER BY coalesce(g.failed_at, g.grabbed_at), g.id",
"describe": {
"columns": [
{
@@ -70,5 +70,5 @@
false
]
},
"hash": "3f252a992c1bc5b18bb4467d1c4fd4137966a469b134fbeb51fb91f67951cbbe"
"hash": "0cfce0fe064cccb1dceb285a50fa77d990b116a95b987e604913cd8953da36ad"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT wanted_languages FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "wanted_languages",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "wanted_languages"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false
]
},
"hash": "0f94b0f839cfcbe4a70ee5be81f452c88dc610d05042d4af811963c0cbc6113e"
}
@@ -0,0 +1,21 @@
{
"db_name": "SQLite",
"query": "SELECT EXISTS(SELECT 1 FROM series WHERE id = ?) AS \"exists!: bool\"",
"describe": {
"columns": [
{
"name": "exists!: bool",
"ordinal": 0,
"type_info": "Integer",
"origin": "Expression"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "2084928704dbb8a2c81692b9aa3b6edff64745945ea4e88d70bcb139e272c857"
}
@@ -0,0 +1,21 @@
{
"db_name": "SQLite",
"query": "SELECT EXISTS(\n SELECT 1 FROM subtitle_files\n WHERE media_file_id = ? AND language = ? AND forced = 0\n ) AS \"exists!: bool\"",
"describe": {
"columns": [
{
"name": "exists!: bool",
"ordinal": 0,
"type_info": "Integer",
"origin": "Expression"
}
],
"parameters": {
"Right": 2
},
"nullable": [
false
]
},
"hash": "233bae03912e50e6bcde4b24a9d7fbe256dbf626e4e236a06ba08a00643566d3"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_files\n (media_file_id, language, origin, provider, candidate_id, engine,\n forced, sdh, synced, sync_rejected, skeleton_aligned, path)\n VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\n ON CONFLICT (path) DO NOTHING\n ON CONFLICT (media_file_id, language, forced, sdh)\n WHERE origin = 'embedded' DO NOTHING\n RETURNING id AS \"id!: i64\"",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "id"
}
}
}
],
"parameters": {
"Right": 12
},
"nullable": [
null
]
},
"hash": "266c8e63e7f81cb07921de180f32b47ac8c4e9d47a31af786742404f8e54f517"
}
@@ -0,0 +1,50 @@
{
"db_name": "SQLite",
"query": "SELECT sr.tmdb_id AS \"tmdb_id!: i64\",\n s.number AS \"season!: i64\",\n e.number AS \"episode!: i64\"\n FROM episodes e\n JOIN seasons s ON s.id = e.season_id\n JOIN series sr ON sr.id = s.series_id\n WHERE e.id = ?",
"describe": {
"columns": [
{
"name": "tmdb_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "tmdb_id"
}
}
},
{
"name": "season!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "seasons",
"name": "number"
}
}
},
{
"name": "episode!: i64",
"ordinal": 2,
"type_info": "Integer",
"origin": {
"Table": {
"table": "episodes",
"name": "number"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false
]
},
"hash": "268cb11282b16494330c1655ba3dfcb4f2506686822711f3c6489c8ddd42b0e7"
}
@@ -0,0 +1,38 @@
{
"db_name": "SQLite",
"query": "SELECT sf.id AS \"id!: i64\", sf.path AS \"path!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id\n WHERE mf.owner_kind = 'movie' AND mf.owner_id = ? AND sf.path IS NOT NULL",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "id"
}
}
},
{
"name": "path!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
true
]
},
"hash": "295476f5ee76f552a0d0727fca01f7602111b1dc5614bf2f8c8626744ecb22db"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "INSERT INTO grabs (release_id, target_kind, target_id, infohash, state)\n VALUES (?, ?, ?, ?, 'sent')\n ON CONFLICT (infohash) DO UPDATE SET\n release_id = excluded.release_id,\n target_kind = excluded.target_kind,\n target_id = excluded.target_id,\n state = 'sent',\n grabbed_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now'),\n imported_at = NULL\n WHERE grabs.state = 'vanished'\n RETURNING id AS \"id!: i64\"",
"query": "INSERT INTO grabs (release_id, target_kind, target_id, infohash, state)\n VALUES (?, ?, ?, ?, 'sent')\n ON CONFLICT (infohash) DO UPDATE SET\n release_id = excluded.release_id,\n target_kind = excluded.target_kind,\n target_id = excluded.target_id,\n state = 'sent',\n grabbed_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now'),\n imported_at = NULL\n WHERE grabs.state = 'vanished'\n OR (grabs.target_kind = 'movie'\n AND NOT EXISTS (SELECT 1 FROM movies WHERE id = grabs.target_id))\n OR (grabs.target_kind = 'season'\n AND NOT EXISTS (SELECT 1 FROM seasons WHERE id = grabs.target_id))\n OR (grabs.target_kind = 'episode'\n AND NOT EXISTS (SELECT 1 FROM episodes WHERE id = grabs.target_id))\n RETURNING id AS \"id!: i64\"",
"describe": {
"columns": [
{
@@ -22,5 +22,5 @@
null
]
},
"hash": "2d0b92ba9aa4bc257286a67f0d37e2182c4478153c9096c1522c5ab23ad472e9"
"hash": "2aeb5f9dab4a62d4cc463a05c0636c379b9c5a1cebee9dfa61df2162f2c64383"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT wanted_languages AS \"wanted_languages!: String\" FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "wanted_languages!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "wanted_languages"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false
]
},
"hash": "2afeefde38499d722f329d1d472bc89524ce6498ccd4098ba97979e0621b840b"
}
@@ -0,0 +1,122 @@
{
"db_name": "SQLite",
"query": "SELECT m.id AS \"movie_id!: i64\", m.tmdb_id AS \"tmdb_id!: i64\",\n m.title AS \"title!: String\", m.year, m.poster_path,\n mf.id AS \"media_file_id!: i64\", sa.language AS \"language!: String\",\n sa.state AS \"state!: arr_db::SubtitleState\", sa.last_failure\n FROM subtitle_attempts sa\n JOIN media_files mf ON mf.id = sa.media_file_id AND mf.owner_kind = 'movie'\n JOIN movies m ON m.id = mf.owner_id\n WHERE sa.state IN ('failed', 'capped', 'unavailable')\n ORDER BY m.title, sa.language",
"describe": {
"columns": [
{
"name": "movie_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "id"
}
}
},
{
"name": "tmdb_id!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "tmdb_id"
}
}
},
{
"name": "title!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "movies",
"name": "title"
}
}
},
{
"name": "year",
"ordinal": 3,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "year"
}
}
},
{
"name": "poster_path",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "movies",
"name": "poster_path"
}
}
},
{
"name": "media_file_id!: i64",
"ordinal": 5,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
},
{
"name": "language!: String",
"ordinal": 6,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "language"
}
}
},
{
"name": "state!: arr_db::SubtitleState",
"ordinal": 7,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "state"
}
}
},
{
"name": "last_failure",
"ordinal": 8,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "last_failure"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
false,
false,
true,
true,
false,
false,
false,
true
]
},
"hash": "2b67abe4db9e00517fecfa53fbad5bbbc633a9f537eaaaa9d46a445d8c25eb2c"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "UPDATE subtitle_attempts\n SET state = 'wanted',\n updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now')\n WHERE media_file_id = ? AND language = ? AND state = 'satisfied'",
"describe": {
"columns": [],
"parameters": {
"Right": 2
},
"nullable": []
},
"hash": "31bed4c9097d6ea2ed189de48440bd0d882873d27450e5d177baed65c74fd7de"
}
@@ -0,0 +1,74 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\",\n path AS \"path!: String\",\n size AS \"size!: i64\",\n owner_kind AS \"owner_kind!: String\",\n owner_id AS \"owner_id!: i64\"\n FROM media_files WHERE id = ?",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
},
{
"name": "path!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "media_files",
"name": "path"
}
}
},
{
"name": "size!: i64",
"ordinal": 2,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "size"
}
}
},
{
"name": "owner_kind!: String",
"ordinal": 3,
"type_info": "Text",
"origin": {
"Table": {
"table": "media_files",
"name": "owner_kind"
}
}
},
{
"name": "owner_id!: i64",
"ordinal": 4,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "owner_id"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
false,
false
]
},
"hash": "32443dc6e5bdd414340443c5d527b2578bcdeebd053b667db35423d829d4bf10"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_attempts (media_file_id, language)\n SELECT id, ? FROM media_files WHERE probed IS NOT NULL\n ON CONFLICT (media_file_id, language) DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Right": 1
},
"nullable": []
},
"hash": "35ace0417f613340f82522549d70d0dffeb680afb57511cb17fa15b3a571b974"
}
@@ -0,0 +1,38 @@
{
"db_name": "SQLite",
"query": "SELECT r.name AS \"name!: String\",\n r.parsed AS \"parsed!: serde_json::Value\"\n FROM grabs g JOIN releases r ON r.id = g.release_id\n WHERE g.target_kind = ? AND g.target_id = ?\n ORDER BY g.imported_at DESC, g.id DESC LIMIT 1",
"describe": {
"columns": [
{
"name": "name!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "releases",
"name": "name"
}
}
},
{
"name": "parsed!: serde_json::Value",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "releases",
"name": "parsed"
}
}
}
],
"parameters": {
"Right": 2
},
"nullable": [
false,
false
]
},
"hash": "432cf38775bc4d7dee1dd9d009e06638ea57f805127f7634f34b8189e831f461"
}
@@ -0,0 +1,98 @@
{
"db_name": "SQLite",
"query": "SELECT a.media_file_id AS \"media_file_id!: i64\",\n a.language AS \"language!: String\",\n a.state AS \"state!: SubtitleState\",\n a.attempts AS \"attempts!: i64\",\n a.last_attempt_at,\n a.last_failure,\n f.path AS \"media_file_path!: String\"\n FROM subtitle_attempts a\n JOIN media_files f ON f.id = a.media_file_id\n WHERE a.state IN ('wanted', 'failed')\n ORDER BY f.created_at DESC, a.last_attempt_at, a.language\n LIMIT ?",
"describe": {
"columns": [
{
"name": "media_file_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "media_file_id"
}
}
},
{
"name": "language!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "language"
}
}
},
{
"name": "state!: SubtitleState",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "state"
}
}
},
{
"name": "attempts!: i64",
"ordinal": 3,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "attempts"
}
}
},
{
"name": "last_attempt_at",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "last_attempt_at"
}
}
},
{
"name": "last_failure",
"ordinal": 5,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "last_failure"
}
}
},
{
"name": "media_file_path!: String",
"ordinal": 6,
"type_info": "Text",
"origin": {
"Table": {
"table": "media_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
false,
true,
true,
false
]
},
"hash": "47ad9162ba373dfbdf061898f5f148e65de261b494453804db912a919a78c2c5"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT tmdb_id AS \"tmdb_id!: i64\" FROM movies WHERE id = ?",
"describe": {
"columns": [
{
"name": "tmdb_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "tmdb_id"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "4f4f037c956ac6b246503354ff9b59db59e36f9a6d59adc0753b95662c9c630a"
}
@@ -0,0 +1,38 @@
{
"db_name": "SQLite",
"query": "SELECT sf.id AS \"id!: i64\", sf.path AS \"path!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id\n JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id\n JOIN seasons se ON se.id = e.season_id\n WHERE se.series_id = ? AND sf.path IS NOT NULL",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "id"
}
}
},
{
"name": "path!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
true
]
},
"hash": "5153426ad8b36b6a2b8c17942dfedf02b1e1f91482c8784357d2ef61c9d160c1"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT sf.path AS \"path!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id\n JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id\n JOIN seasons se ON se.id = e.season_id\n WHERE se.series_id = ? AND sf.path IS NOT NULL",
"describe": {
"columns": [
{
"name": "path!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
true
]
},
"hash": "5f3a663eef0318b38cd864a2e06befa20fba22fc2d281295aed81a851a35ea07"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "DELETE FROM grabs WHERE target_kind = 'movie' AND target_id = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 1
},
"nullable": []
},
"hash": "607f337ce8db7eb277dfc569fb1a68e956a38f1db4462cab02fe26c8c9912a81"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_attempts (media_file_id, language)\n VALUES (?, ?)\n ON CONFLICT (media_file_id, language) DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Right": 2
},
"nullable": []
},
"hash": "6236e1be0e268c678271dd72be52454347da6637f0bf3ecb52dda82cddfe459a"
}
@@ -0,0 +1,98 @@
{
"db_name": "SQLite",
"query": "SELECT m.id AS \"movie_id!: i64\", m.tmdb_id AS \"tmdb_id!: i64\",\n m.title AS \"title!: String\", m.year, m.poster_path,\n mf.id AS \"media_file_id!: i64\", sf.language AS \"language!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id AND mf.owner_kind = 'movie'\n JOIN movies m ON m.id = mf.owner_id\n WHERE sf.sync_rejected = 1\n ORDER BY m.title, sf.language",
"describe": {
"columns": [
{
"name": "movie_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "id"
}
}
},
{
"name": "tmdb_id!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "tmdb_id"
}
}
},
{
"name": "title!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "movies",
"name": "title"
}
}
},
{
"name": "year",
"ordinal": 3,
"type_info": "Integer",
"origin": {
"Table": {
"table": "movies",
"name": "year"
}
}
},
{
"name": "poster_path",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "movies",
"name": "poster_path"
}
}
},
{
"name": "media_file_id!: i64",
"ordinal": 5,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
},
{
"name": "language!: String",
"ordinal": 6,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "language"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
false,
false,
true,
true,
false,
false
]
},
"hash": "69f34468a61979b85b7513056d58e34c743af3ae62d5327d4dea49fe7586a85b"
}
@@ -0,0 +1,158 @@
{
"db_name": "SQLite",
"query": "SELECT s.id AS \"series_id!: i64\", s.tmdb_id AS \"tmdb_id!: i64\",\n s.title AS \"title!: String\", s.year, s.poster_path,\n e.id AS \"episode_id!: i64\", se.number AS \"season_number!: i64\",\n e.number AS \"episode_number!: i64\", mf.id AS \"media_file_id!: i64\",\n sa.language AS \"language!: String\",\n sa.state AS \"state!: arr_db::SubtitleState\", sa.last_failure\n FROM subtitle_attempts sa\n JOIN media_files mf ON mf.id = sa.media_file_id AND mf.owner_kind = 'episode'\n JOIN episodes e ON e.id = mf.owner_id\n JOIN seasons se ON se.id = e.season_id\n JOIN series s ON s.id = se.series_id\n WHERE sa.state IN ('failed', 'capped', 'unavailable')\n ORDER BY s.title, se.number, e.number, sa.language",
"describe": {
"columns": [
{
"name": "series_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "id"
}
}
},
{
"name": "tmdb_id!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "tmdb_id"
}
}
},
{
"name": "title!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "series",
"name": "title"
}
}
},
{
"name": "year",
"ordinal": 3,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "year"
}
}
},
{
"name": "poster_path",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "series",
"name": "poster_path"
}
}
},
{
"name": "episode_id!: i64",
"ordinal": 5,
"type_info": "Integer",
"origin": {
"Table": {
"table": "episodes",
"name": "id"
}
}
},
{
"name": "season_number!: i64",
"ordinal": 6,
"type_info": "Integer",
"origin": {
"Table": {
"table": "seasons",
"name": "number"
}
}
},
{
"name": "episode_number!: i64",
"ordinal": 7,
"type_info": "Integer",
"origin": {
"Table": {
"table": "episodes",
"name": "number"
}
}
},
{
"name": "media_file_id!: i64",
"ordinal": 8,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
},
{
"name": "language!: String",
"ordinal": 9,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "language"
}
}
},
{
"name": "state!: arr_db::SubtitleState",
"ordinal": 10,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "state"
}
}
},
{
"name": "last_failure",
"ordinal": 11,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "last_failure"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
false,
false,
true,
true,
false,
false,
false,
false,
false,
false,
true
]
},
"hash": "6cdb9a9344bbab557433da35471968b6e5cf96b5b220ffb41f10b6d851884803"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_budget_spend (kind, name, day, spent)\n SELECT ?1, ?2, strftime('%Y-%m-%d', 'now'), ?3\n WHERE ?3 <= ?4\n ON CONFLICT (kind, name, day) DO UPDATE\n SET spent = subtitle_budget_spend.spent + excluded.spent\n WHERE subtitle_budget_spend.spent + excluded.spent <= ?4\n RETURNING spent AS \"spent!: i64\"",
"describe": {
"columns": [
{
"name": "spent!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_budget_spend",
"name": "spent"
}
}
}
],
"parameters": {
"Right": 4
},
"nullable": [
false
]
},
"hash": "6e27f3a4c642ac324ef5ae2017a54b93f1d95aa54c46ff7e169278a295d1a2cc"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "DELETE FROM subtitle_attempts WHERE language = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 1
},
"nullable": []
},
"hash": "74178ac3ec41aa8a5ce164103a8d8301cb555eea2fa5d655a7365fd02c557cce"
}
@@ -0,0 +1,74 @@
{
"db_name": "SQLite",
"query": "SELECT wanted_languages AS \"wanted_languages!: String\",\n providers_enabled AS \"providers_enabled!: String\",\n translation_engine AS \"translation_engine: String\",\n provider_daily_budgets AS \"provider_daily_budgets!: String\",\n translator_daily_budgets AS \"translator_daily_budgets!: String\"\n FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "wanted_languages!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "wanted_languages"
}
}
},
{
"name": "providers_enabled!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "providers_enabled"
}
}
},
{
"name": "translation_engine: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "translation_engine"
}
}
},
{
"name": "provider_daily_budgets!: String",
"ordinal": 3,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "provider_daily_budgets"
}
}
},
{
"name": "translator_daily_budgets!: String",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "translator_daily_budgets"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
false,
true,
false,
false
]
},
"hash": "77e55f2aa534468cf6e599d0c01679653553a68f2f1b9b5516f5d2fb42974b05"
}
@@ -0,0 +1,50 @@
{
"db_name": "SQLite",
"query": "SELECT remote_command_timeout_seconds AS \"remote_command_timeout_seconds!: i64\",\n openai_base_url AS \"openai_base_url: String\",\n openai_model AS \"openai_model: String\"\n FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "remote_command_timeout_seconds!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "remote_command_timeout_seconds"
}
}
},
{
"name": "openai_base_url: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "openai_base_url"
}
}
},
{
"name": "openai_model: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "openai_model"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
true,
true
]
},
"hash": "7c501e25106ea8d4ad9f28ca904c446551c6afd68d85e2231742bebd215a1a4b"
}
@@ -0,0 +1,110 @@
{
"db_name": "SQLite",
"query": "SELECT wanted_languages AS \"wanted_languages!: String\",\n providers_enabled AS \"providers_enabled!: String\",\n translation_engine AS \"translation_engine: String\",\n provider_daily_budgets AS \"provider_daily_budgets!: String\",\n translator_daily_budgets AS \"translator_daily_budgets!: String\",\n remote_command_timeout_seconds AS \"remote_command_timeout_seconds!: i64\",\n openai_base_url AS \"openai_base_url: String\",\n openai_model AS \"openai_model: String\"\n FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "wanted_languages!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "wanted_languages"
}
}
},
{
"name": "providers_enabled!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "providers_enabled"
}
}
},
{
"name": "translation_engine: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "translation_engine"
}
}
},
{
"name": "provider_daily_budgets!: String",
"ordinal": 3,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "provider_daily_budgets"
}
}
},
{
"name": "translator_daily_budgets!: String",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "translator_daily_budgets"
}
}
},
{
"name": "remote_command_timeout_seconds!: i64",
"ordinal": 5,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "remote_command_timeout_seconds"
}
}
},
{
"name": "openai_base_url: String",
"ordinal": 6,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "openai_base_url"
}
}
},
{
"name": "openai_model: String",
"ordinal": 7,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "openai_model"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
false,
true,
false,
false,
false,
true,
true
]
},
"hash": "82bb36910db14d5471117285ef2d3a9f8efefc5d400a8f94e1f4aed321284504"
}
@@ -0,0 +1,134 @@
{
"db_name": "SQLite",
"query": "SELECT s.id AS \"series_id!: i64\", s.tmdb_id AS \"tmdb_id!: i64\",\n s.title AS \"title!: String\", s.year, s.poster_path,\n e.id AS \"episode_id!: i64\", se.number AS \"season_number!: i64\",\n e.number AS \"episode_number!: i64\", mf.id AS \"media_file_id!: i64\",\n sf.language AS \"language!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id AND mf.owner_kind = 'episode'\n JOIN episodes e ON e.id = mf.owner_id\n JOIN seasons se ON se.id = e.season_id\n JOIN series s ON s.id = se.series_id\n WHERE sf.sync_rejected = 1\n ORDER BY s.title, se.number, e.number, sf.language",
"describe": {
"columns": [
{
"name": "series_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "id"
}
}
},
{
"name": "tmdb_id!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "tmdb_id"
}
}
},
{
"name": "title!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "series",
"name": "title"
}
}
},
{
"name": "year",
"ordinal": 3,
"type_info": "Integer",
"origin": {
"Table": {
"table": "series",
"name": "year"
}
}
},
{
"name": "poster_path",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "series",
"name": "poster_path"
}
}
},
{
"name": "episode_id!: i64",
"ordinal": 5,
"type_info": "Integer",
"origin": {
"Table": {
"table": "episodes",
"name": "id"
}
}
},
{
"name": "season_number!: i64",
"ordinal": 6,
"type_info": "Integer",
"origin": {
"Table": {
"table": "seasons",
"name": "number"
}
}
},
{
"name": "episode_number!: i64",
"ordinal": 7,
"type_info": "Integer",
"origin": {
"Table": {
"table": "episodes",
"name": "number"
}
}
},
{
"name": "media_file_id!: i64",
"ordinal": 8,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
},
{
"name": "language!: String",
"ordinal": 9,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "language"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false,
false,
false,
true,
true,
false,
false,
false,
false,
false
]
},
"hash": "8543e4ad8c59e83911688ceace72f66d360240a45eee05aaf15dab233e22e7e6"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\" FROM media_files WHERE path = ?",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "8651688f9f999629b4996c0451f87f5a84fabe748849b4b80063af1c18f73917"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "UPDATE subtitle_files\n SET synced = ?, sync_rejected = ?,\n updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now')\n WHERE id = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 3
},
"nullable": []
},
"hash": "874079e2ed732468a1f249e6aa026753c78042e164d11eb7cc014c9500211b64"
}
@@ -0,0 +1,170 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\",\n media_file_id AS \"media_file_id!: i64\",\n language AS \"language!: String\",\n origin AS \"origin!: SubtitleOrigin\",\n provider,\n candidate_id,\n engine,\n forced AS \"forced!: bool\",\n sdh AS \"sdh!: bool\",\n synced AS \"synced!: bool\",\n sync_rejected AS \"sync_rejected!: bool\",\n skeleton_aligned AS \"skeleton_aligned!: bool\",\n path\n FROM subtitle_files\n WHERE media_file_id = ?\n ORDER BY language, id",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "id"
}
}
},
{
"name": "media_file_id!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "media_file_id"
}
}
},
{
"name": "language!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "language"
}
}
},
{
"name": "origin!: SubtitleOrigin",
"ordinal": 3,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "origin"
}
}
},
{
"name": "provider",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "provider"
}
}
},
{
"name": "candidate_id",
"ordinal": 5,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "candidate_id"
}
}
},
{
"name": "engine",
"ordinal": 6,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "engine"
}
}
},
{
"name": "forced!: bool",
"ordinal": 7,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "forced"
}
}
},
{
"name": "sdh!: bool",
"ordinal": 8,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "sdh"
}
}
},
{
"name": "synced!: bool",
"ordinal": 9,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "synced"
}
}
},
{
"name": "sync_rejected!: bool",
"ordinal": 10,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "sync_rejected"
}
}
},
{
"name": "skeleton_aligned!: bool",
"ordinal": 11,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "skeleton_aligned"
}
}
},
{
"name": "path",
"ordinal": 12,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
false,
true,
true,
true,
false,
false,
false,
false,
false,
true
]
},
"hash": "8abb1e992e0972a19e76c7788c8c350951a1d14766088b1cf3fb886fb658d575"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_skeletons (media_file_id, stream_index, language, cues)\n VALUES (?, ?, ?, ?)\n ON CONFLICT (media_file_id, stream_index) DO UPDATE SET\n language = excluded.language,\n cues = excluded.cues",
"describe": {
"columns": [],
"parameters": {
"Right": 4
},
"nullable": []
},
"hash": "8e3c23ed488bc82ff3dd4f6da02d00435525d321b6002b9f31979a39281369ce"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "DELETE FROM subtitle_files WHERE id = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 1
},
"nullable": []
},
"hash": "90480a6b6e724d7e4d05239799ad2f586c57deea100320fe433da007396f1e48"
}
@@ -0,0 +1,50 @@
{
"db_name": "SQLite",
"query": "SELECT media_file_id AS \"media_file_id!: i64\",\n language AS \"language!: String\",\n path\n FROM subtitle_files WHERE id = ?",
"describe": {
"columns": [
{
"name": "media_file_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "media_file_id"
}
}
},
{
"name": "language!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "language"
}
}
},
{
"name": "path",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
true
]
},
"hash": "9091de460ba1cb47c9faa24fe37fee03615e7b23b10ceef88ee12c8deb0dbaaf"
}
@@ -0,0 +1,38 @@
{
"db_name": "SQLite",
"query": "SELECT sf.id AS \"id!: i64\", sf.path AS \"path!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id\n JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id\n JOIN seasons se ON se.id = e.season_id\n JOIN series s ON s.id = se.series_id\n WHERE s.root_id = ? AND sf.path IS NOT NULL\n ORDER BY sf.id",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "id"
}
}
},
{
"name": "path!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
true
]
},
"hash": "92a3d21df1ee60492a1e0742f1425c73a16ee7dd38bfdf2d7166db95d663dbf4"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "UPDATE subtitle_files SET path = ?, updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now') WHERE id = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 2
},
"nullable": []
},
"hash": "9b129ff8a5951c9dfcb9778da0174ac66d4307936323b1eff9440adb4c34b893"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT spent AS \"spent!: i64\" FROM subtitle_budget_spend\n WHERE kind = ? AND name = ? AND day = strftime('%Y-%m-%d', 'now')",
"describe": {
"columns": [
{
"name": "spent!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_budget_spend",
"name": "spent"
}
}
}
],
"parameters": {
"Right": 2
},
"nullable": [
false
]
},
"hash": "9ee9ff74406e333df50076d1c456f0a9672173af252f36e5d75d2a9df99ab68d"
}
@@ -0,0 +1,38 @@
{
"db_name": "SQLite",
"query": "SELECT sf.id AS \"id!: i64\", sf.path AS \"path!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id\n JOIN movies m ON mf.owner_kind = 'movie' AND m.id = mf.owner_id\n WHERE m.root_id = ? AND sf.path IS NOT NULL\n ORDER BY sf.id",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "id"
}
}
},
{
"name": "path!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
true
]
},
"hash": "a1256d6eec88a6ce5c308a7b96d95b8b43100b1f4d8ac4cabc7c3bef9fefd745"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT mf.id AS \"id!: i64\", mf.path AS \"path!: String\"\n FROM media_files mf\n JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id\n JOIN seasons se ON se.id = e.season_id\n WHERE se.series_id = ?",
"query": "SELECT mf.id AS \"id!: i64\", mf.path AS \"path!: String\"\n FROM media_files mf\n JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id\n JOIN seasons se ON se.id = e.season_id\n WHERE se.series_id = ?",
"describe": {
"columns": [
{
@@ -34,5 +34,5 @@
false
]
},
"hash": "89fb1aa140a11bcd9bb082b2dcba50df414a77bc191755d515e5d5905a651adc"
"hash": "a2023f024d53aa5f139da347f22af38d57c6f846bb8eff74958fd4d1299268f8"
}
@@ -0,0 +1,86 @@
{
"db_name": "SQLite",
"query": "SELECT media_file_id AS \"media_file_id!: i64\",\n language AS \"language!: String\",\n state AS \"state!: SubtitleState\",\n attempts AS \"attempts!: i64\",\n last_attempt_at,\n last_failure\n FROM subtitle_attempts\n WHERE media_file_id = ?\n ORDER BY language",
"describe": {
"columns": [
{
"name": "media_file_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "media_file_id"
}
}
},
{
"name": "language!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "language"
}
}
},
{
"name": "state!: SubtitleState",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "state"
}
}
},
{
"name": "attempts!: i64",
"ordinal": 3,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "attempts"
}
}
},
{
"name": "last_attempt_at",
"ordinal": 4,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "last_attempt_at"
}
}
},
{
"name": "last_failure",
"ordinal": 5,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_attempts",
"name": "last_failure"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
false,
true,
true
]
},
"hash": "adf45101bc3ba20d35b59794053029e89d22ada5c7ffecd47187ec34261bc223"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT probed FROM media_files WHERE id = ?",
"describe": {
"columns": [
{
"name": "probed",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "media_files",
"name": "probed"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
true
]
},
"hash": "af10ccd655f39235f525a531e38850b2e60ff713a038ee1e7c04c616b4613b52"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT sf.path AS \"path!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id\n JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id\n WHERE e.season_id = ? AND sf.path IS NOT NULL",
"describe": {
"columns": [
{
"name": "path!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
true
]
},
"hash": "b2d937e5b8381de44fc6c1915e3deb24236c2c8afa25ffec1c7bbe519aac8803"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT providers_enabled AS \"providers_enabled!: String\"\n FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "providers_enabled!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "providers_enabled"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
false
]
},
"hash": "bc6dfe5a5cc639a64e3020c348a2dc74b4298101f8c29d9f8e6397cf79bb83d7"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_attempts (media_file_id, language, state)\n VALUES (?, ?, 'satisfied')\n ON CONFLICT (media_file_id, language) DO UPDATE\n SET state = 'satisfied',\n last_failure = NULL,\n updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now')",
"describe": {
"columns": [],
"parameters": {
"Right": 2
},
"nullable": []
},
"hash": "c5a9e43cfeb315e70bbb603f22acc519d7df3cd0b4d606d0d9056414ac6c503b"
}
@@ -0,0 +1,21 @@
{
"db_name": "SQLite",
"query": "SELECT EXISTS(SELECT 1 FROM movies WHERE id = ?) AS \"exists!: bool\"",
"describe": {
"columns": [
{
"name": "exists!: bool",
"ordinal": 0,
"type_info": "Integer",
"origin": "Expression"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "d81ec62e6c4879d08ce3551c0c71df012bed05ac2a37933adc2deb6552eeeac1"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT path AS \"path!: String\" FROM subtitle_files\n WHERE media_file_id = ? AND language = ? AND path IS NOT NULL",
"describe": {
"columns": [
{
"name": "path!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 2
},
"nullable": [
true
]
},
"hash": "e17b71f67d8b79c54d2d6f32d5e4197d77faac1763b360e8e609b0b9ba5820f1"
}
@@ -0,0 +1,21 @@
{
"db_name": "SQLite",
"query": "SELECT EXISTS(SELECT 1 FROM subtitle_files WHERE path = ?) AS \"exists!: bool\"",
"describe": {
"columns": [
{
"name": "exists!: bool",
"ordinal": 0,
"type_info": "Integer",
"origin": "Expression"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "e3b82dc7b2a3b28753c84864239699cb2007a8101cccc22d223d1f587a5533f8"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\" FROM subtitle_files WHERE path = ?",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "id"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "e5b075d0b7fe5410d3e55d5daf7837d0a9398b1954e713e80a1d82c0a7e172a6"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "DELETE FROM grabs\n WHERE (target_kind = 'season'\n AND target_id IN (SELECT id FROM seasons WHERE series_id = ?))\n OR (target_kind = 'episode'\n AND target_id IN (SELECT e.id FROM episodes e\n JOIN seasons s ON s.id = e.season_id\n WHERE s.series_id = ?))",
"describe": {
"columns": [],
"parameters": {
"Right": 2
},
"nullable": []
},
"hash": "e6fbef670552928dfeeb6f6d442021340acb438a1ed6a7b917be91c9d368cb57"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT translation_engine AS \"translation_engine: String\"\n FROM subtitle_settings WHERE id = 1",
"describe": {
"columns": [
{
"name": "translation_engine: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_settings",
"name": "translation_engine"
}
}
}
],
"parameters": {
"Right": 0
},
"nullable": [
true
]
},
"hash": "e7a0291c2fb8cfe59a02ffef11d9657ea6a40e4079573f569573075a205c50e7"
}
@@ -0,0 +1,21 @@
{
"db_name": "SQLite",
"query": "SELECT EXISTS(SELECT 1 FROM media_files WHERE id = ?) AS \"exists!: bool\"",
"describe": {
"columns": [
{
"name": "exists!: bool",
"ordinal": 0,
"type_info": "Integer",
"origin": "Expression"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "e8066bed6fb53ff90d78f47bb1c9ce8160918d4b35764fd988e5e1dd6c101089"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "UPDATE subtitle_settings SET\n wanted_languages = ?, providers_enabled = ?, translation_engine = ?,\n provider_daily_budgets = ?, translator_daily_budgets = ?,\n remote_command_timeout_seconds = ?,\n openai_base_url = ?, openai_model = ?,\n updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now')\n WHERE id = 1",
"describe": {
"columns": [],
"parameters": {
"Right": 8
},
"nullable": []
},
"hash": "eab2c4e7c21cd5d22a583c36dd2d79590e9ce55801c4b87b8db1b90f7b317225"
}
@@ -0,0 +1,50 @@
{
"db_name": "SQLite",
"query": "SELECT stream_index AS \"stream_index!: i64\",\n language AS \"language!: String\",\n cues AS \"cues!: String\"\n FROM subtitle_skeletons\n WHERE media_file_id = ?\n ORDER BY stream_index",
"describe": {
"columns": [
{
"name": "stream_index!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_skeletons",
"name": "stream_index"
}
}
},
{
"name": "language!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_skeletons",
"name": "language"
}
}
},
{
"name": "cues!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_skeletons",
"name": "cues"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false
]
},
"hash": "eb7df52055a60a00b721e3eef152e1b04a8512a85e8ad3618545c634c23abf31"
}
@@ -0,0 +1,38 @@
{
"db_name": "SQLite",
"query": "SELECT mf.id AS \"media_file_id!: i64\", e.id AS \"episode_id!: i64\"\n FROM media_files mf\n JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id\n JOIN seasons se ON se.id = e.season_id\n WHERE se.series_id = ?\n ORDER BY mf.path",
"describe": {
"columns": [
{
"name": "media_file_id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
},
{
"name": "episode_id!: i64",
"ordinal": 1,
"type_info": "Integer",
"origin": {
"Table": {
"table": "episodes",
"name": "id"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false
]
},
"hash": "f4c3eb506289667ace56da5d28c046163d5f12bd1dca53d3d6fc559f6e9ce393"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT sf.path AS \"path!: String\"\n FROM subtitle_files sf\n JOIN media_files mf ON mf.id = sf.media_file_id\n WHERE mf.owner_kind = 'episode' AND mf.owner_id = ? AND sf.path IS NOT NULL",
"describe": {
"columns": [
{
"name": "path!: String",
"ordinal": 0,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "path"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
true
]
},
"hash": "fa35de1c7e62c68dd1049d7e7e08c825d3ca31eb631744a57594aa2fc08fb945"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\" FROM media_files\n WHERE owner_kind = ? AND owner_id = ? ORDER BY path",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
}
],
"parameters": {
"Right": 2
},
"nullable": [
false
]
},
"hash": "fcba9e1f708e6bf2d8471d0c7d26be0b5ad8a30fa042434ffaaff8c69ccf31de"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\" FROM subtitle_files\n WHERE media_file_id = ? AND language = ? AND origin = 'embedded'\n AND forced = ? AND sdh = ?",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "id"
}
}
}
],
"parameters": {
"Right": 4
},
"nullable": [
false
]
},
"hash": "fe15c4a3774e663b03455300d0baec1dadc6cf995c20df803bc4bf49d1cc7c77"
}
+5 -3
View File
@@ -16,8 +16,9 @@ crates/
├── arr-parse/ release name parsing (no IO)
├── arr-meta/ TMDB client
├── arr-indexer/ Torznab via Prowlarr
├── arr-dl/ Transmission RPC
├── arr-dl/ qBittorrent WebUI API
├── arr-probe/ ffprobe wrapper
├── arr-subs/ subtitle providers, translation, sync
├── arr-db/ sqlx + migrations
├── arr-api/ axum + OpenAPI
├── arr-compat/ Radarr/Sonarr v3 shim for Jellyseerr
@@ -243,8 +244,9 @@ mid-issue.
- **Prowlarr** — `prowlarr` container on the Dokploy host, port 9696. Owns
tracker auth, FlareSolverr and the Cardigann definitions. Not replaced.
- **Transmission** — native in LXC 130 at `10.6.10.45:9091`, RPC
unauthenticated. Download dir `/mnt/media/transmission/complete`.
- **qBittorrent** — `qbittorrent.n62.casa`, WebUI API v2, login required
(`ARR_QBITTORRENT_USERNAME` / `ARR_QBITTORRENT_PASSWORD`). Download dir
`/mnt/media/qbittorrent/complete`.
- **Jellyfin** — native in LXC at `10.6.10.18:8096`. Library roots under
`/mnt/media-v2`.
- **Media** — ZFS, single dataset, bind-mounted as `/mnt/media`. Downloads and
Generated
+61 -3
View File
@@ -32,9 +32,12 @@ version = "0.1.0"
dependencies = [
"arr-core",
"arr-db",
"arr-dl",
"arr-indexer",
"arr-meta",
"arr-parse",
"arr-probe",
"arr-subs",
"axum",
"chrono",
"reqwest",
@@ -88,8 +91,8 @@ dependencies = [
"arr-meta",
"arr-parse",
"arr-probe",
"arr-subs",
"axum",
"base64",
"chrono",
"include_dir",
"mime_guess",
@@ -119,16 +122,17 @@ dependencies = [
"tempfile",
"thiserror",
"tokio",
"tracing",
]
[[package]]
name = "arr-dl"
version = "0.1.0"
dependencies = [
"base64",
"reqwest",
"serde",
"serde_json",
"sha1 0.10.7",
"thiserror",
"tokio",
"url",
@@ -139,12 +143,14 @@ dependencies = [
name = "arr-e2e"
version = "0.1.0"
dependencies = [
"arr-db",
"arr-dl",
"arr-indexer",
"arr-meta",
"chrono",
"reqwest",
"serde_json",
"sqlx",
"tempfile",
"tokio",
"uuid",
@@ -194,11 +200,30 @@ dependencies = [
"arr-core",
"serde",
"serde_json",
"tempfile",
"thiserror",
"tokio",
"tracing",
]
[[package]]
name = "arr-subs"
version = "0.1.0"
dependencies = [
"arr-core",
"arr-parse",
"chardetng",
"encoding_rs",
"reqwest",
"serde",
"serde_json",
"tempfile",
"thiserror",
"tokio",
"tracing",
"wiremock",
]
[[package]]
name = "assert-json-diff"
version = "2.0.2"
@@ -366,6 +391,17 @@ dependencies = [
"rand_core",
]
[[package]]
name = "chardetng"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13de944a44b5064ee5d3a5ceccc49a41bfec50f2580e66f82e87703acdb88b53"
dependencies = [
"cfg-if",
"encoding_rs",
"memchr",
]
[[package]]
name = "chrono"
version = "0.4.45"
@@ -531,6 +567,15 @@ dependencies = [
"serde",
]
[[package]]
name = "encoding_rs"
version = "0.8.35"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75030f3c4f45dafd7586dd6780965a8c7e8e285a5ecb86713e63a79c5b2766f3"
dependencies = [
"cfg-if",
]
[[package]]
name = "equivalent"
version = "1.0.2"
@@ -1498,6 +1543,7 @@ dependencies = [
"base64",
"bytes",
"futures-core",
"futures-util",
"http",
"http-body",
"http-body-util",
@@ -1506,6 +1552,7 @@ dependencies = [
"hyper-util",
"js-sys",
"log",
"mime_guess",
"percent-encoding",
"pin-project-lite",
"quinn",
@@ -1688,6 +1735,17 @@ dependencies = [
"serde",
]
[[package]]
name = "sha1"
version = "0.10.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a978451301f4db1d02937a4ab3ccce137717b81826e79b7d49ffe3244a13c3b8"
dependencies = [
"cfg-if",
"cpufeatures 0.2.17",
"digest 0.10.7",
]
[[package]]
name = "sha1"
version = "0.11.0"
@@ -1887,7 +1945,7 @@ dependencies = [
"log",
"percent-encoding",
"serde",
"sha1",
"sha1 0.11.0",
"sha2 0.11.0",
"sqlx-core",
"thiserror",
+5 -2
View File
@@ -20,13 +20,13 @@ arr-indexer = { path = "crates/arr-indexer" }
arr-meta = { path = "crates/arr-meta" }
arr-parse = { path = "crates/arr-parse" }
arr-probe = { path = "crates/arr-probe" }
arr-subs = { path = "crates/arr-subs" }
# Runtime and HTTP
axum = "0.8"
tokio = { version = "1", features = ["macros", "rt-multi-thread", "sync", "time", "net", "io-util", "fs", "signal", "process"] }
tower-http = { version = "0.6", features = ["trace"] }
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
base64 = "0.22"
reqwest = { version = "0.12", default-features = false, features = ["json", "multipart", "rustls-tls"] }
# Persistence
sqlx = { version = "0.9", default-features = false, features = ["runtime-tokio", "sqlite", "macros", "migrate", "chrono", "json"] }
@@ -47,8 +47,11 @@ quick-xml = { version = "0.37", features = ["serialize"] }
toml = "0.8"
# Odds and ends
chardetng = "1"
chrono = { version = "0.4", default-features = false, features = ["clock", "serde"] }
clap = { version = "4.5", features = ["derive", "env"] }
encoding_rs = "0.8"
sha1 = "0.10"
thiserror = "2"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
+216 -22
View File
@@ -27,7 +27,7 @@ allowed to be narrow.
Explicitly out of scope, permanently unless stated:
- **Authentication.** The perimeter is a VPN plus Authelia at the proxy. The
service binds without auth, same trust model as the existing Transmission RPC.
service binds without auth, same trust model as the rest of the stack.
- **Library migration or filesystem scan.** The service knows only what it put
on disk. Adopting the pre-existing library, if ever wanted, is a one-off
script against both APIs, not a feature.
@@ -46,7 +46,7 @@ Explicitly out of scope, permanently unless stated:
```
┌──────────────┐
TMDB ──────▶│ │
│ arr │────▶ Transmission RPC 10.6.10.45:9091
│ arr │────▶ qBittorrent WebUI qbittorrent.n62.casa
Prowlarr ──────▶│ │
(Torznab) │ (this) │────▶ ffprobe local subprocess
│ │
@@ -61,8 +61,9 @@ Everything already exists except `arr`. Prowlarr keeps owning tracker auth,
Cloudflare bypass via FlareSolverr, rate limiting and the Cardigann
definitions — replacing it buys nothing.
Transmission runs natively in its own LXC (VMID 130, `10.6.10.45`), RPC
unauthenticated, download dir `/mnt/media/transmission/complete`.
qBittorrent is reached at `qbittorrent.n62.casa`, WebUI API v2, download dir
`/mnt/media/qbittorrent/complete`. Its WebUI requires a login, so arr carries
credentials — the one upstream that does.
## 4. Domain model
@@ -517,11 +518,20 @@ leaving RSS matching on.
## 7. Download and import
### 7.1 Transmission
### 7.1 qBittorrent
RPC at `10.6.10.45:9091`, no credentials. Labels are `movies-main`, `tv-kids`
and so on — enough to find things in Transmission's own UI, and distinct from
Radarr's existing `radarr`/`sonarr` labels so both stacks can run side by side.
WebUI API v2 at `qbittorrent.n62.casa`. Labels are qBittorrent **tags**
`movies-main`, `tv-kids` and so on — enough to find things in qBittorrent's own
UI, and distinct from Radarr's existing `radarr`/`sonarr` labels so both stacks
can run side by side. Tags rather than a category because a category also
governs the save path, and arr owns that.
`torrents/add` answers `Ok.` and nothing else: no hash, no name, and no signal
that the torrent was already there. arr therefore derives the v1 infohash from
the magnet or the `.torrent` bytes before the call, and looks the torrent up by
it. That is also what makes an add idempotent across a restart (§8) — the same
release resolves to the same hash, and a second add is recognised as the
duplicate it is.
### 7.2 Hardlink
@@ -540,9 +550,16 @@ at 4K is 40-80 GB per title.
The torrent and the library entry are separate state machines.
Seeding obligation is **per tracker**, configured locally because Prowlarr does
not expose tracker rules — `ratio` and `min_seed_time`. Set `seedRatioLimit`
and `seedIdleLimit` on the torrent at add time and let Transmission enforce
them. A reaper deletes torrents Transmission reports as done seeding.
not expose tracker rules — `ratio` and `min_seed_time`. Set `ratioLimit` and
`inactiveSeedingTimeLimit` on the torrent at add time, with
`shareLimitAction` set to stop rather than delete, and let qBittorrent enforce
them. A reaper deletes torrents qBittorrent stopped on a limit arr set.
Stopped-on-a-limit, not merely stopped: a torrent the operator paused by hand
looks identical in the state field alone, and the reaper deletes data. So the
ratio and idle limits are checked against the torrent's own counters, and a
torrent stopped by a *global* limit reads as still seeding — the safe direction
to be wrong in.
Consequently a hard-failed release is blacklisted and never imported, but its
torrent keeps seeding until the obligation clears. Nothing is deleted early to
@@ -599,11 +616,12 @@ state and act on the gap.
This is idempotent and crash-safe by construction. Kill the process mid-grab and
the next tick recomputes the same gap and continues. A job table would need
retry counts, dead-lettering and reconciliation against Transmission anyway,
because Transmission is an external system that changes underneath the app.
retry counts, dead-lettering and reconciliation against qBittorrent anyway,
because qBittorrent is an external system that changes underneath the app.
Transient state — a search in flight, download progress — is in memory and
rebuilt from Transmission on startup.
rebuilt from qBittorrent on startup. Where that state is ever seen is §9.8:
inline on the row that owns the item, never persisted.
Ticks are staggered: reconcile every 30 s, RSS every 10 min, metadata refresh
daily, reaper every 5 min.
@@ -687,7 +705,7 @@ notifying on everything and being muted within a week.
- **Imported** → to the title's owners. The only good-news notification.
- **Needs a decision** → to the operator alone. Entered the no-PT-source queue,
or the needs-a-decision queue (§5.7).
- **Broken** → to the operator alone. Prowlarr, Transmission or TMDB
- **Broken** → to the operator alone. Prowlarr, qBittorrent or TMDB
unreachable, disk full.
Not notified: grabs, searches, downloads starting or finishing, soft fails.
@@ -741,7 +759,7 @@ inside the app, review text.
opens on.
**The signal chain is a settings section.** The four upstreams — tmdb,
prowlarr, arr, transmission — and their lamps live inside `/settings`, and have
prowlarr, arr, qbittorrent — and their lamps live inside `/settings`, and have
no route of their own. Per-upstream health is something you check when
something is wrong, not a homepage.
@@ -754,6 +772,33 @@ is healthy survives the move and no navigation is needed to get it.
lamp and wordmark, a warning mark only while queues need attention, the centred
search, the version readout and the settings cog.
### 9.8 Downloads on the row
§8 keeps download progress as transient state; nothing in §9 so far says where
it is seen. Without a rule here, a season pack downloading and one that never
started look identical.
**Progress is inline, on the row that owns the item** — a movie row, a season
row for its pack, an episode row. There is no downloads page and no count in
the rail: a download is an attribute of the thing being downloaded, not a place
to visit.
Alongside progress the same row carries the other states a torrent can be in:
**seeding** under §7.3's obligation, **stalled**, **errored**.
**A torrent arr did not grab is never shown.** §2 already rules the service
knows only what it put on disk; qBittorrent's own UI lists the rest.
Nothing is persisted — no progress column, no new table. The snapshot comes
from qBittorrent and dies with the process (§8), refreshed by the UI's normal
polling cadence at roughly 15 s rather than SSE.
At phone width only active-grab progress survives; seeding and stalled shed
with the row's other secondary chips.
Out of scope here: any change to what the reconcile loop does, and any new
state machine — §7.3's two lifecycles stay as they are.
## 10. Persistence
SQLite via `sqlx`, compile-time-checked queries, migrations in `arr-db`.
@@ -763,8 +808,10 @@ service.
Policy lives in the database, not a config file — size targets and DV rules get
tuned by hand during testing and a restart-to-reload loop gets old immediately.
Only bootstrap settings (bind address, Prowlarr URL, Transmission URL, TMDB key,
media root, TMDB response cache directory) come from config/env.
Only bootstrap settings (bind address, Prowlarr URL, qBittorrent URL and
login, TMDB key, media root, TMDB response cache directory) come from
config/env. The qBittorrent password is a secret, so it is env-only and has no
config-file field.
Backup is `sqlite3 .backup` on a timer.
@@ -778,8 +825,9 @@ arr-core domain types, policy engine, scoring no IO, no heavy deps
arr-parse release name parsing no IO
arr-meta TMDB client
arr-indexer Torznab via Prowlarr
arr-dl Transmission RPC
arr-dl qBittorrent WebUI API
arr-probe ffprobe wrapper
arr-subs subtitle providers, translation, sync
arr-db sqlx + migrations
arr-api axum + OpenAPI
arr-compat Radarr/Sonarr v3 shim for Jellyseerr
@@ -834,8 +882,10 @@ and `target/`, keyed on `Cargo.lock` plus `rust-toolchain.toml`. Never build
- Prowlarr and TMDB — `wiremock` with recorded real responses as fixtures.
Never live: trackers rate-limit, and it would leak credentials into CI.
- Transmission — a real container as a CI service. RPC semantics are the most
likely source of surprise and it starts in a second.
- qBittorrent — a real container, started with a seeded config because the
image prints a random WebUI password per boot. Its API semantics are the most
likely source of surprise: `torrents/add` reports nothing back, and required
parameters have changed between major versions.
- `ffprobe` — tiny committed clips, a few KB each. The Jellyfin LXC already has
the right fixtures at `/srv/jellyfin-test`, including a real **DV Profile 5**
clip. That is the test proving the policy engine rejects Profile 5 and accepts
@@ -862,7 +912,7 @@ previous one.
grabbing, derived status.
7. **Owners and notifications** — tags, per-person ntfy topics, filtered views.
8. **Jellyseerr compat**`arr-compat`.
9. **Subtitles** — replaces Bazarr. Out of scope for this document.
9. **Subtitles** — replaces Bazarr. See §15.
Movies before TV because TV adds season packs, air-date calendars and
per-episode state on top of an otherwise identical pipeline. Doing it second
@@ -882,3 +932,147 @@ means that pipeline is already proven.
RSS lane obeys the same guard — it skips packs for any season with episodes
on disk. Whether a re-grab is ever wanted remains unresolved and deliberately
deferred.
## 15. Subtitles
Replaces Bazarr. Phase 9 in §13; the `arr-subs` crate in §11.
**Wanted set.** Global, not per root. Two languages are separately wanted for
every media file: Portuguese — pt-PT preferred, pt-BR accepted — and English.
A file is satisfied for a language when a subtitle in it exists, embedded or as
a sidecar. Satisfying a language is a statement about *viewing* it, and not
about having text in it — the distinction matters where an image track is all
there is, and **Translation** below is where it bites. This is deliberately unlike §5.2's audio rules, which attach to a
root: subtitles carry no blacklist there and none here. pt-BR subtitles are
always fine.
**Embedded tracks.** An embedded subtitle track satisfies its language.
Text-format tracks (`subrip`, `ass`, `mov_text`) are additionally extracted to
a sidecar SRT, because an extracted track is a legal translation source.
Image-format tracks (PGS on BluRay, VobSub on DVD) carry bitmaps, not text:
they satisfy viewing but can never feed a translator, and arr does not OCR
them. `arr-probe` already reports subtitle tracks with resolved languages; the
format is the new fact it must carry.
**Cue skeletons.** What an image track *does* have is exact timings, because
its packet timestamps are the disc's own cue structure. At import — while the
file is being read and hardlinked anyway — arr derives a **cue skeleton** from
each non-forced image track: `ffprobe -show_packets` on the track, packets
paired show-to-clear, no pixel read. The skeleton has timings and no text, and
that is enough, because `alass` matches on interval structure rather than on
words. A sparse skeleton is still a strong reference; a downloaded subtitle and
a retail disc's track never carry the same cues anyway.
The pairing is the whole of it, so it is validated before it is trusted: an
even packet count, every implied duration plausible, and a cue density that
fits the runtime. PGS permits several composition segments per subtitle, and a
track built that way pairs into something that is quietly half a second out —
worse than no skeleton. A track that fails validation gets none, and alignment
falls back to the video.
**Providers.** OpenSubtitles.com, behind one trait.
**Ranking.** A `moviehash` match wins outright. Then an exact release-name
match, then same release group or same source, then uploader rating and
download count as tiebreakers. Every rejected candidate names the rule that
killed it, so the manual view described in §9.3 works unchanged for subtitles.
**Forced and SDH.** A forced track covers only foreign-language lines and
on-screen signs; it never satisfies a want and arr never goes looking for one.
An SDH track is complete and satisfies, ranked below a plain subtitle.
Neither gets its own sidecar name. A language is satisfied by exactly one
sidecar, so `<video>.<lang>.srt` needs no segment distinguishing forced from
plain from SDH — where a plain subtitle exists for a language, the forced one
is ignored rather than kept beside it. The database says the same thing: one
sidecar row per (media file, language).
**Translation.** When no provider has a wanted language, arr translates
immediately — there is no waiting window. The source is an existing subtitle:
a downloaded one, or one extracted from a text-format embedded track. Being
able to translate from an embedded track is a deliberate improvement on
Bazarr, which cannot.
**Fetching a source.** A release whose only subtitle is an image track has no
text source and never will: the track satisfies its language, so nothing is
ever fetched in it, and the track itself cannot be translated from. That is a
deadlock, and it is broken by a narrow carve-out — when a wanted language needs
translating and no text source exists, arr fetches a text subtitle in the image
track's language *even though that language reads as satisfied*. The fetch
obtains a source; it settles no want of its own, and the no-upgrade rule below
does not apply to it. Where that track has a cue skeleton, the fetched subtitle
is aligned against the skeleton rather than the video, which puts it on the
disc's own timings instead of on a heuristic read of the audio. OCR remains a
non-goal: this reaches the same place with real text.
**Translation backends.** Pluggable, each behind its own cargo feature: an
OpenAI-compatible HTTP endpoint, DeepL, Google Translate, and a generic remote
command driven by a configured template (`ssh box claude -p` is one instance
of that template, not a backend of its own). The engine in use is a database
setting, so switching does not need a rebuild when the feature is compiled in.
Subtitles are sent in batches of cues; a reply whose cue count or numbering
does not match the batch is rejected. Timing data never leaves arr.
**No upgrade loop.** Once a language is satisfied — by a machine translation
too — arr stops working on it. A real subtitle appearing later does not
replace anything. Replacement is a manual action from the UI. This is §5.4's
rule applied to subtitles.
The one exception is the source fetch above. It is not an upgrade: the language
it downloads is already satisfied and stays satisfied by the same track it was
before, and what the download settles is a *different* language's gap. Nothing
is replaced, so nothing about the rule changes.
**On disk.** Sidecars live next to the video inside the §7.4 title folder,
named `<video basename>.<lang>.srt`, e.g.
`… - [2160p][WEB-DL][HDR10].pt-PT.srt`. A machine translation carries an extra
`.mt` segment: `… [HDR10].pt-PT.mt.srt`. That keeps §7.4's audit-by-`ls`
property — with the app stopped, the filename says which subtitles are
machine-made. Folder-level delete stays atomic because sidecars are inside the
folder.
`.mt` is the only optional segment. One language, one sidecar: a second
subtitle for a language arr already has is refused, and replacing one is the
manual delete-then-fetch §15's no-upgrade rule already describes.
**Sync.** `alass` runs on every fetched and every translated subtitle. It is a
single small binary invoked like `ffprobe`, so it costs nothing at rest. It
reports no confidence value, so its output is accepted unless it is
implausible — a shift beyond 60 seconds, or cues lost — in which case the
unsynced original is kept and the file is flagged.
The reference is the video, except where a cue skeleton exists, and then it is
the skeleton. A subtitle a skeleton accepted is already on disc-exact timings,
and translation copies those timings over untouched, so the pass that would
otherwise run on the translation is skipped: a second alignment has nothing
left to find and can only move them off.
**Configuration.** Provider credentials and translator API keys are bootstrap
config or environment, per §10 — a secret never becomes a database row. Wanted
languages, chosen engine, per-provider enable and the daily budgets are
database rows edited from `/settings` without a restart.
So is everything needed to point the OpenAI-compatible backend somewhere else:
its **base URL and its model name are database rows too**, not bootstrap
config. That backend is not "OpenAI" — it is any endpoint speaking that shape,
`llama.cpp` and a local gateway included, and which one is in use is a thing to
try and change, not a property of the deployment fixed at start-up. Only the
API key stays in the environment, and an endpoint that needs no key is a valid
configuration.
**Budgets.** A token bucket per provider and per translator, with a configured
daily allowance. The reconcile loop spends it newest-import-first, so enabling
this on an existing library drains the backlog over days instead of hitting
every rate limit at once. Being at the cap is a visible queue state, not an
error.
**Notifications.** No new event classes. §9.5 stands: subtitle fetches never
notify, and a provider or translator being unreachable folds into the existing
"Broken" message to the operator alone.
**Non-goals**, stated here so they do not creep back: OCR of image-based
tracks, transcribing audio when no subtitle exists anywhere, adopting subtitle
files already on disk that arr did not write (§2 already says the service
knows only what it put there — which does mean arr may fetch a second copy
alongside one Bazarr left), and any background loop that upgrades a subtitle
in place.
+46 -2
View File
@@ -8,17 +8,61 @@ RUN pnpm -C web install --frozen-lockfile --config.dangerouslyAllowAllBuilds=tru
COPY web ./web
RUN pnpm -C web build
FROM rust:1-bookworm AS build
# Every translation backend is a default-off cargo feature (DESIGN.md §15), so
# a build without these ships an image that cannot translate at all. All four
# go in: which one is in use is a database setting, and the point of the
# feature switches is the build, not the deployment.
#
# The dependency build and the workspace build are split with cargo-chef so a
# source-only change reuses the dependency layer. The two `cargo` invocations
# must carry identical flags or the cook output is not reusable.
FROM lukemathwalker/cargo-chef:latest-rust-1-bookworm AS chef
WORKDIR /src
ENV ARR_FEATURES=translate-openai,translate-deepl,translate-google,translate-command
FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json
FROM chef AS build
COPY --from=planner /src/recipe.json recipe.json
# cargo-chef's skeleton keeps build scripts, and arr-daemon's panics in release
# when it can find no SPA bundle. A stub satisfies it; the real bundle arrives
# below and the changed ARR_WEB_DIST reruns only that one build script.
RUN mkdir -p /stub/dist && printf '<!doctype html>stub\n' > /stub/dist/index.html
ENV ARR_WEB_DIST=/stub/dist
RUN cargo chef cook --release -p arr-daemon --features "$ARR_FEATURES" \
--recipe-path recipe.json
COPY . .
COPY --from=web /src/web/dist ./web/dist
ENV ARR_WEB_DIST=/src/web/dist
RUN cargo build --release -p arr-daemon
RUN cargo build --release -p arr-daemon --features "$ARR_FEATURES"
# §15 runs `alass` over every fetched and every translated subtitle, and it is
# not in Debian — a release binary is the only way in. Its own stage so the
# runtime image needs no download tool, and so a source change does not refetch
# it. The digest is pinned: an unverified binary from a release page is not
# something to run over the library. The asset is x86-64 only, which is what
# this image targets.
FROM debian:bookworm-slim AS alass
ARG ALASS_VERSION=2.0.0
ARG ALASS_SHA256=7bd0b9ae7e035d3ba940eacffb21243614df36231d47f21f0b4ce42001ab7fcd
RUN apt-get update \
&& apt-get install -y --no-install-recommends curl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
RUN curl -fsSL -o /usr/local/bin/alass \
"https://github.com/kaegi/alass/releases/download/v${ALASS_VERSION}/alass-linux64" \
&& echo "${ALASS_SHA256} /usr/local/bin/alass" | sha256sum -c - \
&& chmod +x /usr/local/bin/alass \
&& alass --version
FROM debian:bookworm-slim
# ffmpeg covers both `ffprobe` for arr-probe and the audio extraction `alass`
# shells out to.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ffmpeg ca-certificates \
&& rm -rf /var/lib/apt/lists/*
COPY --from=alass /usr/local/bin/alass /usr/local/bin/alass
COPY --from=build /src/target/release/arr /usr/local/bin/arr
ENV ARR_BIND_ADDR=0.0.0.0:7878 \
ARR_DATABASE_PATH=/data/arr.db
+34 -12
View File
@@ -15,32 +15,54 @@ fmt-check:
fmt:
cargo fmt --all
# Clippy, warnings denied.
# Clippy, warnings denied. `--all-features` matters: every subtitle
# translation backend sits behind a default-off cargo feature, so without it
# the gate never compiles a line of them (#191, #192).
lint:
cargo clippy --all-targets -- -D warnings
cargo clippy --all-targets --all-features -- -D warnings
# Unused workspace dependencies. Stable toolchain, unlike cargo-udeps.
deps:
cargo machete
# Unit and integration tests, excluding the e2e crate.
# Unit and integration tests, excluding the e2e crate. `--all-features` for
# the same reason `lint` needs it: the translation backends are feature-gated
# and would otherwise never run.
test:
cargo nextest run --workspace --exclude arr-e2e
cargo nextest run --workspace --exclude arr-e2e --all-features
# End-to-end tests. Needs a Transmission container (`just e2e-up`); not part
# of the push gate. Never point this at the production Transmission LXC.
# End-to-end tests. Needs a qBittorrent container (`just e2e-up`); not part
# of the push gate. Never point this at the production qBittorrent.
e2e:
cargo nextest run -p arr-e2e
# Throwaway local Transmission for `just e2e`, mirroring the CI service.
# Throwaway local qBittorrent for `just e2e`, mirroring the CI container.
#
# The config is seeded before first start because qBittorrent prints a random
# WebUI password per boot and has no environment variable to set one; the
# subnet whitelist is what lets the tests talk to it without credentials.
e2e-up:
docker run --rm -d --name arr-e2e-transmission \
-e PUID=1000 -e PGID=1000 -p 9091:9091 \
linuxserver/transmission:latest
#!/usr/bin/env bash
set -euo pipefail
mkdir -p /tmp/arr-e2e-qbt/config/qBittorrent
cat > /tmp/arr-e2e-qbt/config/qBittorrent/qBittorrent.conf <<'CONF'
[Preferences]
WebUI\Port=8080
WebUI\AuthSubnetWhitelistEnabled=true
WebUI\AuthSubnetWhitelist=0.0.0.0/0
WebUI\CSRFProtection=false
WebUI\HostHeaderValidation=false
Downloads\SavePath=/downloads
CONF
docker run --rm -d --name arr-e2e-qbittorrent \
-e PUID=1000 -e PGID=1000 -e WEBUI_PORT=8080 -p 8080:8080 \
-v /tmp/arr-e2e-qbt/config:/config \
linuxserver/qbittorrent:latest
until curl -sf http://127.0.0.1:8080/api/v2/app/version >/dev/null; do sleep 1; done
# Stop and discard the local Transmission container.
# Stop and discard the local qBittorrent container.
e2e-down:
docker stop arr-e2e-transmission
docker stop arr-e2e-qbittorrent
# Frontend lint and typecheck. No-op until the SPA exists (issue #6). Probes
# for package.json, not the directory: `just gen-client` writes into web/ and
+1 -1
View File
@@ -43,7 +43,7 @@ manual search buries the decision under the release name column.
- Self-hosted behind VPN + Authelia; the app itself has no auth layer.
- API-first: the SPA is one client of the OpenAPI-described HTTP API, no
privileged path.
- Coexists with Prowlarr (indexers), Transmission (downloads), Jellyfin
- Coexists with Prowlarr (indexers), qBittorrent (downloads), Jellyfin
(playback), Jellyseerr (requests), ntfy (notifications).
## Capabilities and Constraints
+1 -1
View File
@@ -12,7 +12,7 @@ This is that loop, once, in Rust.
- **Prowlarr** stays — it owns tracker auth, Cloudflare bypass and the indexer
definitions, and replacing it buys nothing.
- **Transmission** stays.
- **qBittorrent** stays.
- **Jellyseerr** stays, talking to a thin Radarr-compatible shim.
## Status
+3
View File
@@ -8,10 +8,13 @@ publish = false
[dependencies]
arr-core = { workspace = true }
arr-dl = { workspace = true }
arr-db = { workspace = true }
arr-indexer = { workspace = true }
arr-meta = { workspace = true }
arr-parse = { workspace = true }
arr-probe = { workspace = true }
arr-subs = { workspace = true }
axum = { workspace = true }
chrono = { workspace = true }
reqwest = { workspace = true }
+231
View File
@@ -0,0 +1,231 @@
//! The live download snapshot, joined to arr-owned grabs only.
use std::time::{Duration, Instant};
use arr_dl::{Torrent, TorrentState};
use axum::extract::State;
use axum::Json;
use serde::Serialize;
use utoipa::ToSchema;
use crate::movies::ApiError;
use crate::state::AppState;
const SNAPSHOT_TTL: Duration = Duration::from_secs(15);
#[derive(Debug, Clone, Copy, Serialize, ToSchema, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum DownloadPhase {
Downloading,
Stalled,
Errored,
Seeding,
Queued,
}
#[derive(Debug, Clone, Serialize, ToSchema)]
pub struct Download {
pub target_kind: String,
pub target_id: i64,
pub infohash: String,
pub name: String,
pub phase: DownloadPhase,
pub progress: f64,
pub download_rate: u64,
pub eta: Option<i64>,
pub error: Option<String>,
}
#[utoipa::path(
get,
path = "/api/downloads",
tag = "system",
responses(
(status = 200, description = "Live downloads started by arr", body = [Download]),
(status = 503, description = "qBittorrent or database unavailable", body = crate::movies::ErrorBody)
)
)]
pub(crate) async fn list(State(state): State<AppState>) -> Result<Json<Vec<Download>>, ApiError> {
let database = state.database().ok_or(ApiError::Unavailable)?;
let qbit = state.qbit().ok_or(ApiError::Upstream("qbit"))?;
let torrents = {
let mut cache = state.download_snapshot().write().await;
if let Some((at, torrents)) = cache.as_ref() {
if at.elapsed() < SNAPSHOT_TTL {
torrents.clone()
} else {
let torrents = qbit
.list_torrents()
.await
.map_err(|_| ApiError::Upstream("qbit"))?;
*cache = Some((Instant::now(), torrents.clone()));
torrents
}
} else {
let torrents = qbit
.list_torrents()
.await
.map_err(|_| ApiError::Upstream("qbit"))?;
*cache = Some((Instant::now(), torrents.clone()));
torrents
}
};
let grabs: Vec<Grab> =
sqlx::query_as("SELECT target_kind, target_id, infohash FROM grabs ORDER BY id")
.fetch_all(database.pool())
.await?;
let mut downloads = Vec::new();
for grab in grabs {
let Some(torrent) = torrents
.iter()
.find(|torrent| torrent.hash.eq_ignore_ascii_case(&grab.infohash))
else {
continue;
};
downloads.push(Download {
target_kind: grab.target_kind,
target_id: grab.target_id,
infohash: torrent.hash.clone(),
name: torrent.name.clone(),
phase: phase(torrent),
progress: torrent.progress,
download_rate: torrent.download_rate,
eta: torrent.eta,
error: torrent.error.clone(),
});
}
Ok(Json(downloads))
}
#[derive(Debug, sqlx::FromRow)]
struct Grab {
target_kind: String,
target_id: i64,
infohash: String,
}
fn phase(torrent: &Torrent) -> DownloadPhase {
if torrent.error.is_some() {
DownloadPhase::Errored
} else {
match torrent.state {
TorrentState::Seeding => DownloadPhase::Seeding,
// qBittorrent's own verdict, not a zero rate: a torrent between
// peers reads zero for a poll or two and finishes fine, and
// §9.8's stalled chip is for the case that never does.
TorrentState::Downloading if torrent.is_stalled => DownloadPhase::Stalled,
TorrentState::Downloading => DownloadPhase::Downloading,
_ => DownloadPhase::Queued,
}
}
}
#[cfg(test)]
mod tests {
use std::path::PathBuf;
use arr_db::Db;
use axum::extract::State;
use serde_json::json;
use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};
use super::{list, phase, DownloadPhase};
use crate::state::{AppState, Upstreams};
#[test]
fn torrent_states_become_the_download_phases() {
let torrent =
|state: arr_dl::TorrentState, rate: u64, error: Option<&str>| arr_dl::Torrent {
name: "name".into(),
hash: "hash".into(),
state,
progress: 0.5,
download_rate: rate,
eta: None,
error: error.map(str::to_owned),
is_stalled: rate == 0,
download_dir: PathBuf::from("/downloads"),
labels: Vec::new(),
is_finished: false,
};
assert_eq!(
phase(&torrent(arr_dl::TorrentState::Downloading, 10, None)),
DownloadPhase::Downloading
);
assert_eq!(
phase(&torrent(arr_dl::TorrentState::Downloading, 0, None)),
DownloadPhase::Stalled
);
assert_eq!(
phase(&torrent(arr_dl::TorrentState::Seeding, 0, None)),
DownloadPhase::Seeding
);
assert_eq!(
phase(&torrent(
arr_dl::TorrentState::Downloading,
10,
Some("disk full")
)),
DownloadPhase::Errored
);
assert_eq!(
phase(&torrent(arr_dl::TorrentState::QueuedToDownload, 0, None)),
DownloadPhase::Queued
);
}
#[tokio::test]
async fn endpoint_joins_only_arr_grabs_and_reuses_the_snapshot() {
let server = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/api/v2/torrents/info"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!([{
"hash": "ABC", "name": "grabbed", "state": "downloading",
"progress": 0.4, "dlspeed": 123, "eta": 60,
"save_path": "/downloads", "tags": ""
}])))
.expect(1)
.mount(&server)
.await;
let directory = tempfile::tempdir().expect("tempdir");
let database = Db::connect(directory.path().join("arr.db"))
.await
.expect("database");
database.migrate().await.expect("migrate");
sqlx::query(
"INSERT INTO releases (id, indexer_id, guid, name, size, download_url, parsed)
VALUES (1, 1, 'guid', 'release', 1, 'magnet:?x', '{}')",
)
.execute(database.pool())
.await
.expect("release");
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash)
VALUES (1, 'movie', 42, 'abc')",
)
.execute(database.pool())
.await
.expect("grab");
let state = AppState::new(Upstreams::new("unused".into(), server.uri()))
.expect("state")
.with_database(database)
.with_qbit(arr_dl::QbitClient::new(&server.uri()).expect("client"));
let first = list(State(state.clone())).await.expect("first").0;
let second = list(State(state)).await.expect("second").0;
assert_eq!(first.len(), 1);
assert_eq!(first[0].target_kind, "movie");
assert_eq!(first[0].target_id, 42);
assert_eq!(first[0].infohash, "ABC");
assert_eq!(first[0].phase, DownloadPhase::Downloading);
assert_eq!(first[0].download_rate, 123);
assert_eq!(second.len(), 1);
}
}
+384 -27
View File
@@ -1,16 +1,20 @@
//! `GET /api/health` — is each of the three upstreams answering.
//!
//! The three probed here are the ones DESIGN.md §9.5 calls "Broken": without
//! Prowlarr nothing is found, without Transmission nothing is fetched, and
//! Prowlarr nothing is found, without qBittorrent nothing is fetched, and
//! without TMDB nothing is identified. The endpoint always answers `200` —
//! the body carries the verdict, so a degraded service can still explain
//! itself to the UI instead of looking like a fourth outage.
use std::ffi::OsStr;
use axum::extract::State;
use axum::Json;
use serde::Serialize;
use utoipa::ToSchema;
use arr_subs::{binary_present, translate as backend_error};
use crate::state::AppState;
/// Whether the service as a whole can do its job.
@@ -68,6 +72,44 @@ impl Check {
}
}
/// One enabled subtitle provider's verdict (#200).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, ToSchema)]
pub struct ProviderCheck {
/// The provider's id as settings name it.
pub id: String,
pub status: Status,
#[serde(skip_serializing_if = "Option::is_none")]
pub detail: Option<String>,
}
/// The subtitle lane's verdicts (DESIGN.md §15, #200). Only what is actually
/// in use appears here: providers nobody enabled and engines nobody selected
/// cannot be broken.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, ToSchema)]
pub struct SubtitleHealth {
/// One entry per provider the settings enable, in that order.
pub providers: Vec<ProviderCheck>,
/// The selected translation engine, or `None` when none is chosen —
/// which is a configuration state, not an outage.
#[serde(skip_serializing_if = "Option::is_none")]
pub translation: Option<Check>,
/// The `alass` sync binary, present at its configured path.
pub alass: Check,
/// The `ffmpeg` extraction binary, present at its configured path.
pub ffmpeg: Check,
}
impl SubtitleHealth {
/// Every verdict the lane carries, for the overall status.
fn statuses(&self) -> impl Iterator<Item = Status> + '_ {
self.providers
.iter()
.map(|provider| provider.status)
.chain(self.translation.iter().map(|check| check.status))
.chain([self.alass.status, self.ffmpeg.status])
}
}
/// The body of `GET /api/health`.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, ToSchema)]
pub struct HealthReport {
@@ -76,11 +118,13 @@ pub struct HealthReport {
#[schema(example = "0.1.0")]
pub version: String,
pub prowlarr: Check,
pub transmission: Check,
pub qbit: Check,
pub tmdb: Check,
pub subtitles: SubtitleHealth,
}
/// Report reachability of Prowlarr, Transmission and TMDB.
/// Report reachability of Prowlarr, qBittorrent and TMDB, plus the
/// subtitle upstreams (#200).
#[utoipa::path(
get,
path = "/api/health",
@@ -90,17 +134,19 @@ pub struct HealthReport {
),
)]
pub async fn health(State(state): State<AppState>) -> Json<HealthReport> {
// Three independent network probes; serialising them would make the
// endpoint as slow as the sum of the timeouts.
let (prowlarr, transmission, tmdb) = tokio::join!(
// Independent network probes; serialising them would make the endpoint
// as slow as the sum of the timeouts.
let (prowlarr, qbit, tmdb, subtitles) = tokio::join!(
probe_prowlarr(&state),
probe_transmission(&state),
probe_qbit(&state),
probe_tmdb(&state),
probe_subtitles(&state)
);
let status = if [prowlarr.status, transmission.status, tmdb.status]
.iter()
.all(|s| *s == Status::Ok)
let status = if [prowlarr.status, qbit.status, tmdb.status]
.into_iter()
.chain(subtitles.statuses())
.all(|check| check == Status::Ok)
{
Health::Ok
} else {
@@ -111,8 +157,9 @@ pub async fn health(State(state): State<AppState>) -> Json<HealthReport> {
status,
version: env!("CARGO_PKG_VERSION").to_string(),
prowlarr,
transmission,
qbit,
tmdb,
subtitles,
})
}
@@ -134,22 +181,21 @@ async fn probe_prowlarr(state: &AppState) -> Check {
}
}
/// Transmission answers an RPC call without a session id with `409` plus the
/// id to retry with. That is a live daemon, so it counts as reachable.
async fn probe_transmission(state: &AppState) -> Check {
let request = state
.http()
.post(&state.upstreams().transmission_url)
.json(&serde_json::json!({ "method": "session-get" }));
match request.send().await {
Ok(response)
if response.status().is_success()
|| response.status() == reqwest::StatusCode::CONFLICT =>
{
Check::ok()
}
Ok(response) => Check::unreachable(format!("http {}", response.status().as_u16())),
Err(err) => Check::unreachable(describe(err)),
/// The probe is a real torrent list through the configured client, not a bare
/// reachability check.
///
/// A `WebUI` that wants credentials answers `403` to anything unauthenticated,
/// so "the port is open" is true of an instance arr cannot use at all. Reading
/// that as healthy once left the lamp green while every grab and every reaper
/// tick failed on a rejected password, which is the one thing this lamp exists
/// to catch.
async fn probe_qbit(state: &AppState) -> Check {
let Some(qbit) = state.qbit() else {
return Check::unconfigured("no qBittorrent client is configured");
};
match qbit.list_torrents().await {
Ok(_) => Check::ok(),
Err(error) => Check::unreachable(error.to_string()),
}
}
@@ -181,3 +227,314 @@ async fn probe_tmdb(state: &AppState) -> Check {
fn describe(err: reqwest::Error) -> String {
err.without_url().to_string()
}
// ---- the subtitle lane (DESIGN.md §15, #200) -----------------------------
/// Probe every subtitle lamp: enabled providers, the selected engine, and
/// the two binaries. The settings row decides what is *in use* — a provider
/// nobody enabled or an engine nobody selected is not probed at all, so it
/// cannot fail a lamp.
async fn probe_subtitles(state: &AppState) -> SubtitleHealth {
let settings = match state.database() {
Some(_) => crate::subtitle_settings::load(state).await.ok(),
None => None,
};
// No readable row means nothing is known to be in use; the lamps stay
// quiet rather than failing on data the operator has not entered yet.
let enabled = settings
.as_ref()
.map(|settings| settings.providers_enabled.as_slice())
.unwrap_or_default();
let engine = settings
.as_ref()
.and_then(|settings| settings.translation_engine.as_deref());
let mut probes = Vec::with_capacity(enabled.len());
for id in enabled {
probes.push(probe_provider(state, id).await);
}
let translation = match engine {
Some(engine) => Some(probe_engine(state, engine).await),
None => None,
};
SubtitleHealth {
providers: probes,
translation,
alass: binary_check("alass", state.syncer().binary_path()),
ffmpeg: binary_check("ffmpeg", state.ffmpeg_binary()),
}
}
/// Reachable and credentials accepted, per enabled provider (#200).
///
/// A provider that is enabled but was never attached is unconfigured, not
/// unreachable: its credentials are bootstrap config that this deployment
/// simply does not have.
async fn probe_provider(state: &AppState, id: &str) -> ProviderCheck {
let check = match state.subtitle_provider(id) {
Some(provider) => check_from(provider.probe().await),
None => Check::unconfigured(format!(
"{id} is enabled but has no credentials — they are bootstrap config, not a setting"
)),
};
ProviderCheck {
id: id.to_owned(),
status: check.status,
detail: check.detail,
}
}
/// Reachable, credentials accepted — for the remote-command backend,
/// "reachable" means the command ran and exited cleanly (#200).
async fn probe_engine(state: &AppState, engine: &str) -> Check {
match state.translation_backend(engine) {
Some(backend) => backend_check(backend.probe().await),
None => Check::unconfigured(format!(
"'{engine}' is selected but unavailable in this build"
)),
}
}
/// One probe verdict, whatever kind of upstream produced it. A refused key
/// gets its own wording because it never resolves by retrying. The two error
/// enums — providers' and backends' — carry the same shape for this purpose.
fn check_from(result: Result<(), arr_subs::Error>) -> Check {
match result {
Ok(()) => Check::ok(),
Err(arr_subs::Error::Unauthorized { .. }) => Check::unreachable("credentials refused"),
Err(error) => Check::unreachable(error.to_string()),
}
}
fn backend_check(result: Result<(), backend_error::Error>) -> Check {
match result {
Ok(()) => Check::ok(),
Err(backend_error::Error::Unauthorized { .. }) => Check::unreachable("credentials refused"),
Err(error) => Check::unreachable(error.to_string()),
}
}
/// A binary lamp (#200): present and executable at its configured path.
/// Nothing is spawned — the endpoint is polled, and starting `ffmpeg` per
/// poll would be neither cheap nor side-effect-free.
fn binary_check(name: &str, binary: &OsStr) -> Check {
if binary_present(binary) {
Check::ok()
} else {
Check::unreachable(format!("{name} not found at {}", binary.to_string_lossy()))
}
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use arr_db::Db;
use arr_subs::translate as backend;
use arr_subs::{
Backend, CandidateId, DownloadFuture, Provider, ProviderId, SearchFuture, SearchRequest,
Syncer,
};
use crate::{router, AppState, Upstreams};
/// A provider whose lamp is what `lamp` says — the probe under test is
/// ours, so the stub never touches the network.
#[derive(Debug)]
struct StubProvider {
id: &'static str,
up: bool,
}
impl Provider for StubProvider {
fn id(&self) -> ProviderId {
ProviderId::new(self.id)
}
fn search<'a>(&'a self, _request: &'a SearchRequest) -> SearchFuture<'a> {
Box::pin(async move { Ok(Vec::new()) })
}
fn download<'a>(&'a self, _id: &'a CandidateId) -> DownloadFuture<'a> {
unreachable!("health probes never download");
}
fn probe(&self) -> arr_subs::ProbeFuture<'_> {
let result = if self.up {
Ok(())
} else {
Err(arr_subs::Error::Unauthorized {
provider: ProviderId::new(self.id),
})
};
Box::pin(async move { result })
}
}
/// Same idea for the selected engine; the closure rebuilds its error per
/// call because probing borrows.
#[derive(Debug)]
struct StubBackend {
#[allow(dead_code)]
detail: &'static str,
}
impl Backend for StubBackend {
fn id(&self) -> backend::BackendId {
backend::BackendId::new("openai")
}
fn supports(&self, _target: &arr_core::Language) -> bool {
true
}
fn translate<'a>(&'a self, _batch: &'a backend::Batch) -> backend::TranslateFuture<'a> {
unreachable!("health probes never translate");
}
fn probe(&self) -> backend::ProbeFuture<'_> {
Box::pin(async move {
Err(backend::Error::Transport {
backend: backend::BackendId::new("openai"),
source: "connection refused".into(),
})
})
}
}
/// Serve the app on an ephemeral port and return its base URL.
async fn serve(state: AppState) -> String {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind");
let address = listener.local_addr().expect("address");
tokio::spawn(async move { axum::serve(listener, router(state)).await.expect("serve") });
format!("http://{address}")
}
/// An app with a migrated database and both binaries somewhere findable.
/// The three classic upstreams point at a dead port; these tests read the
/// subtitle lamps, not theirs.
async fn application() -> (tempfile::TempDir, AppState) {
let dir = tempfile::tempdir().expect("tempdir");
let database = Db::connect(dir.path().join("arr.db"))
.await
.expect("connect");
database.migrate().await.expect("migrate");
let state = AppState::new(Upstreams::new(
"http://127.0.0.1:1".into(),
"http://127.0.0.1:1".into(),
))
.expect("state")
.with_database(database)
.with_syncer(Syncer::new().with_binary("sh"))
.with_ffmpeg_binary("sh");
(dir, state)
}
async fn report(state: AppState) -> serde_json::Value {
let base = serve(state).await;
reqwest::get(format!("{base}/api/health"))
.await
.expect("request health")
.json()
.await
.expect("health body")
}
/// Replace the seeded enabled set. Runtime-checked rather than going
/// through the settings API, whose validation refuses engines this test
/// binary has not compiled in.
async fn set_enabled(state: &AppState, providers: &str, engine: Option<&str>) {
sqlx::query("UPDATE subtitle_settings SET providers_enabled = ?, translation_engine = ?")
.bind(providers)
.bind(engine)
.execute(state.database().expect("database").pool())
.await
.expect("settings update");
}
#[tokio::test]
async fn without_a_settings_row_the_lane_is_quiet_and_binaries_still_checked() {
// No database attached: nothing is known to be in use, so nothing
// may fail a lamp — but a missing binary is a fact about the host.
let state = AppState::new(Upstreams::new(
"http://127.0.0.1:1".into(),
"http://127.0.0.1:1".into(),
))
.expect("state")
.with_syncer(Syncer::new().with_binary("sh"))
.with_ffmpeg_binary("/nowhere/ffmpeg");
let body = report(state).await;
assert_eq!(body["subtitles"]["providers"], serde_json::json!([]));
assert!(body["subtitles"]["translation"].is_null());
assert_eq!(body["subtitles"]["alass"]["status"], "ok");
assert_eq!(body["subtitles"]["ffmpeg"]["status"], "unreachable");
assert_eq!(body["status"], "degraded");
}
#[tokio::test]
async fn an_enabled_provider_without_credentials_is_unconfigured() {
let (_dir, state) = application().await;
// The seed row enables OpenSubtitles; none is attached.
let body = report(state).await;
assert_eq!(
body["subtitles"]["providers"],
serde_json::json!([
{ "id": "opensubtitles", "status": "unconfigured",
"detail": "opensubtitles is enabled but has no credentials — they are bootstrap config, not a setting" },
])
);
assert_eq!(body["status"], "degraded");
}
#[tokio::test]
async fn an_attached_provider_lamp_follows_its_probe() {
let (_dir, state) = application().await;
set_enabled(&state, r#"["ok","bad"]"#, None).await;
let state = state.with_subtitle_providers(vec![
Arc::new(StubProvider { id: "ok", up: true }),
Arc::new(StubProvider {
id: "bad",
up: false,
}),
]);
let body = report(state).await;
assert_eq!(body["subtitles"]["providers"][0]["id"], "ok");
assert_eq!(body["subtitles"]["providers"][0]["status"], "ok");
assert_eq!(body["subtitles"]["providers"][1]["id"], "bad");
assert_eq!(body["subtitles"]["providers"][1]["status"], "unreachable");
assert_eq!(
body["subtitles"]["providers"][1]["detail"],
"credentials refused"
);
}
#[tokio::test]
async fn a_selected_engine_is_probed_only_when_selected() {
let (_dir, state) = application().await;
let body = report(state.clone()).await;
assert!(body["subtitles"]["translation"].is_null(), "{body}");
set_enabled(&state, "[]", Some("openai")).await;
// Selected but never attached: unavailable in this deployment.
let body = report(state).await;
assert_eq!(body["subtitles"]["translation"]["status"], "unconfigured");
}
#[tokio::test]
async fn an_unreachable_engine_fails_the_whole_report() {
let (_dir, state) = application().await;
set_enabled(&state, "[]", Some("openai")).await;
let state = state.with_translation_backends(vec![Arc::new(StubBackend {
detail: "connection refused",
})]);
let body = report(state).await;
assert_eq!(body["subtitles"]["translation"]["status"], "unreachable");
assert_eq!(body["status"], "degraded");
}
}
+88 -47
View File
@@ -6,18 +6,23 @@
//! carrying a `#[utoipa::path]` annotation. A handler added without one fails
//! to compile, and the gate in DESIGN.md §12 fails with it.
mod downloads;
mod health;
pub mod jellyfin;
mod metadata;
mod movies;
mod owners;
mod policies;
#[cfg(test)]
mod privilege;
mod reclassify;
mod relocate;
mod roots;
mod search;
mod series;
mod state;
mod subtitle_settings;
mod subtitles;
mod trailer;
use axum::routing::get;
@@ -27,6 +32,7 @@ use utoipa_axum::router::OpenApiRouter;
use utoipa_axum::routes;
use utoipa_scalar::{Scalar, Servable};
pub use downloads::{Download, DownloadPhase};
pub use health::{Check, Health, HealthReport, Status};
pub use metadata::{MetadataTrailer, MovieMetadata, SeriesMetadata};
pub use movies::{
@@ -46,6 +52,13 @@ pub use state::{
AppState, EpisodeCommand, MetadataCommand, MovieCommand, SeasonCommand, Upstreams,
DEFAULT_TMDB_URL,
};
pub use subtitle_settings::{SubtitleSettings, SubtitleSettingsInput};
pub use subtitles::{
EmbeddedTrack, EpisodeSubtitleGaps, EpisodeSubtitleStatus, MissingSubtitle, MovieSubtitleGaps,
SeasonSubtitleGaps, SeriesSubtitleGaps, Subtitle, SubtitleCandidate, SubtitleExtractInput,
SubtitleGap, SubtitleGrabInput, SubtitleProviderError, SubtitleQueue, SubtitleSearchInput,
SubtitleSearchResults, SubtitleStatus, SubtitleTranslateInput,
};
pub use trailer::{Trailer, TrailerKind};
/// Where the generated document is served, and where `just gen-client` reads
@@ -71,7 +84,8 @@ pub const DOCS_PATH: &str = "/api/docs";
(name = "owners", description = "Owner tags and filtered views (DESIGN.md §4.3)"),
(name = "policies", description = "Quality policies (DESIGN.md §5)"),
(name = "search", description = "Unified title and release search"),
(name = "roots", description = "Root folders and their policies")
(name = "roots", description = "Root folders and their policies"),
(name = "subtitles", description = "Subtitles and their configuration (DESIGN.md §15)")
),
)]
struct ApiDoc;
@@ -80,6 +94,7 @@ struct ApiDoc;
fn api_router() -> OpenApiRouter<AppState> {
OpenApiRouter::with_openapi(ApiDoc::openapi())
.routes(routes!(health::health))
.routes(routes!(downloads::list))
.routes(routes!(movies::list, movies::create))
.routes(routes!(movies::get, movies::update, movies::delete))
.routes(routes!(movies::search))
@@ -118,6 +133,19 @@ fn api_router() -> OpenApiRouter<AppState> {
.routes(routes!(roots::get, roots::update, roots::delete))
.routes(routes!(policies::list, policies::create))
.routes(routes!(policies::get, policies::update, policies::delete))
.routes(routes!(subtitle_settings::get, subtitle_settings::update))
.routes(routes!(subtitles::list_for_media_file))
.routes(routes!(subtitles::list_for_movie))
.routes(routes!(subtitles::list_for_episode))
.routes(routes!(subtitles::status_for_movie))
.routes(routes!(subtitles::status_for_episode))
.routes(routes!(subtitles::status_for_series))
.routes(routes!(subtitles::search))
.routes(routes!(subtitles::grab))
.routes(routes!(subtitles::extract))
.routes(routes!(subtitles::translate))
.routes(routes!(subtitles::delete))
.routes(routes!(subtitles::queue))
}
/// The generated `OpenAPI` document.
@@ -149,8 +177,8 @@ mod tests {
use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};
/// A Prowlarr that answers `/ping`, and a Transmission that answers an
/// RPC call the way a real one does when it has no session id yet.
/// A Prowlarr that answers `/ping`, and a qBittorrent that answers the
/// version call the way a real one does for an unauthenticated caller.
async fn upstreams_up() -> (MockServer, MockServer) {
let prowlarr = MockServer::start().await;
Mock::given(method("GET"))
@@ -161,16 +189,14 @@ mod tests {
.mount(&prowlarr)
.await;
let transmission = MockServer::start().await;
Mock::given(method("POST"))
.and(path("/transmission/rpc"))
.respond_with(
ResponseTemplate::new(409).insert_header("X-Transmission-Session-Id", "abc"),
)
.mount(&transmission)
let qbit = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/api/v2/torrents/info"))
.respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!([])))
.mount(&qbit)
.await;
(prowlarr, transmission)
(prowlarr, qbit)
}
/// Serve the app on an ephemeral port and return its base URL. The server
@@ -197,7 +223,7 @@ mod tests {
#[tokio::test]
async fn all_upstreams_up_is_ok() {
let (prowlarr, transmission) = upstreams_up().await;
let (prowlarr, qbit) = upstreams_up().await;
let tmdb = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/configuration"))
@@ -206,31 +232,31 @@ mod tests {
.await;
let state = AppState::new(
Upstreams::new(
prowlarr.uri(),
format!("{}/transmission/rpc", transmission.uri()),
)
.with_tmdb_url(tmdb.uri())
.with_tmdb_api_key(Some("key".into())),
Upstreams::new(prowlarr.uri(), qbit.uri())
.with_tmdb_url(tmdb.uri())
.with_tmdb_api_key(Some("key".into())),
)
.expect("state");
.expect("state")
.with_qbit(arr_dl::QbitClient::new(&qbit.uri()).expect("client"))
// The subtitle binaries default to `PATH`; pin them to something
// every machine has so this test stays about the classic upstreams.
.with_syncer(arr_subs::Syncer::new().with_binary("sh"))
.with_ffmpeg_binary("sh");
let body = report(state).await;
assert_eq!(body["status"], "ok");
assert_eq!(body["prowlarr"]["status"], "ok");
assert_eq!(body["transmission"]["status"], "ok");
assert_eq!(body["qbit"]["status"], "ok");
assert_eq!(body["tmdb"]["status"], "ok");
assert_eq!(body["version"], env!("CARGO_PKG_VERSION"));
}
#[tokio::test]
async fn a_missing_tmdb_key_is_unconfigured_not_an_outage() {
let (prowlarr, transmission) = upstreams_up().await;
let state = AppState::new(Upstreams::new(
prowlarr.uri(),
format!("{}/transmission/rpc", transmission.uri()),
))
.expect("state");
let (prowlarr, qbit) = upstreams_up().await;
let state = AppState::new(Upstreams::new(prowlarr.uri(), qbit.uri()))
.expect("state")
.with_qbit(arr_dl::QbitClient::new(&qbit.uri()).expect("client"));
let body = report(state).await;
assert_eq!(body["tmdb"]["status"], "unconfigured");
@@ -239,20 +265,18 @@ mod tests {
#[tokio::test]
async fn an_unreachable_upstream_degrades_the_service() {
let (_prowlarr, transmission) = upstreams_up().await;
let (_prowlarr, qbit) = upstreams_up().await;
// Port 1 is privileged and nothing binds it, so the probe gets a
// refused connection immediately instead of waiting out the timeout.
let state = AppState::new(Upstreams::new(
"http://127.0.0.1:1".into(),
format!("{}/transmission/rpc", transmission.uri()),
))
.expect("state");
let state = AppState::new(Upstreams::new("http://127.0.0.1:1".into(), qbit.uri()))
.expect("state")
.with_qbit(arr_dl::QbitClient::new(&qbit.uri()).expect("client"));
let body = report(state).await;
assert_eq!(body["status"], "degraded");
assert_eq!(body["prowlarr"]["status"], "unreachable");
assert_eq!(body["transmission"]["status"], "ok");
assert_eq!(body["qbit"]["status"], "ok");
}
#[tokio::test]
@@ -263,13 +287,9 @@ mod tests {
.respond_with(ResponseTemplate::new(500))
.mount(&prowlarr)
.await;
let transmission = MockServer::start().await;
let qbit = MockServer::start().await;
let state = AppState::new(Upstreams::new(
prowlarr.uri(),
format!("{}/transmission/rpc", transmission.uri()),
))
.expect("state");
let state = AppState::new(Upstreams::new(prowlarr.uri(), qbit.uri())).expect("state");
let body = report(state).await;
assert_eq!(body["prowlarr"]["status"], "unreachable");
@@ -278,7 +298,7 @@ mod tests {
#[tokio::test]
async fn a_failed_tmdb_probe_never_echoes_the_api_key() {
let (prowlarr, transmission) = upstreams_up().await;
let (prowlarr, qbit) = upstreams_up().await;
let tmdb = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/configuration"))
@@ -287,14 +307,12 @@ mod tests {
.await;
let state = AppState::new(
Upstreams::new(
prowlarr.uri(),
format!("{}/transmission/rpc", transmission.uri()),
)
.with_tmdb_url(tmdb.uri())
.with_tmdb_api_key(Some("super-secret".into())),
Upstreams::new(prowlarr.uri(), qbit.uri())
.with_tmdb_url(tmdb.uri())
.with_tmdb_api_key(Some("super-secret".into())),
)
.expect("state");
.expect("state")
.with_qbit(arr_dl::QbitClient::new(&qbit.uri()).expect("client"));
let body = report(state).await;
assert_eq!(body["tmdb"]["status"], "unreachable");
@@ -359,6 +377,23 @@ mod tests {
("/api/policies/{policy_id}", "put"),
("/api/policies/{policy_id}", "delete"),
("/api/roots", "post"),
("/api/settings/subtitles", "get"),
("/api/settings/subtitles", "put"),
("/api/media-files/{media_file_id}/subtitles", "get"),
("/api/movies/{movie_id}/subtitles", "get"),
("/api/episodes/{episode_id}/subtitles", "get"),
("/api/movies/{movie_id}/subtitles/status", "get"),
("/api/episodes/{episode_id}/subtitles/status", "get"),
("/api/series/{series_id}/subtitles/status", "get"),
("/api/media-files/{media_file_id}/subtitles/search", "post"),
("/api/media-files/{media_file_id}/subtitles/grab", "post"),
("/api/media-files/{media_file_id}/subtitles/extract", "post"),
(
"/api/media-files/{media_file_id}/subtitles/translate",
"post",
),
("/api/subtitles/{subtitle_id}", "delete"),
("/api/queues/subtitles", "get"),
] {
assert!(
json["paths"][path][method].is_object(),
@@ -373,6 +408,12 @@ mod tests {
"AttentionQueues",
"SeriesAttention",
"Series",
"Subtitle",
"SubtitleCandidate",
"SubtitleSearchResults",
"SubtitleStatus",
"MissingSubtitle",
"EpisodeSubtitleStatus",
] {
assert!(
json["components"]["schemas"][schema].is_object(),
+169 -14
View File
@@ -180,6 +180,11 @@ pub struct Accepted {
#[derive(Debug, Clone, Serialize, ToSchema)]
pub struct ErrorBody {
pub error: String,
/// A machine-readable discriminant, set only where a client needs to
/// branch on the failure rather than display it (issue #221). `None`
/// everywhere else — the message is for the operator, not the client.
#[serde(skip_serializing_if = "Option::is_none")]
pub code: Option<String>,
}
#[derive(Debug)]
@@ -191,6 +196,22 @@ pub enum ApiError {
OwnerNotFound,
PolicyNotFound,
RootNotFound,
/// A `media_files` row that is not there. Named apart from
/// [`Self::NotFound`] because the subtitle surface (§15) is keyed on
/// files, not on titles, and "movie not found" would misdirect.
MediaFileNotFound,
SubtitleNotFound,
/// A subtitle provider or translation backend could not do what a manual
/// action asked (§15). Carries the message because two providers and
/// four engines are configurable at once and an unnamed failure is
/// unactionable.
SubtitleUpstream(String),
/// A grab named a `candidate_id` the provider no longer recognises
/// (issue #221): a search's results outlive the search itself only in
/// the client's memory, and the provider can expire one at will. Kept
/// apart from [`Self::SubtitleUpstream`] so the panel can offer "search
/// again" from a `code`, not from matching the message text.
SubtitleCandidateExpired,
/// The §9.6 chip outcome: the title exists upstream but has no trailer.
/// Ordinary, so it must stay distinguishable from an upstream failure.
NoTrailer,
@@ -211,38 +232,60 @@ pub enum ApiError {
impl IntoResponse for ApiError {
fn into_response(self) -> Response {
let (status, error) = match self {
Self::NotFound => (StatusCode::NOT_FOUND, "movie not found".to_string()),
Self::SeriesNotFound => (StatusCode::NOT_FOUND, "series not found".to_string()),
Self::SeasonNotFound => (StatusCode::NOT_FOUND, "season not found".to_string()),
Self::EpisodeNotFound => (StatusCode::NOT_FOUND, "episode not found".to_string()),
Self::OwnerNotFound => (StatusCode::NOT_FOUND, "owner not found".to_string()),
Self::PolicyNotFound => (StatusCode::NOT_FOUND, "policy not found".to_string()),
Self::RootNotFound => (StatusCode::NOT_FOUND, "root not found".to_string()),
Self::NoTrailer => (StatusCode::NOT_FOUND, "no trailer".to_string()),
Self::Conflict(error) => (StatusCode::CONFLICT, error),
Self::Invalid(error) => (StatusCode::UNPROCESSABLE_ENTITY, error),
let (status, error, code) = match self {
Self::NotFound => (StatusCode::NOT_FOUND, "movie not found".to_string(), None),
Self::SeriesNotFound => (StatusCode::NOT_FOUND, "series not found".to_string(), None),
Self::SeasonNotFound => (StatusCode::NOT_FOUND, "season not found".to_string(), None),
Self::EpisodeNotFound => (StatusCode::NOT_FOUND, "episode not found".to_string(), None),
Self::OwnerNotFound => (StatusCode::NOT_FOUND, "owner not found".to_string(), None),
Self::PolicyNotFound => (StatusCode::NOT_FOUND, "policy not found".to_string(), None),
Self::RootNotFound => (StatusCode::NOT_FOUND, "root not found".to_string(), None),
Self::MediaFileNotFound => (
StatusCode::NOT_FOUND,
"media file not found".to_string(),
None,
),
Self::SubtitleNotFound => (
StatusCode::NOT_FOUND,
"subtitle not found".to_string(),
None,
),
Self::SubtitleUpstream(error) => (StatusCode::SERVICE_UNAVAILABLE, error, None),
Self::SubtitleCandidateExpired => (
StatusCode::NOT_FOUND,
"candidate no longer exists — search again".to_string(),
Some("candidate_expired".to_string()),
),
Self::NoTrailer => (StatusCode::NOT_FOUND, "no trailer".to_string(), None),
Self::Conflict(error) => (StatusCode::CONFLICT, error, None),
Self::Invalid(error) => (StatusCode::UNPROCESSABLE_ENTITY, error, None),
Self::Unavailable => (
StatusCode::SERVICE_UNAVAILABLE,
"database unavailable".into(),
None,
),
Self::Upstream(name) => (
StatusCode::SERVICE_UNAVAILABLE,
format!("{name} unavailable"),
None,
),
Self::Database(error) => {
tracing::error!(%error, "API database error");
(StatusCode::INTERNAL_SERVER_ERROR, "database error".into())
(
StatusCode::INTERNAL_SERVER_ERROR,
"database error".into(),
None,
)
}
// The message is the caller's: this variant is returned by the
// delete lane and by the relocate lane, and "files not removed"
// is a lie about a move that failed.
Self::Filesystem(error) => {
tracing::error!(%error, "API filesystem error");
(StatusCode::INTERNAL_SERVER_ERROR, error.clone())
(StatusCode::INTERNAL_SERVER_ERROR, error.clone(), None)
}
};
(status, Json(ErrorBody { error })).into_response()
(status, Json(ErrorBody { error, code })).into_response()
}
}
@@ -553,6 +596,14 @@ pub async fn delete(
)
.execute(pool(&state)?)
.await?;
// `grabs.infohash` is unique, so an orphan left here would block the row
// a later grab of the same release needs to write.
sqlx::query!(
"DELETE FROM grabs WHERE target_kind = 'movie' AND target_id = ?",
id
)
.execute(pool(&state)?)
.await?;
let result = sqlx::query!("DELETE FROM movies WHERE id = ?", id)
.execute(pool(&state)?)
.await?;
@@ -1589,6 +1640,46 @@ mod tests {
assert_eq!(orphans, 0, "the file rows go with the files");
}
/// `grabs.infohash` is unique, so a row left behind by a deleted title
/// blocks the row a later grab of the same release needs to write — and
/// the grab records nothing while the new title still reads as
/// downloading.
#[tokio::test]
async fn deleting_a_movie_takes_its_grabs_with_it() {
let (_dir, state, base) = application().await;
let movie = add_movie(&base, 693_134, 1).await;
let id = movie["id"].as_i64().expect("id");
let pool = state.database().expect("database").pool();
sqlx::query(
"INSERT INTO releases (id, indexer_id, guid, name, size, download_url, parsed)
VALUES (1, 1, 'guid', 'release', 1, 'magnet:?x', '{}')",
)
.execute(pool)
.await
.expect("release");
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash, state)
VALUES (1, 'movie', ?, 'deadbeef', 'sent')",
)
.bind(id)
.execute(pool)
.await
.expect("grab");
let response = reqwest::Client::new()
.delete(format!("{base}/api/movies/{id}"))
.send()
.await
.expect("delete");
assert_eq!(response.status(), StatusCode::NO_CONTENT);
let orphans: i64 = sqlx::query_scalar("SELECT count(*) FROM grabs")
.fetch_one(pool)
.await
.expect("count grabs");
assert_eq!(orphans, 0, "no grab outlives the title it was for");
}
/// The guard that keeps a delete inside the library: a path that is not
/// under the title's root is left alone, whatever the row says.
#[tokio::test]
@@ -1704,6 +1795,66 @@ mod tests {
);
}
/// Issue #237: the folder rename #228 wired up already carries a sidecar
/// with it, but the `subtitle_files` row pointing at the old path did not
/// follow — it went stale silently, and `DELETE /api/subtitles/{id}`
/// (#218) would then report success while leaving the real file on disk.
#[tokio::test]
async fn changing_root_moves_the_subtitle_row_with_its_sidecar() {
let (_dir, state, base) = application().await;
let movie = add_movie(&base, 693_134, 1).await;
let id = movie["id"].as_i64().expect("id");
let source = tempfile::tempdir().expect("source root");
let destination = tempfile::tempdir().expect("destination root");
let folder = library_on_disk(&state, id, source.path()).await;
point_root_at(&state, 2, destination.path()).await;
let pool = state.database().expect("database").pool();
let media_file_id: i64 = sqlx::query_scalar(
"SELECT id FROM media_files WHERE owner_kind = 'movie' AND owner_id = ?",
)
.bind(id)
.fetch_one(pool)
.await
.expect("media file id");
let sidecar = folder.join("dune.pt.srt");
sqlx::query(
"INSERT INTO subtitle_files (media_file_id, language, origin, provider, path)
VALUES (?, 'pt-PT', 'provider', 'opensubtitles', ?)",
)
.bind(media_file_id)
.bind(sidecar.to_str().expect("utf-8 path"))
.execute(pool)
.await
.expect("subtitle file");
let updated: serde_json::Value = reqwest::Client::new()
.patch(format!("{base}/api/movies/{id}"))
.json(&serde_json::json!({"root_id": 2}))
.send()
.await
.expect("move root")
.json()
.await
.expect("updated json");
assert_eq!(updated["root_id"], 2);
let path: String =
sqlx::query_scalar("SELECT path FROM subtitle_files WHERE media_file_id = ?")
.bind(media_file_id)
.fetch_one(pool)
.await
.expect("subtitle file row");
assert!(
std::path::Path::new(&path).starts_with(destination.path()),
"the subtitle row follows the file: {path}"
);
assert!(
std::path::Path::new(&path).exists(),
"the rewritten subtitle path describes the disk"
);
}
/// Reported in production: moving a title into a root whose directory has
/// never been written to failed with
/// `could not move '...The Batman...': No such file or directory`, naming
@@ -1841,6 +1992,10 @@ mod tests {
async fn a_failed_rename_leaves_the_row_alone() {
use std::os::unix::fs::PermissionsExt;
if !crate::privilege::mode_bits_bind().await {
crate::privilege::skipped_because_privileged("a_failed_rename_leaves_the_row_alone");
return;
}
let (_dir, state, base) = application().await;
let movie = add_movie(&base, 693_134, 1).await;
let id = movie["id"].as_i64().expect("id");
+36
View File
@@ -0,0 +1,36 @@
//! Whether mode bits actually constrain this process.
//!
//! Four tests force a filesystem failure by freezing a directory to `0o555`
//! and asserting the handler reports it. Mode bits do not constrain a
//! privileged user, so under `root` — which is what the CI container runs
//! as — the operation succeeds and the assertion fails for a reason that has
//! nothing to do with the code under test.
//!
//! The probe asks the filesystem rather than asking for the uid: what the
//! tests depend on is the refusal, not the identity, and a container can
//! hold `CAP_DAC_OVERRIDE` without being uid 0.
/// True when a read-only directory refuses a write to this process.
pub(crate) async fn mode_bits_bind() -> bool {
use std::os::unix::fs::PermissionsExt;
let probe = tempfile::tempdir().expect("probe root");
let frozen = probe.path().join("frozen");
tokio::fs::create_dir(&frozen).await.expect("probe folder");
tokio::fs::set_permissions(&frozen, std::fs::Permissions::from_mode(0o555))
.await
.expect("freeze the probe folder");
let refused = tokio::fs::write(frozen.join("probe"), b"x").await.is_err();
// `TempDir::drop` needs the write back to remove the tree
tokio::fs::set_permissions(&frozen, std::fs::Permissions::from_mode(0o755))
.await
.expect("thaw the probe folder");
refused
}
/// Announces a test that cannot run here, on stderr, which `nextest` prints
/// when the run is given `--no-capture` and swallows otherwise — the same
/// deal every other skipped case in a Rust suite gets.
pub(crate) fn skipped_because_privileged(test: &str) {
eprintln!("{test}: skipped — this process overrides mode bits, so the failure it forces cannot happen");
}
+149 -51
View File
@@ -1,9 +1,12 @@
//! Moving library files when the layout under them changes: a title changing
//! its `root_id` (issue #228), and a root changing its `path` (issue #236).
//! Both rename §7.4 folders and rewrite the `media_files` rows to match, so
//! the layout keeps describing the disk. This module only moves things: the
//! stored verdicts a root change invalidates (§5.1) are re-derived by the
//! calling handler through `reclassify`, after the row commits.
//! Both rename §7.4 folders and rewrite the `media_files` and `subtitle_files`
//! rows to match, so the layout keeps describing the disk. Sidecars (§15) live
//! inside the same folder as the video, so the rename already carries them —
//! only their rows need rewriting (issue #237). This module only moves
//! things: the stored verdicts a root change invalidates (§5.1) are
//! re-derived by the calling handler through `reclassify`, after the row
//! commits.
//!
//! Every root shares one filesystem — one ZFS dataset, bind-mounted — so this
//! is a directory rename, never a copy. Hardlinked files keep their inodes
@@ -35,6 +38,15 @@ pub(crate) enum TitleKind {
Series,
}
/// Which table a rewritten row belongs to — `rewrite_rows` needs this to
/// target the right `UPDATE`, since a `media_files` id and a `subtitle_files`
/// id share no namespace.
#[derive(Debug, Clone, Copy)]
enum FileTable {
Media,
Subtitle,
}
/// One rename from the old root into the new one: a §7.4 title folder, or a
/// loose file sitting straight in the root.
#[derive(Debug)]
@@ -49,7 +61,7 @@ struct PlannedRename {
#[derive(Debug)]
pub(crate) struct Relocation {
performed: Vec<PlannedRename>,
rewrites: Vec<(i64, String)>,
rewrites: Vec<(FileTable, i64, String)>,
/// Every directory level this request materialised for the new root,
/// deepest first. `create_dir_all` can make more than one — moving a
/// root to `/mnt/media-v2/tv/kids` when `/mnt/media-v2` is all that
@@ -112,13 +124,13 @@ pub(crate) async fn relocate_root(
/// destination that already exists, then perform them, undoing what was
/// performed if one fails.
async fn relocate_files(
files: &[(i64, String)],
files: &[(FileTable, i64, String)],
old_root: &str,
new_root: &str,
) -> Result<Relocation, ApiError> {
let mut renames: Vec<PlannedRename> = Vec::new();
let mut rewrites: Vec<(i64, String)> = Vec::new();
for (file_id, path) in files {
let mut rewrites: Vec<(FileTable, i64, String)> = Vec::new();
for (table, file_id, path) in files {
let Some(source) = title_target(old_root, path) else {
// Outside its own root: not ours to move, and the row keeps
// pointing at where the file really is.
@@ -144,7 +156,7 @@ async fn relocate_files(
"non-UTF-8 path under {new_root}"
)));
};
rewrites.push((*file_id, rewritten.to_owned()));
rewrites.push((*table, *file_id, rewritten.to_owned()));
}
// Every destination is checked before anything is renamed, so a conflict
@@ -275,21 +287,34 @@ impl Relocation {
!self.performed.is_empty()
}
/// Point the `media_files` rows at the new root, inside the caller's
/// transaction so they land together with the `root_id` change or not at
/// all.
/// Point the `media_files` and `subtitle_files` rows at the new root,
/// inside the caller's transaction so they land together with the
/// `root_id` change or not at all.
pub(crate) async fn rewrite_rows(
&self,
transaction: &mut sqlx::Transaction<'_, sqlx::Sqlite>,
) -> Result<(), sqlx::Error> {
for (file_id, path) in &self.rewrites {
sqlx::query!(
"UPDATE media_files SET path = ?, updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now') WHERE id = ?",
path,
file_id
)
.execute(&mut **transaction)
.await?;
for (table, file_id, path) in &self.rewrites {
match table {
FileTable::Media => {
sqlx::query!(
"UPDATE media_files SET path = ?, updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now') WHERE id = ?",
path,
file_id
)
.execute(&mut **transaction)
.await?;
}
FileTable::Subtitle => {
sqlx::query!(
"UPDATE subtitle_files SET path = ?, updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now') WHERE id = ?",
path,
file_id
)
.execute(&mut **transaction)
.await?;
}
}
}
Ok(())
}
@@ -346,12 +371,17 @@ async fn root_path(state: &AppState, root_id: i64) -> Result<String, ApiError> {
.await?)
}
/// Every file the service recorded under a root: the rows of every movie in
/// it, and the rows of every episode of every series in it. Ordered so the
/// Every file the service recorded under a root: the `media_files` rows of
/// every movie and episode in it, and the `subtitle_files` sidecars (§15) on
/// those files — an embedded track has no `path` and nothing on disk to
/// carry, so it is excluded rather than rewritten to nothing. Ordered so the
/// renames happen in a stable order, which is what makes a failure part-way
/// through reproducible.
async fn root_files(state: &AppState, root_id: i64) -> Result<Vec<(i64, String)>, ApiError> {
let mut files: Vec<(i64, String)> = sqlx::query!(
async fn root_files(
state: &AppState,
root_id: i64,
) -> Result<Vec<(FileTable, i64, String)>, ApiError> {
let mut files: Vec<(FileTable, i64, String)> = sqlx::query!(
r#"SELECT mf.id AS "id!: i64", mf.path AS "path!: String"
FROM media_files mf
JOIN movies m ON mf.owner_kind = 'movie' AND m.id = mf.owner_id
@@ -362,7 +392,7 @@ async fn root_files(state: &AppState, root_id: i64) -> Result<Vec<(i64, String)>
.fetch_all(pool(state)?)
.await?
.into_iter()
.map(|row| (row.id, row.path))
.map(|row| (FileTable::Media, row.id, row.path))
.collect();
files.extend(
sqlx::query!(
@@ -378,41 +408,109 @@ async fn root_files(state: &AppState, root_id: i64) -> Result<Vec<(i64, String)>
.fetch_all(pool(state)?)
.await?
.into_iter()
.map(|row| (row.id, row.path)),
.map(|row| (FileTable::Media, row.id, row.path)),
);
files.extend(
sqlx::query!(
r#"SELECT sf.id AS "id!: i64", sf.path AS "path!: String"
FROM subtitle_files sf
JOIN media_files mf ON mf.id = sf.media_file_id
JOIN movies m ON mf.owner_kind = 'movie' AND m.id = mf.owner_id
WHERE m.root_id = ? AND sf.path IS NOT NULL
ORDER BY sf.id"#,
root_id
)
.fetch_all(pool(state)?)
.await?
.into_iter()
.map(|row| (FileTable::Subtitle, row.id, row.path)),
);
files.extend(
sqlx::query!(
r#"SELECT sf.id AS "id!: i64", sf.path AS "path!: String"
FROM subtitle_files sf
JOIN media_files mf ON mf.id = sf.media_file_id
JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id
JOIN seasons se ON se.id = e.season_id
JOIN series s ON s.id = se.series_id
WHERE s.root_id = ? AND sf.path IS NOT NULL
ORDER BY sf.id"#,
root_id
)
.fetch_all(pool(state)?)
.await?
.into_iter()
.map(|row| (FileTable::Subtitle, row.id, row.path)),
);
Ok(files)
}
/// Every file the service recorded for the title: a movie's own rows, or the
/// rows of every episode below a series.
/// rows of every episode below a series — plus the `subtitle_files` sidecars
/// (§15) on those files. An embedded track has no `path` and nothing on disk
/// to carry, so it is excluded rather than rewritten to nothing.
async fn title_files(
state: &AppState,
kind: TitleKind,
title_id: i64,
) -> Result<Vec<(i64, String)>, ApiError> {
) -> Result<Vec<(FileTable, i64, String)>, ApiError> {
Ok(match kind {
TitleKind::Movie => sqlx::query!(
r#"SELECT id AS "id!: i64", path AS "path!: String"
FROM media_files WHERE owner_kind = 'movie' AND owner_id = ?"#,
title_id
)
.fetch_all(pool(state)?)
.await?
.into_iter()
.map(|row| (row.id, row.path))
.collect(),
TitleKind::Series => sqlx::query!(
r#"SELECT mf.id AS "id!: i64", mf.path AS "path!: String"
FROM media_files mf
JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id
JOIN seasons se ON se.id = e.season_id
WHERE se.series_id = ?"#,
title_id
)
.fetch_all(pool(state)?)
.await?
.into_iter()
.map(|row| (row.id, row.path))
.collect(),
TitleKind::Movie => {
let mut files: Vec<(FileTable, i64, String)> = sqlx::query!(
r#"SELECT id AS "id!: i64", path AS "path!: String"
FROM media_files WHERE owner_kind = 'movie' AND owner_id = ?"#,
title_id
)
.fetch_all(pool(state)?)
.await?
.into_iter()
.map(|row| (FileTable::Media, row.id, row.path))
.collect();
files.extend(
sqlx::query!(
r#"SELECT sf.id AS "id!: i64", sf.path AS "path!: String"
FROM subtitle_files sf
JOIN media_files mf ON mf.id = sf.media_file_id
WHERE mf.owner_kind = 'movie' AND mf.owner_id = ? AND sf.path IS NOT NULL"#,
title_id
)
.fetch_all(pool(state)?)
.await?
.into_iter()
.map(|row| (FileTable::Subtitle, row.id, row.path)),
);
files
}
TitleKind::Series => {
let mut files: Vec<(FileTable, i64, String)> = sqlx::query!(
r#"SELECT mf.id AS "id!: i64", mf.path AS "path!: String"
FROM media_files mf
JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id
JOIN seasons se ON se.id = e.season_id
WHERE se.series_id = ?"#,
title_id
)
.fetch_all(pool(state)?)
.await?
.into_iter()
.map(|row| (FileTable::Media, row.id, row.path))
.collect();
files.extend(
sqlx::query!(
r#"SELECT sf.id AS "id!: i64", sf.path AS "path!: String"
FROM subtitle_files sf
JOIN media_files mf ON mf.id = sf.media_file_id
JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id
JOIN seasons se ON se.id = e.season_id
WHERE se.series_id = ? AND sf.path IS NOT NULL"#,
title_id
)
.fetch_all(pool(state)?)
.await?
.into_iter()
.map(|row| (FileTable::Subtitle, row.id, row.path)),
);
files
}
})
}
+22 -1
View File
@@ -27,7 +27,7 @@ pub struct Root {
/// The payload for creating or replacing a root.
#[derive(Debug, Clone, Deserialize, ToSchema)]
pub struct RootInput {
/// `movie` or `tv` — the Transmission label and layout prefix (§7.1, §7.4).
/// `movie` or `tv` — the qBittorrent label and layout prefix (§7.1, §7.4).
pub kind: String,
/// `main` or `kids`.
pub audience: String,
@@ -1048,6 +1048,13 @@ mod tests {
async fn one_folder_that_cannot_move_puts_the_others_back() {
use std::os::unix::fs::PermissionsExt;
if !crate::privilege::mode_bits_bind().await {
crate::privilege::skipped_because_privileged(
"one_folder_that_cannot_move_puts_the_others_back",
);
return;
}
let (_dir, state, base) = application().await;
let old = tempfile::tempdir().expect("old root");
let home = tempfile::tempdir().expect("home of the new path");
@@ -1125,6 +1132,13 @@ mod tests {
async fn a_new_root_that_already_existed_survives_a_failed_move() {
use std::os::unix::fs::PermissionsExt;
if !crate::privilege::mode_bits_bind().await {
crate::privilege::skipped_because_privileged(
"a_new_root_that_already_existed_survives_a_failed_move",
);
return;
}
let (_dir, state, base) = application().await;
let old = tempfile::tempdir().expect("old root");
let new = tempfile::tempdir().expect("new root, already there");
@@ -1393,6 +1407,13 @@ mod tests {
async fn a_failed_move_removes_every_level_it_created() {
use std::os::unix::fs::PermissionsExt;
if !crate::privilege::mode_bits_bind().await {
crate::privilege::skipped_because_privileged(
"a_failed_move_removes_every_level_it_created",
);
return;
}
let (_dir, state, base) = application().await;
let old = tempfile::tempdir().expect("old root");
let home = tempfile::tempdir().expect("home of the new path");
+354 -14
View File
@@ -65,9 +65,11 @@ pub struct Series {
pub metadata_refreshed_at: Option<String>,
/// `airing`, `incomplete`, `waiting`, `complete` or `ended` (§4.2).
pub status: String,
/// Episodes currently marked wanted (§4.1 — the only intent).
pub wanted_episodes: i64,
/// Wanted episodes already on disk.
/// Episodes this series accounts for: wanted, on disk, or both (§4.1).
/// A season grabbed once and then untracked keeps its files counted —
/// nothing wanted is not the same as nothing there.
pub total_episodes: i64,
/// Counted episodes already on disk.
pub available_episodes: i64,
}
@@ -317,10 +319,15 @@ fn with_status(row: &SeriesRow, episodes: &[arr_core::Episode], now: SystemTime)
// too. A series reading `complete` next to `42/52 eps` is the confusion
// this avoids. The season number rides on each episode (#131), so no
// parallel slice can be forgotten.
let wanted = episodes
.iter()
.filter(|episode| episode.wanted && episode.season_number != 0);
let available = wanted
//
// The denominator is wanted-or-on-disk, not wanted alone. Untracking a
// season clears `wanted` on every episode (§4.1) while the files stay,
// so counting intent alone reads `0/0 eps` next to a season of green
// check glyphs.
let counted = episodes.iter().filter(|episode| {
episode.season_number != 0 && (episode.wanted || episode.state == MediaState::Available)
});
let available = counted
.clone()
.filter(|episode| episode.state == MediaState::Available);
Series {
@@ -339,7 +346,7 @@ fn with_status(row: &SeriesRow, episodes: &[arr_core::Episode], now: SystemTime)
vote_average: row.vote_average,
metadata_refreshed_at: row.metadata_refreshed_at.clone(),
status: status_name(derive_series_status(&core_series(row), episodes, now)).to_owned(),
wanted_episodes: i64::try_from(wanted.count()).unwrap_or(i64::MAX),
total_episodes: i64::try_from(counted.count()).unwrap_or(i64::MAX),
available_episodes: i64::try_from(available.count()).unwrap_or(i64::MAX),
}
}
@@ -672,6 +679,23 @@ pub async fn delete(
)
.execute(pool(&state)?)
.await?;
// Seasons and episodes go with the series by cascade, but grabs point at
// them polymorphically and have no foreign key to follow. `infohash` is
// unique, so an orphan left here would block a later grab of the same
// release from recording anything.
sqlx::query!(
"DELETE FROM grabs
WHERE (target_kind = 'season'
AND target_id IN (SELECT id FROM seasons WHERE series_id = ?))
OR (target_kind = 'episode'
AND target_id IN (SELECT e.id FROM episodes e
JOIN seasons s ON s.id = e.season_id
WHERE s.series_id = ?))",
id,
id
)
.execute(pool(&state)?)
.await?;
let result = sqlx::query!("DELETE FROM series WHERE id = ?", id)
.execute(pool(&state)?)
.await?;
@@ -708,8 +732,9 @@ enum FileScope {
/// subfolders, sidecar subtitles and artwork go with it. A season or a single
/// episode resolves to the recorded file and nothing else — the title folder
/// holds the siblings this call must not touch, and a season subfolder would
/// have to be re-derived to be named, which §2 forbids. Sidecars beside a
/// removed episode therefore stay; they are not rows this service wrote.
/// have to be re-derived to be named, which §2 forbids. Subtitle sidecars are
/// rows this service wrote too (#186), so they are resolved and unlinked the
/// same way as the video they sit beside (#218).
///
/// The torrent is untouched (§7.3). It keeps seeding under its own rule and
/// the reaper deletes it; a hardlinked file loses only its library name.
@@ -718,7 +743,8 @@ enum FileScope {
/// recorded and can retry rather than losing the record of what is on disk.
async fn remove_library_files(state: &AppState, scope: FileScope) -> Result<(), ApiError> {
let root = scope_root(state, scope).await?;
let paths = scope_paths(state, scope).await?;
let mut paths = scope_paths(state, scope).await?;
paths.extend(subtitle_paths(state, scope).await?);
let mut targets: Vec<std::path::PathBuf> = Vec::new();
for path in &paths {
@@ -838,6 +864,50 @@ async fn scope_paths(state: &AppState, scope: FileScope) -> Result<Vec<String>,
})
}
/// Every subtitle sidecar (#186) this service wrote for the scope. Mirrors
/// [`scope_paths`]: same owner filter, same guard against widening. An
/// embedded track has no `path` — it never touched disk — and is excluded.
async fn subtitle_paths(state: &AppState, scope: FileScope) -> Result<Vec<String>, ApiError> {
Ok(match scope {
FileScope::Series(id) => {
sqlx::query_scalar!(
r#"SELECT sf.path AS "path!: String"
FROM subtitle_files sf
JOIN media_files mf ON mf.id = sf.media_file_id
JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id
JOIN seasons se ON se.id = e.season_id
WHERE se.series_id = ? AND sf.path IS NOT NULL"#,
id
)
.fetch_all(pool(state)?)
.await?
}
FileScope::Season(id) => {
sqlx::query_scalar!(
r#"SELECT sf.path AS "path!: String"
FROM subtitle_files sf
JOIN media_files mf ON mf.id = sf.media_file_id
JOIN episodes e ON mf.owner_kind = 'episode' AND e.id = mf.owner_id
WHERE e.season_id = ? AND sf.path IS NOT NULL"#,
id
)
.fetch_all(pool(state)?)
.await?
}
FileScope::Episode(id) => {
sqlx::query_scalar!(
r#"SELECT sf.path AS "path!: String"
FROM subtitle_files sf
JOIN media_files mf ON mf.id = sf.media_file_id
WHERE mf.owner_kind = 'episode' AND mf.owner_id = ? AND sf.path IS NOT NULL"#,
id
)
.fetch_all(pool(state)?)
.await?
}
})
}
/// The one recorded file, when it really sits inside the root. `None` when it
/// does not, which is the guard that keeps a sub-series delete inside the
/// library it belongs to.
@@ -1602,7 +1672,7 @@ pub async fn season_releases(
/// A grab for this season that downloaded in full and was then condemned at
/// import (#227, §5.7).
///
/// The torrent stays at 100% in Transmission — §7.3 leaves that lifecycle to
/// The torrent stays at 100% in qBittorrent — §7.3 leaves that lifecycle to
/// the reaper — the release is blacklisted and every episode it was covering
/// reopens as a gap. Nothing on screen connected the two, so the season read
/// `0/10` as though no grab had ever been tried. Every fact is already
@@ -1652,6 +1722,22 @@ async fn season_import_failures(
WHERE f.owner_kind = 'episode' AND f.owner_id = e.id
)
)
-- A later attempt owns the headline while it is still in
-- flight. Keep the failed grab recorded; it is simply no
-- longer the current explanation for the gap.
AND NOT EXISTS (
SELECT 1 FROM grabs newer
WHERE newer.target_kind = 'season'
AND newer.target_id = g.target_id
-- 'vanished' is not a live attempt either: the
-- torrent left qBittorrent, so letting it take the
-- headline would leave the season saying nothing at
-- all about a gap that still exists.
AND newer.state NOT IN ('failed', 'vanished')
AND (newer.grabbed_at > coalesce(g.failed_at, g.grabbed_at)
OR (newer.grabbed_at = coalesce(g.failed_at, g.grabbed_at)
AND newer.id > g.id))
)
ORDER BY coalesce(g.failed_at, g.grabbed_at), g.id"#,
series_id
)
@@ -1725,7 +1811,7 @@ pub struct SeasonPackState {
/// #227. The last pack that downloaded in full and was then condemned at
/// import, while the season is still waiting for a file. A deck that
/// cannot say this leaves the season reading as though nothing was ever
/// tried, with the torrent still sitting at 100% in Transmission.
/// tried, with the torrent still sitting at 100% in qBittorrent.
pub import_failure: Option<ImportFailure>,
}
@@ -2057,6 +2143,7 @@ mod tests {
/// season read `0/10` with nothing anywhere saying a grab had been tried.
/// The season row, its deck and the deck's blacklisted release now each
/// carry the failure and the reason it failed on.
#[allow(clippy::too_many_lines)]
#[tokio::test]
async fn an_abandoned_pack_is_visible_on_the_season_and_its_deck() {
let (_dir, state, base) = application().await;
@@ -2115,6 +2202,49 @@ mod tests {
.await
.expect("failed pack grab");
// A newer attempt supersedes the old failure while it is in flight.
let newer_name = "Rick.And.Morty.S08.720p.WEB-DL.x264-NEW";
let newer_release_id = sqlx::query_scalar::<_, i64>(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, verdict)
VALUES (7, 'new-pack', ?, 1000, 'url', ?, 'eligible') RETURNING id",
)
.bind(newer_name)
.bind(serde_json::to_string(&parsed).expect("parsed json"))
.fetch_one(pool)
.await
.expect("newer release");
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash, state, grabbed_at)
VALUES (?, 'season', ?, 'new123', 'sent', '2026-01-03T00:00:00.000Z')",
)
.bind(newer_release_id)
.bind(season_id)
.execute(pool)
.await
.expect("newer in-flight grab");
let seasons = seasons_json(&base, series_id).await;
assert!(
seasons[0]["import_failure"].is_null(),
"a newer in-flight grab supersedes the old failure"
);
let pack_state: serde_json::Value = reqwest::get(format!(
"{base}/api/series/{series_id}/seasons/8/pack-state"
))
.await
.expect("pack state")
.json()
.await
.expect("pack state json");
assert!(
pack_state["import_failure"].is_null(),
"the pack-state endpoint uses the same supersession rule"
);
sqlx::query("DELETE FROM grabs WHERE infohash = 'new123'")
.execute(pool)
.await
.expect("remove test attempt");
// The failure with no blacklist row yet: still a failure, and the
// reason is simply not known. Rows written before the blacklist
// carried one read this way and must keep rendering.
@@ -3003,7 +3133,7 @@ mod tests {
listed[0]["status"], "incomplete",
"§4.2: an aired wanted episode with no file"
);
assert_eq!(listed[0]["wanted_episodes"], 1);
assert_eq!(listed[0]["total_episodes"], 1);
assert_eq!(listed[0]["available_episodes"], 0);
sqlx::query("UPDATE episodes SET state = 'available' WHERE id = ?")
@@ -3534,6 +3664,64 @@ mod tests {
assert_eq!(orphans, 0, "the file rows go with the files");
}
/// Seasons and episodes cascade off the series row, but grabs point at
/// them polymorphically with no foreign key to follow. `grabs.infohash`
/// is unique, so one left behind blocks the row a later grab of the same
/// pack or episode needs to write.
#[tokio::test]
async fn deleting_a_series_takes_its_season_and_episode_grabs_with_it() {
let (_dir, state, base) = application().await;
let root_id = tv_root(&state, "main").await;
let series = add_series(&base, root_id, true).await;
let series_id = series["id"].as_i64().expect("id");
let season = add_season(
&base,
series_id,
1,
serde_json::json!([{"number": 1, "title": "Pilot", "air_date": "2001-01-01"}]),
)
.await;
let season_id = season["id"].as_i64().expect("season id");
let episode_id = season["episodes"][0]["id"].as_i64().expect("episode id");
let pool = state.database().expect("database").pool();
sqlx::query(
"INSERT INTO releases (id, indexer_id, guid, name, size, download_url, parsed)
VALUES (1, 1, 'guid', 'release', 1, 'magnet:?x', '{}')",
)
.execute(pool)
.await
.expect("release");
for (kind, target, hash) in [
("season", season_id, "aaaa"),
("episode", episode_id, "bbbb"),
] {
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash, state)
VALUES (1, ?, ?, ?, 'sent')",
)
.bind(kind)
.bind(target)
.bind(hash)
.execute(pool)
.await
.expect("grab");
}
let response = reqwest::Client::new()
.delete(format!("{base}/api/series/{series_id}"))
.send()
.await
.expect("delete");
assert_eq!(response.status(), StatusCode::NO_CONTENT);
let orphans: i64 = sqlx::query_scalar("SELECT count(*) FROM grabs")
.fetch_one(pool)
.await
.expect("count grabs");
assert_eq!(orphans, 0, "neither scope outlives the series");
}
/// A missing series is 404 before anything touches the disk.
#[tokio::test]
async fn deleting_a_missing_series_is_a_404() {
@@ -3675,6 +3863,42 @@ mod tests {
file
}
/// Same as [`episode_file_on_disk`], plus a §15 sidecar next to it with a
/// `subtitle_files` row pointing at the video's `media_files` row (#218).
async fn episode_file_with_sidecar_on_disk(
state: &AppState,
episode_id: i64,
root: &std::path::Path,
season: i64,
name: &str,
) -> (std::path::PathBuf, std::path::PathBuf) {
let file = episode_file_on_disk(state, episode_id, root, season, name).await;
let pool = state.database().expect("database").pool();
let media_file_id: i64 = sqlx::query_scalar(
"SELECT id FROM media_files WHERE owner_kind = 'episode' AND owner_id = ? AND path = ?",
)
.bind(episode_id)
.bind(file.to_str().expect("utf-8 path"))
.fetch_one(pool)
.await
.expect("media file id");
let sidecar = file.with_extension("pt-PT.srt");
tokio::fs::write(&sidecar, b"subs")
.await
.expect("write sidecar");
sqlx::query(
"INSERT INTO subtitle_files (media_file_id, language, origin, provider, path)
VALUES (?, 'pt-PT', 'provider', 'opensubtitles', ?)",
)
.bind(media_file_id)
.bind(sidecar.to_str().expect("utf-8 path"))
.execute(pool)
.await
.expect("subtitle file");
(file, sidecar)
}
async fn point_root_at(state: &AppState, root_id: i64, path: &std::path::Path) {
sqlx::query("UPDATE roots SET path = ? WHERE id = ?")
.bind(path.to_str().expect("utf-8 root"))
@@ -3877,6 +4101,122 @@ mod tests {
assert_eq!(remaining, vec![second], "only the episode's row goes");
}
/// #218: a season-scoped delete unlinks its subtitle sidecars too — the
/// `subtitle_files` rows go with the video's `media_files` row, but the
/// files on disk do not follow without this.
#[tokio::test]
async fn removing_a_season_takes_its_subtitle_sidecars() {
let (_dir, state, base) = application().await;
let root_id = tv_root(&state, "main").await;
let series = add_series(&base, root_id, true).await;
let series_id = series["id"].as_i64().expect("id");
let root = tempfile::tempdir().expect("root");
point_root_at(&state, root_id, root.path()).await;
let first = add_season(
&base,
series_id,
1,
serde_json::json!([{"number": 1, "title": "The Magic Xylophone"}]),
)
.await;
let second = add_season(
&base,
series_id,
2,
serde_json::json!([{"number": 1, "title": "Dance Mode"}]),
)
.await;
let s01e01 = first["episodes"][0]["id"].as_i64().expect("id");
let s02e01 = second["episodes"][0]["id"].as_i64().expect("id");
let (_, in_scope_sidecar) =
episode_file_with_sidecar_on_disk(&state, s01e01, root.path(), 1, "Bluey - S01E01.mkv")
.await;
let (_, other_sidecar) =
episode_file_with_sidecar_on_disk(&state, s02e01, root.path(), 2, "Bluey - S02E01.mkv")
.await;
let response = reqwest::Client::new()
.delete(format!("{base}/api/series/{series_id}/seasons/1/files"))
.send()
.await
.expect("delete season files");
assert_eq!(response.status(), StatusCode::NO_CONTENT);
assert!(!in_scope_sidecar.exists(), "its sidecar is gone with it");
assert!(
other_sidecar.exists(),
"another season's sidecar is not in scope"
);
let pool = state.database().expect("database").pool();
let orphans: i64 = sqlx::query_scalar("SELECT count(*) FROM subtitle_files")
.fetch_one(pool)
.await
.expect("count subtitle rows");
assert_eq!(
orphans, 1,
"only the deleted season's subtitle row cascades"
);
}
/// #218: the same fix at the narrower episode scope, leaving a sibling
/// episode's sidecar untouched.
#[tokio::test]
async fn removing_one_episode_takes_its_subtitle_sidecar() {
let (_dir, state, base) = application().await;
let root_id = tv_root(&state, "main").await;
let series = add_series(&base, root_id, true).await;
let series_id = series["id"].as_i64().expect("id");
let root = tempfile::tempdir().expect("root");
point_root_at(&state, root_id, root.path()).await;
let season = add_season(
&base,
series_id,
1,
serde_json::json!([
{"number": 1, "title": "The Magic Xylophone"},
{"number": 2, "title": "Hospital"}
]),
)
.await;
let episodes = season["episodes"].as_array().expect("episodes");
let first = episodes
.iter()
.find(|episode| episode["number"] == 1)
.expect("s01e01")["id"]
.as_i64()
.expect("id");
let second = episodes
.iter()
.find(|episode| episode["number"] == 2)
.expect("s01e02")["id"]
.as_i64()
.expect("id");
let (_, first_sidecar) =
episode_file_with_sidecar_on_disk(&state, first, root.path(), 1, "Bluey - S01E01.mkv")
.await;
let (_, second_sidecar) =
episode_file_with_sidecar_on_disk(&state, second, root.path(), 1, "Bluey - S01E02.mkv")
.await;
let response = reqwest::Client::new()
.delete(format!("{base}/api/episodes/{first}/files"))
.send()
.await
.expect("delete episode files");
assert_eq!(response.status(), StatusCode::NO_CONTENT);
assert!(!first_sidecar.exists(), "the episode's sidecar is gone");
assert!(
second_sidecar.exists(),
"its sibling's sidecar is not in scope"
);
}
/// #174: removal is not conditional on there being anything to remove.
/// A scope with no files still clears intent, and still answers 204.
#[tokio::test]
+160 -5
View File
@@ -1,11 +1,16 @@
//! What the API needs to answer a request: one HTTP client and the addresses
//! of the three upstreams the service cannot work without (DESIGN.md §3).
use std::sync::Arc;
use std::time::Duration;
use std::ffi::{OsStr, OsString};
use std::sync::{atomic::AtomicU64, Arc};
use std::time::{Duration, Instant};
use arr_db::Db;
use arr_dl::QbitClient;
use arr_probe::Extractor;
use arr_subs::{Backend, OpenAiEndpoint, Provider, Syncer};
use tokio::sync::mpsc;
use tokio::sync::RwLock;
use crate::jellyfin::JellyfinClient;
@@ -13,6 +18,12 @@ use crate::jellyfin::JellyfinClient;
/// is configurable, so this is a constant that tests point elsewhere.
pub const DEFAULT_TMDB_URL: &str = "https://api.themoviedb.org/3";
/// The `ffmpeg` invoked when nothing else is configured — same default as
/// `arr-probe`'s extractor, which is where it is actually run.
pub const DEFAULT_FFMPEG_BINARY: &str = "ffmpeg";
type DownloadSnapshot = Arc<RwLock<Option<(Instant, Vec<arr_dl::Torrent>)>>>;
/// How long an upstream has to answer a probe before it counts as
/// unreachable. Health is polled by a human waiting on a page.
const PROBE_TIMEOUT: Duration = Duration::from_secs(3);
@@ -22,7 +33,7 @@ const PROBE_TIMEOUT: Duration = Duration::from_secs(3);
pub struct Upstreams {
pub prowlarr_url: String,
pub prowlarr_api_key: Option<String>,
pub transmission_url: String,
pub qbittorrent_url: String,
pub tmdb_url: String,
pub tmdb_api_key: Option<String>,
}
@@ -30,11 +41,11 @@ pub struct Upstreams {
impl Upstreams {
/// Every upstream at its documented default, no keys.
#[must_use]
pub fn new(prowlarr_url: String, transmission_url: String) -> Self {
pub fn new(prowlarr_url: String, qbittorrent_url: String) -> Self {
Self {
prowlarr_url,
prowlarr_api_key: None,
transmission_url,
qbittorrent_url,
tmdb_url: DEFAULT_TMDB_URL.to_string(),
tmdb_api_key: None,
}
@@ -78,7 +89,28 @@ pub struct AppState {
pending_season_commands: Arc<tokio::sync::Mutex<mpsc::Receiver<SeasonCommand>>>,
metadata_commands: mpsc::Sender<MetadataCommand>,
pending_metadata_commands: Arc<tokio::sync::Mutex<mpsc::Receiver<MetadataCommand>>>,
subtitle_providers: Arc<Vec<Arc<dyn Provider>>>,
translation_backends: Arc<Vec<Arc<dyn Backend>>>,
/// The remote-command backend's live timeout, in milliseconds (#219).
/// `None` unless the daemon compiled and configured that backend; the
/// settings API writes it on every edit of the row.
command_timeout: Option<Arc<AtomicU64>>,
/// Where the OpenAI-compatible backend points and which model it names
/// (#220). `None` unless that backend is compiled in and wired up; the
/// settings API repoints it on every edit, and the health lamp probes
/// whatever it currently holds.
openai_endpoint: Option<OpenAiEndpoint>,
/// The configured `ffmpeg` binary, for the health lamps (#200).
ffmpeg_binary: OsString,
/// Runs that same `ffmpeg` to write an embedded text track out as a
/// sidecar (§15, #260). Derived from `ffmpeg_binary` rather than set on
/// its own, so the lamp and the extraction can never disagree about
/// which binary this deployment has.
extractor: Extractor,
jellyfin: Option<JellyfinClient>,
syncer: Syncer,
qbit: Option<QbitClient>,
download_snapshot: DownloadSnapshot,
}
/// Work explicitly requested through the movie API.
@@ -149,7 +181,16 @@ impl AppState {
pending_season_commands: Arc::new(tokio::sync::Mutex::new(pending_season_commands)),
metadata_commands,
pending_metadata_commands: Arc::new(tokio::sync::Mutex::new(pending_metadata_commands)),
subtitle_providers: Arc::new(Vec::new()),
translation_backends: Arc::new(Vec::new()),
command_timeout: None,
openai_endpoint: None,
ffmpeg_binary: DEFAULT_FFMPEG_BINARY.into(),
extractor: Extractor::default(),
jellyfin: None,
syncer: Syncer::default(),
qbit: None,
download_snapshot: Arc::new(RwLock::new(None)),
})
}
@@ -160,6 +201,56 @@ impl AppState {
self
}
/// Attach the shared qBittorrent client used by the API's live download
/// snapshot endpoint.
#[must_use]
pub fn with_qbit(mut self, qbit: QbitClient) -> Self {
self.qbit = Some(qbit);
self
}
/// Attach the subtitle providers this deployment has credentials for
/// (`DESIGN.md` §15).
///
/// Which of them a search actually runs is the `providers_enabled`
/// setting, read per request; this is the narrower fact of which ones
/// exist at all, because credentials are bootstrap config and never
/// reach the database (§10).
#[must_use]
pub fn with_subtitle_providers(mut self, providers: Vec<Arc<dyn Provider>>) -> Self {
self.subtitle_providers = Arc::new(providers);
self
}
/// Attach the translation backends this binary compiled in (§15).
///
/// Empty when no `translate-*` cargo feature is on, which is the default
/// — a manual translation then fails with a message saying so rather
/// than silently doing nothing.
#[must_use]
pub fn with_translation_backends(mut self, backends: Vec<Arc<dyn Backend>>) -> Self {
self.translation_backends = Arc::new(backends);
self
}
/// Attach the cell the remote-command backend re-reads per batch (#219),
/// so an edit of `remote_command_timeout_seconds` reaches it without a
/// restart. Absent when that backend is not configured.
#[must_use]
pub fn with_command_timeout(mut self, timeout: Arc<AtomicU64>) -> Self {
self.command_timeout = Some(timeout);
self
}
/// Attach the cell the OpenAI-compatible backend re-reads per request
/// (#220), so an edit of `openai_base_url` or `openai_model` reaches it
/// — and the health lamp probes it — without a restart.
#[must_use]
pub fn with_openai_endpoint(mut self, endpoint: OpenAiEndpoint) -> Self {
self.openai_endpoint = Some(endpoint);
self
}
/// Attach the Jellyfin client, so a manual subtitle grab or translation
/// can trigger the same library refresh import does (§7.5, §15).
#[must_use]
@@ -172,6 +263,62 @@ impl AppState {
self.jellyfin.as_ref()
}
pub(crate) fn subtitle_provider(&self, id: &str) -> Option<&Arc<dyn Provider>> {
self.subtitle_providers
.iter()
.find(|provider| provider.id().as_str() == id)
}
pub(crate) fn subtitle_providers(&self) -> &[Arc<dyn Provider>] {
&self.subtitle_providers
}
pub(crate) fn translation_backend(&self, id: &str) -> Option<&Arc<dyn Backend>> {
self.translation_backends
.iter()
.find(|backend| backend.id().as_str() == id)
}
pub(crate) fn command_timeout(&self) -> Option<&Arc<AtomicU64>> {
self.command_timeout.as_ref()
}
pub(crate) fn openai_endpoint(&self) -> Option<&OpenAiEndpoint> {
self.openai_endpoint.as_ref()
}
/// Attach the `alass` binary this deployment runs (§15). Defaults to
/// resolving `alass` from `PATH`.
#[must_use]
pub fn with_syncer(mut self, syncer: Syncer) -> Self {
self.syncer = syncer;
self
}
/// Attach the configured `ffmpeg` binary — the health lamps (#200)
/// probe it and the extract lane (#260) runs it. Defaults to resolving
/// `ffmpeg` from `PATH`.
#[must_use]
pub fn with_ffmpeg_binary(mut self, binary: impl Into<OsString>) -> Self {
let binary = binary.into();
self.extractor = Extractor::new().with_binary(binary.clone());
self.ffmpeg_binary = binary;
self
}
/// The configured `ffmpeg` binary.
pub(crate) fn ffmpeg_binary(&self) -> &OsStr {
&self.ffmpeg_binary
}
pub(crate) fn syncer(&self) -> &Syncer {
&self.syncer
}
pub(crate) fn extractor(&self) -> &Extractor {
&self.extractor
}
/// Wait for the next manual movie action in the daemon's reconcile loop.
///
/// # Errors
@@ -221,6 +368,14 @@ impl AppState {
self.database.as_ref()
}
pub(crate) fn qbit(&self) -> Option<&QbitClient> {
self.qbit.as_ref()
}
pub(crate) fn download_snapshot(&self) -> &DownloadSnapshot {
&self.download_snapshot
}
pub(crate) fn send_movie_command(
&self,
command: MovieCommand,
+761
View File
@@ -0,0 +1,761 @@
//! The runtime-editable half of subtitle configuration (`DESIGN.md` §15,
//! §10, issue #198). Provider credentials, translator keys, the
//! remote-command template and the `alass`/`ffmpeg` paths never reach here —
//! those are config/env, per §10, and this surface would leak them into a
//! `sqlite3 .backup` on a timer if it did.
//!
//! A single row rather than a CRUD collection: the wanted set, enabled
//! providers and translation engine are global, not per root (§15).
use std::collections::BTreeMap;
use axum::extract::rejection::JsonRejection;
use axum::extract::State;
use axum::Json;
use serde::{Deserialize, Serialize};
use utoipa::ToSchema;
use crate::movies::{pool, ApiError, ErrorBody};
use crate::policies::parsed;
use crate::state::AppState;
/// The subtitle settings row, plus which translation engines this binary
/// actually has compiled in.
#[derive(Debug, Clone, Serialize, ToSchema)]
pub struct SubtitleSettings {
pub wanted_languages: Vec<String>,
pub providers_enabled: Vec<String>,
pub translation_engine: Option<String>,
pub provider_daily_budgets: BTreeMap<String, u32>,
pub translator_daily_budgets: BTreeMap<String, u32>,
pub remote_command_timeout_seconds: u32,
/// Where the OpenAI-compatible backend points (#220). `null` means its
/// own default, `https://api.openai.com/v1/` — that backend is anything
/// speaking the shape, so a `llama.cpp` address belongs here. Not a
/// secret: the API key stays in the environment (§10).
pub openai_base_url: Option<String>,
/// The model that backend names. `null` means its own default.
pub openai_model: Option<String>,
/// Engines DESIGN.md §15 knows about that this binary compiled in.
/// `translation_engine` is always a member of this list or `null` — a
/// backend whose cargo feature is missing is never selectable.
pub available_engines: Vec<String>,
}
/// The payload for replacing the settings row. Same shape as
/// [`SubtitleSettings`] minus `available_engines`, which is a fact about the
/// binary, not something an operator sets.
#[derive(Debug, Clone, Deserialize, ToSchema)]
pub struct SubtitleSettingsInput {
pub wanted_languages: Vec<String>,
pub providers_enabled: Vec<String>,
pub translation_engine: Option<String>,
#[serde(default)]
pub provider_daily_budgets: BTreeMap<String, u32>,
#[serde(default)]
pub translator_daily_budgets: BTreeMap<String, u32>,
pub remote_command_timeout_seconds: u32,
#[serde(default)]
pub openai_base_url: Option<String>,
#[serde(default)]
pub openai_model: Option<String>,
}
impl SubtitleSettingsInput {
/// Validate against the vocabulary the running binary actually knows.
/// Every failure names its field so a rejected edit is fixable without
/// reading the schema.
fn validate(&self) -> Result<(), String> {
if self.wanted_languages.is_empty() {
return Err("wanted_languages: must not be empty".into());
}
if self.wanted_languages.iter().any(String::is_empty) {
return Err("wanted_languages: language tags must not be empty".into());
}
let mut seen = std::collections::BTreeSet::new();
for lang in &self.wanted_languages {
if !seen.insert(lang.as_str()) {
return Err(format!("wanted_languages: '{lang}' appears twice"));
}
}
if self.providers_enabled.iter().any(String::is_empty) {
return Err("providers_enabled: provider ids must not be empty".into());
}
let mut seen = std::collections::BTreeSet::new();
for provider in &self.providers_enabled {
if !seen.insert(provider.as_str()) {
return Err(format!("providers_enabled: '{provider}' appears twice"));
}
}
if let Some(engine) = &self.translation_engine {
if !arr_subs::ENGINES.contains(&engine.as_str()) {
return Err(format!(
"translation_engine: '{engine}' is not a known engine"
));
}
if !arr_subs::compiled_engines().contains(&engine.as_str()) {
return Err(format!(
"translation_engine: '{engine}' is not compiled into this binary"
));
}
}
if self.remote_command_timeout_seconds == 0 {
return Err("remote_command_timeout_seconds: must be greater than zero".into());
}
// #220: an unparseable base URL is rejected here rather than at the
// next translation, where it would surface as an engine that has
// quietly stopped working. Validated whether or not this build
// compiled the backend in — the column exists either way.
if let Some(base_url) = blank_to_none(self.openai_base_url.as_deref()) {
arr_subs::OpenAiEndpoint::new(Some(base_url), None).map_err(|error| {
// The backend's own Display talks about a reply that came
// back wrong; here nothing was ever sent, so only the
// reason belongs in the message.
let reason = match &error {
arr_subs::translate::Error::Malformed { detail, .. } => detail.clone(),
other => other.to_string(),
};
format!("openai_base_url: {reason}")
})?;
}
Ok(())
}
fn into_columns(self) -> Result<SettingsColumns, ApiError> {
fn json(value: impl serde::Serialize) -> Result<String, ApiError> {
serde_json::to_string(&value).map_err(|error| {
tracing::error!(%error, "subtitle settings serialisation failed");
ApiError::Database("serialisation failed".into())
})
}
Ok(SettingsColumns {
wanted_languages: json(&self.wanted_languages)?,
providers_enabled: json(&self.providers_enabled)?,
translation_engine: self.translation_engine,
provider_daily_budgets: json(&self.provider_daily_budgets)?,
translator_daily_budgets: json(&self.translator_daily_budgets)?,
remote_command_timeout_seconds: i64::from(self.remote_command_timeout_seconds),
// An empty field means "use the backend's default", which is the
// NULL the migration describes — not an endpoint named "".
openai_base_url: blank_to_none(self.openai_base_url.as_deref()).map(str::to_owned),
openai_model: blank_to_none(self.openai_model.as_deref()).map(str::to_owned),
})
}
}
/// The row as the table stores it, before JSON parsing.
struct SettingsColumns {
wanted_languages: String,
providers_enabled: String,
translation_engine: Option<String>,
provider_daily_budgets: String,
translator_daily_budgets: String,
remote_command_timeout_seconds: i64,
openai_base_url: Option<String>,
openai_model: Option<String>,
}
/// A field the operator left empty is absent, not an empty setting.
fn blank_to_none(value: Option<&str>) -> Option<&str> {
value.map(str::trim).filter(|value| !value.is_empty())
}
fn column<T: serde::de::DeserializeOwned>(
column: &'static str,
value: &str,
) -> Result<T, ApiError> {
serde_json::from_str(value).map_err(|error| {
tracing::error!(column, %error, "subtitle settings column holds unexpected JSON");
ApiError::Database(format!("subtitle settings column {column} is not valid"))
})
}
impl SettingsColumns {
fn into_settings(self) -> Result<SubtitleSettings, ApiError> {
Ok(SubtitleSettings {
wanted_languages: column("wanted_languages", &self.wanted_languages)?,
providers_enabled: column("providers_enabled", &self.providers_enabled)?,
translation_engine: self.translation_engine,
provider_daily_budgets: column("provider_daily_budgets", &self.provider_daily_budgets)?,
translator_daily_budgets: column(
"translator_daily_budgets",
&self.translator_daily_budgets,
)?,
remote_command_timeout_seconds: u32::try_from(self.remote_command_timeout_seconds)
.unwrap_or(0),
openai_base_url: self.openai_base_url,
openai_model: self.openai_model,
available_engines: arr_subs::compiled_engines()
.into_iter()
.map(str::to_owned)
.collect(),
})
}
}
/// Read the row and parse it. Shared with the health lamps (#200), which
/// need the enabled set and the chosen engine but not the budgets.
pub(crate) async fn load(state: &AppState) -> Result<SubtitleSettings, ApiError> {
let row = sqlx::query_as!(
SettingsColumns,
r#"SELECT wanted_languages AS "wanted_languages!: String",
providers_enabled AS "providers_enabled!: String",
translation_engine AS "translation_engine: String",
provider_daily_budgets AS "provider_daily_budgets!: String",
translator_daily_budgets AS "translator_daily_budgets!: String",
remote_command_timeout_seconds AS "remote_command_timeout_seconds!: i64",
openai_base_url AS "openai_base_url: String",
openai_model AS "openai_model: String"
FROM subtitle_settings WHERE id = 1"#
)
.fetch_one(pool(state)?)
.await?;
row.into_settings()
}
#[utoipa::path(
get, path = "/api/settings/subtitles", tag = "subtitles",
responses(
(status = 200, body = SubtitleSettings),
(status = 500, body = ErrorBody),
(status = 503, body = ErrorBody)
)
)]
pub async fn get(State(state): State<AppState>) -> Result<Json<SubtitleSettings>, ApiError> {
Ok(Json(load(&state).await?))
}
#[utoipa::path(
put, path = "/api/settings/subtitles", tag = "subtitles", request_body = SubtitleSettingsInput,
responses(
(status = 200, body = SubtitleSettings),
(status = 422, body = ErrorBody),
(status = 500, body = ErrorBody),
(status = 503, body = ErrorBody)
)
)]
pub async fn update(
State(state): State<AppState>,
body: Result<Json<SubtitleSettingsInput>, JsonRejection>,
) -> Result<Json<SubtitleSettings>, ApiError> {
let input = parsed(body)?;
input.validate().map_err(ApiError::Invalid)?;
let new_wanted = input.wanted_languages.clone();
let columns = input.into_columns()?;
let timeout_seconds = columns.remote_command_timeout_seconds;
let openai_base_url = columns.openai_base_url.clone();
let openai_model = columns.openai_model.clone();
let previous_wanted: String =
sqlx::query_scalar!("SELECT wanted_languages FROM subtitle_settings WHERE id = 1")
.fetch_one(pool(&state)?)
.await?;
let dropped: Vec<String> = column::<Vec<String>>("wanted_languages", &previous_wanted)?
.into_iter()
.filter(|language| !new_wanted.contains(language))
.collect();
let mut transaction = pool(&state)?.begin().await?;
sqlx::query!(
r#"UPDATE subtitle_settings SET
wanted_languages = ?, providers_enabled = ?, translation_engine = ?,
provider_daily_budgets = ?, translator_daily_budgets = ?,
remote_command_timeout_seconds = ?,
openai_base_url = ?, openai_model = ?,
updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now')
WHERE id = 1"#,
columns.wanted_languages,
columns.providers_enabled,
columns.translation_engine,
columns.provider_daily_budgets,
columns.translator_daily_budgets,
columns.remote_command_timeout_seconds,
columns.openai_base_url,
columns.openai_model,
)
.execute(&mut *transaction)
.await?;
// #224: a dropped language's attempt bookkeeping (backoff counter,
// `last_failure`) must not resurrect if the language is re-added later.
// Subtitle files stay — only the attempt rows are wanted-set-scoped.
for language in &dropped {
sqlx::query!("DELETE FROM subtitle_attempts WHERE language = ?", language)
.execute(&mut *transaction)
.await?;
}
transaction.commit().await?;
// Issue #219: the row alone never reaches the running backend. Push it
// into the cell the remote-command translator re-reads per batch; the
// cell counts milliseconds, the row counts seconds.
if let Some(timeout) = state.command_timeout() {
timeout.store(
u64::try_from(timeout_seconds)
.unwrap_or(u64::MAX)
.saturating_mul(1_000),
std::sync::atomic::Ordering::Relaxed,
);
}
// #220: the same path for the OpenAI-compatible backend. `validate` has
// already parsed the base URL, so this cannot fail for a reason the
// operator has not been told about.
if let Some(endpoint) = state.openai_endpoint() {
if let Err(error) = endpoint.set(openai_base_url.as_deref(), openai_model.as_deref()) {
tracing::error!(%error, "validated openai endpoint failed to apply");
}
}
Ok(Json(load(&state).await?))
}
#[cfg(test)]
mod tests {
use axum::http::StatusCode;
use crate::{router, AppState, Upstreams};
async fn application() -> (tempfile::TempDir, String) {
let (dir, base, _timeout) = application_with_timeout().await;
(dir, base)
}
/// The same app, with the remote-command backend's live timeout cell
/// attached — what the daemon wires up when that backend is configured.
async fn application_with_timeout() -> (
tempfile::TempDir,
String,
std::sync::Arc<std::sync::atomic::AtomicU64>,
) {
use std::sync::{atomic::AtomicU64, Arc};
let dir = tempfile::tempdir().expect("tempdir");
let database = arr_db::Db::connect(dir.path().join("arr.db"))
.await
.expect("connect database");
database.migrate().await.expect("migrate database");
let timeout = Arc::new(AtomicU64::new(30_000));
let state = AppState::new(Upstreams::new(
"http://127.0.0.1:1".into(),
"http://127.0.0.1:1".into(),
))
.expect("state")
.with_database(database)
.with_command_timeout(timeout.clone());
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind");
let address = listener.local_addr().expect("address");
let app = router(state);
tokio::spawn(async move { axum::serve(listener, app).await.expect("serve") });
(dir, format!("http://{address}"), timeout)
}
/// The same app, with the OpenAI-compatible backend's live endpoint
/// attached — what the daemon wires up when that backend is compiled in.
async fn application_with_openai() -> (tempfile::TempDir, String, arr_subs::OpenAiEndpoint) {
let dir = tempfile::tempdir().expect("tempdir");
let database = arr_db::Db::connect(dir.path().join("arr.db"))
.await
.expect("connect database");
database.migrate().await.expect("migrate database");
let endpoint = arr_subs::OpenAiEndpoint::new(None, None).expect("defaults resolve");
let state = AppState::new(Upstreams::new(
"http://127.0.0.1:1".into(),
"http://127.0.0.1:1".into(),
))
.expect("state")
.with_database(database)
.with_openai_endpoint(endpoint.clone());
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind");
let address = listener.local_addr().expect("address");
let app = router(state);
tokio::spawn(async move { axum::serve(listener, app).await.expect("serve") });
(dir, format!("http://{address}"), endpoint)
}
/// The same app, with the underlying pool exposed so a test can seed or
/// inspect rows the API surface does not read back directly.
async fn application_with_pool() -> (tempfile::TempDir, String, sqlx::SqlitePool) {
let dir = tempfile::tempdir().expect("tempdir");
let database = arr_db::Db::connect(dir.path().join("arr.db"))
.await
.expect("connect database");
database.migrate().await.expect("migrate database");
let pool = database.pool().clone();
let state = AppState::new(Upstreams::new(
"http://127.0.0.1:1".into(),
"http://127.0.0.1:1".into(),
))
.expect("state")
.with_database(database);
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind");
let address = listener.local_addr().expect("address");
let app = router(state);
tokio::spawn(async move { axum::serve(listener, app).await.expect("serve") });
(dir, format!("http://{address}"), pool)
}
fn valid_input() -> serde_json::Value {
serde_json::json!({
"wanted_languages": ["pt-PT", "en"],
"providers_enabled": ["opensubtitles"],
"translation_engine": null,
"provider_daily_budgets": { "opensubtitles": 100 },
"translator_daily_budgets": {},
"remote_command_timeout_seconds": 45
})
}
#[tokio::test]
async fn the_seeded_row_reads_back_with_the_engines_this_build_compiled() {
let (_dir, base) = application().await;
let settings: serde_json::Value = reqwest::get(format!("{base}/api/settings/subtitles"))
.await
.expect("get settings")
.json()
.await
.expect("settings json");
assert_eq!(
settings["wanted_languages"],
serde_json::json!(["pt-PT", "en"])
);
assert_eq!(
settings["providers_enabled"],
serde_json::json!(["opensubtitles"])
);
assert!(settings["translation_engine"].is_null());
// `available_engines` is a fact about the build, not about the seed
// row, so it tracks the `translate-*` features rather than a literal.
assert_eq!(
settings["available_engines"],
serde_json::json!(arr_subs::compiled_engines())
);
}
#[tokio::test]
async fn the_settings_round_trip_through_a_put() {
let (_dir, base) = application().await;
let updated: serde_json::Value = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&valid_input())
.send()
.await
.expect("put settings")
.json()
.await
.expect("updated json");
assert_eq!(
updated["provider_daily_budgets"],
serde_json::json!({ "opensubtitles": 100 })
);
assert_eq!(updated["remote_command_timeout_seconds"], 45);
let refetched: serde_json::Value = reqwest::get(format!("{base}/api/settings/subtitles"))
.await
.expect("get settings")
.json()
.await
.expect("settings json");
assert_eq!(refetched, updated);
}
/// #224: dropping a language from `wanted_languages` must clear its
/// `subtitle_attempts` rows — otherwise re-adding it later resurrects a
/// stale backoff counter as though the attempts had just happened.
#[tokio::test]
async fn dropping_a_language_clears_its_attempt_rows() {
let (_dir, base, pool) = application_with_pool().await;
sqlx::query(
"INSERT INTO media_files (id, owner_kind, owner_id, path, size)
VALUES (1, 'movie', 1, 'x.mkv', 1)",
)
.execute(&pool)
.await
.expect("media file");
sqlx::query(
"INSERT INTO subtitle_attempts
(media_file_id, language, state, attempts, last_attempt_at, last_failure)
VALUES (1, 'en', 'failed', 3, '2024-01-01T00:00:00Z', 'no provider match')",
)
.execute(&pool)
.await
.expect("dropped-language attempt");
sqlx::query(
"INSERT INTO subtitle_attempts (media_file_id, language, state) VALUES (1, 'pt-PT', 'wanted')",
)
.execute(&pool)
.await
.expect("kept-language attempt");
let mut payload = valid_input();
payload["wanted_languages"] = serde_json::json!(["pt-PT"]);
reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings")
.error_for_status()
.expect("valid input accepted");
let remaining: Vec<String> =
sqlx::query_scalar("SELECT language FROM subtitle_attempts ORDER BY language")
.fetch_all(&pool)
.await
.expect("attempts");
assert_eq!(remaining, vec!["pt-PT".to_string()]);
}
/// Issue #219: the row alone never reaches the running backend, so a PUT
/// must push its value into the cell the command translator re-reads.
#[tokio::test]
async fn a_put_updates_the_live_command_timeout() {
let (_dir, base, timeout) = application_with_timeout().await;
assert_eq!(
timeout.load(std::sync::atomic::Ordering::Relaxed),
30_000,
"seeded from the row at startup"
);
reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&valid_input())
.send()
.await
.expect("put settings")
.error_for_status()
.expect("valid input accepted");
assert_eq!(
timeout.load(std::sync::atomic::Ordering::Relaxed),
45_000,
"the edited value reaches the running backend"
);
}
/// #220: the two OpenAI-compatible endpoint fields are ordinary settings
/// — they round-trip, and an omitted or empty one reads back as `null`,
/// which the backend takes as "use your own default".
#[tokio::test]
async fn the_openai_endpoint_round_trips_and_blanks_read_back_null() {
let (_dir, base) = application().await;
let client = reqwest::Client::new();
let seeded: serde_json::Value = reqwest::get(format!("{base}/api/settings/subtitles"))
.await
.expect("get settings")
.json()
.await
.expect("settings json");
assert!(seeded["openai_base_url"].is_null());
assert!(seeded["openai_model"].is_null());
let mut payload = valid_input();
payload["openai_base_url"] = serde_json::json!("http://127.0.0.1:8080/v1");
payload["openai_model"] = serde_json::json!("qwen2.5:7b");
let updated: serde_json::Value = client
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings")
.json()
.await
.expect("updated json");
assert_eq!(updated["openai_base_url"], "http://127.0.0.1:8080/v1");
assert_eq!(updated["openai_model"], "qwen2.5:7b");
// An emptied field means "back to the default", not an endpoint
// named "" — the settings form sends an empty input, not a null.
payload["openai_base_url"] = serde_json::json!("");
payload["openai_model"] = serde_json::json!(" ");
let cleared: serde_json::Value = client
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings")
.json()
.await
.expect("updated json");
assert!(cleared["openai_base_url"].is_null());
assert!(cleared["openai_model"].is_null());
}
/// #220: a base URL that does not parse is a 422 naming the field, the
/// same shape `translation_engine` already rejects with.
#[tokio::test]
async fn a_base_url_that_does_not_parse_is_a_422_naming_the_field() {
let (_dir, base) = application().await;
let mut payload = valid_input();
payload["openai_base_url"] = serde_json::json!("not a url");
let response = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings");
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
let body: serde_json::Value = response.json().await.expect("error json");
assert_eq!(
body["error"], "openai_base_url: bad base URL: relative URL without a base",
"the message names the field and the reason, not a reply that never came"
);
}
/// #220, the point of the issue: the row alone never reaches the running
/// backend, so a PUT must repoint the cell it re-reads per request —
/// which is also what the health lamp probes.
#[tokio::test]
async fn a_put_repoints_the_live_openai_endpoint() {
let (_dir, base, endpoint) = application_with_openai().await;
assert_eq!(endpoint.base_url(), arr_subs::OPENAI_DEFAULT_BASE_URL);
assert_eq!(endpoint.model(), arr_subs::OPENAI_DEFAULT_MODEL);
let mut payload = valid_input();
payload["openai_base_url"] = serde_json::json!("http://127.0.0.1:8080/v1");
payload["openai_model"] = serde_json::json!("qwen2.5:7b");
reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings")
.error_for_status()
.expect("valid input accepted");
assert_eq!(endpoint.base_url(), "http://127.0.0.1:8080/v1/");
assert_eq!(endpoint.model(), "qwen2.5:7b");
}
/// Whether a known engine is selectable depends on which `translate-*`
/// features this binary was built with, so the test asks the build rather
/// than assuming. With no feature on, every engine is uncompiled and must
/// be refused; with all of them on there is nothing to refuse, and the
/// complementary truth — a compiled engine is accepted — is what holds.
#[tokio::test]
async fn an_uncompiled_engine_is_a_422_naming_the_field() {
let compiled = arr_subs::compiled_engines();
let Some(uncompiled) = arr_subs::ENGINES
.iter()
.find(|engine| !compiled.contains(*engine))
else {
return a_compiled_engine_is_accepted().await;
};
let (_dir, base) = application().await;
let mut payload = valid_input();
payload["translation_engine"] = serde_json::json!(uncompiled);
let response = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings");
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
let body: serde_json::Value = response.json().await.expect("error body");
assert!(
body["error"]
.as_str()
.expect("error text")
.contains("translation_engine"),
"{body}"
);
}
/// The other side of the feature gate: an engine this binary *did*
/// compile in is selectable. Called directly when no engine is uncompiled.
async fn a_compiled_engine_is_accepted() {
let compiled = arr_subs::compiled_engines();
let Some(engine) = compiled.first() else {
return;
};
let (_dir, base) = application().await;
let mut payload = valid_input();
payload["translation_engine"] = serde_json::json!(engine);
let response = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings");
assert_eq!(response.status(), StatusCode::OK, "{engine} is compiled in");
}
#[tokio::test]
async fn an_unknown_engine_name_is_a_422() {
let (_dir, base) = application().await;
let mut payload = valid_input();
payload["translation_engine"] = serde_json::json!("bing-translate");
let response = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings");
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
}
#[tokio::test]
async fn every_field_validates_by_name() {
let (_dir, base) = application().await;
let with = |patch: &dyn Fn(&mut serde_json::Value)| {
let mut payload = valid_input();
patch(&mut payload);
payload
};
let cases: Vec<(serde_json::Value, &str)> = vec![
(
with(&|payload| payload["wanted_languages"] = serde_json::json!([])),
"wanted_languages",
),
(
with(&|payload| {
payload["wanted_languages"] = serde_json::json!(["pt-PT", "pt-PT"]);
}),
"wanted_languages",
),
(
with(&|payload| {
payload["providers_enabled"] = serde_json::json!(["opensubtitles", ""]);
}),
"providers_enabled",
),
(
with(&|payload| payload["remote_command_timeout_seconds"] = serde_json::json!(0)),
"remote_command_timeout_seconds",
),
];
for (payload, field) in cases {
let response = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.json(&payload)
.send()
.await
.expect("put settings");
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
let body: serde_json::Value = response.json().await.expect("error body");
let error = body["error"].as_str().expect("error text");
assert!(error.contains(field), "{field}: {error}");
}
}
#[tokio::test]
async fn malformed_json_is_422_not_400_or_500() {
let (_dir, base) = application().await;
let response = reqwest::Client::new()
.put(format!("{base}/api/settings/subtitles"))
.header("content-type", "application/json")
.body("{not json")
.send()
.await
.expect("malformed put");
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
}
}
File diff suppressed because it is too large Load Diff
+29
View File
@@ -97,6 +97,21 @@ pub fn episode_file_name(
}
}
/// A subtitle sidecar's filename (§15): the video's name, the language, and
/// `srt` — `… [2160p][WEB-DL][HDR10].pt-PT.srt`. A machine translation
/// carries an extra `.mt` segment so `ls` says which subtitles arr made.
#[must_use]
pub fn subtitle_name(video_name: &str, language: &Language, machine_translated: bool) -> String {
let stem = video_name
.rsplit_once('.')
.map_or(video_name, |(stem, _)| stem);
if machine_translated {
format!("{stem}.{language}.mt.srt")
} else {
format!("{stem}.{language}.srt")
}
}
/// The §7.4 attribute tags, in a fixed order: resolution, source, HDR,
/// Portuguese audio.
///
@@ -215,6 +230,20 @@ mod tests {
);
}
/// The §15 sidecar names: plain for a real subtitle, `.mt` for arr's own.
#[test]
fn subtitle_sidecars_carry_the_language_and_the_mt_marker() {
let video = "Dune Part Two (2024) [tmdbid-693134] - [2160p][WEB-DL][HDR10].mkv";
assert_eq!(
subtitle_name(video, &Language::PortuguesePortugal, false),
"Dune Part Two (2024) [tmdbid-693134] - [2160p][WEB-DL][HDR10].pt-PT.srt"
);
assert_eq!(
subtitle_name(video, &Language::PortugueseBrazil, true),
"Dune Part Two (2024) [tmdbid-693134] - [2160p][WEB-DL][HDR10].pt-BR.mt.srt"
);
}
/// The kids audit surface: a pt-PT track is tagged, SDR is not.
#[test]
fn portuguese_audio_is_tagged_and_sdr_is_not() {
+100 -2
View File
@@ -11,6 +11,7 @@ pub mod matching;
pub mod policy;
pub mod score;
pub mod status;
pub mod subs;
pub mod tracking;
pub use arr_parse::NameClaims as ParsedRelease;
@@ -24,6 +25,10 @@ pub use matching::{
};
pub use score::{Score, ScoreWeights};
pub use status::{derive_series_status, SeriesStatus};
pub use subs::{
rank as rank_subtitles, RankedSubtitle, SubtitleCandidate, SubtitleRule, SubtitleTarget,
SubtitleVerdict,
};
macro_rules! id_type {
($name:ident) => {
@@ -252,7 +257,7 @@ pub enum MediaState {
Missing,
Downloading,
Available,
/// #108: `wanted` was cleared after the grab vanished from Transmission.
/// #108: `wanted` was cleared after the grab vanished from qBittorrent.
/// Distinct from `Missing` so it does not read as an open gap.
Parked,
}
@@ -318,9 +323,82 @@ pub struct AudioTrack {
pub handler_name: Option<String>,
}
/// How an embedded subtitle track is encoded (DESIGN.md §15).
///
/// The split that matters is text versus bitmap: a text track extracts to a
/// sidecar SRT and can feed a translator, a bitmap one satisfies its language
/// for viewing and nothing more. There is no OCR.
#[derive(Clone, Copy, Debug, Eq, Hash, Ord, PartialEq, PartialOrd)]
pub enum SubtitleCodec {
/// `subrip`, the format sidecars are written in.
SubRip,
/// Advanced `SubStation` Alpha, and SSA with it.
Ass,
/// MP4's timed text.
MovText,
/// Presentation graphics — the bitmap track on `BluRay`.
Pgs,
/// `VobSub` — the bitmap track on DVD.
VobSub,
/// Anything else `ffprobe` names that is not one of the above.
Other,
}
impl fmt::Display for SubtitleCodec {
/// `ffprobe`'s codec name, so the probe column spells what the file said.
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str(match self {
Self::SubRip => "subrip",
Self::Ass => "ass",
Self::MovText => "mov_text",
Self::Pgs => "hdmv_pgs_subtitle",
Self::VobSub => "dvd_subtitle",
Self::Other => "other",
})
}
}
impl SubtitleCodec {
/// §15: only these extract to SRT and may become a translation source.
#[must_use]
pub const fn is_text(self) -> bool {
matches!(self, Self::SubRip | Self::Ass | Self::MovText)
}
/// §15's bitmap tracks. They carry no text, but their packet timings are
/// exact for the release they came off, which is what a cue skeleton is
/// derived from (#268).
#[must_use]
pub const fn is_image(self) -> bool {
matches!(self, Self::Pgs | Self::VobSub)
}
/// The inverse of [`Display`](fmt::Display): read back a codec that was
/// written out under its `ffprobe` name. `ffprobe`'s own aliases are
/// accepted alongside, so a column written by an older probe still
/// resolves. Anything unrecognised is [`Self::Other`], which extracts
/// nothing — the same answer as a bitmap track.
#[must_use]
pub fn from_probe_name(name: &str) -> Self {
match name {
"subrip" | "srt" => Self::SubRip,
"ass" | "ssa" => Self::Ass,
"mov_text" => Self::MovText,
"hdmv_pgs_subtitle" => Self::Pgs,
"dvd_subtitle" => Self::VobSub,
_ => Self::Other,
}
}
}
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct SubtitleTrack {
pub language: Language,
pub codec: SubtitleCodec,
/// Foreign lines and on-screen signs only. Never satisfies a want (§15).
pub forced: bool,
/// Complete, with sound descriptions. Satisfies, ranked below plain (§15).
pub sdh: bool,
}
#[derive(Clone, Debug, Eq, PartialEq)]
@@ -437,7 +515,7 @@ pub struct MovieOwner {
#[cfg(test)]
mod tests {
use super::{DolbyVisionProfile, HdrFormat, Language, Rule, Verdict};
use super::{DolbyVisionProfile, HdrFormat, Language, Rule, SubtitleCodec, Verdict};
#[test]
fn portuguese_variants_are_distinct() {
@@ -459,6 +537,26 @@ mod tests {
);
}
#[test]
fn probe_names_round_trip_through_the_codec() {
for codec in [
SubtitleCodec::SubRip,
SubtitleCodec::Ass,
SubtitleCodec::MovText,
SubtitleCodec::Pgs,
SubtitleCodec::VobSub,
] {
assert_eq!(SubtitleCodec::from_probe_name(&codec.to_string()), codec);
}
assert_eq!(SubtitleCodec::from_probe_name("srt"), SubtitleCodec::SubRip);
assert_eq!(SubtitleCodec::from_probe_name("ssa"), SubtitleCodec::Ass);
assert_eq!(
SubtitleCodec::from_probe_name("dvb_subtitle"),
SubtitleCodec::Other
);
assert!(!SubtitleCodec::from_probe_name("other").is_text());
}
#[test]
fn non_eligible_verdicts_name_the_rule() {
let rule = Rule::DolbyVisionProfile(DolbyVisionProfile {
+61 -1
View File
@@ -237,6 +237,16 @@ impl PolicyRule for SourceRule {
/// automatically and its import is recorded as a §5.7 waiver. This mirrors
/// [`ResolutionRule`], where an override moves a failure between hard and
/// soft and never makes the rule stop applying.
///
/// **The floor only ever rejects pre-grab.** It exists to keep the ranking
/// from picking mud when something better is in the same search (§5.5), and
/// that job is finished once a release is grabbed: at import there is
/// nothing left to choose between, so a hard fail there condemns a file
/// already on disk and blacklists the release under §5.7. Size is also not
/// post-download evidence — §5.6 gives `ffprobe` real audio, HDR, codec and
/// duration, and the size was on the release before the grab. So after
/// download the floor is a soft fail whatever the overrides say: the file
/// imports and carries a waiver naming the floor it missed.
#[derive(Clone, Copy, Debug, Default)]
pub struct SizeRule;
@@ -254,7 +264,10 @@ impl PolicyRule for SizeRule {
context.runtime_minutes,
) {
None => RuleEvaluation::Unknown(RuleKind::Size),
Some(true) if context.overrides.allow_below_floor => {
Some(true)
if context.overrides.allow_below_floor
|| context.candidate.phase() == EvaluationPhase::PostDownload =>
{
RuleEvaluation::SoftFail(Rule::Size)
}
Some(true) => RuleEvaluation::HardFail(Rule::Size),
@@ -752,6 +765,50 @@ mod tests {
);
}
/// The floor is a selection filter (§5.5), so it only rejects while
/// there is still a selection to make. A pack accepted pre-grab on its
/// per-episode average carries episodes that individually fall short;
/// hard-failing them at import blacklists a release the operator chose
/// and reopens every episode, over a number that was known before the
/// grab. Post-download the floor waives instead, with no override.
#[test]
fn below_the_floor_after_download_waives_rather_than_condemning_the_release() {
let policy = banded_policy();
let overrides = TitleOverrides::default();
let claims = claims(Some(ClaimedResolution::P1080), Some(ClaimedSource::WebDl));
// Pre-grab the same size is still a rejection: nothing is on disk
// yet and a better candidate may be one row down.
assert_eq!(
evaluate(
&policy,
&overrides,
&en(),
Candidate::PreGrab(&claims),
Some(1 << 30),
1,
0,
)
.verdict,
Verdict::Rejected(Rule::Size)
);
let media = probed(Resolution::R1080p, Some(Source::WebDl));
assert_eq!(
evaluate(
&policy,
&overrides,
&en(),
Candidate::PostDownload(&media),
Some(1 << 30),
1,
0,
)
.verdict,
Verdict::Waived(Rule::Size)
);
}
#[test]
fn allow_below_floor_says_nothing_about_a_release_that_clears_the_floor() {
let overrides = TitleOverrides {
@@ -1241,6 +1298,9 @@ mod tests {
let mut media = probed_audio(vec![track(Language::PortuguesePortugal)]);
media.subtitle_tracks = vec![SubtitleTrack {
language: Language::PortugueseBrazil,
codec: crate::SubtitleCodec::SubRip,
forced: false,
sdh: false,
}];
assert_eq!(
verdict_for(&kids_policy(), &en(), Candidate::PostDownload(&media)),
+345
View File
@@ -0,0 +1,345 @@
//! Ranking subtitle candidates against a media file (`DESIGN.md` §15).
//!
//! Pure and IO-free, alongside the release scoring in §5.5: the input is the
//! facts already known about the file plus a list of candidates as reported
//! by a provider, the output is those candidates ordered best first, each
//! carrying the verdict that put it there.
//!
//! The verdict vocabulary mirrors the release path — [`SubtitleVerdict`]
//! carries `Eligible` or `Rejected` a named [`SubtitleRule`], not a string
//! built for a log line — so §9.3's manual view needs no second concept for
//! subtitles. There is no `waived` bucket: nothing about a subtitle is worth
//! overriding by hand.
use crate::Source;
/// What is known about the file a subtitle candidate is ranked against.
#[derive(Clone, Copy, Debug, Default)]
pub struct SubtitleTarget<'a> {
/// The file's own `moviehash`, when computed.
pub moviehash: Option<&'a str>,
/// The exact name of the release that produced the file.
pub release_name: Option<&'a str>,
/// The release group claimed by that release name.
pub release_group: Option<&'a str>,
/// The source tier claimed by that release name.
pub source: Option<Source>,
}
/// One subtitle candidate, as reported by a provider.
#[derive(Clone, Copy, Debug, Default)]
pub struct SubtitleCandidate<'a> {
/// Covers foreign-language lines and on-screen signs only; never
/// satisfies a want (§15's "Forced and SDH").
pub forced: bool,
/// Complete and satisfies a want, but ranks below a plain subtitle for
/// the same language.
pub hearing_impaired: bool,
pub moviehash: Option<&'a str>,
pub release_name: Option<&'a str>,
pub release_group: Option<&'a str>,
pub source: Option<Source>,
pub uploader_rating: f64,
pub download_count: u64,
}
/// Why a subtitle candidate was rejected.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum SubtitleRule {
/// A forced track covers foreign-language lines only and cannot satisfy
/// a want (§15).
Forced,
}
impl SubtitleRule {
/// The stable name shared by the API and UI, so one rule reads the same
/// everywhere (mirrors [`crate::Rule::name`]).
#[must_use]
pub const fn name(&self) -> &'static str {
match self {
Self::Forced => "forced",
}
}
}
/// The verdict for one subtitle candidate.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum SubtitleVerdict {
Eligible,
Rejected(SubtitleRule),
}
/// One candidate paired with the verdict that placed it in the ranking.
#[derive(Clone, Copy, Debug)]
pub struct RankedSubtitle<'a> {
/// Position of this candidate in the slice handed to [`rank`]. Ranking
/// reorders, and a candidate carries no identity of its own, so this is
/// how a caller maps a result back to the provider candidate it came
/// from — and therefore to the id it must ask the provider to download.
pub index: usize,
pub candidate: SubtitleCandidate<'a>,
pub verdict: SubtitleVerdict,
}
/// The ordering key for an eligible candidate, most significant field first.
/// Every field is a boolean win or an integer tiebreaker, so `Ord` alone
/// gives the ranking — no float, no partial order.
type TierKey = (bool, bool, bool, bool, i64, u64);
fn tier_key(candidate: &SubtitleCandidate<'_>, target: &SubtitleTarget<'_>) -> TierKey {
let plain = !candidate.hearing_impaired;
let moviehash_match = matches_ci(candidate.moviehash, target.moviehash);
let exact_release_match = matches_ci(candidate.release_name, target.release_name);
let group_or_source_match = matches_ci(candidate.release_group, target.release_group)
|| matches_source(candidate.source, target.source);
// Scaled to hundredths so the tiebreaker is an integer, not a float.
// Ratings sit in 0..10, nowhere near i64's range.
#[allow(clippy::cast_possible_truncation)]
let rating = (candidate.uploader_rating * 100.0).round() as i64;
(
plain,
moviehash_match,
exact_release_match,
group_or_source_match,
rating,
candidate.download_count,
)
}
fn matches_ci(candidate: Option<&str>, target: Option<&str>) -> bool {
match (candidate, target) {
(Some(candidate), Some(target)) => candidate.eq_ignore_ascii_case(target),
_ => false,
}
}
fn matches_source(candidate: Option<Source>, target: Option<Source>) -> bool {
matches!((candidate, target), (Some(candidate), Some(target)) if candidate == target)
}
/// Rank candidates against a target, best first.
///
/// A forced candidate is rejected outright and sorts after every eligible
/// one, in the order it was given (§15: it can never satisfy a want). Among
/// eligible candidates, a plain subtitle always outranks a hearing-impaired
/// one for the same language; within that split, a `moviehash` match wins
/// outright, then an exact release-name match, then a shared release group
/// or source, then uploader rating and download count as tiebreakers.
#[must_use]
pub fn rank<'a>(
target: &SubtitleTarget<'_>,
candidates: &[SubtitleCandidate<'a>],
) -> Vec<RankedSubtitle<'a>> {
let mut ranked: Vec<RankedSubtitle<'a>> = candidates
.iter()
.enumerate()
.map(|(index, candidate)| RankedSubtitle {
index,
candidate: *candidate,
verdict: if candidate.forced {
SubtitleVerdict::Rejected(SubtitleRule::Forced)
} else {
SubtitleVerdict::Eligible
},
})
.collect();
ranked.sort_by(|a, b| match (a.verdict, b.verdict) {
(SubtitleVerdict::Rejected(_), SubtitleVerdict::Rejected(_)) => std::cmp::Ordering::Equal,
(SubtitleVerdict::Rejected(_), SubtitleVerdict::Eligible) => std::cmp::Ordering::Greater,
(SubtitleVerdict::Eligible, SubtitleVerdict::Rejected(_)) => std::cmp::Ordering::Less,
(SubtitleVerdict::Eligible, SubtitleVerdict::Eligible) => {
tier_key(&b.candidate, target).cmp(&tier_key(&a.candidate, target))
}
});
ranked
}
#[cfg(test)]
mod tests {
use super::*;
fn target() -> SubtitleTarget<'static> {
SubtitleTarget {
moviehash: Some("abc123"),
release_name: Some("Movie.2024.1080p.WEB-DL-GROUP"),
release_group: Some("GROUP"),
source: Some(Source::WebDl),
}
}
fn plain() -> SubtitleCandidate<'static> {
SubtitleCandidate {
uploader_rating: 5.0,
download_count: 100,
..Default::default()
}
}
#[test]
fn ranking_reports_where_each_candidate_came_from() {
// Ranking reorders, and a candidate carries no id of its own, so the
// index is the only way back to the provider candidate — and so to
// the id the provider is asked to download.
let weak = plain();
let strong = SubtitleCandidate {
moviehash: Some("abc123"),
..plain()
};
let ranked = rank(&target(), &[weak, strong]);
assert_eq!(ranked[0].index, 1);
assert_eq!(ranked[1].index, 0);
}
#[test]
fn a_rejected_candidate_still_reports_its_index() {
let forced = SubtitleCandidate {
forced: true,
..plain()
};
let ranked = rank(&target(), &[forced, plain()]);
assert_eq!(ranked[0].index, 1);
assert_eq!(ranked[1].index, 0);
assert_eq!(
ranked[1].verdict,
SubtitleVerdict::Rejected(SubtitleRule::Forced)
);
}
#[test]
fn a_moviehash_match_wins_outright_over_every_other_tier() {
let hash_match = SubtitleCandidate {
moviehash: Some("abc123"),
uploader_rating: 0.0,
download_count: 0,
..plain()
};
let everything_else = SubtitleCandidate {
release_name: Some("Movie.2024.1080p.WEB-DL-GROUP"),
release_group: Some("GROUP"),
source: Some(Source::WebDl),
uploader_rating: 10.0,
download_count: 1_000_000,
..plain()
};
let ranked = rank(&target(), &[everything_else, hash_match]);
assert!(matches!(ranked[0].candidate.moviehash, Some("abc123")));
assert_eq!(ranked[0].verdict, SubtitleVerdict::Eligible);
}
#[test]
fn an_exact_release_name_match_outranks_group_or_source_alone() {
let exact_name = SubtitleCandidate {
release_name: Some("Movie.2024.1080p.WEB-DL-GROUP"),
..plain()
};
let group_only = SubtitleCandidate {
release_group: Some("GROUP"),
uploader_rating: 10.0,
download_count: 1_000_000,
..plain()
};
let ranked = rank(&target(), &[group_only, exact_name]);
assert_eq!(
ranked[0].candidate.release_name,
Some("Movie.2024.1080p.WEB-DL-GROUP")
);
}
#[test]
fn same_source_alone_outranks_no_match_at_all() {
let same_source = SubtitleCandidate {
source: Some(Source::WebDl),
..plain()
};
let no_match = SubtitleCandidate {
uploader_rating: 10.0,
download_count: 1_000_000,
..plain()
};
let ranked = rank(&target(), &[no_match, same_source]);
assert_eq!(ranked[0].candidate.source, Some(Source::WebDl));
}
#[test]
#[allow(clippy::float_cmp)]
fn rating_then_downloads_break_ties() {
let low = SubtitleCandidate {
uploader_rating: 3.0,
download_count: 50,
..plain()
};
let high_rating = SubtitleCandidate {
uploader_rating: 8.0,
download_count: 10,
..plain()
};
let ranked = rank(&target(), &[low, high_rating]);
assert_eq!(ranked[0].candidate.uploader_rating, 8.0);
let same_rating_more_downloads = SubtitleCandidate {
uploader_rating: 8.0,
download_count: 9_000,
..plain()
};
let ranked = rank(&target(), &[high_rating, same_rating_more_downloads]);
assert_eq!(ranked[0].candidate.download_count, 9_000);
}
#[test]
fn a_forced_candidate_is_never_eligible() {
let forced = SubtitleCandidate {
forced: true,
moviehash: Some("abc123"),
..plain()
};
let ranked = rank(&target(), &[forced]);
assert_eq!(
ranked[0].verdict,
SubtitleVerdict::Rejected(SubtitleRule::Forced)
);
assert_eq!(SubtitleRule::Forced.name(), "forced");
}
#[test]
fn an_sdh_candidate_ranks_below_any_plain_candidate() {
let sdh_with_hash_match = SubtitleCandidate {
hearing_impaired: true,
moviehash: Some("abc123"),
uploader_rating: 10.0,
download_count: 1_000_000,
..plain()
};
let plain_with_nothing = SubtitleCandidate {
uploader_rating: 0.0,
download_count: 0,
..plain()
};
let ranked = rank(&target(), &[sdh_with_hash_match, plain_with_nothing]);
assert!(!ranked[0].candidate.hearing_impaired);
assert!(ranked[1].candidate.hearing_impaired);
}
#[test]
fn rejected_candidates_sort_after_every_eligible_one_in_input_order() {
let forced_a = SubtitleCandidate {
forced: true,
download_count: 1,
..plain()
};
let forced_b = SubtitleCandidate {
forced: true,
download_count: 2,
..plain()
};
let eligible = plain();
let ranked = rank(&target(), &[forced_a, eligible, forced_b]);
assert_eq!(ranked[0].verdict, SubtitleVerdict::Eligible);
assert_eq!(ranked[1].candidate.download_count, 1);
assert_eq!(ranked[2].candidate.download_count, 2);
}
}
+11 -1
View File
@@ -10,6 +10,16 @@ publish = false
name = "arr"
path = "src/main.rs"
# Forwards straight to `arr-subs`' own features (DESIGN.md §15): which
# translators a build ships is a compile-time choice, off by default, same as
# the crate that implements them.
[features]
default = []
translate-openai = ["arr-subs/translate-openai"]
translate-deepl = ["arr-subs/translate-deepl"]
translate-google = ["arr-subs/translate-google"]
translate-command = ["arr-subs/translate-command"]
[dependencies]
arr-api = { workspace = true }
arr-compat = { workspace = true }
@@ -20,6 +30,7 @@ arr-indexer = { workspace = true }
arr-meta = { workspace = true }
arr-parse = { workspace = true }
arr-probe = { workspace = true }
arr-subs = { workspace = true }
axum = { workspace = true }
chrono = { workspace = true }
include_dir = { workspace = true }
@@ -36,7 +47,6 @@ tracing = { workspace = true }
tracing-subscriber = { workspace = true }
[dev-dependencies]
base64 = { workspace = true }
tempfile = { workspace = true }
wiremock = { workspace = true }
+215 -25
View File
@@ -1,5 +1,5 @@
//! §9.5 *broken* → the operator alone: Prowlarr, Transmission or TMDB
//! unreachable.
//! §9.5 *broken* → the operator alone: Prowlarr, qBittorrent or TMDB
//! unreachable, or a subtitle lamp failing (#200).
//!
//! Edge-triggered: notifies once when an upstream stops answering, and
//! silently re-arms once it answers again. There is no "fixed" notification —
@@ -24,57 +24,126 @@ const PROBE_TIMEOUT: Duration = Duration::from_secs(5);
pub struct Upstreams {
pub prowlarr_url: String,
pub prowlarr_api_key: Option<String>,
pub transmission_url: String,
pub qbittorrent_url: String,
pub tmdb_url: String,
pub tmdb_api_key: Option<String>,
}
/// The subtitle upstreams (#200): the providers and engines this deployment
/// has credentials for, and the two binaries with their configured paths.
///
/// What is *in use* is read from the settings row per tick; this only holds
/// what could ever answer.
#[derive(Debug, Clone)]
pub struct SubtitleUpstreams {
pub providers: Vec<Arc<dyn arr_subs::Provider>>,
pub backends: Vec<Arc<dyn arr_subs::Backend>>,
pub alass_path: std::path::PathBuf,
pub ffmpeg_path: std::path::PathBuf,
}
impl SubtitleUpstreams {
/// The lamps as `(name, reachable)` pairs. Only what the settings row
/// has in use is probed — a provider nobody enabled cannot be broken.
async fn probe(&self, database: &Db) -> Vec<(String, bool)> {
let (providers_enabled, engine) = match crate::subtitles::load_settings(database).await {
Ok(settings) => (settings.providers_enabled, settings.translation_engine),
// An unreadable row says nothing about any upstream; skipping the
// whole lane beats notifying on our own database.
Err(error) => {
tracing::warn!(%error, "subtitle settings unreadable; subtitle lamps skipped");
return Vec::new();
}
};
let mut lamps = Vec::new();
for id in &providers_enabled {
let reachable = match self
.providers
.iter()
.find(|p| p.id().as_str() == id.as_str())
{
Some(provider) => provider.probe().await.is_ok(),
None => false,
};
lamps.push((id.clone(), reachable));
}
if let Some(engine) = engine {
let reachable = match self.backends.iter().find(|b| b.id().as_str() == engine) {
Some(backend) => backend.probe().await.is_ok(),
None => false,
};
lamps.push((engine, reachable));
}
lamps.push((
"alass".to_owned(),
arr_subs::binary_present(std::ffi::OsStr::new(&self.alass_path)),
));
lamps.push((
"ffmpeg".to_owned(),
arr_subs::binary_present(std::ffi::OsStr::new(&self.ffmpeg_path)),
));
lamps
}
}
#[derive(Debug)]
pub struct BrokenAction {
http: Client,
upstreams: Upstreams,
subtitles: SubtitleUpstreams,
notifier: Notifier,
operator_topic: String,
/// Which upstreams are currently notified as broken. Transient — a
/// restart re-probes and re-notifies whatever is still down.
broken: Arc<Mutex<HashSet<&'static str>>>,
broken: Arc<Mutex<HashSet<String>>>,
}
impl BrokenAction {
#[must_use]
pub fn new(upstreams: Upstreams, notifier: Notifier, operator_topic: String) -> Self {
pub fn new(
upstreams: Upstreams,
subtitles: SubtitleUpstreams,
notifier: Notifier,
operator_topic: String,
) -> Self {
Self {
http: Client::new(),
upstreams,
subtitles,
notifier,
operator_topic,
broken: Arc::new(Mutex::new(HashSet::new())),
}
}
async fn tick(&self) -> Vec<Outcome> {
let (prowlarr, transmission, tmdb) = tokio::join!(
async fn tick(&self, database: &Db) -> Vec<Outcome> {
let (prowlarr, qbit, tmdb, subtitles) = tokio::join!(
self.probe_prowlarr(),
self.probe_transmission(),
self.probe_qbit(),
self.probe_tmdb(),
self.subtitles.probe(database),
);
let mut outcomes = Vec::new();
outcomes.extend(self.notify_transition("prowlarr", prowlarr).await);
outcomes.extend(self.notify_transition("transmission", transmission).await);
outcomes.extend(self.notify_transition("qbit", qbit).await);
outcomes.extend(self.notify_transition("tmdb", tmdb).await);
for (name, reachable) in subtitles {
outcomes.extend(self.notify_transition(&name, reachable).await);
}
outcomes
}
/// `reachable` is `true` when the upstream answered, or when it needs no
/// key and none is configured (not an outage — see `probe_tmdb`).
async fn notify_transition(&self, name: &'static str, reachable: bool) -> Option<Outcome> {
async fn notify_transition(&self, name: &str, reachable: bool) -> Option<Outcome> {
let mut broken = self.broken.lock().await;
if reachable {
broken.remove(name);
return None;
}
if !broken.insert(name) {
if !broken.insert(name.to_owned()) {
return None;
}
match self
@@ -109,13 +178,13 @@ impl BrokenAction {
matches!(request.send().await, Ok(response) if response.status().is_success())
}
/// Transmission answers an RPC call without a session id with `409` plus
/// qBittorrent answers an RPC call without a session id with `409` plus
/// the id to retry with. That is a live daemon, so it counts as
/// reachable.
async fn probe_transmission(&self) -> bool {
async fn probe_qbit(&self) -> bool {
let request = self
.http
.post(&self.upstreams.transmission_url)
.post(&self.upstreams.qbittorrent_url)
.timeout(PROBE_TIMEOUT)
.json(&serde_json::json!({ "method": "session-get" }));
matches!(
@@ -153,8 +222,8 @@ impl Action for BrokenAction {
"broken"
}
fn run<'a>(&'a self, _database: &'a Db) -> ActionFuture<'a> {
Box::pin(async move { Ok(self.tick().await) })
fn run<'a>(&'a self, database: &'a Db) -> ActionFuture<'a> {
Box::pin(async move { Ok(self.tick(database).await) })
}
}
@@ -166,24 +235,50 @@ mod tests {
use super::*;
fn upstreams(prowlarr_url: String, transmission_url: String) -> Upstreams {
fn upstreams(prowlarr_url: String, qbittorrent_url: String) -> Upstreams {
Upstreams {
prowlarr_url,
prowlarr_api_key: None,
transmission_url,
qbittorrent_url,
tmdb_url: "http://127.0.0.1:1".to_string(),
tmdb_api_key: None,
}
}
/// A migrated database whose settings enable nothing — the classic
/// upstreams under test here must not share the tick with subtitle
/// lamps the seed row would otherwise put in use.
async fn database() -> (tempfile::TempDir, Db) {
let dir = tempfile::tempdir().unwrap();
let db = Db::connect(dir.path().join("broken-test.db"))
.await
.unwrap();
db.migrate().await.unwrap();
sqlx::query("UPDATE subtitle_settings SET providers_enabled = '[]'")
.execute(db.pool())
.await
.unwrap();
(dir, db)
}
fn subtitles() -> SubtitleUpstreams {
SubtitleUpstreams {
providers: Vec::new(),
backends: Vec::new(),
// Present on every machine that runs these tests.
alass_path: "sh".into(),
ffmpeg_path: "sh".into(),
}
}
#[tokio::test]
async fn an_unreachable_upstream_notifies_once_then_re_arms() {
let prowlarr = MockServer::start().await;
// No mock mounted: every request 404s, which counts as unreachable.
let transmission = MockServer::start().await;
let qbit = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(409))
.mount(&transmission)
.mount(&qbit)
.await;
let ntfy = MockServer::start().await;
@@ -192,14 +287,16 @@ mod tests {
.mount(&ntfy)
.await;
let (_dir, db) = database().await;
let action = BrokenAction::new(
upstreams(prowlarr.uri(), transmission.uri()),
upstreams(prowlarr.uri(), qbit.uri()),
subtitles(),
Notifier::new(ntfy.uri()).unwrap(),
"operator-topic".to_string(),
);
let first = action.tick().await;
let second = action.tick().await;
let first = action.tick(&db).await;
let second = action.tick(&db).await;
assert_eq!(first.len(), 1, "notifies on the tick it goes unreachable");
assert_eq!(second.len(), 0, "does not repeat while still broken");
@@ -209,12 +306,105 @@ mod tests {
.respond_with(ResponseTemplate::new(200))
.mount(&prowlarr)
.await;
let recovered = action.tick().await;
let recovered = action.tick(&db).await;
assert_eq!(recovered.len(), 0, "recovery is silent, no fourth event");
// Take it down again: a fresh outage re-arms and notifies again.
prowlarr.reset().await;
let broken_again = action.tick().await;
let broken_again = action.tick(&db).await;
assert_eq!(broken_again.len(), 1, "re-arms after recovering");
}
/// #200: an enabled provider nobody configured is a broken lamp like any
/// other, and it notifies once — then stays quiet while it stays broken.
#[tokio::test]
async fn an_enabled_but_missing_subtitle_provider_notifies_once() {
let ntfy = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200))
.mount(&ntfy)
.await;
let dir = tempfile::tempdir().unwrap();
let db = Db::connect(dir.path().join("broken-subs.db"))
.await
.unwrap();
db.migrate().await.unwrap();
// The seed row enables OpenSubtitles, which is not attached to this action.
let prowlarr = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/ping"))
.respond_with(ResponseTemplate::new(200))
.mount(&prowlarr)
.await;
let qbit = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(409))
.mount(&qbit)
.await;
let action = BrokenAction::new(
upstreams(prowlarr.uri(), qbit.uri()),
subtitles(),
Notifier::new(ntfy.uri()).unwrap(),
"operator-topic".to_string(),
);
let first = action.tick(&db).await;
let second = action.tick(&db).await;
assert_eq!(first.len(), 1, "one lamp per missing provider");
assert_eq!(second.len(), 0, "does not repeat while still broken");
// Attaching nothing but disabling them silences the lamps.
sqlx::query("UPDATE subtitle_settings SET providers_enabled = '[]'")
.execute(db.pool())
.await
.unwrap();
let third = action.tick(&db).await;
assert_eq!(third.len(), 0, "a disabled provider cannot be broken");
}
/// #200: binaries are judged at their configured paths.
#[tokio::test]
async fn a_missing_binary_is_a_broken_lamp() {
let ntfy = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200))
.mount(&ntfy)
.await;
let (_dir, db) = database().await;
let prowlarr = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/ping"))
.respond_with(ResponseTemplate::new(200))
.mount(&prowlarr)
.await;
let qbit = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(409))
.mount(&qbit)
.await;
let subs = SubtitleUpstreams {
alass_path: "/nowhere/alass".into(),
..subtitles()
};
let action = BrokenAction::new(
upstreams(prowlarr.uri(), qbit.uri()),
subs,
Notifier::new(ntfy.uri()).unwrap(),
"operator-topic".to_string(),
);
let first = action.tick(&db).await;
assert_eq!(first.len(), 1, "only the missing binary notifies");
assert!(
first
.iter()
.any(|outcome| format!("{outcome:?}").contains("alass")),
"{first:?}"
);
}
}
+295 -17
View File
@@ -20,7 +20,12 @@ pub const ENV_DATABASE_PATH: &str = "ARR_DATABASE_PATH";
pub const ENV_MEDIA_ROOT: &str = "ARR_MEDIA_ROOT";
pub const ENV_PROWLARR_URL: &str = "ARR_PROWLARR_URL";
pub const ENV_PROWLARR_API_KEY: &str = "ARR_PROWLARR_API_KEY";
pub const ENV_TRANSMISSION_URL: &str = "ARR_TRANSMISSION_URL";
pub const ENV_QBITTORRENT_URL: &str = "ARR_QBITTORRENT_URL";
// The WebUI password is a secret, so both halves are env-only (§10) and
// neither has a `ConfigFile` field. Leaving them unset is valid: it is the
// right shape for an instance that bypasses authentication for arr's address.
pub const ENV_QBITTORRENT_USERNAME: &str = "ARR_QBITTORRENT_USERNAME";
pub const ENV_QBITTORRENT_PASSWORD: &str = "ARR_QBITTORRENT_PASSWORD";
pub const ENV_DOWNLOAD_DIR: &str = "ARR_DOWNLOAD_DIR";
pub const ENV_SEED_RATIO_LIMIT: &str = "ARR_SEED_RATIO_LIMIT";
pub const ENV_SEED_IDLE_LIMIT_MINUTES: &str = "ARR_SEED_IDLE_LIMIT_MINUTES";
@@ -30,22 +35,42 @@ pub const ENV_JELLYFIN_URL: &str = "ARR_JELLYFIN_URL";
pub const ENV_JELLYFIN_API_KEY: &str = "ARR_JELLYFIN_API_KEY";
pub const ENV_NTFY_URL: &str = "ARR_NTFY_URL";
pub const ENV_NTFY_OPERATOR_TOPIC: &str = "ARR_NTFY_OPERATOR_TOPIC";
// DESIGN.md §15 subtitle bootstrap. Credentials are secrets (env-only,
// below); base URLs, the remote-command template and the binary paths may
// also sit in the config file.
pub const ENV_OPENSUBTITLES_API_KEY: &str = "ARR_OPENSUBTITLES_API_KEY";
pub const ENV_OPENSUBTITLES_USERNAME: &str = "ARR_OPENSUBTITLES_USERNAME";
pub const ENV_OPENSUBTITLES_PASSWORD: &str = "ARR_OPENSUBTITLES_PASSWORD";
// The OpenAI-compatible backend's base URL and model are `subtitle_settings`
// rows, not bootstrap keys (#220): that backend is anything speaking the
// shape, and which endpoint is in use is something the operator changes from
// `/settings`. Only the key is here, because §10 keeps secrets out of the
// database — and an endpoint needing no key at all is valid.
pub const ENV_TRANSLATE_OPENAI_API_KEY: &str = "ARR_TRANSLATE_OPENAI_API_KEY";
pub const ENV_TRANSLATE_DEEPL_API_KEY: &str = "ARR_TRANSLATE_DEEPL_API_KEY";
pub const ENV_TRANSLATE_DEEPL_BASE_URL: &str = "ARR_TRANSLATE_DEEPL_BASE_URL";
pub const ENV_TRANSLATE_GOOGLE_API_KEY: &str = "ARR_TRANSLATE_GOOGLE_API_KEY";
pub const ENV_TRANSLATE_GOOGLE_BASE_URL: &str = "ARR_TRANSLATE_GOOGLE_BASE_URL";
pub const ENV_TRANSLATE_COMMAND_TEMPLATE: &str = "ARR_TRANSLATE_COMMAND_TEMPLATE";
pub const ENV_ALASS_PATH: &str = "ARR_ALASS_PATH";
pub const ENV_FFMPEG_PATH: &str = "ARR_FFMPEG_PATH";
pub const DEFAULT_BIND_ADDR: &str = "0.0.0.0:7878";
pub const DEFAULT_DATABASE_PATH: &str = "arr.db";
pub const DEFAULT_MEDIA_ROOT: &str = "/mnt/media";
pub const DEFAULT_PROWLARR_URL: &str = "http://localhost:9696";
pub const DEFAULT_TRANSMISSION_URL: &str = "http://localhost:9091/transmission/rpc";
/// Transmission's own view of the download directory (DESIGN.md §3). It
pub const DEFAULT_QBITTORRENT_URL: &str = "http://localhost:8080";
/// qBittorrent's own view of the download directory (DESIGN.md §3). It
/// shares the media dataset with the library so hardlinks work (§7.2).
pub const DEFAULT_DOWNLOAD_DIR: &str = "/mnt/media/transmission/complete";
pub const DEFAULT_DOWNLOAD_DIR: &str = "/mnt/media/qbittorrent/complete";
/// Seeding obligation defaults (§7.3), applied to every torrent at add time
/// and enforced by Transmission. Per-tracker rules are issue #25.
/// and enforced by qBittorrent. Per-tracker rules are issue #25.
pub const DEFAULT_SEED_RATIO_LIMIT: f64 = 1.0;
pub const DEFAULT_SEED_IDLE_LIMIT_MINUTES: u64 = 4320;
pub const DEFAULT_JELLYFIN_URL: &str = "http://localhost:8096";
pub const DEFAULT_NTFY_URL: &str = "http://localhost";
pub const DEFAULT_ALASS_PATH: &str = "alass";
pub const DEFAULT_FFMPEG_PATH: &str = "ffmpeg";
#[derive(Debug, thiserror::Error)]
pub enum ConfigError {
#[error("io: {0}")]
@@ -77,7 +102,7 @@ struct ConfigFile {
#[serde(default)]
prowlarr_url: Option<String>,
#[serde(default)]
transmission_url: Option<String>,
qbittorrent_url: Option<String>,
#[serde(default)]
download_dir: Option<PathBuf>,
#[serde(default)]
@@ -92,6 +117,16 @@ struct ConfigFile {
ntfy_url: Option<String>,
#[serde(default)]
ntfy_operator_topic: Option<String>,
#[serde(default)]
translate_deepl_base_url: Option<String>,
#[serde(default)]
translate_google_base_url: Option<String>,
#[serde(default)]
translate_command_template: Option<String>,
#[serde(default)]
alass_path: Option<PathBuf>,
#[serde(default)]
ffmpeg_path: Option<PathBuf>,
}
#[derive(Debug, Clone, Copy, PartialEq, Deserialize)]
@@ -121,7 +156,9 @@ pub struct EnvOverrides {
pub media_root: Option<String>,
pub prowlarr_url: Option<String>,
pub prowlarr_api_key: Option<String>,
pub transmission_url: Option<String>,
pub qbittorrent_url: Option<String>,
pub qbittorrent_username: Option<String>,
pub qbittorrent_password: Option<String>,
pub download_dir: Option<String>,
pub seed_ratio_limit: Option<String>,
pub seed_idle_limit_minutes: Option<String>,
@@ -131,6 +168,17 @@ pub struct EnvOverrides {
pub jellyfin_api_key: Option<String>,
pub ntfy_url: Option<String>,
pub ntfy_operator_topic: Option<String>,
pub opensubtitles_api_key: Option<String>,
pub opensubtitles_username: Option<String>,
pub opensubtitles_password: Option<String>,
pub translate_openai_api_key: Option<String>,
pub translate_deepl_api_key: Option<String>,
pub translate_deepl_base_url: Option<String>,
pub translate_google_api_key: Option<String>,
pub translate_google_base_url: Option<String>,
pub translate_command_template: Option<String>,
pub alass_path: Option<String>,
pub ffmpeg_path: Option<String>,
}
impl EnvOverrides {
@@ -142,7 +190,9 @@ impl EnvOverrides {
media_root: std::env::var(ENV_MEDIA_ROOT).ok(),
prowlarr_url: std::env::var(ENV_PROWLARR_URL).ok(),
prowlarr_api_key: std::env::var(ENV_PROWLARR_API_KEY).ok(),
transmission_url: std::env::var(ENV_TRANSMISSION_URL).ok(),
qbittorrent_url: std::env::var(ENV_QBITTORRENT_URL).ok(),
qbittorrent_username: std::env::var(ENV_QBITTORRENT_USERNAME).ok(),
qbittorrent_password: std::env::var(ENV_QBITTORRENT_PASSWORD).ok(),
download_dir: std::env::var(ENV_DOWNLOAD_DIR).ok(),
seed_ratio_limit: std::env::var(ENV_SEED_RATIO_LIMIT).ok(),
seed_idle_limit_minutes: std::env::var(ENV_SEED_IDLE_LIMIT_MINUTES).ok(),
@@ -152,6 +202,17 @@ impl EnvOverrides {
jellyfin_api_key: std::env::var(ENV_JELLYFIN_API_KEY).ok(),
ntfy_url: std::env::var(ENV_NTFY_URL).ok(),
ntfy_operator_topic: std::env::var(ENV_NTFY_OPERATOR_TOPIC).ok(),
opensubtitles_api_key: std::env::var(ENV_OPENSUBTITLES_API_KEY).ok(),
opensubtitles_username: std::env::var(ENV_OPENSUBTITLES_USERNAME).ok(),
opensubtitles_password: std::env::var(ENV_OPENSUBTITLES_PASSWORD).ok(),
translate_openai_api_key: std::env::var(ENV_TRANSLATE_OPENAI_API_KEY).ok(),
translate_deepl_api_key: std::env::var(ENV_TRANSLATE_DEEPL_API_KEY).ok(),
translate_deepl_base_url: std::env::var(ENV_TRANSLATE_DEEPL_BASE_URL).ok(),
translate_google_api_key: std::env::var(ENV_TRANSLATE_GOOGLE_API_KEY).ok(),
translate_google_base_url: std::env::var(ENV_TRANSLATE_GOOGLE_BASE_URL).ok(),
translate_command_template: std::env::var(ENV_TRANSLATE_COMMAND_TEMPLATE).ok(),
alass_path: std::env::var(ENV_ALASS_PATH).ok(),
ffmpeg_path: std::env::var(ENV_FFMPEG_PATH).ok(),
}
}
}
@@ -164,8 +225,13 @@ pub struct Config {
pub media_root: PathBuf,
pub prowlarr_url: String,
pub prowlarr_api_key: Option<String>,
pub transmission_url: String,
/// Where Transmission puts completed downloads, in Transmission's own
pub qbittorrent_url: String,
/// The `WebUI` login. `None` for an instance that bypasses authentication
/// for arr's address; a username without a password (or the reverse) is
/// treated as no credentials at all.
pub qbittorrent_username: Option<String>,
pub qbittorrent_password: Option<String>,
/// Where qBittorrent puts completed downloads, in qBittorrent's own
/// namespace (§7.1).
pub download_dir: PathBuf,
pub seed_ratio_limit: f64,
@@ -182,6 +248,84 @@ pub struct Config {
/// *broken*. `None` means those two notifications are skipped — there is
/// no sensible default topic name to fall back to.
pub ntfy_operator_topic: Option<String>,
/// §15. `None` means OpenSubtitles.com search runs unauthenticated,
/// which its API allows at a lower rate.
pub opensubtitles_api_key: Option<String>,
pub opensubtitles_username: Option<String>,
pub opensubtitles_password: Option<String>,
/// §15. `None` is valid: `llama.cpp` and a local gateway serve without
/// authentication. Where that backend points and which model it names
/// are `subtitle_settings` rows, read at start-up and re-read on every
/// edit (#220), not bootstrap config.
pub translate_openai_api_key: Option<String>,
pub translate_deepl_api_key: Option<String>,
/// `None` means the backend's own built-in default when it lands (#192).
pub translate_deepl_base_url: Option<String>,
pub translate_google_api_key: Option<String>,
/// `None` means the backend's own built-in default when it lands (#193).
pub translate_google_base_url: Option<String>,
/// §15. The generic remote-command backend's invocation template, e.g.
/// `ssh box claude -p`. `None` means that backend is unconfigured.
pub translate_command_template: Option<String>,
/// §15. `alass` runs on every fetched and every translated subtitle; a
/// bare name resolves through `PATH`, matching `arr-probe`'s `ffprobe`.
pub alass_path: PathBuf,
/// §15. Extracts text-format embedded tracks to sidecar SRTs.
pub ffmpeg_path: PathBuf,
}
/// §15/§10 subtitle bootstrap: provider and translator credentials, base
/// URLs, the remote-command template, and binary paths. Resolved separately
/// from [`Config::resolve`] for the same reason `arr-daemon::api_state` is
/// split out of `run` — one field per provider or backend, and `resolve` is
/// already at the too-many-lines limit.
struct SubtitleBootstrap {
opensubtitles_api_key: Option<String>,
opensubtitles_username: Option<String>,
opensubtitles_password: Option<String>,
translate_openai_api_key: Option<String>,
translate_deepl_api_key: Option<String>,
translate_deepl_base_url: Option<String>,
translate_google_api_key: Option<String>,
translate_google_base_url: Option<String>,
translate_command_template: Option<String>,
alass_path: PathBuf,
ffmpeg_path: PathBuf,
}
fn resolve_subtitle_bootstrap(env: &EnvOverrides, file: &ConfigFile) -> SubtitleBootstrap {
SubtitleBootstrap {
opensubtitles_api_key: env.opensubtitles_api_key.clone(),
opensubtitles_username: env.opensubtitles_username.clone(),
opensubtitles_password: env.opensubtitles_password.clone(),
translate_openai_api_key: env.translate_openai_api_key.clone(),
translate_deepl_api_key: env.translate_deepl_api_key.clone(),
translate_deepl_base_url: env
.translate_deepl_base_url
.clone()
.or_else(|| file.translate_deepl_base_url.clone()),
translate_google_api_key: env.translate_google_api_key.clone(),
translate_google_base_url: env
.translate_google_base_url
.clone()
.or_else(|| file.translate_google_base_url.clone()),
translate_command_template: env
.translate_command_template
.clone()
.or_else(|| file.translate_command_template.clone()),
alass_path: env
.alass_path
.clone()
.map(PathBuf::from)
.or_else(|| file.alass_path.clone())
.unwrap_or_else(|| PathBuf::from(DEFAULT_ALASS_PATH)),
ffmpeg_path: env
.ffmpeg_path
.clone()
.map(PathBuf::from)
.or_else(|| file.ffmpeg_path.clone())
.unwrap_or_else(|| PathBuf::from(DEFAULT_FFMPEG_PATH)),
}
}
impl Config {
@@ -197,6 +341,7 @@ impl Config {
Some(path) => ConfigFile::load(Path::new(path))?,
None => ConfigFile::default(),
};
let subtitles = resolve_subtitle_bootstrap(&env, &file);
let bind_addr = match &env.bind_addr {
Some(raw) => parse_bind_addr(raw, ENV_BIND_ADDR)?,
@@ -231,10 +376,12 @@ impl Config {
.or(file.prowlarr_url)
.unwrap_or_else(|| DEFAULT_PROWLARR_URL.to_string()),
prowlarr_api_key: env.prowlarr_api_key,
transmission_url: env
.transmission_url
.or(file.transmission_url)
.unwrap_or_else(|| DEFAULT_TRANSMISSION_URL.to_string()),
qbittorrent_url: env
.qbittorrent_url
.or(file.qbittorrent_url)
.unwrap_or_else(|| DEFAULT_QBITTORRENT_URL.to_string()),
qbittorrent_username: env.qbittorrent_username,
qbittorrent_password: env.qbittorrent_password,
download_dir: env
.download_dir
.map(PathBuf::from)
@@ -263,6 +410,17 @@ impl Config {
.or(file.ntfy_url)
.unwrap_or_else(|| DEFAULT_NTFY_URL.to_string()),
ntfy_operator_topic: env.ntfy_operator_topic.or(file.ntfy_operator_topic),
opensubtitles_api_key: subtitles.opensubtitles_api_key,
opensubtitles_username: subtitles.opensubtitles_username,
opensubtitles_password: subtitles.opensubtitles_password,
translate_openai_api_key: subtitles.translate_openai_api_key,
translate_deepl_api_key: subtitles.translate_deepl_api_key,
translate_deepl_base_url: subtitles.translate_deepl_base_url,
translate_google_api_key: subtitles.translate_google_api_key,
translate_google_base_url: subtitles.translate_google_base_url,
translate_command_template: subtitles.translate_command_template,
alass_path: subtitles.alass_path,
ffmpeg_path: subtitles.ffmpeg_path,
})
}
}
@@ -287,6 +445,28 @@ fn parse_bind_addr(raw: &str, env: &'static str) -> Result<SocketAddr, ConfigErr
mod tests {
use super::*;
/// §10 keeps secrets out of the config file, so the `WebUI` login is
/// env-only and absent by default — which is the shape an instance that
/// bypasses authentication for arr's address wants.
#[test]
fn the_qbittorrent_login_is_env_only_and_optional() {
let config = Config::resolve(EnvOverrides::default()).unwrap();
assert_eq!(config.qbittorrent_username, None);
assert_eq!(config.qbittorrent_password, None);
let config = Config::resolve(EnvOverrides {
qbittorrent_username: Some("arr".into()),
qbittorrent_password: Some("secret".into()),
..EnvOverrides::default()
})
.unwrap();
assert_eq!(config.qbittorrent_username.as_deref(), Some("arr"));
let error = toml::from_str::<ConfigFile>("qbittorrent_password = \"secret\"")
.expect_err("the file has no field for it");
assert!(error.to_string().contains("qbittorrent_password"));
}
#[test]
fn empty_environment_resolves_to_defaults() {
let config = Config::resolve(EnvOverrides::default()).unwrap();
@@ -294,7 +474,7 @@ mod tests {
assert_eq!(config.database_path, PathBuf::from(DEFAULT_DATABASE_PATH));
assert_eq!(config.media_root, PathBuf::from(DEFAULT_MEDIA_ROOT));
assert_eq!(config.prowlarr_url, DEFAULT_PROWLARR_URL);
assert_eq!(config.transmission_url, DEFAULT_TRANSMISSION_URL);
assert_eq!(config.qbittorrent_url, DEFAULT_QBITTORRENT_URL);
assert_eq!(config.download_dir, PathBuf::from(DEFAULT_DOWNLOAD_DIR));
assert!((config.seed_ratio_limit - DEFAULT_SEED_RATIO_LIMIT).abs() < f64::EPSILON);
assert_eq!(
@@ -308,6 +488,10 @@ mod tests {
assert_eq!(config.prowlarr_api_key, None);
assert_eq!(config.tmdb_api_key, None);
assert_eq!(config.jellyfin_api_key, None);
assert_eq!(config.opensubtitles_api_key, None);
assert_eq!(config.translate_command_template, None);
assert_eq!(config.alass_path, PathBuf::from(DEFAULT_ALASS_PATH));
assert_eq!(config.ffmpeg_path, PathBuf::from(DEFAULT_FFMPEG_PATH));
}
#[test]
@@ -345,7 +529,7 @@ prowlarr_url = "http://prowlarr.internal:9696"
assert_eq!(config.media_root, PathBuf::from("/tank/media"));
assert_eq!(config.prowlarr_url, "http://prowlarr.internal:9696");
// Untouched fields still default.
assert_eq!(config.transmission_url, DEFAULT_TRANSMISSION_URL);
assert_eq!(config.qbittorrent_url, DEFAULT_QBITTORRENT_URL);
}
#[test]
@@ -376,6 +560,100 @@ prowlarr_url = "http://prowlarr.internal:9696"
assert_eq!(config.jellyfin_api_key.as_deref(), Some("secret-3"));
}
/// §15: `OpenSubtitles` and every translator's credentials are secrets, so
/// this mirrors [`secrets_come_only_from_env`] for them.
#[test]
fn subtitle_secrets_come_only_from_env() {
let env = EnvOverrides {
opensubtitles_api_key: Some("os-key".into()),
opensubtitles_username: Some("os-user".into()),
opensubtitles_password: Some("os-pass".into()),
translate_openai_api_key: Some("oa-key".into()),
translate_deepl_api_key: Some("dl-key".into()),
translate_google_api_key: Some("gg-key".into()),
..EnvOverrides::default()
};
let config = Config::resolve(env).unwrap();
assert_eq!(config.opensubtitles_api_key.as_deref(), Some("os-key"));
assert_eq!(config.opensubtitles_username.as_deref(), Some("os-user"));
assert_eq!(config.opensubtitles_password.as_deref(), Some("os-pass"));
assert_eq!(config.translate_openai_api_key.as_deref(), Some("oa-key"));
assert_eq!(config.translate_deepl_api_key.as_deref(), Some("dl-key"));
assert_eq!(config.translate_google_api_key.as_deref(), Some("gg-key"));
}
#[test]
fn a_subtitle_secret_in_the_config_file_is_a_parse_error() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("arr.toml");
std::fs::write(&path, "opensubtitles_api_key = \"leaked\"\n").unwrap();
let env = EnvOverrides {
config_file: Some(path.to_string_lossy().into_owned()),
..EnvOverrides::default()
};
assert!(matches!(
Config::resolve(env),
Err(ConfigError::TomlDecode(_))
));
}
/// §15: base URLs, the remote-command template and the binary paths are
/// not secrets, so the file, the environment and their precedence all
/// apply the same way they do for `jellyfin_url` and friends.
#[test]
fn subtitle_non_secret_bootstrap_comes_from_file_or_env() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("arr.toml");
std::fs::write(
&path,
r#"
translate_command_template = "ssh box claude -p"
alass_path = "/usr/local/bin/alass"
"#,
)
.unwrap();
let env = EnvOverrides {
config_file: Some(path.to_string_lossy().into_owned()),
..EnvOverrides::default()
};
let config = Config::resolve(env.clone()).unwrap();
assert_eq!(
config.translate_command_template.as_deref(),
Some("ssh box claude -p")
);
assert_eq!(config.alass_path, PathBuf::from("/usr/local/bin/alass"));
// ffmpeg_path was not set anywhere, so it still defaults.
assert_eq!(config.ffmpeg_path, PathBuf::from(DEFAULT_FFMPEG_PATH));
let config = Config::resolve(EnvOverrides {
ffmpeg_path: Some("/opt/bin/ffmpeg".into()),
..env
})
.unwrap();
assert_eq!(config.ffmpeg_path, PathBuf::from("/opt/bin/ffmpeg"));
}
/// #220 retired `translate_openai_base_url` and `translate_openai_model`
/// — both are `subtitle_settings` rows now. `deny_unknown_fields` turns
/// a config file still carrying them into a parse error, so an operator
/// who upgrades sees the move rather than a setting silently ignored.
#[test]
fn the_retired_openai_endpoint_keys_are_a_parse_error() {
for key in ["translate_openai_base_url", "translate_openai_model"] {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("arr.toml");
std::fs::write(&path, format!("{key} = \"x\"\n")).unwrap();
let env = EnvOverrides {
config_file: Some(path.to_string_lossy().into_owned()),
..EnvOverrides::default()
};
assert!(
matches!(Config::resolve(env), Err(ConfigError::TomlDecode(_))),
"{key} must no longer be accepted"
);
}
}
#[test]
fn tmdb_url_is_an_env_only_seam() {
let config = Config::resolve(EnvOverrides::default()).unwrap();
+200 -228
View File
@@ -1,5 +1,5 @@
//! The grab pipeline: close the "wanted movie, no file" gap by searching,
//! scoring, picking a winner and sending it to Transmission. See DESIGN.md
//! scoring, picking a winner and sending it to qBittorrent. See DESIGN.md
//! §5.4, §6.2, §7.1, §7.3 and §8.
//!
//! There is no grab delay and there is no job queue. The gap is recomputed
@@ -7,7 +7,7 @@
//! restarting converges instead of double-grabbing:
//!
//! - a title with a live `grabs` row is not a gap, so it is never re-searched;
//! - Transmission's `torrent-add` is keyed on the infohash, so re-sending the
//! - qBittorrent's `torrent-add` is keyed on the infohash, so re-sending the
//! same release returns the torrent that is already there rather than a
//! second one;
//! - the `grabs` row is written from that response, so a crash between the add
@@ -22,7 +22,7 @@ use arr_core::policy::{evaluate, Candidate};
use arr_core::score::{claimed_episode_count, score};
use arr_core::{Language, Policy, TitleOverrides, Verdict};
use arr_db::{blacklist, Blacklist, Db, MoviePolicy};
use arr_dl::{AddTorrent, TorrentSource, TransmissionClient};
use arr_dl::{AddTorrent, QbitClient, TorrentSource};
use arr_indexer::{Download, ProwlarrClient, SearchRelease, SearchRequest};
use arr_meta::TmdbClient;
@@ -37,7 +37,7 @@ const MOVIES_PER_TICK: i64 = 5;
/// Seeding obligations, per tracker in principle (§7.3) and per install in
/// practice until issue #25 gives them a home. Both are set on the torrent at
/// add time and enforced by Transmission.
/// add time and enforced by qBittorrent.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct SeedingLimits {
pub ratio: f64,
@@ -77,8 +77,8 @@ pub enum GrabError {
Metadata(#[from] arr_meta::Error),
#[error("movie {0} has an invalid TMDB id")]
InvalidTmdbId(i64),
#[error("transmission: {0}")]
Transmission(#[from] arr_dl::Error),
#[error("qbit: {0}")]
Qbit(#[from] arr_dl::Error),
#[error("download link: {0}")]
Download(#[from] arr_indexer::DownloadError),
#[error("release {name}: {source}")]
@@ -102,13 +102,13 @@ impl GrabAction {
#[must_use]
pub fn new(
prowlarr: ProwlarrClient,
transmission: TransmissionClient,
qbit: QbitClient,
download_dir: PathBuf,
seeding: SeedingRules,
) -> Self {
Self {
indexers: IndexerDirectory::new(prowlarr.clone()),
grabber: Grabber::new(prowlarr.clone(), transmission, download_dir, seeding),
grabber: Grabber::new(prowlarr.clone(), qbit, download_dir, seeding),
prowlarr,
tmdb: None,
}
@@ -427,7 +427,7 @@ impl GrabAction {
/// The manual one-click grab (§9.3, issue #107): the release is already
/// chosen, so this skips search and scoring and sends it straight to
/// Transmission.
/// qBittorrent.
pub(crate) async fn grab_release_now(
&self,
database: &Db,
@@ -460,17 +460,17 @@ impl GrabAction {
}
}
/// Sending a chosen release to Transmission and recording the grab.
/// Sending a chosen release to qBittorrent and recording the grab.
///
/// Targeted search and RSS (§6.2) differ in how a title is chosen and in
/// whether a failed attempt counts toward a backoff; from the winning
/// release onward they are the same writes, so they share this.
#[derive(Debug)]
pub(crate) struct Grabber {
/// Resolves the winner's indexer link before it is sent on: Transmission
/// Resolves the winner's indexer link before it is sent on: qBittorrent
/// cannot reach Prowlarr and arr can (issue #100).
prowlarr: ProwlarrClient,
transmission: TransmissionClient,
qbit: QbitClient,
download_dir: PathBuf,
seeding: SeedingRules,
}
@@ -532,22 +532,22 @@ impl GrabScope {
impl Grabber {
pub(crate) fn new(
prowlarr: ProwlarrClient,
transmission: TransmissionClient,
qbit: QbitClient,
download_dir: PathBuf,
seeding: SeedingRules,
) -> Self {
Self {
prowlarr,
transmission,
qbit,
download_dir,
seeding,
}
}
/// Move grabs Transmission reports as complete out of `sent`, whatever
/// Move grabs qBittorrent reports as complete out of `sent`, whatever
/// they target.
///
/// Transmission is authoritative and its view is rebuilt on every tick
/// qBittorrent is authoritative and its view is rebuilt on every tick
/// rather than cached (§8), so this is also what reconstructs in-flight
/// state after a restart. Both grab actions call it; whichever runs first
/// does the work and the other finds nothing.
@@ -564,7 +564,7 @@ impl Grabber {
}
let torrents: HashMap<String, f64> = self
.transmission
.qbit
.list_torrents()
.await?
.into_iter()
@@ -603,7 +603,7 @@ impl Grabber {
Ok(outcomes)
}
/// §86/#108: a `sent` grab whose infohash Transmission no longer reports
/// §86/#108: a `sent` grab whose infohash qBittorrent no longer reports
/// — removed by hand, not a policy failure. Marked `vanished` rather than
/// `failed` so it does not feed the `needs_decision` queue (attention.rs).
/// Nothing is blacklisted, since the release itself never failed policy,
@@ -624,10 +624,10 @@ impl Grabber {
grab_id,
target_kind,
target_id,
"torrent vanished from Transmission; target parked"
"torrent vanished from qBittorrent; target parked"
);
Ok(Outcome::new(
format!("grab {grab_id} sent, torrent vanished from Transmission"),
format!("grab {grab_id} sent, torrent vanished from qBittorrent"),
format!("parked {target_kind} {target_id}"),
))
}
@@ -646,12 +646,12 @@ impl Grabber {
}
}
/// Resolve the winner's indexer link and add it to Transmission.
/// Resolve the winner's indexer link and add it to qBittorrent.
///
/// The link is resolved here rather than passed on, because Transmission
/// The link is resolved here rather than passed on, because qBittorrent
/// has no route to Prowlarr and cannot follow its redirect to a magnet
/// (issue #100).
async fn send_to_transmission(
async fn send_to_qbit(
&self,
winner: &Eligible,
loaded: &MoviePolicy,
@@ -659,7 +659,7 @@ impl Grabber {
let seeding = self.seeding.for_indexer(winner.indexer_id);
let source = torrent_source(self.prowlarr.download(&winner.download_url).await?);
Ok(self
.transmission
.qbit
.add_torrent(AddTorrent {
source,
label: label(loaded),
@@ -670,10 +670,10 @@ impl Grabber {
.await?)
}
/// Add the winning release to Transmission and record the grab.
/// Add the winning release to qBittorrent and record the grab.
///
/// The search still counts as an attempt (§6.2) on every exit that is
/// not a completed grab — a Transmission error or a blacklisted-infohash
/// not a completed grab — a qBittorrent error or a blacklisted-infohash
/// drop must not leave the same release to repeat next tick with no
/// backoff.
pub(crate) async fn send_winner(
@@ -684,7 +684,7 @@ impl Grabber {
blacklist: &Blacklist,
winner: Eligible,
) -> Result<Option<Outcome>, GrabError> {
let added = match self.send_to_transmission(&winner, loaded).await {
let added = match self.send_to_qbit(&winner, loaded).await {
Ok(added) => added,
Err(error) => {
self.record_attempt(database, target).await?;
@@ -694,7 +694,7 @@ impl Grabber {
let infohash = added.hash.to_ascii_lowercase();
// §6.3's second key. A `.torrent` link hides its infohash until
// Transmission has fetched it, so the same blacklisted torrent can
// qBittorrent has fetched it, so the same blacklisted torrent can
// reach here under a new name.
if blacklist.blocks_infohash(&infohash) {
self.record_attempt(database, target).await?;
@@ -707,6 +707,14 @@ impl Grabber {
// with its own earlier row. A `sent`/`downloaded` row is the restart
// case and is left alone; a `vanished` one (§86) is reclaimed, since
// nothing blacklisted the release.
//
// A row whose target no longer exists is reclaimed too, whatever its
// state. Deleting a title used to leave its grabs behind, and one of
// those orphans holding the infohash meant a later grab of the same
// release silently recorded nothing: the title flipped to
// `downloading` while the only row still pointed at the deleted one,
// so it never imported. Titles now take their grabs with them, and
// this clause heals the databases that already have orphans.
let target_kind = target.scope.target_kind();
let target_id = target.scope.target_id();
let inserted = sqlx::query!(
@@ -720,6 +728,12 @@ impl Grabber {
grabbed_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now'),
imported_at = NULL
WHERE grabs.state = 'vanished'
OR (grabs.target_kind = 'movie'
AND NOT EXISTS (SELECT 1 FROM movies WHERE id = grabs.target_id))
OR (grabs.target_kind = 'season'
AND NOT EXISTS (SELECT 1 FROM seasons WHERE id = grabs.target_id))
OR (grabs.target_kind = 'episode'
AND NOT EXISTS (SELECT 1 FROM episodes WHERE id = grabs.target_id))
RETURNING id AS "id!: i64""#,
winner.id,
target_kind,
@@ -782,7 +796,7 @@ impl Grabber {
/// Undo a grab whose infohash turned out to be blacklisted (§6.3).
///
/// The name is added to the blacklist so the next tick stops at the cheap
/// check instead of paying Transmission again, and no `grabs` row is
/// check instead of paying qBittorrent again, and no `grabs` row is
/// written, which leaves the title a gap for the next candidate.
async fn drop_blacklisted_torrent(
&self,
@@ -822,7 +836,7 @@ impl Grabber {
} else {
// This tick added it seconds ago, so it carries no seeding
// obligation and has nothing on disk worth keeping.
self.transmission.remove_torrent(added.id, true).await?;
self.qbit.remove_torrent(&added.hash, true).await?;
tracing::warn!(
title = target.title,
release = release_name,
@@ -1509,7 +1523,7 @@ pub(crate) async fn park_target(
}
/// `movies-main`, `tv-kids` (§7.1). Distinct from Radarr's own labels, so
/// both stacks can run against one Transmission.
/// both stacks can run against one qBittorrent.
fn label(loaded: &MoviePolicy) -> String {
label_for_root(&loaded.root_kind, &loaded.root_audience)
}
@@ -1519,7 +1533,7 @@ pub(crate) fn label_for_root(kind: &str, audience: &str) -> String {
format!("{kind}-{audience}")
}
/// A resolved download, in the shape Transmission takes it.
/// A resolved download, in the shape qBittorrent takes it.
fn torrent_source(download: Download) -> TorrentSource {
match download {
Download::Magnet(uri) => TorrentSource::Magnet(uri),
@@ -1572,6 +1586,18 @@ pub(crate) mod test_downloads {
feed.replace(links, &format!("{}{PREFIX}", server.uri()))
}
/// The infohash the mock indexer's magnet for `name` carries.
///
/// Deterministic and distinct per release, so duplicates still collapse
/// and a test can blacklist a hash before the tick that would grab it.
pub(crate) fn infohash_for(name: &str) -> String {
let mut hash: u64 = 5381;
for byte in name.as_bytes() {
hash = hash.wrapping_mul(33) ^ u64::from(*byte);
}
format!("{hash:040x}")
}
/// Prowlarr's live behaviour: the download endpoint 302s to a magnet.
pub(crate) async fn mount(server: &MockServer) {
Mock::given(method("GET"))
@@ -1592,16 +1618,9 @@ pub(crate) mod test_downloads {
.next()
.unwrap_or_default()
.to_owned();
// Deterministic stand-in for the infohash the tracker would give,
// and distinct per release so duplicates still collapse.
let mut hash: u64 = 5381;
for byte in name.as_bytes() {
hash = hash.wrapping_mul(33) ^ u64::from(*byte);
}
ResponseTemplate::new(302).insert_header(
"location",
format!("magnet:?xt=urn:btih:{hash:040x}&dn={name}"),
)
let hash = super::test_downloads::infohash_for(&name);
ResponseTemplate::new(302)
.insert_header("location", format!("magnet:?xt=urn:btih:{hash}&dn={name}"))
}
}
}
@@ -1609,127 +1628,16 @@ pub(crate) mod test_downloads {
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use base64::Engine as _;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use serde_json::{json, Value};
use serde_json::json;
use wiremock::matchers::{method, path, query_param};
use wiremock::{Mock, MockServer, Request, Respond, ResponseTemplate};
use wiremock::{Mock, MockServer, ResponseTemplate};
use crate::qbit_fake::FakeQbit;
use super::*;
/// A Transmission that dedupes on the infohash, like the real one: the
/// same source added twice is one torrent and a `torrent-duplicate`
/// response. That is the property the restart case leans on.
#[derive(Clone, Debug, Default)]
struct FakeTransmission {
torrents: Arc<Mutex<Vec<FakeTorrent>>>,
}
#[derive(Clone, Debug)]
struct FakeTorrent {
id: i64,
hash: String,
source: String,
labels: Vec<String>,
progress: f64,
}
impl FakeTransmission {
fn torrents(&self) -> Vec<FakeTorrent> {
self.torrents.lock().unwrap().clone()
}
fn complete_all(&self) {
for torrent in self.torrents.lock().unwrap().iter_mut() {
torrent.progress = 1.0;
}
}
fn add(&self, arguments: &Value) -> ResponseTemplate {
// A magnet arrives as `filename`, a torrent body as base64
// `metainfo` — either identifies the torrent for the fake.
let source = arguments["filename"]
.as_str()
.or_else(|| arguments["metainfo"].as_str())
.unwrap_or_default()
.to_owned();
let labels: Vec<String> = arguments["labels"]
.as_array()
.map(|values| {
values
.iter()
.filter_map(|value| value.as_str().map(ToOwned::to_owned))
.collect()
})
.unwrap_or_default();
let mut torrents = self.torrents.lock().unwrap();
if let Some(existing) = torrents.iter().find(|torrent| torrent.source == source) {
return success(&json!({"torrent-duplicate": {
"id": existing.id, "name": existing.source, "hashString": existing.hash
}}));
}
let id = i64::try_from(torrents.len()).unwrap() + 1;
// Deterministic stand-in for the real infohash, which likewise
// comes out the same for the same torrent.
let hash = format!("{:040x}", id * 7);
torrents.push(FakeTorrent {
id,
hash: hash.clone(),
source: source.clone(),
labels,
progress: 0.0,
});
success(&json!({"torrent-added": {"id": id, "name": source, "hashString": hash}}))
}
}
impl Respond for FakeTransmission {
fn respond(&self, request: &Request) -> ResponseTemplate {
let body: Value = serde_json::from_slice(&request.body).unwrap();
let arguments = &body["arguments"];
match body["method"].as_str().unwrap_or_default() {
"torrent-add" => self.add(arguments),
"torrent-get" => {
let torrents: Vec<Value> = self
.torrents()
.into_iter()
.map(|torrent| {
json!({
"id": torrent.id, "name": torrent.source,
"hashString": torrent.hash, "status": 4,
"percentDone": torrent.progress,
"downloadDir": "/downloads", "labels": torrent.labels
})
})
.collect();
success(&json!({"torrents": torrents}))
}
"torrent-remove" => {
let removed: Vec<i64> = arguments["ids"]
.as_array()
.map(|ids| ids.iter().filter_map(serde_json::Value::as_i64).collect())
.unwrap_or_default();
self.torrents
.lock()
.unwrap()
.retain(|torrent| !removed.contains(&torrent.id));
success(&json!({}))
}
_ => success(&json!({})),
}
}
}
fn success(arguments: &Value) -> ResponseTemplate {
ResponseTemplate::new(200).set_body_json(json!({
"result": "success", "arguments": arguments
}))
}
const RSS: &str = r#"<rss><channel>
<item>
<title>Dune.Part.Two.2024.2160p.WEB-DL.DDP5.1.Atmos</title>
@@ -1846,14 +1754,8 @@ mod tests {
server
}
async fn transmission() -> (MockServer, FakeTransmission) {
let server = MockServer::start().await;
let fake = FakeTransmission::default();
Mock::given(method("POST"))
.respond_with(fake.clone())
.mount(&server)
.await;
(server, fake)
async fn qbit() -> (MockServer, FakeQbit) {
FakeQbit::start().await
}
async fn wanted_movie() -> (tempfile::TempDir, Db) {
@@ -1871,11 +1773,11 @@ mod tests {
(dir, database)
}
fn action(prowlarr: &MockServer, transmission: &MockServer) -> GrabAction {
fn action(prowlarr: &MockServer, qbit: &MockServer) -> GrabAction {
GrabAction::new(
ProwlarrClient::new(prowlarr.uri(), "key").unwrap(),
TransmissionClient::new(&transmission.uri()).unwrap(),
PathBuf::from("/mnt/media/transmission/complete"),
QbitClient::new(&qbit.uri()).unwrap(),
PathBuf::from("/mnt/media/qbittorrent/complete"),
SeedingRules::new(
SeedingLimits {
ratio: 1.5,
@@ -1888,10 +1790,10 @@ mod tests {
fn action_with_tmdb(
prowlarr: &MockServer,
transmission: &MockServer,
qbit: &MockServer,
metadata: &MockServer,
) -> GrabAction {
action(prowlarr, transmission).with_tmdb(Arc::new(
action(prowlarr, qbit).with_tmdb(Arc::new(
TmdbClient::builder("key")
.base_url(format!("{}/3/", metadata.uri()))
.build()
@@ -1945,7 +1847,7 @@ mod tests {
async fn a_wanted_movie_ends_with_one_torrent_and_one_grab() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1971,10 +1873,10 @@ mod tests {
async fn a_restart_mid_flight_does_not_grab_twice() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
// The crash: Transmission has the torrent, the database does not know.
// The crash: qBittorrent has the torrent, the database does not know.
sqlx::query("DELETE FROM grabs")
.execute(database.pool())
.await
@@ -1996,7 +1898,7 @@ mod tests {
async fn a_second_tick_grabs_nothing_new() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
@@ -2013,26 +1915,17 @@ mod tests {
async fn the_label_and_both_seed_limits_are_set_at_add_time() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
assert_eq!(fake.torrents()[0].labels, vec!["movies-main".to_owned()]);
let add = downloader
.received_requests()
.await
.unwrap()
.into_iter()
.filter_map(|request| serde_json::from_slice::<Value>(&request.body).ok())
.find(|body| body["method"] == "torrent-add")
.expect("torrent-add");
assert_eq!(add["arguments"]["seedRatioLimit"], json!(1.5));
assert_eq!(add["arguments"]["seedIdleLimit"], json!(60));
assert_eq!(add["arguments"]["seedRatioMode"], json!(1));
assert_eq!(add["arguments"]["seedIdleMode"], json!(1));
let added = &fake.torrents()[0];
assert_eq!(added.labels, vec!["movies-main".to_owned()]);
assert!((added.ratio_limit - 1.5).abs() < f64::EPSILON);
assert_eq!(added.idle_limit_minutes, 60);
assert_eq!(
add["arguments"]["download-dir"],
json!("/mnt/media/transmission/complete")
added.save_path,
String::from("/mnt/media/qbittorrent/complete")
);
}
@@ -2058,7 +1951,7 @@ mod tests {
async fn every_candidate_is_recorded_with_its_verdict() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -2081,13 +1974,13 @@ mod tests {
assert_eq!(attempts, 0);
}
/// Transmission is authoritative (§8): a completed torrent moves its grab
/// qBittorrent is authoritative (§8): a completed torrent moves its grab
/// out of `sent` without the process having watched it happen.
#[tokio::test]
async fn a_completed_torrent_moves_its_grab_to_downloaded() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
@@ -2099,14 +1992,14 @@ mod tests {
}
/// #108, overriding §86: a torrent removed by hand — gone from
/// Transmission before it finished — parks the movie instead of
/// qBittorrent before it finished — parks the movie instead of
/// reopening the gap, and is marked `vanished` rather than `failed` so
/// it never counts toward the `needs_decision` queue (attention.rs).
#[tokio::test]
async fn a_torrent_removed_by_hand_parks_the_movie() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
fake.torrents.lock().unwrap().clear();
@@ -2131,7 +2024,7 @@ mod tests {
async fn a_parked_movie_is_not_regrabbed() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
@@ -2154,7 +2047,7 @@ mod tests {
async fn a_rewanted_movie_can_regrab_the_same_infohash() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
@@ -2179,6 +2072,83 @@ mod tests {
assert_eq!(state, "downloading");
}
/// A grab row left behind by a deleted title holds the release's unique
/// infohash. Re-adding the title and grabbing the same release used to
/// record nothing at all: the conflict hit a row in `sent`, which is only
/// reclaimed when it is `vanished`, so the movie flipped to `downloading`
/// while the one row still pointed at the title that no longer existed —
/// and nothing ever imported for the new one.
#[tokio::test]
async fn an_orphaned_grab_is_reclaimed_by_the_title_that_regrabs_it() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
assert_eq!(grabs(&database).await.len(), 1);
// The title is deleted the way an old build left it: row gone, grab
// behind, still reading `sent`.
sqlx::query("UPDATE grabs SET target_id = 99, state = 'sent'")
.execute(database.pool())
.await
.unwrap();
sqlx::query(
"UPDATE movies SET state = 'missing', wanted = 1, search_attempts = 0,
last_searched_at = NULL WHERE id = 1",
)
.execute(database.pool())
.await
.unwrap();
let outcomes = action.tick(&database).await.unwrap();
assert_eq!(outcomes.len(), 1, "the regrab records a grab");
assert_eq!(fake.torrents().len(), 1, "same release, same torrent");
let grabs = grabs(&database).await;
assert_eq!(grabs.len(), 1, "the orphan is reclaimed, not duplicated");
assert_eq!(grabs[0].0, 1, "and it now belongs to the live movie");
}
/// The other half: a live title's row is still off limits, so a restart
/// mid-flight does not let one title steal another's torrent.
#[tokio::test]
async fn a_grab_belonging_to_a_live_title_is_not_stolen() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, _fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
// A second live movie, and the grab reassigned to it.
sqlx::query(
"INSERT INTO movies (id, tmdb_id, title, year, original_language, root_id, state)
SELECT 2, 999, 'Other', 2020, 'en', id, 'downloading'
FROM roots WHERE kind = 'movie' AND audience = 'main'",
)
.execute(database.pool())
.await
.unwrap();
sqlx::query("UPDATE grabs SET target_id = 2, state = 'sent'")
.execute(database.pool())
.await
.unwrap();
sqlx::query(
"UPDATE movies SET state = 'missing', wanted = 1, search_attempts = 0,
last_searched_at = NULL WHERE id = 1",
)
.execute(database.pool())
.await
.unwrap();
action.tick(&database).await.unwrap();
let grabs = grabs(&database).await;
assert_eq!(grabs.len(), 1);
assert_eq!(grabs[0].0, 2, "movie 2 keeps the torrent it is downloading");
}
/// §5.2: the language rules are expressed against the title's original
/// language, and guessing it is worse than waiting for it.
#[tokio::test]
@@ -2189,7 +2159,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -2212,7 +2182,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -2240,7 +2210,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -2253,23 +2223,24 @@ mod tests {
assert_eq!(rule.as_deref(), Some("blacklisted"));
}
/// §6.3's second key. A `.torrent` link hides its infohash until
/// Transmission has fetched it, so the blacklisted torrent is only
/// recognised after the add — and must not leave a grab behind.
/// §6.3's second key. The infohash is only known once the indexer link
/// has been resolved, which happens at grab time, so a blacklisted
/// torrent under a new release name is recognised after the add — and
/// must not leave a grab behind.
#[tokio::test]
async fn a_blacklisted_infohash_never_becomes_a_grab() {
let (_dir, database) = wanted_movie().await;
// What the fake hands back for the first torrent it accepts.
// The infohash the winning release's magnet carries.
blacklist::add(
database.pool(),
Some(&format!("{:040x}", 7)),
Some(&test_downloads::infohash_for("good.torrent")),
"Some.Older.Name.Of.The.Same.Torrent",
"required_audio",
)
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -2279,7 +2250,7 @@ mod tests {
"a torrent added this tick and then found blacklisted is removed"
);
// Recorded under its new name, so the next tick stops before paying
// Transmission again.
// qBittorrent again.
let blacklist = Blacklist::load(database.pool()).await.unwrap();
assert!(blacklist.blocks_name("Dune.Part.Two.2024.2160p.WEB-DL.DDP5.1.Atmos"));
// And the cached row stops reading eligible straight away, so §9.3's
@@ -2329,7 +2300,7 @@ mod tests {
async fn a_manual_search_on_an_available_movie_refreshes_the_deck() {
let (_dir, database) = available_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcome = action(&indexer, &downloader)
.search_now(&database, 1)
@@ -2351,7 +2322,7 @@ mod tests {
let (_dir, database) = available_movie().await;
let indexer = prowlarr().await;
let metadata = tmdb(UNRELEASED_METADATA).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action_with_tmdb(&indexer, &downloader, &metadata)
.search_now(&database, 1)
@@ -2367,7 +2338,7 @@ mod tests {
async fn a_manual_search_with_a_grab_in_flight_does_not_grab_again() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
assert_eq!(grabs(&database).await.len(), 1);
@@ -2384,7 +2355,7 @@ mod tests {
async fn a_manual_search_on_a_wanted_movie_still_grabs() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcome = action(&indexer, &downloader)
.search_now(&database, 1)
@@ -2405,7 +2376,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcome = action(&indexer, &downloader)
.search_now(&database, 1)
@@ -2427,7 +2398,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -2441,7 +2412,7 @@ mod tests {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let metadata = tmdb(UNRELEASED_METADATA).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
let action = action_with_tmdb(&indexer, &downloader, &metadata);
for _ in 0..3 {
@@ -2461,7 +2432,7 @@ mod tests {
let (_dir, database) = wanted_movie().await;
let indexer = empty_prowlarr().await;
let metadata = tmdb(RELEASED_METADATA).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
let action = action_with_tmdb(&indexer, &downloader, &metadata);
action.tick(&database).await.unwrap();
@@ -2513,7 +2484,7 @@ mod tests {
let (_dir, database) = wanted_movie().await;
let indexer = empty_prowlarr().await;
let metadata = tmdb(RELEASED_METADATA).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action_with_tmdb(&indexer, &downloader, &metadata)
.tick(&database)
@@ -2568,7 +2539,7 @@ mod tests {
"vote_average": 8.152,"#,
);
let metadata = tmdb(&artwork_metadata).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action_with_tmdb(&indexer, &downloader, &metadata)
.tick(&database)
@@ -2629,7 +2600,7 @@ mod tests {
"vote_average": 8.152,"#,
)
};
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action_with_tmdb(&indexer, &downloader, &tmdb(&unrated()).await)
.tick(&database)
@@ -2690,7 +2661,7 @@ mod tests {
.unwrap();
let indexer = empty_prowlarr().await;
let metadata = tmdb(&RELEASED_METADATA.replace("Dune Part Two", "Dune: Part Two")).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action_with_tmdb(&indexer, &downloader, &metadata)
.tick(&database)
@@ -2713,7 +2684,7 @@ mod tests {
async fn indexer_discovery_is_cached_across_ticks() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
@@ -2744,7 +2715,7 @@ mod tests {
)
.mount(&indexer)
.await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let mut action = action(&indexer, &downloader);
action.indexers.discovery_timeout = Duration::from_millis(50);
@@ -2761,7 +2732,7 @@ mod tests {
assert!(fake.torrents().is_empty());
}
/// §100: Transmission has no route to the indexer, so a download link
/// §100: qBittorrent has no route to the indexer, so a download link
/// that answers with the torrent itself is forwarded as inline metainfo
/// and the link never leaves arr.
#[tokio::test]
@@ -2779,13 +2750,14 @@ mod tests {
)
.mount(&indexer)
.await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
let sent = base64::engine::general_purpose::STANDARD
.decode(&fake.torrents()[0].source)
.expect("the torrent is sent as base64 metainfo");
let sent = fake.torrents()[0]
.metainfo
.clone()
.expect("the torrent file is sent inline");
assert_eq!(sent, TORRENT);
let fetched_with_key = indexer
.received_requests()
@@ -2798,12 +2770,12 @@ mod tests {
}
/// A link that redirects to a magnet — Prowlarr's usual answer — reaches
/// Transmission as the magnet, which it can act on without the indexer.
/// qBittorrent as the magnet, which it can act on without the indexer.
#[tokio::test]
async fn a_link_that_redirects_to_a_magnet_is_sent_as_the_magnet() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
+244 -79
View File
@@ -5,7 +5,7 @@
//! The torrent's own files are never moved, renamed or deleted — the torrent
//! and the library entry are separate lifecycles (§7.3). A hard-failed
//! release is blacklisted and its grab marked failed, but the torrent keeps
//! seeding until Transmission's own limits clear it.
//! seeding until qBittorrent's own limits clear it.
//!
//! Everything here is idempotent from domain rows (§8): a grab in
//! `downloaded` with no import recorded is the gap, and re-running any prefix
@@ -22,8 +22,8 @@ use arr_core::layout;
use arr_core::policy::{evaluate, Candidate};
use arr_core::{ProbedMedia, Rule, Source, Verdict};
use arr_db::Db;
use arr_dl::TransmissionClient;
use arr_probe::Prober;
use arr_dl::QbitClient;
use arr_probe::{Prober, Skeletons};
use crate::notify::Notifier;
use crate::reconcile::{Action, ActionFuture, Outcome};
@@ -36,8 +36,8 @@ pub enum ImportError {
Database(#[from] sqlx::Error),
#[error("policy: {0}")]
Policy(#[from] arr_db::PolicyError),
#[error("transmission: {0}")]
Transmission(#[from] arr_dl::Error),
#[error("qbit: {0}")]
Qbit(#[from] arr_dl::Error),
#[error("probe: {0}")]
Probe(#[from] arr_probe::Error),
#[error("blocking task: {0}")]
@@ -73,8 +73,13 @@ enum ProbeOutcome {
/// the §7.4 layout.
#[derive(Debug)]
pub struct ImportAction {
transmission: TransmissionClient,
qbit: QbitClient,
prober: Prober,
/// §15, #268. Reads the packet timings of image-format subtitle tracks
/// into a cue skeleton. Here rather than in the subtitle lane because
/// deriving one costs a full demux, and import is where the file is
/// being read and hardlinked anyway.
skeletons: Skeletons,
jellyfin: JellyfinClient,
notifier: Notifier,
/// The operator's ntfy topic (DESIGN.md §9.5), for the *broken*
@@ -96,7 +101,7 @@ pub struct ImportAction {
probed: std::sync::Arc<tokio::sync::Mutex<HashMap<PathBuf, ProbeOutcome>>>,
}
/// A movie grab Transmission finished downloading, not yet imported.
/// A movie grab qBittorrent finished downloading, not yet imported.
#[derive(Debug, Clone)]
struct PendingImport {
grab_id: i64,
@@ -112,15 +117,16 @@ struct PendingImport {
impl ImportAction {
#[must_use]
pub fn new(
transmission: TransmissionClient,
qbit: QbitClient,
prober: Prober,
jellyfin: JellyfinClient,
notifier: Notifier,
operator_topic: Option<String>,
) -> Self {
Self {
transmission,
qbit,
prober,
skeletons: Skeletons::new(),
jellyfin,
notifier,
operator_topic,
@@ -176,6 +182,94 @@ impl ImportAction {
Ok(files)
}
/// Derive the cue skeleton of every non-forced image-format subtitle
/// track in a file that has just been placed (§15, #268).
///
/// Detached, like the probes above: one derivation is a full demux and
/// the reconcile lane's tick budget is 25 s. Nothing downstream waits on
/// it — a skeleton that never lands, because the process died or because
/// the packets did not pair, reads exactly like a track that has none,
/// and the subtitle lane aligns against the video instead.
async fn spawn_skeletons(&self, database: &Db, placed: &Path, feature: &arr_probe::ProbedFile) {
let tracks: Vec<(usize, String)> = feature
.media
.subtitle_tracks
.iter()
.enumerate()
.filter(|(_, track)| !track.forced && track.codec.is_image())
.map(|(index, track)| (index, track.language.to_string()))
.collect();
if tracks.is_empty() {
return;
}
let path_text = placed.to_string_lossy().into_owned();
let media_file_id = match sqlx::query_scalar!(
r#"SELECT id AS "id!: i64" FROM media_files WHERE path = ?"#,
path_text
)
.fetch_optional(database.pool())
.await
{
Ok(Some(id)) => id,
Ok(None) => return,
Err(error) => {
tracing::warn!(path = %placed.display(), %error, "no media file row to hang a cue skeleton on");
return;
}
};
let skeletons = self.skeletons.clone();
let database = database.clone();
let video = placed.to_path_buf();
let runtime = feature.duration;
tokio::spawn(async move {
for (stream_index, language) in tracks {
let outcome = skeletons.derive(&video, stream_index, Some(runtime)).await;
let index = i64::try_from(stream_index).unwrap_or(i64::MAX);
match outcome {
Ok(arr_probe::SkeletonOutcome::Derived(skeleton)) => {
let cues: Vec<(i64, i64)> = skeleton
.cues
.iter()
.map(|cue| (millis(cue.start), millis(cue.end)))
.collect();
if let Err(error) = arr_db::skeletons::record(
database.pool(),
media_file_id,
index,
&language,
&cues,
)
.await
{
tracing::warn!(%error, media_file_id, stream_index, "cue skeleton not recorded");
} else {
tracing::info!(
media_file_id,
stream_index,
cues = cues.len(),
"cue skeleton derived"
);
}
}
Ok(arr_probe::SkeletonOutcome::Implausible(reason)) => tracing::info!(
media_file_id,
stream_index,
%reason,
"cue skeleton refused, alignment falls back to the video"
),
Err(error) => tracing::warn!(
media_file_id,
stream_index,
%error,
"cue skeleton could not be read"
),
}
}
});
}
/// Drop a settled grab's probe results — imported or blacklisted, they
/// will not be needed again.
async fn forget_probes(&self, paths: &[PathBuf]) {
@@ -345,6 +439,7 @@ impl ImportAction {
tokio::task::spawn_blocking(move || place(&source_path, &link_target)).await??;
record_import(database, pending, &feature, waiver.as_ref(), &destination).await?;
self.spawn_skeletons(database, &destination, &feature).await;
self.forget_probes(&paths).await;
self.refresh_jellyfin().await;
let path_text = destination.to_string_lossy().into_owned();
@@ -388,7 +483,7 @@ impl ImportAction {
}
}
/// The torrent's files as safe local paths, or `None` when Transmission
/// The torrent's files as safe local paths, or `None` when qBittorrent
/// no longer has the torrent.
///
/// Torrent-declared names are untrusted input: an absolute or
@@ -399,13 +494,13 @@ impl ImportAction {
grab_id: i64,
infohash: &str,
) -> Result<Option<Vec<PathBuf>>, ImportError> {
let Some(content) = self.transmission.torrent_content(infohash).await? else {
// Gone from Transmission — the caller marks the grab vanished
let Some(content) = self.qbit.torrent_content(infohash).await? else {
// Gone from qBittorrent — the caller marks the grab vanished
// and parks the target (#108).
tracing::warn!(
grab_id,
infohash,
"downloaded grab has no torrent in Transmission; not importing"
"downloaded grab has no torrent in qBittorrent; not importing"
);
return Ok(None);
};
@@ -591,6 +686,7 @@ impl ImportAction {
record_episode_import(database, episode.id, feature, waiver.as_ref(), &destination)
.await?;
self.spawn_skeletons(database, &destination, feature).await;
imported += 1;
tracing::info!(
grab_id = pending.grab_id,
@@ -671,7 +767,7 @@ impl ImportAction {
))
}
/// §86/#108: a `downloaded` grab whose torrent Transmission no longer
/// §86/#108: a `downloaded` grab whose torrent qBittorrent no longer
/// reports — removed by hand, not a policy failure. Marked `vanished`
/// rather than `failed` so it does not feed the `needs_decision` queue
/// (attention.rs). Nothing is blacklisted, since the release itself
@@ -692,11 +788,11 @@ impl ImportAction {
movie_id = pending.movie_id,
title = pending.title,
release = pending.release_name,
"torrent vanished from Transmission; movie parked"
"torrent vanished from qBittorrent; movie parked"
);
Ok(Outcome::new(
format!(
"grab {} downloaded, torrent vanished from Transmission",
"grab {} downloaded, torrent vanished from qBittorrent",
pending.grab_id
),
format!("parked movie {}", pending.movie_id),
@@ -726,11 +822,11 @@ impl ImportAction {
grab_id = pending.grab_id,
series = pending.series_title,
release = pending.release_name,
"torrent vanished from Transmission; target parked"
"torrent vanished from qBittorrent; target parked"
);
Ok(Outcome::new(
format!(
"grab {} downloaded, torrent vanished from Transmission",
"grab {} downloaded, torrent vanished from qBittorrent",
pending.grab_id
),
format!("parked {target_kind} {target_id}"),
@@ -897,7 +993,7 @@ async fn record_import(
Ok(())
}
/// The gap, straight out of the domain rows (§8): a movie grab Transmission
/// The gap, straight out of the domain rows (§8): a movie grab qBittorrent
/// finished that no import has settled.
async fn pending_imports(database: &Db) -> Result<Vec<PendingImport>, ImportError> {
let rows = sqlx::query!(
@@ -935,7 +1031,7 @@ async fn pending_imports(database: &Db) -> Result<Vec<PendingImport>, ImportErro
.collect())
}
/// A TV grab Transmission finished downloading — one episode or a season
/// A TV grab qBittorrent finished downloading — one episode or a season
/// pack — not yet imported.
#[derive(Debug, Clone)]
struct PendingTvImport {
@@ -1200,6 +1296,11 @@ async fn record_episode_import(
Ok(())
}
/// Milliseconds, saturating. A cue past 292 million years does not exist.
fn millis(value: std::time::Duration) -> i64 {
i64::try_from(value.as_millis()).unwrap_or(i64::MAX)
}
/// The `probed` column (§4, §5.6): what `ffprobe` found, in the spellings the
/// policy columns use.
fn probed_json(media: &ProbedMedia) -> serde_json::Value {
@@ -1219,7 +1320,14 @@ fn probed_json(media: &ProbedMedia) -> serde_json::Value {
"sub_tracks": media
.subtitle_tracks
.iter()
.map(|track| serde_json::json!({ "language": track.language.to_string() }))
.map(|track| {
serde_json::json!({
"language": track.language.to_string(),
"codec": track.codec.to_string(),
"forced": track.forced,
"sdh": track.sdh,
})
})
.collect::<Vec<_>>(),
})
}
@@ -1390,18 +1498,39 @@ mod tests {
path
}
async fn harness(media_json: &str) -> Harness {
harness_with(
media_json,
json!([
{"name": "Dune/Dune.mkv", "length": 13, "bytesCompleted": 13},
{"name": "Dune/Dune.nfo", "length": 10, "bytesCompleted": 10}
]),
)
.await
/// A qBittorrent holding exactly one torrent, `INFOHASH`, with `files`
/// under `save_path`.
async fn qbit(save_path: &str, files: &[(&str, u64)]) -> MockServer {
let server = MockServer::start().await;
let torrent = json!({
"hash": INFOHASH, "name": RELEASE_NAME, "state": "uploading",
"progress": 1.0, "save_path": save_path, "tags": "movies-main",
"ratio": 0.1, "ratio_limit": 1.5, "seeding_time": 1,
"seeding_time_limit": -1, "inactive_seeding_time_limit": -1,
"last_activity": 0
});
Mock::given(method("GET"))
.and(path("/api/v2/torrents/info"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!([torrent])))
.mount(&server)
.await;
let files: Vec<serde_json::Value> = files
.iter()
.map(|(name, size)| json!({"name": name, "size": size}))
.collect();
Mock::given(method("GET"))
.and(path("/api/v2/torrents/files"))
.respond_with(ResponseTemplate::new(200).set_body_json(files))
.mount(&server)
.await;
server
}
async fn harness_with(media_json: &str, files: serde_json::Value) -> Harness {
async fn harness(media_json: &str) -> Harness {
harness_with(media_json, &[("Dune/Dune.mkv", 13), ("Dune/Dune.nfo", 10)]).await
}
async fn harness_with(media_json: &str, files: &[(&str, u64)]) -> Harness {
let dir = tempfile::tempdir().unwrap();
let downloads = dir.path().join("downloads");
let library = dir.path().join("library");
@@ -1446,18 +1575,7 @@ mod tests {
.unwrap();
insert_owner(&database, "movie", 1, "Alice", "alice-topic").await;
let server = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"result": "success",
"arguments": {"torrents": [{
"hashString": INFOHASH,
"downloadDir": downloads.to_string_lossy(),
"files": files
}]}
})))
.mount(&server)
.await;
let server = qbit(&downloads.to_string_lossy(), files).await;
let jellyfin_server = start_jellyfin_server().await;
let ntfy_server = start_ntfy_server().await;
@@ -1466,7 +1584,7 @@ mod tests {
let jellyfin = JellyfinClient::new(jellyfin_server.uri(), None).unwrap();
let notifier = Notifier::new(ntfy_server.uri()).unwrap();
let action = ImportAction::new(
TransmissionClient::new(&server.uri()).unwrap(),
QbitClient::new(&server.uri()).unwrap(),
prober,
jellyfin,
notifier,
@@ -1720,7 +1838,7 @@ mod tests {
assert!(outcomes.is_empty());
}
/// #108, overriding §86: a `downloaded` grab whose torrent Transmission
/// #108, overriding §86: a `downloaded` grab whose torrent qBittorrent
/// no longer reports — removed by hand, not a policy failure — is marked
/// `vanished` and parks the movie (`wanted` cleared) instead of
/// reopening it as a gap, without touching the blacklist.
@@ -1757,17 +1875,15 @@ mod tests {
.unwrap();
let server = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"result": "success",
"arguments": {"torrents": []}
})))
Mock::given(method("GET"))
.and(path("/api/v2/torrents/info"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!([])))
.mount(&server)
.await;
let jellyfin_server = start_jellyfin_server().await;
let ntfy_server = start_ntfy_server().await;
let action = ImportAction::new(
TransmissionClient::new(&server.uri()).unwrap(),
QbitClient::new(&server.uri()).unwrap(),
Prober::new().with_binary(fake_ffprobe(dir.path(), HDR10_PROBE)),
JellyfinClient::new(jellyfin_server.uri(), None).unwrap(),
Notifier::new(ntfy_server.uri()).unwrap(),
@@ -1802,11 +1918,11 @@ mod tests {
async fn hostile_torrent_paths_never_leave_the_download_root() {
let h = harness_with(
HDR10_PROBE,
json!([
{"name": "../outside.mkv", "length": 13, "bytesCompleted": 13},
{"name": "/tmp/absolute.mkv", "length": 13, "bytesCompleted": 13},
{"name": "Dune/Dune.mkv", "length": 13, "bytesCompleted": 13}
]),
&[
("../outside.mkv", 13),
("/tmp/absolute.mkv", 13),
("Dune/Dune.mkv", 13),
],
)
.await;
@@ -1825,11 +1941,7 @@ mod tests {
/// hard fail, not an escape.
#[tokio::test]
async fn a_torrent_of_only_hostile_paths_hard_fails() {
let h = harness_with(
HDR10_PROBE,
json!([{"name": "../../etc/passwd", "length": 13, "bytesCompleted": 13}]),
)
.await;
let h = harness_with(HDR10_PROBE, &[("../../etc/passwd", 13)]).await;
let outcomes = h.action.tick(&h.database).await.unwrap();
@@ -1878,7 +1990,7 @@ mod tests {
std::fs::set_permissions(&script, permissions).unwrap();
let action = ImportAction::new(
TransmissionClient::new("http://127.0.0.1:1").unwrap(),
QbitClient::new("http://127.0.0.1:1").unwrap(),
Prober::new().with_binary(&script),
JellyfinClient::new("http://127.0.0.1:1", None).unwrap(),
Notifier::new("http://127.0.0.1:1").unwrap(),
@@ -1925,7 +2037,7 @@ mod tests {
permissions.set_mode(0o755);
std::fs::set_permissions(&script, permissions).unwrap();
let action = ImportAction::new(
TransmissionClient::new("http://127.0.0.1:1").unwrap(),
QbitClient::new("http://127.0.0.1:1").unwrap(),
Prober::new().with_binary(&script),
JellyfinClient::new("http://127.0.0.1:1", None).unwrap(),
Notifier::new("http://127.0.0.1:1").unwrap(),
@@ -1973,6 +2085,18 @@ mod tests {
]
}"#;
/// The same file, one gigabyte instead of ten: under the 2160p floor.
/// A pack accepted pre-grab on its per-episode average carries episodes
/// like this whenever one runs short.
const TV_THIN_PROBE: &str = r#"{
"format": {"format_name": "matroska,webm", "duration": "3300.0", "size": "1073741824"},
"streams": [
{"codec_type": "video", "codec_name": "hevc", "width": 3840, "height": 1600,
"color_transfer": "smpte2084"},
{"codec_type": "audio", "codec_name": "eac3", "tags": {"language": "eng"}}
]
}"#;
const PACK_RELEASE_NAME: &str = "Fallout.S01.2160p.WEB-DL.DDP5.1.Atmos";
struct TvHarness {
@@ -2051,21 +2175,14 @@ mod tests {
.unwrap();
insert_owner(&database, "series", 1, "Bob", "bob-topic").await;
let server = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"result": "success",
"arguments": {"torrents": [{
"hashString": INFOHASH,
"downloadDir": downloads.to_string_lossy(),
"files": [
{"name": "Fallout.S01/Fallout.S01E01.mkv", "length": 2, "bytesCompleted": 2},
{"name": "Fallout.S01/Fallout.S01E02.mkv", "length": 2, "bytesCompleted": 2}
]
}]}
})))
.mount(&server)
.await;
let server = qbit(
&downloads.to_string_lossy(),
&[
("Fallout.S01/Fallout.S01E01.mkv", 2),
("Fallout.S01/Fallout.S01E02.mkv", 2),
],
)
.await;
let jellyfin_server = start_jellyfin_server().await;
let ntfy_server = start_ntfy_server().await;
@@ -2073,7 +2190,7 @@ mod tests {
let jellyfin = JellyfinClient::new(jellyfin_server.uri(), None).unwrap();
let notifier = Notifier::new(ntfy_server.uri()).unwrap();
let action = ImportAction::new(
TransmissionClient::new(&server.uri()).unwrap(),
QbitClient::new(&server.uri()).unwrap(),
prober,
jellyfin,
notifier,
@@ -2186,6 +2303,54 @@ mod tests {
assert_eq!(grab_state, "imported");
}
/// §5.5's floor is a selection filter, so it condemns nothing after the
/// download. A pack whose episodes come in under the band imports and
/// records the waiver: blacklisting it would throw away a file already
/// on disk over a number that was on the release before the grab, and
/// §6.3 would then outlast any override the operator later writes.
#[tokio::test]
async fn a_pack_under_the_floor_imports_with_a_waiver_rather_than_blacklisting() {
let h = tv_harness(TV_THIN_PROBE).await;
h.action.tick(&h.database).await.unwrap();
let blacklisted: i64 = sqlx::query_scalar("SELECT count(*) FROM blacklist")
.fetch_one(h.database.pool())
.await
.unwrap();
assert_eq!(blacklisted, 0, "the floor blacklists nothing post-download");
let waivers: Vec<Option<String>> =
sqlx::query_scalar("SELECT waiver FROM media_files ORDER BY path")
.fetch_all(h.database.pool())
.await
.unwrap();
assert_eq!(
waivers,
vec![
Some(r#"{"rule":"size"}"#.to_owned()),
Some(r#"{"rule":"size"}"#.to_owned())
],
"§5.7: import it, record the waiver, surface it"
);
let states: Vec<(String, bool)> =
sqlx::query_as("SELECT state, wanted FROM episodes ORDER BY number")
.fetch_all(h.database.pool())
.await
.unwrap();
assert_eq!(
states,
vec![
("available".to_owned(), true),
("available".to_owned(), true)
]
);
let grab_state: String = sqlx::query_scalar("SELECT state FROM grabs")
.fetch_one(h.database.pool())
.await
.unwrap();
assert_eq!(grab_state, "imported");
}
/// The third acceptance case, import side: a pack whose file hard-fails
/// blacklists that release and reopens the episodes — it does not
/// blacklist or block the season.
+355 -63
View File
@@ -9,15 +9,18 @@ mod indexers;
mod manual;
mod metadata;
mod notify;
#[cfg(test)]
mod qbit_fake;
mod reaper;
pub mod reconcile;
mod rss;
mod series_refresh;
mod subtitles;
mod tv_grab;
mod web;
use std::process::ExitCode;
use std::sync::Arc;
use std::sync::{atomic::AtomicU64, Arc};
use arr_api::{AppState, Upstreams};
use arr_compat::CompatState;
@@ -33,6 +36,7 @@ use reaper::ReaperAction;
use reconcile::{ReconcileLoop, Tick};
use rss::RssAction;
use series_refresh::SeriesRefreshAction;
use subtitles::SubtitleAction;
use tower_http::trace::TraceLayer;
use tv_grab::TvGrabAction;
@@ -90,8 +94,8 @@ enum Error {
Tmdb(#[from] arr_meta::Error),
#[error("prowlarr client: {0}")]
Prowlarr(#[from] arr_indexer::Error),
#[error("transmission client: {0}")]
Transmission(#[from] arr_dl::Error),
#[error("qbit client: {0}")]
Qbit(#[from] arr_dl::Error),
#[error("jellyfin client: {0}")]
Jellyfin(#[from] arr_api::jellyfin::Error),
#[error("ntfy client: {0}")]
@@ -107,12 +111,62 @@ enum Error {
BackgroundTask(#[from] tokio::task::JoinError),
}
/// Everything the HTTP layer needs, assembled from config.
///
/// Split out of [`run`] because it grows a line per upstream the API learns
/// to talk to, and `run` is already at the too-many-lines limit.
fn api_state(
config: &Config,
database: &Db,
jellyfin: arr_api::jellyfin::JellyfinClient,
translators: &Translators,
qbit: &arr_dl::QbitClient,
) -> Result<AppState, Error> {
let mut upstreams = Upstreams::new(config.prowlarr_url.clone(), config.qbittorrent_url.clone())
.with_prowlarr_api_key(config.prowlarr_api_key.clone())
.with_tmdb_api_key(config.tmdb_api_key.clone());
if let Some(tmdb_url) = config.tmdb_url.clone() {
upstreams = upstreams.with_tmdb_url(tmdb_url);
}
let mut state = AppState::new(upstreams)?
.with_database(database.clone())
.with_qbit(qbit.clone())
.with_subtitle_providers(subtitle_providers(
config.opensubtitles_api_key.clone(),
config.opensubtitles_username.clone(),
config.opensubtitles_password.clone(),
))
.with_translation_backends(translators.backends.clone())
.with_jellyfin(jellyfin)
.with_syncer(arr_subs::Syncer::new().with_binary(config.alass_path.clone()))
// The health lamp and the extract lane (#260) run the same binary,
// so the API is told which one this deployment has rather than
// resolving `ffmpeg` from `PATH` behind the operator's back.
.with_ffmpeg_binary(config.ffmpeg_path.clone());
if let Some(timeout) = &translators.command_timeout {
state = state.with_command_timeout(Arc::clone(timeout));
}
if let Some(endpoint) = &translators.openai_endpoint {
state = state.with_openai_endpoint(endpoint.clone());
}
Ok(state)
}
#[allow(clippy::too_many_lines)]
async fn run() -> Result<(), Error> {
let config = Config::load()?;
let database = Db::connect(&config.database_path).await?;
database.migrate().await?;
let transmission = arr_dl::TransmissionClient::new(&config.transmission_url)?;
let qbit = match (&config.qbittorrent_username, &config.qbittorrent_password) {
(Some(username), Some(password)) => arr_dl::QbitClient::with_credentials(
&config.qbittorrent_url,
username.clone(),
password.clone(),
),
_ => arr_dl::QbitClient::new(&config.qbittorrent_url),
}?;
let tmdb = if let Some(key) = &config.tmdb_api_key {
let mut client = TmdbClient::builder(key.clone());
if let Some(url) = &config.tmdb_url {
@@ -124,11 +178,19 @@ async fn run() -> Result<(), Error> {
};
let notifier = Notifier::new(config.ntfy_url.clone())?;
let api_jellyfin = jellyfin_client(&config)?;
let (reconcile, manual_grab, manual_tv) =
reconcile_loop(&database, &config, &transmission, tmdb.as_ref(), &notifier)?;
// Issue #176: the on-demand half of the metadata lane needs its own
// handle — the sweep's `SeriesRefreshAction` is owned by `ReconcileLoop`,
// and the compat shim takes the other clone below.
// Built once and shared: the API's translate handler and the reconcile
// lane must see the same backends, or the command translator's live
// timeout cell (#219) would fork.
let translators = translation_backends(&config);
seed_translator_settings(&database, &translators).await?;
let (reconcile, manual_grab, manual_tv) = reconcile_loop(
&database,
&config,
&qbit,
tmdb.as_ref(),
&notifier,
&translators,
)?;
let metadata_tmdb = tmdb.clone();
// Jellyseerr's Radarr shim (DESIGN.md §9.4) reads the same database and
@@ -138,16 +200,7 @@ async fn run() -> Result<(), Error> {
compat = compat.with_tmdb(tmdb);
}
let mut upstreams = Upstreams::new(config.prowlarr_url, config.transmission_url)
.with_prowlarr_api_key(config.prowlarr_api_key)
.with_tmdb_api_key(config.tmdb_api_key);
if let Some(tmdb_url) = config.tmdb_url {
upstreams = upstreams.with_tmdb_url(tmdb_url);
}
let state = AppState::new(upstreams)?
.with_database(database.clone())
.with_jellyfin(api_jellyfin);
let state = api_state(&config, &database, api_jellyfin, &translators, &qbit)?;
let app = arr_api::router(state.clone())
.merge(arr_compat::router(compat))
.fallback(web::serve)
@@ -224,6 +277,48 @@ async fn run() -> Result<(), Error> {
}
}
/// Seed the translators' live settings from the row, so a restart does not
/// fall back to compiled-in defaults until the next settings edit (issues
/// #219 and #220): the command translator's timeout, and where the
/// OpenAI-compatible backend points plus which model it names.
///
/// The row is guaranteed to exist — migration `0025` seeds it — and the
/// timeout is guaranteed positive by its column CHECK. The two `OpenAI`
/// columns are nullable, and NULL means the backend's own default.
async fn seed_translator_settings(database: &Db, translators: &Translators) -> Result<(), Error> {
if translators.command_timeout.is_none() && translators.openai_endpoint.is_none() {
return Ok(());
}
let row = sqlx::query!(
r#"SELECT remote_command_timeout_seconds AS "remote_command_timeout_seconds!: i64",
openai_base_url AS "openai_base_url: String",
openai_model AS "openai_model: String"
FROM subtitle_settings WHERE id = 1"#
)
.fetch_one(database.pool())
.await?;
if let Some(cell) = &translators.command_timeout {
cell.store(
u64::try_from(row.remote_command_timeout_seconds)
.unwrap_or(u64::MAX)
.saturating_mul(1_000),
std::sync::atomic::Ordering::Relaxed,
);
}
if let Some(endpoint) = &translators.openai_endpoint {
// A row that does not parse must not stop the daemon booting: the
// API validates on write, so this only fires for a hand-edited
// database. The backend stays at its default and the health lamp
// says so.
if let Err(error) =
endpoint.set(row.openai_base_url.as_deref(), row.openai_model.as_deref())
{
tracing::warn!(%error, "stored OpenAI endpoint is unusable; keeping the default");
}
}
Ok(())
}
/// Wire the reconcile lanes (DESIGN.md §8). Grab and RSS both need a
/// Prowlarr key and grab needs TMDB as well; a lane whose upstream is not
/// configured stays unregistered rather than failing every tick.
@@ -236,9 +331,10 @@ async fn run() -> Result<(), Error> {
fn reconcile_loop(
database: &Db,
config: &Config,
transmission: &arr_dl::TransmissionClient,
qbit: &arr_dl::QbitClient,
tmdb: Option<&Arc<TmdbClient>>,
notifier: &Notifier,
translators: &Translators,
) -> Result<(ReconcileLoop, Option<GrabAction>, Option<TvGrabAction>), Error> {
let reconcile = ReconcileLoop::new(database.clone());
let seeding = SeedingRules::new(
@@ -266,27 +362,16 @@ fn reconcile_loop(
.map(|key| arr_indexer::ProwlarrClient::new(config.prowlarr_url.clone(), key))
.transpose()?;
let (reconcile, manual_grab) = register_movie_grab(
reconcile,
prowlarr.as_ref(),
transmission,
config,
&seeding,
tmdb,
);
let (reconcile, manual_grab) =
register_movie_grab(reconcile, prowlarr.as_ref(), qbit, config, &seeding, tmdb);
let (mut reconcile, manual_tv) =
register_tv_grab(reconcile, prowlarr.as_ref(), transmission, config, &seeding);
register_tv_grab(reconcile, prowlarr.as_ref(), qbit, config, &seeding);
// RSS needs no TMDB: it matches what the feeds already carry against the
// wanted list (§6.2).
if let Some(prowlarr) = prowlarr {
reconcile = reconcile.register(
Tick::Rss,
RssAction::new(
prowlarr,
transmission.clone(),
config.download_dir.clone(),
seeding,
),
RssAction::new(prowlarr, qbit.clone(), config.download_dir.clone(), seeding),
);
}
// §8: metadata refresh is its own daily lane, staggered against the
@@ -302,7 +387,7 @@ fn reconcile_loop(
reconcile = reconcile.register(
Tick::Reconcile,
ImportAction::new(
transmission.clone(),
qbit.clone(),
arr_probe::Prober::new(),
jellyfin,
notifier.clone(),
@@ -310,6 +395,12 @@ fn reconcile_loop(
),
);
// §15: subtitle gaps are reconciled from the same rows the API writes.
reconcile = reconcile.register(
Tick::Reconcile,
subtitle_action(config, notifier, translators)?,
);
// §9.5 *needs a decision* and *broken* both go to the operator alone;
// without a topic configured there is nowhere to send them.
if let Some(operator_topic) = &config.ntfy_operator_topic {
@@ -317,19 +408,12 @@ fn reconcile_loop(
Tick::Reconcile,
AttentionAction::new(notifier.clone(), operator_topic.clone()),
);
let broken_upstreams = broken::Upstreams {
prowlarr_url: config.prowlarr_url.clone(),
prowlarr_api_key: config.prowlarr_api_key.clone(),
transmission_url: config.transmission_url.clone(),
tmdb_url: config
.tmdb_url
.clone()
.unwrap_or_else(|| arr_api::DEFAULT_TMDB_URL.to_string()),
tmdb_api_key: config.tmdb_api_key.clone(),
};
reconcile = reconcile.register(
Tick::Reconcile,
BrokenAction::new(broken_upstreams, notifier.clone(), operator_topic.clone()),
reconcile = register_broken(
reconcile,
config,
translators,
notifier,
operator_topic.clone(),
);
} else {
tracing::warn!(
@@ -337,10 +421,51 @@ fn reconcile_loop(
);
}
let reconcile = reconcile.register(Tick::Reaper, ReaperAction::new(transmission.clone()));
let reconcile = reconcile.register(Tick::Reaper, ReaperAction::new(qbit.clone()));
Ok((reconcile, manual_grab, manual_tv))
}
/// The *broken* lane (#200 included): Prowlarr, qBittorrent and TMDB, plus
/// the subtitle lamps — an enabled provider, the selected engine, or a
/// missing `alass`/`ffmpeg` all fold into the same operator message.
fn register_broken(
reconcile: ReconcileLoop,
config: &Config,
translators: &Translators,
notifier: &Notifier,
operator_topic: String,
) -> ReconcileLoop {
let broken_upstreams = broken::Upstreams {
prowlarr_url: config.prowlarr_url.clone(),
prowlarr_api_key: config.prowlarr_api_key.clone(),
qbittorrent_url: config.qbittorrent_url.clone(),
tmdb_url: config
.tmdb_url
.clone()
.unwrap_or_else(|| arr_api::DEFAULT_TMDB_URL.to_string()),
tmdb_api_key: config.tmdb_api_key.clone(),
};
let broken_subtitles = broken::SubtitleUpstreams {
providers: subtitle_providers(
config.opensubtitles_api_key.clone(),
config.opensubtitles_username.clone(),
config.opensubtitles_password.clone(),
),
backends: translators.backends.clone(),
alass_path: config.alass_path.clone(),
ffmpeg_path: config.ffmpeg_path.clone(),
};
reconcile.register(
Tick::Reconcile,
BrokenAction::new(
broken_upstreams,
broken_subtitles,
notifier.clone(),
operator_topic,
),
)
}
/// Register the TV grab lane on `reconcile` and hand back a second,
/// independent instance for `manual::run` (issue #132). `None` when Prowlarr
/// is not configured. TV grabbing needs no TMDB at grab time: air dates are
@@ -349,7 +474,7 @@ fn reconcile_loop(
fn register_tv_grab(
mut reconcile: ReconcileLoop,
prowlarr: Option<&arr_indexer::ProwlarrClient>,
transmission: &arr_dl::TransmissionClient,
qbit: &arr_dl::QbitClient,
config: &Config,
seeding: &SeedingRules,
) -> (ReconcileLoop, Option<TvGrabAction>) {
@@ -359,7 +484,7 @@ fn register_tv_grab(
let tv_grab_action = || {
TvGrabAction::new(
prowlarr.clone(),
transmission.clone(),
qbit.clone(),
config.download_dir.clone(),
seeding.clone(),
)
@@ -377,7 +502,7 @@ fn register_tv_grab(
fn register_movie_grab(
mut reconcile: ReconcileLoop,
prowlarr: Option<&arr_indexer::ProwlarrClient>,
transmission: &arr_dl::TransmissionClient,
qbit: &arr_dl::QbitClient,
config: &Config,
seeding: &SeedingRules,
tmdb: Option<&Arc<TmdbClient>>,
@@ -388,29 +513,23 @@ fn register_movie_grab(
};
reconcile = reconcile.register(
Tick::Reconcile,
movie_grab_action(prowlarr, transmission, config, seeding, tmdb),
movie_grab_action(prowlarr, qbit, config, seeding, tmdb),
);
let manual_grab = Some(movie_grab_action(
prowlarr,
transmission,
config,
seeding,
tmdb,
));
let manual_grab = Some(movie_grab_action(prowlarr, qbit, config, seeding, tmdb));
(reconcile, manual_grab)
}
/// The movie grab lane, built fresh for each caller.
fn movie_grab_action(
prowlarr: &arr_indexer::ProwlarrClient,
transmission: &arr_dl::TransmissionClient,
qbit: &arr_dl::QbitClient,
config: &Config,
seeding: &SeedingRules,
tmdb: &Arc<TmdbClient>,
) -> GrabAction {
GrabAction::new(
prowlarr.clone(),
transmission.clone(),
qbit.clone(),
config.download_dir.clone(),
seeding.clone(),
)
@@ -452,3 +571,176 @@ fn jellyfin_client(config: &Config) -> Result<arr_api::jellyfin::JellyfinClient,
config.jellyfin_api_key.clone(),
)?)
}
/// The §15 reconcile lane: closes subtitle gaps from the attempt rows.
///
/// Takes the same [`Translators`] the API is given, built once at startup, so
/// a translation behaves identically whether the reconcile sweep or the manual
/// endpoint asked for it. Which of the offered backends actually runs is the
/// `translation_engine` database setting, read per translation; with none
/// compiled in the translate step records "not compiled" and backs off rather
/// than failing obscurely.
fn subtitle_action(
config: &Config,
notifier: &Notifier,
translators: &Translators,
) -> Result<SubtitleAction, Error> {
let action = SubtitleAction::new(
subtitle_providers(
config.opensubtitles_api_key.clone(),
config.opensubtitles_username.clone(),
config.opensubtitles_password.clone(),
),
translators.backends.clone(),
arr_subs::Syncer::new().with_binary(config.alass_path.clone()),
arr_probe::Extractor::new().with_binary(config.ffmpeg_path.clone()),
jellyfin_client(config)?,
);
Ok(match &config.ntfy_operator_topic {
Some(topic) => action.with_notifier(notifier.clone(), topic.clone()),
None => action,
})
}
/// The subtitle providers this deployment can reach (DESIGN.md §15).
///
/// Credentials are bootstrap config and never reach the database (§10), so
/// which providers *exist* is decided here, once, at startup; which of them a
/// search *runs* is the `providers_enabled` setting the API reads per
/// request. OpenSubtitles.com needs a registered API key to be called at all,
/// so without one it is not offered.
fn subtitle_providers(
opensubtitles_api_key: Option<String>,
username: Option<String>,
password: Option<String>,
) -> Vec<std::sync::Arc<dyn arr_subs::Provider>> {
let mut providers: Vec<std::sync::Arc<dyn arr_subs::Provider>> = Vec::new();
let Some(api_key) = opensubtitles_api_key else {
tracing::info!("no OpenSubtitles.com API key configured; that provider is off");
return providers;
};
match arr_subs::OpenSubtitles::new(arr_subs::OpenSubtitlesConfig {
api_key,
username,
password,
}) {
Ok(opensubtitles) => providers.push(std::sync::Arc::new(opensubtitles)),
Err(error) => tracing::warn!(%error, "OpenSubtitles.com not available"),
}
providers
}
/// The translation backends this deployment can offer, built once at startup
/// and shared by the API and the reconcile lane (DESIGN.md §15, issue #216).
///
/// Which cargo features this binary was built with decides what could ever
/// be here (`compiled_engines`); credentials decide what actually is, same
/// split `subtitle_providers` makes for search. Which one of these a
/// translation *uses* is the `translation_engine` database setting, read per
/// request (#198) — this only decides which ids exist to be picked.
///
/// When the remote-command backend is one of them, its live timeout cell
/// rides along (#219): the API writes it on every settings edit, so the row
/// reaches the running process without a restart.
struct Translators {
backends: Vec<std::sync::Arc<dyn arr_subs::Backend>>,
command_timeout: Option<Arc<AtomicU64>>,
/// The OpenAI-compatible backend's live endpoint (#220), when that
/// backend is compiled in. Its base URL and model are database rows, so
/// this rides along the same way the command timeout does.
openai_endpoint: Option<arr_subs::OpenAiEndpoint>,
}
#[cfg_attr(
not(any(
feature = "translate-openai",
feature = "translate-deepl",
feature = "translate-google",
feature = "translate-command"
)),
allow(unused_variables, unused_mut)
)]
fn translation_backends(config: &Config) -> Translators {
let mut backends: Vec<std::sync::Arc<dyn arr_subs::Backend>> = Vec::new();
let mut command_timeout: Option<Arc<AtomicU64>> = None;
let mut openai_endpoint: Option<arr_subs::OpenAiEndpoint> = None;
#[cfg(feature = "translate-openai")]
{
// Built at the backend's own defaults and repointed from the
// settings row a moment later (#220). Never gated on the API key:
// `llama.cpp` serves without authentication, so a base URL and no
// key is a valid configuration (DESIGN.md §15).
let openai_config = arr_subs::OpenAiConfig {
api_key: config.translate_openai_api_key.clone(),
};
match arr_subs::OpenAi::new(openai_config) {
Ok(backend) => {
openai_endpoint = Some(backend.endpoint());
backends.push(std::sync::Arc::new(backend));
}
Err(error) => tracing::warn!(%error, "OpenAI-compatible translator not available"),
}
}
#[cfg(feature = "translate-deepl")]
{
if let Some(auth_key) = config.translate_deepl_api_key.clone() {
let deepl_config = arr_subs::DeepLConfig { auth_key };
let backend = match &config.translate_deepl_base_url {
Some(base_url) => arr_subs::DeepL::with_base_url(deepl_config, base_url),
None => arr_subs::DeepL::new(deepl_config),
};
match backend {
Ok(backend) => backends.push(std::sync::Arc::new(backend)),
Err(error) => tracing::warn!(%error, "DeepL not available"),
}
} else {
tracing::info!("no DeepL auth key configured; that translator is off");
}
}
#[cfg(feature = "translate-google")]
{
if let Some(api_key) = config.translate_google_api_key.clone() {
let google_config = arr_subs::GoogleConfig { api_key };
let backend = match &config.translate_google_base_url {
Some(base_url) => arr_subs::Google::with_base_url(google_config, base_url),
None => arr_subs::Google::new(google_config),
};
match backend {
Ok(backend) => backends.push(std::sync::Arc::new(backend)),
Err(error) => tracing::warn!(%error, "Google Translate not available"),
}
} else {
tracing::info!("no Google Translate API key configured; that translator is off");
}
}
#[cfg(feature = "translate-command")]
{
if let Some(template) = config.translate_command_template.clone() {
let command_config = arr_subs::CommandConfig {
template,
timeout: arr_subs::COMMAND_DEFAULT_TIMEOUT,
};
match arr_subs::Command::new(command_config) {
Ok(backend) => {
command_timeout = Some(backend.timeout_cell());
backends.push(std::sync::Arc::new(backend));
}
Err(error) => tracing::warn!(%error, "remote-command translator not available"),
}
} else {
tracing::info!("no remote-command template configured; that translator is off");
}
}
Translators {
backends,
command_timeout,
openai_endpoint,
}
}
+12 -20
View File
@@ -1,7 +1,7 @@
//! Drains `AppState`'s three command channels — the daemon-side consumer
//! DESIGN.md §6.2 and §9.3 assume exists (issues #107 and #132). A `Search`
//! resets backoff and re-runs the targeted-search + grab lane immediately;
//! `Grab` sends an already-chosen release straight to Transmission, skipping
//! `Grab` sends an already-chosen release straight to qBittorrent, skipping
//! search. Movies, episodes and seasons share one lane because they share
//! one operator waiting on a 202.
@@ -149,7 +149,7 @@ mod tests {
use std::path::PathBuf;
use arr_db::Db;
use arr_dl::TransmissionClient;
use arr_dl::QbitClient;
use arr_indexer::ProwlarrClient;
use wiremock::matchers::{method, path, path_regex, query_param};
use wiremock::{Mock, MockServer, ResponseTemplate};
@@ -172,23 +172,15 @@ mod tests {
(dir, database)
}
async fn transmission() -> MockServer {
let server = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({
"result": "success",
"arguments": {"torrent-added": {"id": 1, "name": "x", "hashString": "aaaa"}}
})))
.mount(&server)
.await;
server
async fn qbit() -> MockServer {
crate::qbit_fake::FakeQbit::start().await.0
}
fn grab_action(indexer: &MockServer, transmission: &MockServer) -> GrabAction {
fn grab_action(indexer: &MockServer, qbit: &MockServer) -> GrabAction {
GrabAction::new(
ProwlarrClient::new(indexer.uri(), "key").unwrap(),
TransmissionClient::new(&transmission.uri()).unwrap(),
PathBuf::from("/mnt/media/transmission/complete"),
QbitClient::new(&qbit.uri()).unwrap(),
PathBuf::from("/mnt/media/qbittorrent/complete"),
SeedingRules::new(
SeedingLimits {
ratio: 1.5,
@@ -239,8 +231,8 @@ mod tests {
)
.mount(&indexer)
.await;
let transmission = transmission().await;
let grab = grab_action(&indexer, &transmission);
let qbit = qbit().await;
let grab = grab_action(&indexer, &qbit);
handle_movie(&grab, &database, MovieCommand::Search { movie_id: 1 })
.await
@@ -269,7 +261,7 @@ mod tests {
}
/// Issue #107: a `Grab` command sends the already-chosen release straight
/// to Transmission — no indexer search at all.
/// to qBittorrent — no indexer search at all.
#[tokio::test]
async fn a_grab_command_sends_the_chosen_release_without_searching() {
let (_dir, database) = database_with_wanted_movie().await;
@@ -282,8 +274,8 @@ mod tests {
))
.mount(&indexer)
.await;
let transmission = transmission().await;
let grab = grab_action(&indexer, &transmission);
let qbit = qbit().await;
let grab = grab_action(&indexer, &qbit);
let release_id = sqlx::query(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, score, verdict)
+342
View File
@@ -0,0 +1,342 @@
//! An in-memory qBittorrent for the daemon's tests.
//!
//! Shared rather than per-module because every lane that grabs needs the same
//! two behaviours to be right, and both are easy to fake wrongly: `torrents/add`
//! answers `Ok.` with no hash, so the fake has to derive the same infohash the
//! client did, and adding the same source twice has to collapse to one torrent
//! or the restart and re-grab tests prove nothing.
#![allow(clippy::unwrap_used, dead_code)]
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use arr_dl::TorrentSource;
use serde_json::{json, Value};
use wiremock::matchers::any;
use wiremock::{Mock, MockServer, Request, Respond, ResponseTemplate};
#[derive(Clone, Debug)]
pub(crate) struct FakeTorrent {
pub hash: String,
pub name: String,
/// The magnet URI, or `<inline .torrent>` for a torrent sent as bytes.
pub source: String,
/// The bytes of a torrent sent inline, so a test can prove arr forwarded
/// the file rather than the indexer link.
pub metainfo: Option<Vec<u8>>,
pub labels: Vec<String>,
pub save_path: String,
pub progress: f64,
pub state: String,
pub ratio_limit: f64,
pub idle_limit_minutes: i64,
/// `(path, size)`, relative to `save_path`.
pub files: Vec<(String, u64)>,
}
#[derive(Clone, Debug, Default)]
pub(crate) struct FakeQbit {
pub torrents: Arc<Mutex<Vec<FakeTorrent>>>,
/// Every source ever added, in order, kept across removals.
pub added: Arc<Mutex<Vec<String>>>,
}
impl FakeQbit {
/// Start a mock server backed by a fresh fake.
pub(crate) async fn start() -> (MockServer, Self) {
let server = MockServer::start().await;
let fake = Self::default();
Mock::given(any())
.respond_with(fake.clone())
.mount(&server)
.await;
(server, fake)
}
pub(crate) fn torrents(&self) -> Vec<FakeTorrent> {
self.torrents.lock().unwrap().clone()
}
pub(crate) fn complete_all(&self) {
for torrent in self.torrents.lock().unwrap().iter_mut() {
torrent.progress = 1.0;
torrent.state = "uploading".into();
}
}
/// Give every torrent the same file list and download directory, for the
/// import lane.
pub(crate) fn set_contents(&self, save_path: &str, files: &[(&str, u64)]) {
for torrent in self.torrents.lock().unwrap().iter_mut() {
torrent.save_path = save_path.to_owned();
torrent.files = files
.iter()
.map(|(path, size)| ((*path).to_owned(), *size))
.collect();
}
}
fn add(&self, request: &Request) -> ResponseTemplate {
let parts = multipart(request);
let (source, name, metainfo) = if let Some(urls) = parts.get("urls") {
let uri = String::from_utf8_lossy(urls).trim().to_owned();
let name = uri.split_once("dn=").map_or_else(
|| uri.clone(),
|(_, rest)| rest.split('&').next().unwrap_or_default().to_owned(),
);
(TorrentSource::Magnet(uri), name, None)
} else {
let bytes = parts.get("torrents").cloned().unwrap_or_default();
(
TorrentSource::Metainfo(bytes.clone()),
"<inline .torrent>".to_owned(),
Some(bytes),
)
};
let Ok(hash) = arr_dl::infohash(&source) else {
// The real client refuses these before the call, so reaching here
// means the fixture is malformed.
return ResponseTemplate::new(415).set_body_string("invalid torrent");
};
let display = match &source {
TorrentSource::Magnet(uri) => uri.clone(),
TorrentSource::Metainfo(_) => "<inline .torrent>".to_owned(),
};
self.added.lock().unwrap().push(display.clone());
let mut torrents = self.torrents.lock().unwrap();
if !torrents.iter().any(|torrent| torrent.hash == hash) {
torrents.push(FakeTorrent {
hash,
name,
source: display,
metainfo,
labels: text(&parts, "tags")
.split(',')
.filter(|tag| !tag.is_empty())
.map(ToOwned::to_owned)
.collect(),
save_path: text(&parts, "savepath"),
progress: 0.0,
state: "downloading".into(),
ratio_limit: -1.0,
idle_limit_minutes: -1,
files: Vec::new(),
});
}
ResponseTemplate::new(200).set_body_string("Ok.")
}
fn info(&self, request: &Request) -> ResponseTemplate {
let wanted = query(request, "hashes");
let torrents: Vec<Value> = self
.torrents()
.into_iter()
.filter(|torrent| {
wanted
.as_ref()
.is_none_or(|hash| torrent.hash.eq_ignore_ascii_case(hash))
})
.map(|torrent| {
json!({
"hash": torrent.hash,
"name": torrent.name,
"state": torrent.state,
"progress": torrent.progress,
"dlspeed": 0,
"save_path": torrent.save_path,
"tags": torrent.labels.join(","),
"ratio": 0.0,
"ratio_limit": torrent.ratio_limit,
"seeding_time": 0,
"seeding_time_limit": -1,
"inactive_seeding_time_limit": torrent.idle_limit_minutes,
"last_activity": 0
})
})
.collect();
ResponseTemplate::new(200).set_body_json(torrents)
}
fn files(&self, request: &Request) -> ResponseTemplate {
let hash = query(request, "hash").unwrap_or_default();
let files: Vec<Value> = self
.torrents()
.into_iter()
.find(|torrent| torrent.hash.eq_ignore_ascii_case(&hash))
.map(|torrent| {
torrent
.files
.into_iter()
.map(|(path, size)| json!({"name": path, "size": size}))
.collect()
})
.unwrap_or_default();
ResponseTemplate::new(200).set_body_json(files)
}
fn mutate(&self, path: &str, request: &Request) -> ResponseTemplate {
let form = urlencoded(request);
let hashes = form.get("hashes").cloned().unwrap_or_default();
for torrent in self.torrents.lock().unwrap().iter_mut() {
if !hashes
.split('|')
.any(|hash| hash.eq_ignore_ascii_case(&torrent.hash))
{
continue;
}
match path {
"addTags" => {
for tag in form.get("tags").cloned().unwrap_or_default().split(',') {
if !tag.is_empty() && !torrent.labels.iter().any(|held| held == tag) {
torrent.labels.push(tag.to_owned());
}
}
}
"setLocation" => {
torrent.save_path = form.get("location").cloned().unwrap_or_default();
}
"setShareLimits" => {
torrent.ratio_limit = form
.get("ratioLimit")
.and_then(|value| value.parse().ok())
.unwrap_or(-1.0);
torrent.idle_limit_minutes = form
.get("inactiveSeedingTimeLimit")
.and_then(|value| value.parse().ok())
.unwrap_or(-1);
}
_ => {}
}
}
ResponseTemplate::new(200).set_body_string("")
}
fn delete(&self, request: &Request) -> ResponseTemplate {
let form = urlencoded(request);
let hashes = form.get("hashes").cloned().unwrap_or_default();
self.torrents.lock().unwrap().retain(|torrent| {
!hashes
.split('|')
.any(|hash| hash.eq_ignore_ascii_case(&torrent.hash))
});
ResponseTemplate::new(200).set_body_string("")
}
}
impl Respond for FakeQbit {
fn respond(&self, request: &Request) -> ResponseTemplate {
let path = request.url.path().to_owned();
match path.rsplit('/').next().unwrap_or_default() {
"add" => self.add(request),
"info" => self.info(request),
"files" => self.files(request),
"delete" => self.delete(request),
endpoint @ ("addTags" | "setLocation" | "setShareLimits") => {
self.mutate(endpoint, request)
}
_ => ResponseTemplate::new(200).set_body_string("Ok."),
}
}
}
fn query(request: &Request, key: &str) -> Option<String> {
request
.url
.query_pairs()
.find(|(name, _)| name == key)
.map(|(_, value)| value.into_owned())
}
fn urlencoded(request: &Request) -> HashMap<String, String> {
String::from_utf8_lossy(&request.body)
.split('&')
.filter_map(|pair| pair.split_once('='))
.map(|(key, value)| (decode(key), decode(value)))
.collect()
}
fn decode(input: &str) -> String {
let bytes = input.replace('+', " ").into_bytes();
let mut out = Vec::with_capacity(bytes.len());
let mut at = 0;
while at < bytes.len() {
if bytes[at] == b'%' && at + 2 < bytes.len() {
if let Some(byte) = std::str::from_utf8(&bytes[at + 1..at + 3])
.ok()
.and_then(|pair| u8::from_str_radix(pair, 16).ok())
{
out.push(byte);
at += 3;
continue;
}
}
out.push(bytes[at]);
at += 1;
}
String::from_utf8_lossy(&out).into_owned()
}
fn text(parts: &HashMap<String, Vec<u8>>, key: &str) -> String {
parts
.get(key)
.map(|bytes| String::from_utf8_lossy(bytes).into_owned())
.unwrap_or_default()
}
/// The named parts of a `multipart/form-data` body.
fn multipart(request: &Request) -> HashMap<String, Vec<u8>> {
let Some(boundary) = request
.headers
.get("content-type")
.and_then(|value| value.to_str().ok())
.and_then(|value| value.split("boundary=").nth(1))
.map(|value| format!("--{}", value.trim_matches('"')))
else {
return HashMap::new();
};
let mut parts = HashMap::new();
for section in split(&request.body, boundary.as_bytes()).skip(1) {
let Some(header_end) = find(section, b"\r\n\r\n") else {
continue;
};
let headers = String::from_utf8_lossy(&section[..header_end]);
let Some(name) = headers
.split("name=\"")
.nth(1)
.and_then(|rest| rest.split('"').next())
else {
continue;
};
let body = &section[header_end + 4..];
let body = body.strip_suffix(b"\r\n").unwrap_or(body);
parts.insert(name.to_owned(), body.to_vec());
}
parts
}
fn split<'a>(haystack: &'a [u8], needle: &'a [u8]) -> impl Iterator<Item = &'a [u8]> {
let mut rest = haystack;
std::iter::from_fn(move || {
if rest.is_empty() {
return None;
}
if let Some(at) = find(rest, needle) {
let (section, tail) = rest.split_at(at);
rest = &tail[needle.len()..];
Some(section)
} else {
let section = rest;
rest = &[];
Some(section)
}
})
}
fn find(haystack: &[u8], needle: &[u8]) -> Option<usize> {
haystack
.windows(needle.len())
.position(|window| window == needle)
}
+84 -55
View File
@@ -1,36 +1,36 @@
//! Removes arr torrents only after Transmission says their seeding obligation
//! Removes arr torrents only after qBittorrent says their seeding obligation
//! is complete. The library/import state is deliberately not consulted (§7.3).
use std::collections::HashSet;
use arr_db::Db;
use arr_dl::TransmissionClient;
use arr_dl::QbitClient;
use crate::grab::label_for_root;
use crate::reconcile::{Action, ActionFuture, Outcome};
#[derive(Debug)]
pub struct ReaperAction {
transmission: TransmissionClient,
qbit: QbitClient,
}
impl ReaperAction {
#[must_use]
pub fn new(transmission: TransmissionClient) -> Self {
Self { transmission }
pub fn new(qbit: QbitClient) -> Self {
Self { qbit }
}
async fn tick(&self, labels: &HashSet<String>) -> Result<Vec<Outcome>, arr_dl::Error> {
let torrents = self.transmission.list_torrents().await?;
let torrents = self.qbit.list_torrents().await?;
let mut outcomes = Vec::new();
for torrent in torrents {
if !torrent.is_finished || !torrent.labels.iter().any(|label| labels.contains(label)) {
continue;
}
self.transmission.remove_torrent(torrent.id, true).await?;
self.qbit.remove_torrent(&torrent.hash, true).await?;
outcomes.push(Outcome::new(
format!("torrent {} finished seeding", torrent.hash),
format!("removed torrent {} and its download data", torrent.id),
format!("removed torrent {} and its download data", torrent.name),
));
}
Ok(outcomes)
@@ -64,85 +64,114 @@ mod tests {
use std::sync::{Arc, Mutex};
use serde_json::{json, Value};
use wiremock::matchers::any;
use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, Request, Respond, ResponseTemplate};
use super::*;
/// Records what was asked to be deleted, so a test can assert the reaper
/// touched one torrent and not its neighbours.
#[derive(Clone)]
struct Transmission {
torrents: Value,
removed: Arc<Mutex<Vec<Value>>>,
}
struct Deletions(Arc<Mutex<Vec<String>>>);
impl Respond for Transmission {
impl Respond for Deletions {
fn respond(&self, request: &Request) -> ResponseTemplate {
let body: Value = serde_json::from_slice(&request.body).unwrap();
let arguments = match body["method"].as_str().unwrap() {
"torrent-get" => json!({"torrents": self.torrents}),
"torrent-remove" => {
self.removed.lock().unwrap().push(body["arguments"].clone());
json!({})
}
method => panic!("unexpected method {method}"),
};
ResponseTemplate::new(200)
.insert_header("x-transmission-session-id", "session")
.set_body_json(json!({"result": "success", "arguments": arguments}))
let body = String::from_utf8_lossy(&request.body).into_owned();
self.0.lock().unwrap().push(body);
ResponseTemplate::new(200).set_body_string("")
}
}
/// A torrent qBittorrent stopped on the ratio limit arr set on it, which
/// is what "finished seeding" means to the reaper.
fn torrent(id: i64, label: &str, finished: bool) -> Value {
json!({
"id": id, "name": format!("torrent-{id}"), "hashString": format!("hash-{id}"),
"status": if finished { 0 } else { 6 }, "percentDone": 1.0,
"downloadDir": "/downloads", "labels": [label], "isFinished": finished
"hash": format!("hash-{id}"), "name": format!("torrent-{id}"),
"state": if finished { "stoppedUP" } else { "uploading" },
"progress": 1.0, "save_path": "/downloads", "tags": label,
"ratio": 2.0, "ratio_limit": 1.5,
"seeding_time": 10, "seeding_time_limit": -1,
"inactive_seeding_time_limit": -1, "last_activity": 0
})
}
async fn qbit(server: &MockServer, torrents: Value) -> Arc<Mutex<Vec<String>>> {
let deleted = Arc::new(Mutex::new(Vec::new()));
Mock::given(method("GET"))
.and(path("/api/v2/torrents/info"))
.respond_with(ResponseTemplate::new(200).set_body_json(torrents))
.mount(server)
.await;
Mock::given(method("POST"))
.and(path("/api/v2/torrents/delete"))
.respond_with(Deletions(Arc::clone(&deleted)))
.mount(server)
.await;
deleted
}
#[tokio::test]
async fn removes_only_finished_arr_torrents_with_data() {
let server = MockServer::start().await;
let removed = Arc::new(Mutex::new(Vec::new()));
Mock::given(any())
.respond_with(Transmission {
torrents: json!([
torrent(1, "movies-main", false),
torrent(2, "movies-main", true),
torrent(3, "radarr", true)
]),
removed: Arc::clone(&removed),
})
.mount(&server)
.await;
let action = ReaperAction::new(TransmissionClient::new(&server.uri()).unwrap());
let deleted = qbit(
&server,
json!([
torrent(1, "movies-main", false),
torrent(2, "movies-main", true),
torrent(3, "radarr", true)
]),
)
.await;
let action = ReaperAction::new(QbitClient::new(&server.uri()).unwrap());
let labels = HashSet::from(["movies-main".to_owned(), "movies-kids".to_owned()]);
let outcomes = action.tick(&labels).await.unwrap();
assert_eq!(outcomes.len(), 1);
assert_eq!(
*removed.lock().unwrap(),
[json!({"ids": [2], "delete-local-data": true})]
let deleted = deleted.lock().unwrap();
assert_eq!(deleted.len(), 1);
assert!(
deleted[0].contains("hashes=hash-2"),
"asked: {}",
deleted[0]
);
assert!(
deleted[0].contains("deleteFiles=true"),
"asked: {}",
deleted[0]
);
}
#[tokio::test]
async fn finished_torrent_does_not_need_a_grab_or_import_row() {
let server = MockServer::start().await;
let removed = Arc::new(Mutex::new(Vec::new()));
Mock::given(any())
.respond_with(Transmission {
torrents: json!([torrent(9, "movies-kids", true)]),
removed: Arc::clone(&removed),
})
.mount(&server)
.await;
let action = ReaperAction::new(TransmissionClient::new(&server.uri()).unwrap());
let deleted = qbit(&server, json!([torrent(9, "movies-kids", true)])).await;
let action = ReaperAction::new(QbitClient::new(&server.uri()).unwrap());
let labels = HashSet::from(["movies-kids".to_owned()]);
action.tick(&labels).await.unwrap();
assert_eq!(removed.lock().unwrap().len(), 1);
assert_eq!(deleted.lock().unwrap().len(), 1);
}
#[tokio::test]
async fn a_torrent_stopped_by_hand_is_left_alone() {
let server = MockServer::start().await;
let deleted = qbit(
&server,
json!([{
"hash": "hash-4", "name": "paused by the operator", "state": "stoppedUP",
"progress": 1.0, "save_path": "/downloads", "tags": "movies-main",
"ratio": 0.2, "ratio_limit": 1.5,
"seeding_time": 10, "seeding_time_limit": -1,
"inactive_seeding_time_limit": -1, "last_activity": 0
}]),
)
.await;
let action = ReaperAction::new(QbitClient::new(&server.uri()).unwrap());
let labels = HashSet::from(["movies-main".to_owned()]);
assert!(action.tick(&labels).await.unwrap().is_empty());
assert!(deleted.lock().unwrap().is_empty());
}
}
+10 -10
View File
@@ -287,11 +287,11 @@ mod tests {
use super::*;
#[derive(Clone, Debug, Default)]
struct FakeTransmission {
struct FakeQbit {
torrents: Arc<Mutex<HashMap<i64, String>>>,
}
impl FakeTransmission {
impl FakeQbit {
async fn find_or_add(&self, movie_id: i64) -> (String, bool) {
let mut torrents = self.torrents.lock().await;
if let Some(hash) = torrents.get(&movie_id) {
@@ -310,7 +310,7 @@ mod tests {
#[derive(Debug)]
struct FakeGrabAction {
transmission: FakeTransmission,
qbit: FakeQbit,
release_id: i64,
fail_after_add: bool,
}
@@ -356,11 +356,11 @@ mod tests {
return Ok(Vec::new());
};
let (infohash, added) = self.transmission.find_or_add(movie_id).await;
let (infohash, added) = self.qbit.find_or_add(movie_id).await;
if added && self.fail_after_add {
return Err(Box::new(io::Error::new(
io::ErrorKind::Interrupted,
"simulated process death after Transmission accepted the torrent",
"simulated process death after qBittorrent accepted the torrent",
)) as ActionError);
}
@@ -448,25 +448,25 @@ mod tests {
#[tokio::test]
async fn restart_converges_without_duplicate_external_work() {
let (_directory, database, release_id) = seeded_database().await;
let transmission = FakeTransmission::default();
let qbit = FakeQbit::default();
let interrupted = ReconcileLoop::new(database.clone()).register(
Tick::Reconcile,
FakeGrabAction {
transmission: transmission.clone(),
qbit: qbit.clone(),
release_id,
fail_after_add: true,
},
);
let first = interrupted.run_tick(Tick::Reconcile).await;
assert_eq!(first.failures, 1);
assert_eq!(transmission.len().await, 1);
assert_eq!(qbit.len().await, 1);
drop(interrupted);
let restarted = ReconcileLoop::new(database.clone()).register(
Tick::Reconcile,
FakeGrabAction {
transmission: transmission.clone(),
qbit: qbit.clone(),
release_id,
fail_after_add: false,
},
@@ -476,7 +476,7 @@ mod tests {
assert_eq!(recovered.actions_taken, 1);
assert_eq!(stable.actions_taken, 0);
assert_eq!(transmission.len().await, 1);
assert_eq!(qbit.len().await, 1);
let grabs: i64 = sqlx::query_scalar("SELECT count(*) FROM grabs")
.fetch_one(database.pool())
.await
+22 -62
View File
@@ -28,7 +28,7 @@ use arr_core::matching::{
};
use arr_core::{EpisodeId, Language, MovieId};
use arr_db::{Blacklist, Db, MoviePolicy, TitlePolicy};
use arr_dl::TransmissionClient;
use arr_dl::QbitClient;
use arr_indexer::{ProwlarrClient, SearchRelease, SearchRequest};
use crate::grab::{
@@ -50,13 +50,13 @@ impl RssAction {
#[must_use]
pub fn new(
prowlarr: ProwlarrClient,
transmission: TransmissionClient,
qbit: QbitClient,
download_dir: PathBuf,
seeding: SeedingRules,
) -> Self {
Self {
indexers: IndexerDirectory::new(prowlarr.clone()),
grabber: Grabber::new(prowlarr.clone(), transmission, download_dir, seeding),
grabber: Grabber::new(prowlarr.clone(), qbit, download_dir, seeding),
prowlarr,
}
}
@@ -714,15 +714,15 @@ async fn load_grabbable(
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use std::sync::{Arc, Mutex};
use serde_json::{json, Value};
use serde_json::json;
use wiremock::matchers::{method, path, query_param, query_param_is_missing};
use wiremock::{Mock, MockServer, Request, Respond, ResponseTemplate};
use wiremock::{Mock, MockServer, ResponseTemplate};
use super::*;
use crate::grab::{test_downloads, SeedingLimits};
use arr_dl::TransmissionClient;
use crate::qbit_fake::FakeQbit;
use arr_dl::QbitClient;
/// The recorded feed: two releases of one wanted title, one that only a
/// TMDB id identifies, one near miss, one TV item and one film nobody
@@ -773,40 +773,6 @@ mod tests {
const INDEXERS: [i64; 2] = [7, 9];
/// Enough of Transmission to add a torrent and list nothing back.
#[derive(Clone, Debug, Default)]
struct FakeTransmission {
added: Arc<Mutex<Vec<String>>>,
}
impl Respond for FakeTransmission {
fn respond(&self, request: &Request) -> ResponseTemplate {
let body: Value = serde_json::from_slice(&request.body).unwrap();
match body["method"].as_str().unwrap_or_default() {
"torrent-add" => {
let source = body["arguments"]["filename"]
.as_str()
.unwrap_or_default()
.to_owned();
let mut added = self.added.lock().unwrap();
added.push(source.clone());
let id = i64::try_from(added.len()).unwrap();
success(&json!({"torrent-added": {
"id": id, "name": source, "hashString": format!("{id:040x}")
}}))
}
"torrent-get" => success(&json!({"torrents": []})),
_ => success(&json!({})),
}
}
}
fn success(arguments: &Value) -> ResponseTemplate {
ResponseTemplate::new(200).set_body_json(json!({
"result": "success", "arguments": arguments
}))
}
/// Two indexers, both advertising a text search, both serving the same
/// feed to an empty query.
async fn prowlarr(feed: &str) -> MockServer {
@@ -842,14 +808,8 @@ mod tests {
server
}
async fn transmission() -> (MockServer, FakeTransmission) {
let server = MockServer::start().await;
let fake = FakeTransmission::default();
Mock::given(method("POST"))
.respond_with(fake.clone())
.mount(&server)
.await;
(server, fake)
async fn qbit() -> (MockServer, FakeQbit) {
FakeQbit::start().await
}
/// `(tmdb_id, title, year, blocked)`.
@@ -874,11 +834,11 @@ mod tests {
(dir, database)
}
fn action(prowlarr: &MockServer, transmission: &MockServer) -> RssAction {
fn action(prowlarr: &MockServer, qbit: &MockServer) -> RssAction {
RssAction::new(
ProwlarrClient::new(prowlarr.uri(), "key").unwrap(),
TransmissionClient::new(&transmission.uri()).unwrap(),
PathBuf::from("/mnt/media/transmission/complete"),
QbitClient::new(&qbit.uri()).unwrap(),
PathBuf::from("/mnt/media/qbittorrent/complete"),
SeedingRules::new(
SeedingLimits {
ratio: 1.0,
@@ -975,7 +935,7 @@ mod tests {
])
.await;
let indexer = prowlarr(FEED).await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1019,7 +979,7 @@ mod tests {
async fn a_near_miss_is_not_grabbed() {
let (_dir, database) = wanted(&[(9_999_999, "Dune: Part Three", 2024, false)]).await;
let indexer = prowlarr(FEED).await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1038,7 +998,7 @@ mod tests {
])
.await;
let indexer = prowlarr(FEED).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1055,7 +1015,7 @@ mod tests {
async fn a_blocked_title_still_matches_rss() {
let (_dir, database) = wanted(&[(693_134, "Dune: Part Two", 2024, true)]).await;
let indexer = prowlarr(FEED).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1073,7 +1033,7 @@ mod tests {
])
.await;
let indexer = prowlarr(FEED).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1103,7 +1063,7 @@ mod tests {
let (_dir, database) = wanted(&[]).await;
let (_season_id, episodes) = wanted_series(&database, &["2024-04-11", "2099-01-01"]).await;
let indexer = prowlarr(TV_FEED).await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1149,7 +1109,7 @@ mod tests {
"Unrelated.S01E03.2160p.WEB-DL-GROUP</title>",
);
let indexer = prowlarr(&pack_only).await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1197,7 +1157,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr(TV_FEED).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1237,7 +1197,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr(TV_FEED).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1269,7 +1229,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr(TV_FEED).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More