Compare commits

...

10 Commits

Author SHA1 Message Date
Miguel Palhas d97cdfd557 fix(subs): ask OpenSubtitles for SRT at download
ci / web (push) Successful in 35s
ci / rust (push) Successful in 1m44s
e2e / e2e (push) Failing after 1m51s
ci / image (push) Successful in 2m23s
A search entry often carries no `format`, so `format_of(None)` produced
`Other("")`, that was stashed at search time and handed back as the
download's format, and conversion had no parser to pick:

    4073669 could not be converted from  to SRT: no parser for ""

`POST /download` takes `sub_format`, and the API converts on its side, so
asking for srt makes the answer srt whatever the uploader posted. The
format guessed from the search entry is dead weight and goes with it.

The download mock matched only `file_id`, so it never exercised what the
real API returns.

Closes #272

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 10:10:06 +01:00
Miguel Palhas 7bdbecd68a fix(subs): log in to OpenSubtitles with a JSON body
ci / web (push) Successful in 56s
e2e / e2e (push) Failing after 2m1s
ci / rust (push) Successful in 2m4s
ci / image (push) Successful in 4m8s
`login` posted the credentials as HTTP basic auth with no request body.
The API takes them as JSON, answers basic auth with a 401, and the lane
reported that back as "opensubtitles rejected the configured
credentials" — for correct credentials.

Search was unaffected and hid this: it authenticates with the Api-Key
header alone and never logs in, so candidates were found and only the
download failed.

`mount_login` matched on method and path, so the mock answered a token
to any request shape. It now matches the body.

Closes #271

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 09:57:58 +01:00
Miguel Palhas 3a5e768029 fix(subs): search OpenSubtitles by the real id params
ci / web (push) Successful in 55s
e2e / e2e (push) Failing after 1m41s
ci / rust (push) Successful in 1m48s
ci / image (push) Successful in 2m22s
The id lane sent `tmdb_movie_id` and `tmdb_series_id`. Neither is a
parameter of this API, and unknown ones are ignored rather than refused,
so every search silently degraded to a `moviehash` lookup: one result for
a release someone had already hashed, none at all for anything else.

Two further requirements the API documents and answers a 301 to when
missed: parameters sorted by name, and language codes lowercase and
sorted.

The integration tests asserted the old names against a mock, which
answers whatever it is asked for. They pinned the bug rather than
catching it.

Closes #270

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 09:46:43 +01:00
Miguel Palhas 6cd2902341 feat(subs): translate from an image-only release
ci / web (push) Successful in 1m0s
e2e / e2e (push) Failing after 3m17s
ci / rust (push) Successful in 4m0s
ci / image (push) Successful in 3m56s
A release whose only subtitle is a PGS or VobSub track was stuck both
ways: the track satisfied English so no English SRT was ever fetched,
and bitmaps can never feed a translator. §15 is amended to separate
satisfying viewing from providing a translation source, and to carve a
fetch made to obtain a source out of the no-upgrade rule.

Timings come from the disc rather than from alass guessing at the audio.
At import, each non-forced image track's packet timestamps are paired
show-to-clear into a cue skeleton and stored; the fetched source is then
aligned against that skeleton, and the translation made from it skips
the post-translation pass, which could only move disc-exact timings off.

Pairing is validated before it is trusted — even packet count, plausible
durations, sane density for the runtime — because PGS allows several
composition segments per subtitle and a slipped pairing is quietly half
a second out. A track that fails validation gets no skeleton and falls
back to aligning against the video, as does any file imported before
this: there is no backfill.

Closes #268

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-30 11:53:26 +01:00
Miguel Palhas 015b90a458 fix(api): a deleted title takes its grabs with it
ci / web (push) Successful in 1m3s
e2e / e2e (push) Failing after 1m59s
ci / rust (push) Successful in 2m36s
ci / image (push) Successful in 2m20s
`grabs.infohash` is unique and grabs point at their target
polymorphically, so nothing cascaded them off a deleted movie or series.
An orphan holding an infohash then blocked the row a later grab of the
same release needed: the upsert only reclaims a `vanished` row, so the
insert did nothing while the new title was still flipped to
`downloading`. It never imported, and `/api/downloads` attributed the
torrent to a title that no longer existed.

Deletes now clear the grabs, and the upsert also reclaims a row whose
target is gone whatever its state — which heals the databases that
already have orphans. A row belonging to a live title stays off limits,
so the restart case still cannot let one title steal another's torrent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 20:34:58 +01:00
Miguel Palhas 9ecbfde656 fix(dl): only log in when qBittorrent asks
ci / web (push) Successful in 32s
e2e / e2e (push) Failing after 1m56s
ci / rust (push) Successful in 2m0s
ci / image (push) Successful in 2m27s
Logging in before the first call turned a stale or wrong password into a
hard failure on calls that would have succeeded: an instance that bypasses
authentication for arr's address answers everything, and never needed the
credentials at all. Authentication is now established on the first 403.

The health probe was reading a 403 from `app/version` as healthy, on the
reasoning that a live daemon is all the lamp claims. That is exactly the
state arr cannot use, and it left the lamp green while every grab and every
reaper tick failed on rejected credentials. It now lists torrents through
the configured client, so the lamp fails when arr's own calls do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 18:42:43 +01:00
Miguel Palhas ab4f95b166 feat(dl): replace Transmission with qBittorrent
ci / web (push) Successful in 32s
ci / rust (push) Successful in 3m26s
e2e / e2e (push) Failing after 5m2s
ci / image (push) Successful in 4m1s
qBittorrent's add call answers `Ok.` and nothing else — no hash, no name,
no duplicate signal — so arr derives the v1 infohash from the magnet or the
`.torrent` bytes before the call and looks the torrent up by it. That also
drops Transmission's numeric torrent id: the hash is the only identity now.

The reaper needs "stopped because a share limit was reached", and
qBittorrent's state field cannot tell that apart from a hand-paused torrent.
So the ratio and idle counters are checked against the limits arr set, and a
torrent stopped by a global limit reads as still seeding rather than being
deleted.

The WebUI needs a login, so `ARR_QBITTORRENT_USERNAME` and
`ARR_QBITTORRENT_PASSWORD` join the env-only secrets; leaving both unset is
valid for an instance that whitelists arr's subnet.

Verified against qBittorrent 5 (WebAPI 2.15.1) in a container: it rejects
`setShareLimits` without `shareLimitAction`, which 4.x ignores, so it is
always sent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 18:13:50 +01:00
Miguel Palhas afe167ca7b ci: enable the deploy trigger
ci / web (push) Successful in 59s
ci / rust (push) Successful in 1m47s
e2e / e2e (push) Failing after 2m7s
ci / image (push) Successful in 2m29s
Reverts the temporary docker-branch gating used to probe the runner, and
restores the deploy step now that the image build is verified.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 15:48:35 +01:00
Miguel Palhas a9848fddce ci: build the production image in CI
ci / web (push) Successful in 33s
ci / rust (push) Successful in 1m9s
ci / image (push) Successful in 4m59s
Dokploy built from source on the host, so every deploy paid a full cold
cargo build and the layer cache died with the builder. CI now builds and
pushes to the Gitea registry, and the deploy is a pull.

The Gitea push webhook fires before Actions starts, so autoDeploy on the
Dokploy application is off and this job triggers the deploy itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 15:37:50 +01:00
Miguel Palhas 4d347cd285 fix(db): restore migration 0025 to what was applied
ci / web (push) Successful in 1m8s
ci / rust (push) Successful in 2m17s
e2e / e2e (push) Failing after 2m41s
4cec2b6 edited an already-applied migration to drop podnapisi from the
subtitle_settings default. sqlx checksums migrations, so every boot since
has failed with "migration 25 was previously applied but has been
modified" — the daemon exits 1, swarm keeps the old task, and three
commits never reached production.

0033 already strips podnapisi from existing rows, and it runs after 0025
seeds row 1, so a fresh database still ends up with opensubtitles alone.
The edit changed nothing except the checksum.
2026-08-29 14:24:57 +01:00
65 changed files with 4102 additions and 1383 deletions
+66
View File
@@ -119,3 +119,69 @@ jobs:
- name: design tokens
if: steps.probe.outputs.present == 'true'
run: pnpm -C web run check-tokens
# Build the production image here rather than on the Dokploy host: the gate
# above has to pass first, the layer cache lives in the registry instead of
# being thrown away every deploy, and Dokploy's job shrinks to a pull.
#
# This job is also the deploy trigger. The Gitea push webhook fires the
# instant the commit lands, long before the image exists, so autoDeploy on
# the Dokploy application must stay off — the deploy is the last step here.
image:
needs: [rust, web]
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
env:
IMAGE: git.naps.pt/yolo/arr
DOKPLOY_APP_ID: uEaWG5JDpVIrJloG5rMAP
steps:
- uses: actions/checkout@v4
- name: Docker CLI and buildx
run: |
apt-get update
apt-get install -y --no-install-recommends ca-certificates curl gnupg
. /etc/os-release
install -m 0755 -d /etc/apt/keyrings
curl -fsSL "https://download.docker.com/linux/$ID/gpg" \
-o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc]" \
"https://download.docker.com/linux/$ID $VERSION_CODENAME stable" \
> /etc/apt/sources.list.d/docker.list
apt-get update
apt-get install -y --no-install-recommends docker-ce-cli docker-buildx-plugin
docker version
- name: Registry login
run: |
printf '%s' "${{ secrets.REGISTRY_TOKEN }}" \
| docker login git.naps.pt -u "${{ secrets.REGISTRY_USER }}" --password-stdin
- name: Buildx builder
# The default docker driver cannot export a cache. docker-container can.
run: docker buildx create --name arr --driver docker-container --use
- name: Build and push
run: |
docker buildx build \
--tag "$IMAGE:sha-$GITHUB_SHA" \
--tag "$IMAGE:main" \
--cache-from "type=registry,ref=$IMAGE:buildcache" \
--cache-to "type=registry,ref=$IMAGE:buildcache,mode=max" \
--push .
- name: Deploy
# Pinned to the commit sha, not :main — swarm does not reliably re-pull
# an unchanged tag name, and an older sha is the rollback.
run: |
# The package is public (org yolo is public), so Dokploy pulls with
# no credentials and the registry fields stay null.
curl -fsS -X POST https://dokploy.n62.casa/api/application.update \
-H "x-api-key: ${{ secrets.DOKPLOY_API_KEY }}" \
-H 'content-type: application/json' \
-d "{\"applicationId\":\"$DOKPLOY_APP_ID\",\"dockerImage\":\"$IMAGE:sha-$GITHUB_SHA\"}"
curl -fsS -X POST https://dokploy.n62.casa/api/application.deploy \
-H "x-api-key: ${{ secrets.DOKPLOY_API_KEY }}" \
-H 'content-type: application/json' \
-d "{\"applicationId\":\"$DOKPLOY_APP_ID\",\"title\":\"ci ${GITHUB_SHA:0:8}\"}"
+26 -10
View File
@@ -26,15 +26,6 @@ jobs:
e2e:
runs-on: ubuntu-latest
services:
transmission:
image: linuxserver/transmission:latest
env:
PUID: "1000"
PGID: "1000"
ports:
- 9091:9091
steps:
- uses: actions/checkout@v4
@@ -81,6 +72,31 @@ jobs:
ln -sf "$HOME/.cargo/bin/alass-cli" "$HOME/.cargo/bin/alass"
}
- name: qBittorrent
# Not a `services:` container: qBittorrent has no environment
# variable for the WebUI password and prints a random one per boot,
# so the only scriptable login is a config seeded before first start,
# which a service container cannot be given.
run: |
mkdir -p /tmp/qbt/config/qBittorrent
cat > /tmp/qbt/config/qBittorrent/qBittorrent.conf <<'CONF'
[Preferences]
WebUI\Port=8080
WebUI\AuthSubnetWhitelistEnabled=true
WebUI\AuthSubnetWhitelist=0.0.0.0/0
WebUI\CSRFProtection=false
WebUI\HostHeaderValidation=false
Downloads\SavePath=/downloads
CONF
docker run -d --name arr-e2e-qbittorrent --network host \
-e PUID=1000 -e PGID=1000 -e WEBUI_PORT=8080 \
-v /tmp/qbt/config:/config \
linuxserver/qbittorrent:latest
for _ in $(seq 1 60); do
curl -sf http://127.0.0.1:8080/api/v2/app/version && break
sleep 2
done
- name: build daemon
# The harness spawns this binary; building it here keeps the compile
# out of the first test's boot window. `translate-command` is the
@@ -89,5 +105,5 @@ jobs:
- name: e2e
env:
TRANSMISSION_RPC_URL: http://transmission:9091/transmission/rpc
QBITTORRENT_URL: http://127.0.0.1:8080
run: cargo nextest run -p arr-e2e
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT g.target_id AS \"season_id!: i64\",\n r.name AS \"name!: String\",\n g.infohash AS \"infohash!: String\",\n g.failed_at,\n g.grabbed_at AS \"grabbed_at!: String\"\n FROM grabs g\n JOIN releases r ON r.id = g.release_id\n JOIN seasons s ON s.id = g.target_id\n WHERE g.target_kind = 'season'\n AND g.state = 'failed'\n AND s.series_id = ?\n AND EXISTS (\n SELECT 1 FROM episodes e\n WHERE e.season_id = s.id AND e.wanted\n AND NOT EXISTS (\n SELECT 1 FROM media_files f\n WHERE f.owner_kind = 'episode' AND f.owner_id = e.id\n )\n )\n -- A later attempt owns the headline while it is still in\n -- flight. Keep the failed grab recorded; it is simply no\n -- longer the current explanation for the gap.\n AND NOT EXISTS (\n SELECT 1 FROM grabs newer\n WHERE newer.target_kind = 'season'\n AND newer.target_id = g.target_id\n -- 'vanished' is not a live attempt either: the\n -- torrent left Transmission, so letting it take the\n -- headline would leave the season saying nothing at\n -- all about a gap that still exists.\n AND newer.state NOT IN ('failed', 'vanished')\n AND (newer.grabbed_at > coalesce(g.failed_at, g.grabbed_at)\n OR (newer.grabbed_at = coalesce(g.failed_at, g.grabbed_at)\n AND newer.id > g.id))\n )\n ORDER BY coalesce(g.failed_at, g.grabbed_at), g.id",
"query": "SELECT g.target_id AS \"season_id!: i64\",\n r.name AS \"name!: String\",\n g.infohash AS \"infohash!: String\",\n g.failed_at,\n g.grabbed_at AS \"grabbed_at!: String\"\n FROM grabs g\n JOIN releases r ON r.id = g.release_id\n JOIN seasons s ON s.id = g.target_id\n WHERE g.target_kind = 'season'\n AND g.state = 'failed'\n AND s.series_id = ?\n AND EXISTS (\n SELECT 1 FROM episodes e\n WHERE e.season_id = s.id AND e.wanted\n AND NOT EXISTS (\n SELECT 1 FROM media_files f\n WHERE f.owner_kind = 'episode' AND f.owner_id = e.id\n )\n )\n -- A later attempt owns the headline while it is still in\n -- flight. Keep the failed grab recorded; it is simply no\n -- longer the current explanation for the gap.\n AND NOT EXISTS (\n SELECT 1 FROM grabs newer\n WHERE newer.target_kind = 'season'\n AND newer.target_id = g.target_id\n -- 'vanished' is not a live attempt either: the\n -- torrent left qBittorrent, so letting it take the\n -- headline would leave the season saying nothing at\n -- all about a gap that still exists.\n AND newer.state NOT IN ('failed', 'vanished')\n AND (newer.grabbed_at > coalesce(g.failed_at, g.grabbed_at)\n OR (newer.grabbed_at = coalesce(g.failed_at, g.grabbed_at)\n AND newer.id > g.id))\n )\n ORDER BY coalesce(g.failed_at, g.grabbed_at), g.id",
"describe": {
"columns": [
{
@@ -70,5 +70,5 @@
false
]
},
"hash": "594d1391c0e9d8a0dded5a354e0dc438bfdbac096ef23b5c3b816afdc03684ae"
"hash": "0cfce0fe064cccb1dceb285a50fa77d990b116a95b987e604913cd8953da36ad"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_files\n (media_file_id, language, origin, provider, candidate_id, engine,\n forced, sdh, synced, sync_rejected, path)\n VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\n ON CONFLICT (path) DO NOTHING\n ON CONFLICT (media_file_id, language, forced, sdh)\n WHERE origin = 'embedded' DO NOTHING\n RETURNING id AS \"id!: i64\"",
"query": "INSERT INTO subtitle_files\n (media_file_id, language, origin, provider, candidate_id, engine,\n forced, sdh, synced, sync_rejected, skeleton_aligned, path)\n VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\n ON CONFLICT (path) DO NOTHING\n ON CONFLICT (media_file_id, language, forced, sdh)\n WHERE origin = 'embedded' DO NOTHING\n RETURNING id AS \"id!: i64\"",
"describe": {
"columns": [
{
@@ -16,11 +16,11 @@
}
],
"parameters": {
"Right": 11
"Right": 12
},
"nullable": [
null
]
},
"hash": "3d48b78768bf8dc2e337d0935889a5654089f38bb7ee2a15b7e19f4ed5540262"
"hash": "266c8e63e7f81cb07921de180f32b47ac8c4e9d47a31af786742404f8e54f517"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "INSERT INTO grabs (release_id, target_kind, target_id, infohash, state)\n VALUES (?, ?, ?, ?, 'sent')\n ON CONFLICT (infohash) DO UPDATE SET\n release_id = excluded.release_id,\n target_kind = excluded.target_kind,\n target_id = excluded.target_id,\n state = 'sent',\n grabbed_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now'),\n imported_at = NULL\n WHERE grabs.state = 'vanished'\n RETURNING id AS \"id!: i64\"",
"query": "INSERT INTO grabs (release_id, target_kind, target_id, infohash, state)\n VALUES (?, ?, ?, ?, 'sent')\n ON CONFLICT (infohash) DO UPDATE SET\n release_id = excluded.release_id,\n target_kind = excluded.target_kind,\n target_id = excluded.target_id,\n state = 'sent',\n grabbed_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now'),\n imported_at = NULL\n WHERE grabs.state = 'vanished'\n OR (grabs.target_kind = 'movie'\n AND NOT EXISTS (SELECT 1 FROM movies WHERE id = grabs.target_id))\n OR (grabs.target_kind = 'season'\n AND NOT EXISTS (SELECT 1 FROM seasons WHERE id = grabs.target_id))\n OR (grabs.target_kind = 'episode'\n AND NOT EXISTS (SELECT 1 FROM episodes WHERE id = grabs.target_id))\n RETURNING id AS \"id!: i64\"",
"describe": {
"columns": [
{
@@ -22,5 +22,5 @@
null
]
},
"hash": "2d0b92ba9aa4bc257286a67f0d37e2182c4478153c9096c1522c5ab23ad472e9"
"hash": "2aeb5f9dab4a62d4cc463a05c0636c379b9c5a1cebee9dfa61df2162f2c64383"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "DELETE FROM grabs WHERE target_kind = 'movie' AND target_id = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 1
},
"nullable": []
},
"hash": "607f337ce8db7eb277dfc569fb1a68e956a38f1db4462cab02fe26c8c9912a81"
}
@@ -0,0 +1,26 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\" FROM media_files WHERE path = ?",
"describe": {
"columns": [
{
"name": "id!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "media_files",
"name": "id"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "8651688f9f999629b4996c0451f87f5a84fabe748849b4b80063af1c18f73917"
}
@@ -1,6 +1,6 @@
{
"db_name": "SQLite",
"query": "SELECT id AS \"id!: i64\",\n media_file_id AS \"media_file_id!: i64\",\n language AS \"language!: String\",\n origin AS \"origin!: SubtitleOrigin\",\n provider,\n candidate_id,\n engine,\n forced AS \"forced!: bool\",\n sdh AS \"sdh!: bool\",\n synced AS \"synced!: bool\",\n sync_rejected AS \"sync_rejected!: bool\",\n path\n FROM subtitle_files\n WHERE media_file_id = ?\n ORDER BY language, id",
"query": "SELECT id AS \"id!: i64\",\n media_file_id AS \"media_file_id!: i64\",\n language AS \"language!: String\",\n origin AS \"origin!: SubtitleOrigin\",\n provider,\n candidate_id,\n engine,\n forced AS \"forced!: bool\",\n sdh AS \"sdh!: bool\",\n synced AS \"synced!: bool\",\n sync_rejected AS \"sync_rejected!: bool\",\n skeleton_aligned AS \"skeleton_aligned!: bool\",\n path\n FROM subtitle_files\n WHERE media_file_id = ?\n ORDER BY language, id",
"describe": {
"columns": [
{
@@ -125,8 +125,19 @@
}
},
{
"name": "path",
"name": "skeleton_aligned!: bool",
"ordinal": 11,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_files",
"name": "skeleton_aligned"
}
}
},
{
"name": "path",
"ordinal": 12,
"type_info": "Text",
"origin": {
"Table": {
@@ -151,8 +162,9 @@
false,
false,
false,
false,
true
]
},
"hash": "62d625c322c8c64b48317cdbf466d74043708d8e12997e0efec23a18eaaaaace"
"hash": "8abb1e992e0972a19e76c7788c8c350951a1d14766088b1cf3fb886fb658d575"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "INSERT INTO subtitle_skeletons (media_file_id, stream_index, language, cues)\n VALUES (?, ?, ?, ?)\n ON CONFLICT (media_file_id, stream_index) DO UPDATE SET\n language = excluded.language,\n cues = excluded.cues",
"describe": {
"columns": [],
"parameters": {
"Right": 4
},
"nullable": []
},
"hash": "8e3c23ed488bc82ff3dd4f6da02d00435525d321b6002b9f31979a39281369ce"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "DELETE FROM grabs\n WHERE (target_kind = 'season'\n AND target_id IN (SELECT id FROM seasons WHERE series_id = ?))\n OR (target_kind = 'episode'\n AND target_id IN (SELECT e.id FROM episodes e\n JOIN seasons s ON s.id = e.season_id\n WHERE s.series_id = ?))",
"describe": {
"columns": [],
"parameters": {
"Right": 2
},
"nullable": []
},
"hash": "e6fbef670552928dfeeb6f6d442021340acb438a1ed6a7b917be91c9d368cb57"
}
@@ -0,0 +1,50 @@
{
"db_name": "SQLite",
"query": "SELECT stream_index AS \"stream_index!: i64\",\n language AS \"language!: String\",\n cues AS \"cues!: String\"\n FROM subtitle_skeletons\n WHERE media_file_id = ?\n ORDER BY stream_index",
"describe": {
"columns": [
{
"name": "stream_index!: i64",
"ordinal": 0,
"type_info": "Integer",
"origin": {
"Table": {
"table": "subtitle_skeletons",
"name": "stream_index"
}
}
},
{
"name": "language!: String",
"ordinal": 1,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_skeletons",
"name": "language"
}
}
},
{
"name": "cues!: String",
"ordinal": 2,
"type_info": "Text",
"origin": {
"Table": {
"table": "subtitle_skeletons",
"name": "cues"
}
}
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false
]
},
"hash": "eb7df52055a60a00b721e3eef152e1b04a8512a85e8ad3618545c634c23abf31"
}
+4 -3
View File
@@ -16,7 +16,7 @@ crates/
├── arr-parse/ release name parsing (no IO)
├── arr-meta/ TMDB client
├── arr-indexer/ Torznab via Prowlarr
├── arr-dl/ Transmission RPC
├── arr-dl/ qBittorrent WebUI API
├── arr-probe/ ffprobe wrapper
├── arr-subs/ subtitle providers, translation, sync
├── arr-db/ sqlx + migrations
@@ -244,8 +244,9 @@ mid-issue.
- **Prowlarr** — `prowlarr` container on the Dokploy host, port 9696. Owns
tracker auth, FlareSolverr and the Cardigann definitions. Not replaced.
- **Transmission** — native in LXC 130 at `10.6.10.45:9091`, RPC
unauthenticated. Download dir `/mnt/media/transmission/complete`.
- **qBittorrent** — `qbittorrent.n62.casa`, WebUI API v2, login required
(`ARR_QBITTORRENT_USERNAME` / `ARR_QBITTORRENT_PASSWORD`). Download dir
`/mnt/media/qbittorrent/complete`.
- **Jellyfin** — native in LXC at `10.6.10.18:8096`. Library roots under
`/mnt/media-v2`.
- **Media** — ZFS, single dataset, bind-mounted as `/mnt/media`. Downloads and
Generated
+16 -3
View File
@@ -93,7 +93,6 @@ dependencies = [
"arr-probe",
"arr-subs",
"axum",
"base64",
"chrono",
"include_dir",
"mime_guess",
@@ -123,16 +122,17 @@ dependencies = [
"tempfile",
"thiserror",
"tokio",
"tracing",
]
[[package]]
name = "arr-dl"
version = "0.1.0"
dependencies = [
"base64",
"reqwest",
"serde",
"serde_json",
"sha1 0.10.7",
"thiserror",
"tokio",
"url",
@@ -1543,6 +1543,7 @@ dependencies = [
"base64",
"bytes",
"futures-core",
"futures-util",
"http",
"http-body",
"http-body-util",
@@ -1551,6 +1552,7 @@ dependencies = [
"hyper-util",
"js-sys",
"log",
"mime_guess",
"percent-encoding",
"pin-project-lite",
"quinn",
@@ -1733,6 +1735,17 @@ dependencies = [
"serde",
]
[[package]]
name = "sha1"
version = "0.10.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a978451301f4db1d02937a4ab3ccce137717b81826e79b7d49ffe3244a13c3b8"
dependencies = [
"cfg-if",
"cpufeatures 0.2.17",
"digest 0.10.7",
]
[[package]]
name = "sha1"
version = "0.11.0"
@@ -1932,7 +1945,7 @@ dependencies = [
"log",
"percent-encoding",
"serde",
"sha1",
"sha1 0.11.0",
"sha2 0.11.0",
"sqlx-core",
"thiserror",
+2 -2
View File
@@ -26,8 +26,7 @@ arr-subs = { path = "crates/arr-subs" }
axum = "0.8"
tokio = { version = "1", features = ["macros", "rt-multi-thread", "sync", "time", "net", "io-util", "fs", "signal", "process"] }
tower-http = { version = "0.6", features = ["trace"] }
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
base64 = "0.22"
reqwest = { version = "0.12", default-features = false, features = ["json", "multipart", "rustls-tls"] }
# Persistence
sqlx = { version = "0.9", default-features = false, features = ["runtime-tokio", "sqlite", "macros", "migrate", "chrono", "json"] }
@@ -52,6 +51,7 @@ chardetng = "1"
chrono = { version = "0.4", default-features = false, features = ["clock", "serde"] }
clap = { version = "4.5", features = ["derive", "env"] }
encoding_rs = "0.8"
sha1 = "0.10"
thiserror = "2"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
+86 -24
View File
@@ -27,7 +27,7 @@ allowed to be narrow.
Explicitly out of scope, permanently unless stated:
- **Authentication.** The perimeter is a VPN plus Authelia at the proxy. The
service binds without auth, same trust model as the existing Transmission RPC.
service binds without auth, same trust model as the rest of the stack.
- **Library migration or filesystem scan.** The service knows only what it put
on disk. Adopting the pre-existing library, if ever wanted, is a one-off
script against both APIs, not a feature.
@@ -46,7 +46,7 @@ Explicitly out of scope, permanently unless stated:
```
┌──────────────┐
TMDB ──────▶│ │
│ arr │────▶ Transmission RPC 10.6.10.45:9091
│ arr │────▶ qBittorrent WebUI qbittorrent.n62.casa
Prowlarr ──────▶│ │
(Torznab) │ (this) │────▶ ffprobe local subprocess
│ │
@@ -61,8 +61,9 @@ Everything already exists except `arr`. Prowlarr keeps owning tracker auth,
Cloudflare bypass via FlareSolverr, rate limiting and the Cardigann
definitions — replacing it buys nothing.
Transmission runs natively in its own LXC (VMID 130, `10.6.10.45`), RPC
unauthenticated, download dir `/mnt/media/transmission/complete`.
qBittorrent is reached at `qbittorrent.n62.casa`, WebUI API v2, download dir
`/mnt/media/qbittorrent/complete`. Its WebUI requires a login, so arr carries
credentials — the one upstream that does.
## 4. Domain model
@@ -517,11 +518,20 @@ leaving RSS matching on.
## 7. Download and import
### 7.1 Transmission
### 7.1 qBittorrent
RPC at `10.6.10.45:9091`, no credentials. Labels are `movies-main`, `tv-kids`
and so on — enough to find things in Transmission's own UI, and distinct from
Radarr's existing `radarr`/`sonarr` labels so both stacks can run side by side.
WebUI API v2 at `qbittorrent.n62.casa`. Labels are qBittorrent **tags**
`movies-main`, `tv-kids` and so on — enough to find things in qBittorrent's own
UI, and distinct from Radarr's existing `radarr`/`sonarr` labels so both stacks
can run side by side. Tags rather than a category because a category also
governs the save path, and arr owns that.
`torrents/add` answers `Ok.` and nothing else: no hash, no name, and no signal
that the torrent was already there. arr therefore derives the v1 infohash from
the magnet or the `.torrent` bytes before the call, and looks the torrent up by
it. That is also what makes an add idempotent across a restart (§8) — the same
release resolves to the same hash, and a second add is recognised as the
duplicate it is.
### 7.2 Hardlink
@@ -540,9 +550,16 @@ at 4K is 40-80 GB per title.
The torrent and the library entry are separate state machines.
Seeding obligation is **per tracker**, configured locally because Prowlarr does
not expose tracker rules — `ratio` and `min_seed_time`. Set `seedRatioLimit`
and `seedIdleLimit` on the torrent at add time and let Transmission enforce
them. A reaper deletes torrents Transmission reports as done seeding.
not expose tracker rules — `ratio` and `min_seed_time`. Set `ratioLimit` and
`inactiveSeedingTimeLimit` on the torrent at add time, with
`shareLimitAction` set to stop rather than delete, and let qBittorrent enforce
them. A reaper deletes torrents qBittorrent stopped on a limit arr set.
Stopped-on-a-limit, not merely stopped: a torrent the operator paused by hand
looks identical in the state field alone, and the reaper deletes data. So the
ratio and idle limits are checked against the torrent's own counters, and a
torrent stopped by a *global* limit reads as still seeding — the safe direction
to be wrong in.
Consequently a hard-failed release is blacklisted and never imported, but its
torrent keeps seeding until the obligation clears. Nothing is deleted early to
@@ -599,11 +616,11 @@ state and act on the gap.
This is idempotent and crash-safe by construction. Kill the process mid-grab and
the next tick recomputes the same gap and continues. A job table would need
retry counts, dead-lettering and reconciliation against Transmission anyway,
because Transmission is an external system that changes underneath the app.
retry counts, dead-lettering and reconciliation against qBittorrent anyway,
because qBittorrent is an external system that changes underneath the app.
Transient state — a search in flight, download progress — is in memory and
rebuilt from Transmission on startup. Where that state is ever seen is §9.8:
rebuilt from qBittorrent on startup. Where that state is ever seen is §9.8:
inline on the row that owns the item, never persisted.
Ticks are staggered: reconcile every 30 s, RSS every 10 min, metadata refresh
@@ -688,7 +705,7 @@ notifying on everything and being muted within a week.
- **Imported** → to the title's owners. The only good-news notification.
- **Needs a decision** → to the operator alone. Entered the no-PT-source queue,
or the needs-a-decision queue (§5.7).
- **Broken** → to the operator alone. Prowlarr, Transmission or TMDB
- **Broken** → to the operator alone. Prowlarr, qBittorrent or TMDB
unreachable, disk full.
Not notified: grabs, searches, downloads starting or finishing, soft fails.
@@ -742,7 +759,7 @@ inside the app, review text.
opens on.
**The signal chain is a settings section.** The four upstreams — tmdb,
prowlarr, arr, transmission — and their lamps live inside `/settings`, and have
prowlarr, arr, qbittorrent — and their lamps live inside `/settings`, and have
no route of their own. Per-upstream health is something you check when
something is wrong, not a homepage.
@@ -770,10 +787,10 @@ Alongside progress the same row carries the other states a torrent can be in:
**seeding** under §7.3's obligation, **stalled**, **errored**.
**A torrent arr did not grab is never shown.** §2 already rules the service
knows only what it put on disk; Transmission's own UI lists the rest.
knows only what it put on disk; qBittorrent's own UI lists the rest.
Nothing is persisted — no progress column, no new table. The snapshot comes
from Transmission and dies with the process (§8), refreshed by the UI's normal
from qBittorrent and dies with the process (§8), refreshed by the UI's normal
polling cadence at roughly 15 s rather than SSE.
At phone width only active-grab progress survives; seeding and stalled shed
@@ -791,8 +808,10 @@ service.
Policy lives in the database, not a config file — size targets and DV rules get
tuned by hand during testing and a restart-to-reload loop gets old immediately.
Only bootstrap settings (bind address, Prowlarr URL, Transmission URL, TMDB key,
media root, TMDB response cache directory) come from config/env.
Only bootstrap settings (bind address, Prowlarr URL, qBittorrent URL and
login, TMDB key, media root, TMDB response cache directory) come from
config/env. The qBittorrent password is a secret, so it is env-only and has no
config-file field.
Backup is `sqlite3 .backup` on a timer.
@@ -806,7 +825,7 @@ arr-core domain types, policy engine, scoring no IO, no heavy deps
arr-parse release name parsing no IO
arr-meta TMDB client
arr-indexer Torznab via Prowlarr
arr-dl Transmission RPC
arr-dl qBittorrent WebUI API
arr-probe ffprobe wrapper
arr-subs subtitle providers, translation, sync
arr-db sqlx + migrations
@@ -863,8 +882,10 @@ and `target/`, keyed on `Cargo.lock` plus `rust-toolchain.toml`. Never build
- Prowlarr and TMDB — `wiremock` with recorded real responses as fixtures.
Never live: trackers rate-limit, and it would leak credentials into CI.
- Transmission — a real container as a CI service. RPC semantics are the most
likely source of surprise and it starts in a second.
- qBittorrent — a real container, started with a seeded config because the
image prints a random WebUI password per boot. Its API semantics are the most
likely source of surprise: `torrents/add` reports nothing back, and required
parameters have changed between major versions.
- `ffprobe` — tiny committed clips, a few KB each. The Jellyfin LXC already has
the right fixtures at `/srv/jellyfin-test`, including a real **DV Profile 5**
clip. That is the test proving the policy engine rejects Profile 5 and accepts
@@ -919,7 +940,9 @@ Replaces Bazarr. Phase 9 in §13; the `arr-subs` crate in §11.
**Wanted set.** Global, not per root. Two languages are separately wanted for
every media file: Portuguese — pt-PT preferred, pt-BR accepted — and English.
A file is satisfied for a language when a subtitle in it exists, embedded or as
a sidecar. This is deliberately unlike §5.2's audio rules, which attach to a
a sidecar. Satisfying a language is a statement about *viewing* it, and not
about having text in it — the distinction matters where an image track is all
there is, and **Translation** below is where it bites. This is deliberately unlike §5.2's audio rules, which attach to a
root: subtitles carry no blacklist there and none here. pt-BR subtitles are
always fine.
@@ -931,6 +954,22 @@ they satisfy viewing but can never feed a translator, and arr does not OCR
them. `arr-probe` already reports subtitle tracks with resolved languages; the
format is the new fact it must carry.
**Cue skeletons.** What an image track *does* have is exact timings, because
its packet timestamps are the disc's own cue structure. At import — while the
file is being read and hardlinked anyway — arr derives a **cue skeleton** from
each non-forced image track: `ffprobe -show_packets` on the track, packets
paired show-to-clear, no pixel read. The skeleton has timings and no text, and
that is enough, because `alass` matches on interval structure rather than on
words. A sparse skeleton is still a strong reference; a downloaded subtitle and
a retail disc's track never carry the same cues anyway.
The pairing is the whole of it, so it is validated before it is trusted: an
even packet count, every implied duration plausible, and a cue density that
fits the runtime. PGS permits several composition segments per subtitle, and a
track built that way pairs into something that is quietly half a second out —
worse than no skeleton. A track that fails validation gets none, and alignment
falls back to the video.
**Providers.** OpenSubtitles.com, behind one trait.
**Ranking.** A `moviehash` match wins outright. Then an exact release-name
@@ -954,6 +993,18 @@ a downloaded one, or one extracted from a text-format embedded track. Being
able to translate from an embedded track is a deliberate improvement on
Bazarr, which cannot.
**Fetching a source.** A release whose only subtitle is an image track has no
text source and never will: the track satisfies its language, so nothing is
ever fetched in it, and the track itself cannot be translated from. That is a
deadlock, and it is broken by a narrow carve-out — when a wanted language needs
translating and no text source exists, arr fetches a text subtitle in the image
track's language *even though that language reads as satisfied*. The fetch
obtains a source; it settles no want of its own, and the no-upgrade rule below
does not apply to it. Where that track has a cue skeleton, the fetched subtitle
is aligned against the skeleton rather than the video, which puts it on the
disc's own timings instead of on a heuristic read of the audio. OCR remains a
non-goal: this reaches the same place with real text.
**Translation backends.** Pluggable, each behind its own cargo feature: an
OpenAI-compatible HTTP endpoint, DeepL, Google Translate, and a generic remote
command driven by a configured template (`ssh box claude -p` is one instance
@@ -967,6 +1018,11 @@ too — arr stops working on it. A real subtitle appearing later does not
replace anything. Replacement is a manual action from the UI. This is §5.4's
rule applied to subtitles.
The one exception is the source fetch above. It is not an upgrade: the language
it downloads is already satisfied and stays satisfied by the same track it was
before, and what the download settles is a *different* language's gap. Nothing
is replaced, so nothing about the rule changes.
**On disk.** Sidecars live next to the video inside the §7.4 title folder,
named `<video basename>.<lang>.srt`, e.g.
`… - [2160p][WEB-DL][HDR10].pt-PT.srt`. A machine translation carries an extra
@@ -985,6 +1041,12 @@ reports no confidence value, so its output is accepted unless it is
implausible — a shift beyond 60 seconds, or cues lost — in which case the
unsynced original is kept and the file is flagged.
The reference is the video, except where a cue skeleton exists, and then it is
the skeleton. A subtitle a skeleton accepted is already on disc-exact timings,
and translation copies those timings over untouched, so the pass that would
otherwise run on the translation is skipped: a second alignment has nothing
left to find and can only move them off.
**Configuration.** Provider credentials and translator API keys are bootstrap
config or environment, per §10 — a secret never becomes a database row. Wanted
languages, chosen engine, per-provider enable and the daily budgets are
+25 -7
View File
@@ -8,17 +8,35 @@ RUN pnpm -C web install --frozen-lockfile --config.dangerouslyAllowAllBuilds=tru
COPY web ./web
RUN pnpm -C web build
FROM rust:1-bookworm AS build
# Every translation backend is a default-off cargo feature (DESIGN.md §15), so
# a build without these ships an image that cannot translate at all. All four
# go in: which one is in use is a database setting, and the point of the
# feature switches is the build, not the deployment.
#
# The dependency build and the workspace build are split with cargo-chef so a
# source-only change reuses the dependency layer. The two `cargo` invocations
# must carry identical flags or the cook output is not reusable.
FROM lukemathwalker/cargo-chef:latest-rust-1-bookworm AS chef
WORKDIR /src
ENV ARR_FEATURES=translate-openai,translate-deepl,translate-google,translate-command
FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json
FROM chef AS build
COPY --from=planner /src/recipe.json recipe.json
# cargo-chef's skeleton keeps build scripts, and arr-daemon's panics in release
# when it can find no SPA bundle. A stub satisfies it; the real bundle arrives
# below and the changed ARR_WEB_DIST reruns only that one build script.
RUN mkdir -p /stub/dist && printf '<!doctype html>stub\n' > /stub/dist/index.html
ENV ARR_WEB_DIST=/stub/dist
RUN cargo chef cook --release -p arr-daemon --features "$ARR_FEATURES" \
--recipe-path recipe.json
COPY . .
COPY --from=web /src/web/dist ./web/dist
ENV ARR_WEB_DIST=/src/web/dist
# Every translation backend is a default-off cargo feature (DESIGN.md §15),
# so a build without this line ships an image that cannot translate at all.
# All four go in: which one is in use is a database setting, and the point of
# the feature switches is the build, not the deployment.
RUN cargo build --release -p arr-daemon \
--features translate-openai,translate-deepl,translate-google,translate-command
RUN cargo build --release -p arr-daemon --features "$ARR_FEATURES"
# §15 runs `alass` over every fetched and every translated subtitle, and it is
# not in Debian — a release binary is the only way in. Its own stage so the
+26 -8
View File
@@ -31,20 +31,38 @@ deps:
test:
cargo nextest run --workspace --exclude arr-e2e --all-features
# End-to-end tests. Needs a Transmission container (`just e2e-up`); not part
# of the push gate. Never point this at the production Transmission LXC.
# End-to-end tests. Needs a qBittorrent container (`just e2e-up`); not part
# of the push gate. Never point this at the production qBittorrent.
e2e:
cargo nextest run -p arr-e2e
# Throwaway local Transmission for `just e2e`, mirroring the CI service.
# Throwaway local qBittorrent for `just e2e`, mirroring the CI container.
#
# The config is seeded before first start because qBittorrent prints a random
# WebUI password per boot and has no environment variable to set one; the
# subnet whitelist is what lets the tests talk to it without credentials.
e2e-up:
docker run --rm -d --name arr-e2e-transmission \
-e PUID=1000 -e PGID=1000 -p 9091:9091 \
linuxserver/transmission:latest
#!/usr/bin/env bash
set -euo pipefail
mkdir -p /tmp/arr-e2e-qbt/config/qBittorrent
cat > /tmp/arr-e2e-qbt/config/qBittorrent/qBittorrent.conf <<'CONF'
[Preferences]
WebUI\Port=8080
WebUI\AuthSubnetWhitelistEnabled=true
WebUI\AuthSubnetWhitelist=0.0.0.0/0
WebUI\CSRFProtection=false
WebUI\HostHeaderValidation=false
Downloads\SavePath=/downloads
CONF
docker run --rm -d --name arr-e2e-qbittorrent \
-e PUID=1000 -e PGID=1000 -e WEBUI_PORT=8080 -p 8080:8080 \
-v /tmp/arr-e2e-qbt/config:/config \
linuxserver/qbittorrent:latest
until curl -sf http://127.0.0.1:8080/api/v2/app/version >/dev/null; do sleep 1; done
# Stop and discard the local Transmission container.
# Stop and discard the local qBittorrent container.
e2e-down:
docker stop arr-e2e-transmission
docker stop arr-e2e-qbittorrent
# Frontend lint and typecheck. No-op until the SPA exists (issue #6). Probes
# for package.json, not the directory: `just gen-client` writes into web/ and
+1 -1
View File
@@ -43,7 +43,7 @@ manual search buries the decision under the release name column.
- Self-hosted behind VPN + Authelia; the app itself has no auth layer.
- API-first: the SPA is one client of the OpenAPI-described HTTP API, no
privileged path.
- Coexists with Prowlarr (indexers), Transmission (downloads), Jellyfin
- Coexists with Prowlarr (indexers), qBittorrent (downloads), Jellyfin
(playback), Jellyseerr (requests), ntfy (notifications).
## Capabilities and Constraints
+1 -1
View File
@@ -12,7 +12,7 @@ This is that loop, once, in Rust.
- **Prowlarr** stays — it owns tracker auth, Cloudflare bypass and the indexer
definitions, and replacing it buys nothing.
- **Transmission** stays.
- **qBittorrent** stays.
- **Jellyseerr** stays, talking to a thin Radarr-compatible shim.
## Status
+16 -30
View File
@@ -42,14 +42,12 @@ pub struct Download {
tag = "system",
responses(
(status = 200, description = "Live downloads started by arr", body = [Download]),
(status = 503, description = "Transmission or database unavailable", body = crate::movies::ErrorBody)
(status = 503, description = "qBittorrent or database unavailable", body = crate::movies::ErrorBody)
)
)]
pub(crate) async fn list(State(state): State<AppState>) -> Result<Json<Vec<Download>>, ApiError> {
let database = state.database().ok_or(ApiError::Unavailable)?;
let transmission = state
.transmission()
.ok_or(ApiError::Upstream("transmission"))?;
let qbit = state.qbit().ok_or(ApiError::Upstream("qbit"))?;
let torrents = {
let mut cache = state.download_snapshot().write().await;
@@ -57,18 +55,18 @@ pub(crate) async fn list(State(state): State<AppState>) -> Result<Json<Vec<Downl
if at.elapsed() < SNAPSHOT_TTL {
torrents.clone()
} else {
let torrents = transmission
let torrents = qbit
.list_torrents()
.await
.map_err(|_| ApiError::Upstream("transmission"))?;
.map_err(|_| ApiError::Upstream("qbit"))?;
*cache = Some((Instant::now(), torrents.clone()));
torrents
}
} else {
let torrents = transmission
let torrents = qbit
.list_torrents()
.await
.map_err(|_| ApiError::Upstream("transmission"))?;
.map_err(|_| ApiError::Upstream("qbit"))?;
*cache = Some((Instant::now(), torrents.clone()));
torrents
}
@@ -116,7 +114,7 @@ fn phase(torrent: &Torrent) -> DownloadPhase {
} else {
match torrent.state {
TorrentState::Seeding => DownloadPhase::Seeding,
// Transmission's own verdict, not a zero rate: a torrent between
// qBittorrent's own verdict, not a zero rate: a torrent between
// peers reads zero for a poll or two and finishes fine, and
// §9.8's stalled chip is for the case that never does.
TorrentState::Downloading if torrent.is_stalled => DownloadPhase::Stalled,
@@ -133,7 +131,7 @@ mod tests {
use arr_db::Db;
use axum::extract::State;
use serde_json::json;
use wiremock::matchers::{body_partial_json, method};
use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};
use super::{list, phase, DownloadPhase};
@@ -143,7 +141,6 @@ mod tests {
fn torrent_states_become_the_download_phases() {
let torrent =
|state: arr_dl::TorrentState, rate: u64, error: Option<&str>| arr_dl::Torrent {
id: 1,
name: "name".into(),
hash: "hash".into(),
state,
@@ -186,24 +183,13 @@ mod tests {
#[tokio::test]
async fn endpoint_joins_only_arr_grabs_and_reuses_the_snapshot() {
let server = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(
ResponseTemplate::new(409).insert_header("x-transmission-session-id", "session"),
)
.up_to_n_times(1)
.mount(&server)
.await;
Mock::given(method("POST"))
.and(body_partial_json(json!({"method": "torrent-get"})))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"result": "success",
"arguments": {"torrents": [{
"id": 1, "name": "grabbed", "hashString": "ABC",
"status": 4, "percentDone": 0.4, "rateDownload": 123,
"eta": 60, "errorString": "", "downloadDir": "/downloads",
"labels": [], "isFinished": false
}]}
})))
Mock::given(method("GET"))
.and(path("/api/v2/torrents/info"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!([{
"hash": "ABC", "name": "grabbed", "state": "downloading",
"progress": 0.4, "dlspeed": 123, "eta": 60,
"save_path": "/downloads", "tags": ""
}])))
.expect(1)
.mount(&server)
.await;
@@ -231,7 +217,7 @@ mod tests {
let state = AppState::new(Upstreams::new("unused".into(), server.uri()))
.expect("state")
.with_database(database)
.with_transmission(arr_dl::TransmissionClient::new(&server.uri()).expect("client"));
.with_qbit(arr_dl::QbitClient::new(&server.uri()).expect("client"));
let first = list(State(state.clone())).await.expect("first").0;
let second = list(State(state)).await.expect("second").0;
assert_eq!(first.len(), 1);
+22 -23
View File
@@ -1,7 +1,7 @@
//! `GET /api/health` — is each of the three upstreams answering.
//!
//! The three probed here are the ones DESIGN.md §9.5 calls "Broken": without
//! Prowlarr nothing is found, without Transmission nothing is fetched, and
//! Prowlarr nothing is found, without qBittorrent nothing is fetched, and
//! without TMDB nothing is identified. The endpoint always answers `200` —
//! the body carries the verdict, so a degraded service can still explain
//! itself to the UI instead of looking like a fourth outage.
@@ -118,12 +118,12 @@ pub struct HealthReport {
#[schema(example = "0.1.0")]
pub version: String,
pub prowlarr: Check,
pub transmission: Check,
pub qbit: Check,
pub tmdb: Check,
pub subtitles: SubtitleHealth,
}
/// Report reachability of Prowlarr, Transmission and TMDB, plus the
/// Report reachability of Prowlarr, qBittorrent and TMDB, plus the
/// subtitle upstreams (#200).
#[utoipa::path(
get,
@@ -136,14 +136,14 @@ pub struct HealthReport {
pub async fn health(State(state): State<AppState>) -> Json<HealthReport> {
// Independent network probes; serialising them would make the endpoint
// as slow as the sum of the timeouts.
let (prowlarr, transmission, tmdb, subtitles) = tokio::join!(
let (prowlarr, qbit, tmdb, subtitles) = tokio::join!(
probe_prowlarr(&state),
probe_transmission(&state),
probe_qbit(&state),
probe_tmdb(&state),
probe_subtitles(&state)
);
let status = if [prowlarr.status, transmission.status, tmdb.status]
let status = if [prowlarr.status, qbit.status, tmdb.status]
.into_iter()
.chain(subtitles.statuses())
.all(|check| check == Status::Ok)
@@ -157,7 +157,7 @@ pub async fn health(State(state): State<AppState>) -> Json<HealthReport> {
status,
version: env!("CARGO_PKG_VERSION").to_string(),
prowlarr,
transmission,
qbit,
tmdb,
subtitles,
})
@@ -181,22 +181,21 @@ async fn probe_prowlarr(state: &AppState) -> Check {
}
}
/// Transmission answers an RPC call without a session id with `409` plus the
/// id to retry with. That is a live daemon, so it counts as reachable.
async fn probe_transmission(state: &AppState) -> Check {
let request = state
.http()
.post(&state.upstreams().transmission_url)
.json(&serde_json::json!({ "method": "session-get" }));
match request.send().await {
Ok(response)
if response.status().is_success()
|| response.status() == reqwest::StatusCode::CONFLICT =>
{
Check::ok()
}
Ok(response) => Check::unreachable(format!("http {}", response.status().as_u16())),
Err(err) => Check::unreachable(describe(err)),
/// The probe is a real torrent list through the configured client, not a bare
/// reachability check.
///
/// A `WebUI` that wants credentials answers `403` to anything unauthenticated,
/// so "the port is open" is true of an instance arr cannot use at all. Reading
/// that as healthy once left the lamp green while every grab and every reaper
/// tick failed on a rejected password, which is the one thing this lamp exists
/// to catch.
async fn probe_qbit(state: &AppState) -> Check {
let Some(qbit) = state.qbit() else {
return Check::unconfigured("no qBittorrent client is configured");
};
match qbit.list_torrents().await {
Ok(_) => Check::ok(),
Err(error) => Check::unreachable(error.to_string()),
}
}
+31 -45
View File
@@ -177,8 +177,8 @@ mod tests {
use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};
/// A Prowlarr that answers `/ping`, and a Transmission that answers an
/// RPC call the way a real one does when it has no session id yet.
/// A Prowlarr that answers `/ping`, and a qBittorrent that answers the
/// version call the way a real one does for an unauthenticated caller.
async fn upstreams_up() -> (MockServer, MockServer) {
let prowlarr = MockServer::start().await;
Mock::given(method("GET"))
@@ -189,16 +189,14 @@ mod tests {
.mount(&prowlarr)
.await;
let transmission = MockServer::start().await;
Mock::given(method("POST"))
.and(path("/transmission/rpc"))
.respond_with(
ResponseTemplate::new(409).insert_header("X-Transmission-Session-Id", "abc"),
)
.mount(&transmission)
let qbit = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/api/v2/torrents/info"))
.respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!([])))
.mount(&qbit)
.await;
(prowlarr, transmission)
(prowlarr, qbit)
}
/// Serve the app on an ephemeral port and return its base URL. The server
@@ -225,7 +223,7 @@ mod tests {
#[tokio::test]
async fn all_upstreams_up_is_ok() {
let (prowlarr, transmission) = upstreams_up().await;
let (prowlarr, qbit) = upstreams_up().await;
let tmdb = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/configuration"))
@@ -234,14 +232,12 @@ mod tests {
.await;
let state = AppState::new(
Upstreams::new(
prowlarr.uri(),
format!("{}/transmission/rpc", transmission.uri()),
)
.with_tmdb_url(tmdb.uri())
.with_tmdb_api_key(Some("key".into())),
Upstreams::new(prowlarr.uri(), qbit.uri())
.with_tmdb_url(tmdb.uri())
.with_tmdb_api_key(Some("key".into())),
)
.expect("state")
.with_qbit(arr_dl::QbitClient::new(&qbit.uri()).expect("client"))
// The subtitle binaries default to `PATH`; pin them to something
// every machine has so this test stays about the classic upstreams.
.with_syncer(arr_subs::Syncer::new().with_binary("sh"))
@@ -250,19 +246,17 @@ mod tests {
let body = report(state).await;
assert_eq!(body["status"], "ok");
assert_eq!(body["prowlarr"]["status"], "ok");
assert_eq!(body["transmission"]["status"], "ok");
assert_eq!(body["qbit"]["status"], "ok");
assert_eq!(body["tmdb"]["status"], "ok");
assert_eq!(body["version"], env!("CARGO_PKG_VERSION"));
}
#[tokio::test]
async fn a_missing_tmdb_key_is_unconfigured_not_an_outage() {
let (prowlarr, transmission) = upstreams_up().await;
let state = AppState::new(Upstreams::new(
prowlarr.uri(),
format!("{}/transmission/rpc", transmission.uri()),
))
.expect("state");
let (prowlarr, qbit) = upstreams_up().await;
let state = AppState::new(Upstreams::new(prowlarr.uri(), qbit.uri()))
.expect("state")
.with_qbit(arr_dl::QbitClient::new(&qbit.uri()).expect("client"));
let body = report(state).await;
assert_eq!(body["tmdb"]["status"], "unconfigured");
@@ -271,20 +265,18 @@ mod tests {
#[tokio::test]
async fn an_unreachable_upstream_degrades_the_service() {
let (_prowlarr, transmission) = upstreams_up().await;
let (_prowlarr, qbit) = upstreams_up().await;
// Port 1 is privileged and nothing binds it, so the probe gets a
// refused connection immediately instead of waiting out the timeout.
let state = AppState::new(Upstreams::new(
"http://127.0.0.1:1".into(),
format!("{}/transmission/rpc", transmission.uri()),
))
.expect("state");
let state = AppState::new(Upstreams::new("http://127.0.0.1:1".into(), qbit.uri()))
.expect("state")
.with_qbit(arr_dl::QbitClient::new(&qbit.uri()).expect("client"));
let body = report(state).await;
assert_eq!(body["status"], "degraded");
assert_eq!(body["prowlarr"]["status"], "unreachable");
assert_eq!(body["transmission"]["status"], "ok");
assert_eq!(body["qbit"]["status"], "ok");
}
#[tokio::test]
@@ -295,13 +287,9 @@ mod tests {
.respond_with(ResponseTemplate::new(500))
.mount(&prowlarr)
.await;
let transmission = MockServer::start().await;
let qbit = MockServer::start().await;
let state = AppState::new(Upstreams::new(
prowlarr.uri(),
format!("{}/transmission/rpc", transmission.uri()),
))
.expect("state");
let state = AppState::new(Upstreams::new(prowlarr.uri(), qbit.uri())).expect("state");
let body = report(state).await;
assert_eq!(body["prowlarr"]["status"], "unreachable");
@@ -310,7 +298,7 @@ mod tests {
#[tokio::test]
async fn a_failed_tmdb_probe_never_echoes_the_api_key() {
let (prowlarr, transmission) = upstreams_up().await;
let (prowlarr, qbit) = upstreams_up().await;
let tmdb = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/configuration"))
@@ -319,14 +307,12 @@ mod tests {
.await;
let state = AppState::new(
Upstreams::new(
prowlarr.uri(),
format!("{}/transmission/rpc", transmission.uri()),
)
.with_tmdb_url(tmdb.uri())
.with_tmdb_api_key(Some("super-secret".into())),
Upstreams::new(prowlarr.uri(), qbit.uri())
.with_tmdb_url(tmdb.uri())
.with_tmdb_api_key(Some("super-secret".into())),
)
.expect("state");
.expect("state")
.with_qbit(arr_dl::QbitClient::new(&qbit.uri()).expect("client"));
let body = report(state).await;
assert_eq!(body["tmdb"]["status"], "unreachable");
+48
View File
@@ -596,6 +596,14 @@ pub async fn delete(
)
.execute(pool(&state)?)
.await?;
// `grabs.infohash` is unique, so an orphan left here would block the row
// a later grab of the same release needs to write.
sqlx::query!(
"DELETE FROM grabs WHERE target_kind = 'movie' AND target_id = ?",
id
)
.execute(pool(&state)?)
.await?;
let result = sqlx::query!("DELETE FROM movies WHERE id = ?", id)
.execute(pool(&state)?)
.await?;
@@ -1632,6 +1640,46 @@ mod tests {
assert_eq!(orphans, 0, "the file rows go with the files");
}
/// `grabs.infohash` is unique, so a row left behind by a deleted title
/// blocks the row a later grab of the same release needs to write — and
/// the grab records nothing while the new title still reads as
/// downloading.
#[tokio::test]
async fn deleting_a_movie_takes_its_grabs_with_it() {
let (_dir, state, base) = application().await;
let movie = add_movie(&base, 693_134, 1).await;
let id = movie["id"].as_i64().expect("id");
let pool = state.database().expect("database").pool();
sqlx::query(
"INSERT INTO releases (id, indexer_id, guid, name, size, download_url, parsed)
VALUES (1, 1, 'guid', 'release', 1, 'magnet:?x', '{}')",
)
.execute(pool)
.await
.expect("release");
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash, state)
VALUES (1, 'movie', ?, 'deadbeef', 'sent')",
)
.bind(id)
.execute(pool)
.await
.expect("grab");
let response = reqwest::Client::new()
.delete(format!("{base}/api/movies/{id}"))
.send()
.await
.expect("delete");
assert_eq!(response.status(), StatusCode::NO_CONTENT);
let orphans: i64 = sqlx::query_scalar("SELECT count(*) FROM grabs")
.fetch_one(pool)
.await
.expect("count grabs");
assert_eq!(orphans, 0, "no grab outlives the title it was for");
}
/// The guard that keeps a delete inside the library: a path that is not
/// under the title's root is left alone, whatever the row says.
#[tokio::test]
+1 -1
View File
@@ -27,7 +27,7 @@ pub struct Root {
/// The payload for creating or replacing a root.
#[derive(Debug, Clone, Deserialize, ToSchema)]
pub struct RootInput {
/// `movie` or `tv` — the Transmission label and layout prefix (§7.1, §7.4).
/// `movie` or `tv` — the qBittorrent label and layout prefix (§7.1, §7.4).
pub kind: String,
/// `main` or `kids`.
pub audience: String,
+78 -3
View File
@@ -679,6 +679,23 @@ pub async fn delete(
)
.execute(pool(&state)?)
.await?;
// Seasons and episodes go with the series by cascade, but grabs point at
// them polymorphically and have no foreign key to follow. `infohash` is
// unique, so an orphan left here would block a later grab of the same
// release from recording anything.
sqlx::query!(
"DELETE FROM grabs
WHERE (target_kind = 'season'
AND target_id IN (SELECT id FROM seasons WHERE series_id = ?))
OR (target_kind = 'episode'
AND target_id IN (SELECT e.id FROM episodes e
JOIN seasons s ON s.id = e.season_id
WHERE s.series_id = ?))",
id,
id
)
.execute(pool(&state)?)
.await?;
let result = sqlx::query!("DELETE FROM series WHERE id = ?", id)
.execute(pool(&state)?)
.await?;
@@ -1655,7 +1672,7 @@ pub async fn season_releases(
/// A grab for this season that downloaded in full and was then condemned at
/// import (#227, §5.7).
///
/// The torrent stays at 100% in Transmission — §7.3 leaves that lifecycle to
/// The torrent stays at 100% in qBittorrent — §7.3 leaves that lifecycle to
/// the reaper — the release is blacklisted and every episode it was covering
/// reopens as a gap. Nothing on screen connected the two, so the season read
/// `0/10` as though no grab had ever been tried. Every fact is already
@@ -1713,7 +1730,7 @@ async fn season_import_failures(
WHERE newer.target_kind = 'season'
AND newer.target_id = g.target_id
-- 'vanished' is not a live attempt either: the
-- torrent left Transmission, so letting it take the
-- torrent left qBittorrent, so letting it take the
-- headline would leave the season saying nothing at
-- all about a gap that still exists.
AND newer.state NOT IN ('failed', 'vanished')
@@ -1794,7 +1811,7 @@ pub struct SeasonPackState {
/// #227. The last pack that downloaded in full and was then condemned at
/// import, while the season is still waiting for a file. A deck that
/// cannot say this leaves the season reading as though nothing was ever
/// tried, with the torrent still sitting at 100% in Transmission.
/// tried, with the torrent still sitting at 100% in qBittorrent.
pub import_failure: Option<ImportFailure>,
}
@@ -3647,6 +3664,64 @@ mod tests {
assert_eq!(orphans, 0, "the file rows go with the files");
}
/// Seasons and episodes cascade off the series row, but grabs point at
/// them polymorphically with no foreign key to follow. `grabs.infohash`
/// is unique, so one left behind blocks the row a later grab of the same
/// pack or episode needs to write.
#[tokio::test]
async fn deleting_a_series_takes_its_season_and_episode_grabs_with_it() {
let (_dir, state, base) = application().await;
let root_id = tv_root(&state, "main").await;
let series = add_series(&base, root_id, true).await;
let series_id = series["id"].as_i64().expect("id");
let season = add_season(
&base,
series_id,
1,
serde_json::json!([{"number": 1, "title": "Pilot", "air_date": "2001-01-01"}]),
)
.await;
let season_id = season["id"].as_i64().expect("season id");
let episode_id = season["episodes"][0]["id"].as_i64().expect("episode id");
let pool = state.database().expect("database").pool();
sqlx::query(
"INSERT INTO releases (id, indexer_id, guid, name, size, download_url, parsed)
VALUES (1, 1, 'guid', 'release', 1, 'magnet:?x', '{}')",
)
.execute(pool)
.await
.expect("release");
for (kind, target, hash) in [
("season", season_id, "aaaa"),
("episode", episode_id, "bbbb"),
] {
sqlx::query(
"INSERT INTO grabs (release_id, target_kind, target_id, infohash, state)
VALUES (1, ?, ?, ?, 'sent')",
)
.bind(kind)
.bind(target)
.bind(hash)
.execute(pool)
.await
.expect("grab");
}
let response = reqwest::Client::new()
.delete(format!("{base}/api/series/{series_id}"))
.send()
.await
.expect("delete");
assert_eq!(response.status(), StatusCode::NO_CONTENT);
let orphans: i64 = sqlx::query_scalar("SELECT count(*) FROM grabs")
.fetch_one(pool)
.await
.expect("count grabs");
assert_eq!(orphans, 0, "neither scope outlives the series");
}
/// A missing series is 404 before anything touches the disk.
#[tokio::test]
async fn deleting_a_missing_series_is_a_404() {
+11 -11
View File
@@ -6,7 +6,7 @@ use std::sync::{atomic::AtomicU64, Arc};
use std::time::{Duration, Instant};
use arr_db::Db;
use arr_dl::TransmissionClient;
use arr_dl::QbitClient;
use arr_probe::Extractor;
use arr_subs::{Backend, OpenAiEndpoint, Provider, Syncer};
use tokio::sync::mpsc;
@@ -33,7 +33,7 @@ const PROBE_TIMEOUT: Duration = Duration::from_secs(3);
pub struct Upstreams {
pub prowlarr_url: String,
pub prowlarr_api_key: Option<String>,
pub transmission_url: String,
pub qbittorrent_url: String,
pub tmdb_url: String,
pub tmdb_api_key: Option<String>,
}
@@ -41,11 +41,11 @@ pub struct Upstreams {
impl Upstreams {
/// Every upstream at its documented default, no keys.
#[must_use]
pub fn new(prowlarr_url: String, transmission_url: String) -> Self {
pub fn new(prowlarr_url: String, qbittorrent_url: String) -> Self {
Self {
prowlarr_url,
prowlarr_api_key: None,
transmission_url,
qbittorrent_url,
tmdb_url: DEFAULT_TMDB_URL.to_string(),
tmdb_api_key: None,
}
@@ -109,7 +109,7 @@ pub struct AppState {
extractor: Extractor,
jellyfin: Option<JellyfinClient>,
syncer: Syncer,
transmission: Option<TransmissionClient>,
qbit: Option<QbitClient>,
download_snapshot: DownloadSnapshot,
}
@@ -189,7 +189,7 @@ impl AppState {
extractor: Extractor::default(),
jellyfin: None,
syncer: Syncer::default(),
transmission: None,
qbit: None,
download_snapshot: Arc::new(RwLock::new(None)),
})
}
@@ -201,11 +201,11 @@ impl AppState {
self
}
/// Attach the shared Transmission client used by the API's live download
/// Attach the shared qBittorrent client used by the API's live download
/// snapshot endpoint.
#[must_use]
pub fn with_transmission(mut self, transmission: TransmissionClient) -> Self {
self.transmission = Some(transmission);
pub fn with_qbit(mut self, qbit: QbitClient) -> Self {
self.qbit = Some(qbit);
self
}
@@ -368,8 +368,8 @@ impl AppState {
self.database.as_ref()
}
pub(crate) fn transmission(&self) -> Option<&TransmissionClient> {
self.transmission.as_ref()
pub(crate) fn qbit(&self) -> Option<&QbitClient> {
self.qbit.as_ref()
}
pub(crate) fn download_snapshot(&self) -> &DownloadSnapshot {
+9 -1
View File
@@ -257,7 +257,7 @@ pub enum MediaState {
Missing,
Downloading,
Available,
/// #108: `wanted` was cleared after the grab vanished from Transmission.
/// #108: `wanted` was cleared after the grab vanished from qBittorrent.
/// Distinct from `Missing` so it does not read as an open gap.
Parked,
}
@@ -365,6 +365,14 @@ impl SubtitleCodec {
matches!(self, Self::SubRip | Self::Ass | Self::MovText)
}
/// §15's bitmap tracks. They carry no text, but their packet timings are
/// exact for the release they came off, which is what a cue skeleton is
/// derived from (#268).
#[must_use]
pub const fn is_image(self) -> bool {
matches!(self, Self::Pgs | Self::VobSub)
}
/// The inverse of [`Display`](fmt::Display): read back a codec that was
/// written out under its `ffprobe` name. `ffprobe`'s own aliases are
/// accepted alongside, so a column written by an older probe still
-1
View File
@@ -47,7 +47,6 @@ tracing = { workspace = true }
tracing-subscriber = { workspace = true }
[dev-dependencies]
base64 = { workspace = true }
tempfile = { workspace = true }
wiremock = { workspace = true }
+19 -19
View File
@@ -1,4 +1,4 @@
//! §9.5 *broken* → the operator alone: Prowlarr, Transmission or TMDB
//! §9.5 *broken* → the operator alone: Prowlarr, qBittorrent or TMDB
//! unreachable, or a subtitle lamp failing (#200).
//!
//! Edge-triggered: notifies once when an upstream stops answering, and
@@ -24,7 +24,7 @@ const PROBE_TIMEOUT: Duration = Duration::from_secs(5);
pub struct Upstreams {
pub prowlarr_url: String,
pub prowlarr_api_key: Option<String>,
pub transmission_url: String,
pub qbittorrent_url: String,
pub tmdb_url: String,
pub tmdb_api_key: Option<String>,
}
@@ -118,16 +118,16 @@ impl BrokenAction {
}
async fn tick(&self, database: &Db) -> Vec<Outcome> {
let (prowlarr, transmission, tmdb, subtitles) = tokio::join!(
let (prowlarr, qbit, tmdb, subtitles) = tokio::join!(
self.probe_prowlarr(),
self.probe_transmission(),
self.probe_qbit(),
self.probe_tmdb(),
self.subtitles.probe(database),
);
let mut outcomes = Vec::new();
outcomes.extend(self.notify_transition("prowlarr", prowlarr).await);
outcomes.extend(self.notify_transition("transmission", transmission).await);
outcomes.extend(self.notify_transition("qbit", qbit).await);
outcomes.extend(self.notify_transition("tmdb", tmdb).await);
for (name, reachable) in subtitles {
outcomes.extend(self.notify_transition(&name, reachable).await);
@@ -178,13 +178,13 @@ impl BrokenAction {
matches!(request.send().await, Ok(response) if response.status().is_success())
}
/// Transmission answers an RPC call without a session id with `409` plus
/// qBittorrent answers an RPC call without a session id with `409` plus
/// the id to retry with. That is a live daemon, so it counts as
/// reachable.
async fn probe_transmission(&self) -> bool {
async fn probe_qbit(&self) -> bool {
let request = self
.http
.post(&self.upstreams.transmission_url)
.post(&self.upstreams.qbittorrent_url)
.timeout(PROBE_TIMEOUT)
.json(&serde_json::json!({ "method": "session-get" }));
matches!(
@@ -235,11 +235,11 @@ mod tests {
use super::*;
fn upstreams(prowlarr_url: String, transmission_url: String) -> Upstreams {
fn upstreams(prowlarr_url: String, qbittorrent_url: String) -> Upstreams {
Upstreams {
prowlarr_url,
prowlarr_api_key: None,
transmission_url,
qbittorrent_url,
tmdb_url: "http://127.0.0.1:1".to_string(),
tmdb_api_key: None,
}
@@ -275,10 +275,10 @@ mod tests {
async fn an_unreachable_upstream_notifies_once_then_re_arms() {
let prowlarr = MockServer::start().await;
// No mock mounted: every request 404s, which counts as unreachable.
let transmission = MockServer::start().await;
let qbit = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(409))
.mount(&transmission)
.mount(&qbit)
.await;
let ntfy = MockServer::start().await;
@@ -289,7 +289,7 @@ mod tests {
let (_dir, db) = database().await;
let action = BrokenAction::new(
upstreams(prowlarr.uri(), transmission.uri()),
upstreams(prowlarr.uri(), qbit.uri()),
subtitles(),
Notifier::new(ntfy.uri()).unwrap(),
"operator-topic".to_string(),
@@ -337,14 +337,14 @@ mod tests {
.respond_with(ResponseTemplate::new(200))
.mount(&prowlarr)
.await;
let transmission = MockServer::start().await;
let qbit = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(409))
.mount(&transmission)
.mount(&qbit)
.await;
let action = BrokenAction::new(
upstreams(prowlarr.uri(), transmission.uri()),
upstreams(prowlarr.uri(), qbit.uri()),
subtitles(),
Notifier::new(ntfy.uri()).unwrap(),
"operator-topic".to_string(),
@@ -381,10 +381,10 @@ mod tests {
.respond_with(ResponseTemplate::new(200))
.mount(&prowlarr)
.await;
let transmission = MockServer::start().await;
let qbit = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(409))
.mount(&transmission)
.mount(&qbit)
.await;
let subs = SubtitleUpstreams {
@@ -392,7 +392,7 @@ mod tests {
..subtitles()
};
let action = BrokenAction::new(
upstreams(prowlarr.uri(), transmission.uri()),
upstreams(prowlarr.uri(), qbit.uri()),
subs,
Notifier::new(ntfy.uri()).unwrap(),
"operator-topic".to_string(),
+54 -16
View File
@@ -20,7 +20,12 @@ pub const ENV_DATABASE_PATH: &str = "ARR_DATABASE_PATH";
pub const ENV_MEDIA_ROOT: &str = "ARR_MEDIA_ROOT";
pub const ENV_PROWLARR_URL: &str = "ARR_PROWLARR_URL";
pub const ENV_PROWLARR_API_KEY: &str = "ARR_PROWLARR_API_KEY";
pub const ENV_TRANSMISSION_URL: &str = "ARR_TRANSMISSION_URL";
pub const ENV_QBITTORRENT_URL: &str = "ARR_QBITTORRENT_URL";
// The WebUI password is a secret, so both halves are env-only (§10) and
// neither has a `ConfigFile` field. Leaving them unset is valid: it is the
// right shape for an instance that bypasses authentication for arr's address.
pub const ENV_QBITTORRENT_USERNAME: &str = "ARR_QBITTORRENT_USERNAME";
pub const ENV_QBITTORRENT_PASSWORD: &str = "ARR_QBITTORRENT_PASSWORD";
pub const ENV_DOWNLOAD_DIR: &str = "ARR_DOWNLOAD_DIR";
pub const ENV_SEED_RATIO_LIMIT: &str = "ARR_SEED_RATIO_LIMIT";
pub const ENV_SEED_IDLE_LIMIT_MINUTES: &str = "ARR_SEED_IDLE_LIMIT_MINUTES";
@@ -54,12 +59,12 @@ pub const DEFAULT_BIND_ADDR: &str = "0.0.0.0:7878";
pub const DEFAULT_DATABASE_PATH: &str = "arr.db";
pub const DEFAULT_MEDIA_ROOT: &str = "/mnt/media";
pub const DEFAULT_PROWLARR_URL: &str = "http://localhost:9696";
pub const DEFAULT_TRANSMISSION_URL: &str = "http://localhost:9091/transmission/rpc";
/// Transmission's own view of the download directory (DESIGN.md §3). It
pub const DEFAULT_QBITTORRENT_URL: &str = "http://localhost:8080";
/// qBittorrent's own view of the download directory (DESIGN.md §3). It
/// shares the media dataset with the library so hardlinks work (§7.2).
pub const DEFAULT_DOWNLOAD_DIR: &str = "/mnt/media/transmission/complete";
pub const DEFAULT_DOWNLOAD_DIR: &str = "/mnt/media/qbittorrent/complete";
/// Seeding obligation defaults (§7.3), applied to every torrent at add time
/// and enforced by Transmission. Per-tracker rules are issue #25.
/// and enforced by qBittorrent. Per-tracker rules are issue #25.
pub const DEFAULT_SEED_RATIO_LIMIT: f64 = 1.0;
pub const DEFAULT_SEED_IDLE_LIMIT_MINUTES: u64 = 4320;
pub const DEFAULT_JELLYFIN_URL: &str = "http://localhost:8096";
@@ -97,7 +102,7 @@ struct ConfigFile {
#[serde(default)]
prowlarr_url: Option<String>,
#[serde(default)]
transmission_url: Option<String>,
qbittorrent_url: Option<String>,
#[serde(default)]
download_dir: Option<PathBuf>,
#[serde(default)]
@@ -151,7 +156,9 @@ pub struct EnvOverrides {
pub media_root: Option<String>,
pub prowlarr_url: Option<String>,
pub prowlarr_api_key: Option<String>,
pub transmission_url: Option<String>,
pub qbittorrent_url: Option<String>,
pub qbittorrent_username: Option<String>,
pub qbittorrent_password: Option<String>,
pub download_dir: Option<String>,
pub seed_ratio_limit: Option<String>,
pub seed_idle_limit_minutes: Option<String>,
@@ -183,7 +190,9 @@ impl EnvOverrides {
media_root: std::env::var(ENV_MEDIA_ROOT).ok(),
prowlarr_url: std::env::var(ENV_PROWLARR_URL).ok(),
prowlarr_api_key: std::env::var(ENV_PROWLARR_API_KEY).ok(),
transmission_url: std::env::var(ENV_TRANSMISSION_URL).ok(),
qbittorrent_url: std::env::var(ENV_QBITTORRENT_URL).ok(),
qbittorrent_username: std::env::var(ENV_QBITTORRENT_USERNAME).ok(),
qbittorrent_password: std::env::var(ENV_QBITTORRENT_PASSWORD).ok(),
download_dir: std::env::var(ENV_DOWNLOAD_DIR).ok(),
seed_ratio_limit: std::env::var(ENV_SEED_RATIO_LIMIT).ok(),
seed_idle_limit_minutes: std::env::var(ENV_SEED_IDLE_LIMIT_MINUTES).ok(),
@@ -216,8 +225,13 @@ pub struct Config {
pub media_root: PathBuf,
pub prowlarr_url: String,
pub prowlarr_api_key: Option<String>,
pub transmission_url: String,
/// Where Transmission puts completed downloads, in Transmission's own
pub qbittorrent_url: String,
/// The `WebUI` login. `None` for an instance that bypasses authentication
/// for arr's address; a username without a password (or the reverse) is
/// treated as no credentials at all.
pub qbittorrent_username: Option<String>,
pub qbittorrent_password: Option<String>,
/// Where qBittorrent puts completed downloads, in qBittorrent's own
/// namespace (§7.1).
pub download_dir: PathBuf,
pub seed_ratio_limit: f64,
@@ -362,10 +376,12 @@ impl Config {
.or(file.prowlarr_url)
.unwrap_or_else(|| DEFAULT_PROWLARR_URL.to_string()),
prowlarr_api_key: env.prowlarr_api_key,
transmission_url: env
.transmission_url
.or(file.transmission_url)
.unwrap_or_else(|| DEFAULT_TRANSMISSION_URL.to_string()),
qbittorrent_url: env
.qbittorrent_url
.or(file.qbittorrent_url)
.unwrap_or_else(|| DEFAULT_QBITTORRENT_URL.to_string()),
qbittorrent_username: env.qbittorrent_username,
qbittorrent_password: env.qbittorrent_password,
download_dir: env
.download_dir
.map(PathBuf::from)
@@ -429,6 +445,28 @@ fn parse_bind_addr(raw: &str, env: &'static str) -> Result<SocketAddr, ConfigErr
mod tests {
use super::*;
/// §10 keeps secrets out of the config file, so the `WebUI` login is
/// env-only and absent by default — which is the shape an instance that
/// bypasses authentication for arr's address wants.
#[test]
fn the_qbittorrent_login_is_env_only_and_optional() {
let config = Config::resolve(EnvOverrides::default()).unwrap();
assert_eq!(config.qbittorrent_username, None);
assert_eq!(config.qbittorrent_password, None);
let config = Config::resolve(EnvOverrides {
qbittorrent_username: Some("arr".into()),
qbittorrent_password: Some("secret".into()),
..EnvOverrides::default()
})
.unwrap();
assert_eq!(config.qbittorrent_username.as_deref(), Some("arr"));
let error = toml::from_str::<ConfigFile>("qbittorrent_password = \"secret\"")
.expect_err("the file has no field for it");
assert!(error.to_string().contains("qbittorrent_password"));
}
#[test]
fn empty_environment_resolves_to_defaults() {
let config = Config::resolve(EnvOverrides::default()).unwrap();
@@ -436,7 +474,7 @@ mod tests {
assert_eq!(config.database_path, PathBuf::from(DEFAULT_DATABASE_PATH));
assert_eq!(config.media_root, PathBuf::from(DEFAULT_MEDIA_ROOT));
assert_eq!(config.prowlarr_url, DEFAULT_PROWLARR_URL);
assert_eq!(config.transmission_url, DEFAULT_TRANSMISSION_URL);
assert_eq!(config.qbittorrent_url, DEFAULT_QBITTORRENT_URL);
assert_eq!(config.download_dir, PathBuf::from(DEFAULT_DOWNLOAD_DIR));
assert!((config.seed_ratio_limit - DEFAULT_SEED_RATIO_LIMIT).abs() < f64::EPSILON);
assert_eq!(
@@ -491,7 +529,7 @@ prowlarr_url = "http://prowlarr.internal:9696"
assert_eq!(config.media_root, PathBuf::from("/tank/media"));
assert_eq!(config.prowlarr_url, "http://prowlarr.internal:9696");
// Untouched fields still default.
assert_eq!(config.transmission_url, DEFAULT_TRANSMISSION_URL);
assert_eq!(config.qbittorrent_url, DEFAULT_QBITTORRENT_URL);
}
#[test]
+200 -228
View File
@@ -1,5 +1,5 @@
//! The grab pipeline: close the "wanted movie, no file" gap by searching,
//! scoring, picking a winner and sending it to Transmission. See DESIGN.md
//! scoring, picking a winner and sending it to qBittorrent. See DESIGN.md
//! §5.4, §6.2, §7.1, §7.3 and §8.
//!
//! There is no grab delay and there is no job queue. The gap is recomputed
@@ -7,7 +7,7 @@
//! restarting converges instead of double-grabbing:
//!
//! - a title with a live `grabs` row is not a gap, so it is never re-searched;
//! - Transmission's `torrent-add` is keyed on the infohash, so re-sending the
//! - qBittorrent's `torrent-add` is keyed on the infohash, so re-sending the
//! same release returns the torrent that is already there rather than a
//! second one;
//! - the `grabs` row is written from that response, so a crash between the add
@@ -22,7 +22,7 @@ use arr_core::policy::{evaluate, Candidate};
use arr_core::score::{claimed_episode_count, score};
use arr_core::{Language, Policy, TitleOverrides, Verdict};
use arr_db::{blacklist, Blacklist, Db, MoviePolicy};
use arr_dl::{AddTorrent, TorrentSource, TransmissionClient};
use arr_dl::{AddTorrent, QbitClient, TorrentSource};
use arr_indexer::{Download, ProwlarrClient, SearchRelease, SearchRequest};
use arr_meta::TmdbClient;
@@ -37,7 +37,7 @@ const MOVIES_PER_TICK: i64 = 5;
/// Seeding obligations, per tracker in principle (§7.3) and per install in
/// practice until issue #25 gives them a home. Both are set on the torrent at
/// add time and enforced by Transmission.
/// add time and enforced by qBittorrent.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct SeedingLimits {
pub ratio: f64,
@@ -77,8 +77,8 @@ pub enum GrabError {
Metadata(#[from] arr_meta::Error),
#[error("movie {0} has an invalid TMDB id")]
InvalidTmdbId(i64),
#[error("transmission: {0}")]
Transmission(#[from] arr_dl::Error),
#[error("qbit: {0}")]
Qbit(#[from] arr_dl::Error),
#[error("download link: {0}")]
Download(#[from] arr_indexer::DownloadError),
#[error("release {name}: {source}")]
@@ -102,13 +102,13 @@ impl GrabAction {
#[must_use]
pub fn new(
prowlarr: ProwlarrClient,
transmission: TransmissionClient,
qbit: QbitClient,
download_dir: PathBuf,
seeding: SeedingRules,
) -> Self {
Self {
indexers: IndexerDirectory::new(prowlarr.clone()),
grabber: Grabber::new(prowlarr.clone(), transmission, download_dir, seeding),
grabber: Grabber::new(prowlarr.clone(), qbit, download_dir, seeding),
prowlarr,
tmdb: None,
}
@@ -427,7 +427,7 @@ impl GrabAction {
/// The manual one-click grab (§9.3, issue #107): the release is already
/// chosen, so this skips search and scoring and sends it straight to
/// Transmission.
/// qBittorrent.
pub(crate) async fn grab_release_now(
&self,
database: &Db,
@@ -460,17 +460,17 @@ impl GrabAction {
}
}
/// Sending a chosen release to Transmission and recording the grab.
/// Sending a chosen release to qBittorrent and recording the grab.
///
/// Targeted search and RSS (§6.2) differ in how a title is chosen and in
/// whether a failed attempt counts toward a backoff; from the winning
/// release onward they are the same writes, so they share this.
#[derive(Debug)]
pub(crate) struct Grabber {
/// Resolves the winner's indexer link before it is sent on: Transmission
/// Resolves the winner's indexer link before it is sent on: qBittorrent
/// cannot reach Prowlarr and arr can (issue #100).
prowlarr: ProwlarrClient,
transmission: TransmissionClient,
qbit: QbitClient,
download_dir: PathBuf,
seeding: SeedingRules,
}
@@ -532,22 +532,22 @@ impl GrabScope {
impl Grabber {
pub(crate) fn new(
prowlarr: ProwlarrClient,
transmission: TransmissionClient,
qbit: QbitClient,
download_dir: PathBuf,
seeding: SeedingRules,
) -> Self {
Self {
prowlarr,
transmission,
qbit,
download_dir,
seeding,
}
}
/// Move grabs Transmission reports as complete out of `sent`, whatever
/// Move grabs qBittorrent reports as complete out of `sent`, whatever
/// they target.
///
/// Transmission is authoritative and its view is rebuilt on every tick
/// qBittorrent is authoritative and its view is rebuilt on every tick
/// rather than cached (§8), so this is also what reconstructs in-flight
/// state after a restart. Both grab actions call it; whichever runs first
/// does the work and the other finds nothing.
@@ -564,7 +564,7 @@ impl Grabber {
}
let torrents: HashMap<String, f64> = self
.transmission
.qbit
.list_torrents()
.await?
.into_iter()
@@ -603,7 +603,7 @@ impl Grabber {
Ok(outcomes)
}
/// §86/#108: a `sent` grab whose infohash Transmission no longer reports
/// §86/#108: a `sent` grab whose infohash qBittorrent no longer reports
/// — removed by hand, not a policy failure. Marked `vanished` rather than
/// `failed` so it does not feed the `needs_decision` queue (attention.rs).
/// Nothing is blacklisted, since the release itself never failed policy,
@@ -624,10 +624,10 @@ impl Grabber {
grab_id,
target_kind,
target_id,
"torrent vanished from Transmission; target parked"
"torrent vanished from qBittorrent; target parked"
);
Ok(Outcome::new(
format!("grab {grab_id} sent, torrent vanished from Transmission"),
format!("grab {grab_id} sent, torrent vanished from qBittorrent"),
format!("parked {target_kind} {target_id}"),
))
}
@@ -646,12 +646,12 @@ impl Grabber {
}
}
/// Resolve the winner's indexer link and add it to Transmission.
/// Resolve the winner's indexer link and add it to qBittorrent.
///
/// The link is resolved here rather than passed on, because Transmission
/// The link is resolved here rather than passed on, because qBittorrent
/// has no route to Prowlarr and cannot follow its redirect to a magnet
/// (issue #100).
async fn send_to_transmission(
async fn send_to_qbit(
&self,
winner: &Eligible,
loaded: &MoviePolicy,
@@ -659,7 +659,7 @@ impl Grabber {
let seeding = self.seeding.for_indexer(winner.indexer_id);
let source = torrent_source(self.prowlarr.download(&winner.download_url).await?);
Ok(self
.transmission
.qbit
.add_torrent(AddTorrent {
source,
label: label(loaded),
@@ -670,10 +670,10 @@ impl Grabber {
.await?)
}
/// Add the winning release to Transmission and record the grab.
/// Add the winning release to qBittorrent and record the grab.
///
/// The search still counts as an attempt (§6.2) on every exit that is
/// not a completed grab — a Transmission error or a blacklisted-infohash
/// not a completed grab — a qBittorrent error or a blacklisted-infohash
/// drop must not leave the same release to repeat next tick with no
/// backoff.
pub(crate) async fn send_winner(
@@ -684,7 +684,7 @@ impl Grabber {
blacklist: &Blacklist,
winner: Eligible,
) -> Result<Option<Outcome>, GrabError> {
let added = match self.send_to_transmission(&winner, loaded).await {
let added = match self.send_to_qbit(&winner, loaded).await {
Ok(added) => added,
Err(error) => {
self.record_attempt(database, target).await?;
@@ -694,7 +694,7 @@ impl Grabber {
let infohash = added.hash.to_ascii_lowercase();
// §6.3's second key. A `.torrent` link hides its infohash until
// Transmission has fetched it, so the same blacklisted torrent can
// qBittorrent has fetched it, so the same blacklisted torrent can
// reach here under a new name.
if blacklist.blocks_infohash(&infohash) {
self.record_attempt(database, target).await?;
@@ -707,6 +707,14 @@ impl Grabber {
// with its own earlier row. A `sent`/`downloaded` row is the restart
// case and is left alone; a `vanished` one (§86) is reclaimed, since
// nothing blacklisted the release.
//
// A row whose target no longer exists is reclaimed too, whatever its
// state. Deleting a title used to leave its grabs behind, and one of
// those orphans holding the infohash meant a later grab of the same
// release silently recorded nothing: the title flipped to
// `downloading` while the only row still pointed at the deleted one,
// so it never imported. Titles now take their grabs with them, and
// this clause heals the databases that already have orphans.
let target_kind = target.scope.target_kind();
let target_id = target.scope.target_id();
let inserted = sqlx::query!(
@@ -720,6 +728,12 @@ impl Grabber {
grabbed_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now'),
imported_at = NULL
WHERE grabs.state = 'vanished'
OR (grabs.target_kind = 'movie'
AND NOT EXISTS (SELECT 1 FROM movies WHERE id = grabs.target_id))
OR (grabs.target_kind = 'season'
AND NOT EXISTS (SELECT 1 FROM seasons WHERE id = grabs.target_id))
OR (grabs.target_kind = 'episode'
AND NOT EXISTS (SELECT 1 FROM episodes WHERE id = grabs.target_id))
RETURNING id AS "id!: i64""#,
winner.id,
target_kind,
@@ -782,7 +796,7 @@ impl Grabber {
/// Undo a grab whose infohash turned out to be blacklisted (§6.3).
///
/// The name is added to the blacklist so the next tick stops at the cheap
/// check instead of paying Transmission again, and no `grabs` row is
/// check instead of paying qBittorrent again, and no `grabs` row is
/// written, which leaves the title a gap for the next candidate.
async fn drop_blacklisted_torrent(
&self,
@@ -822,7 +836,7 @@ impl Grabber {
} else {
// This tick added it seconds ago, so it carries no seeding
// obligation and has nothing on disk worth keeping.
self.transmission.remove_torrent(added.id, true).await?;
self.qbit.remove_torrent(&added.hash, true).await?;
tracing::warn!(
title = target.title,
release = release_name,
@@ -1509,7 +1523,7 @@ pub(crate) async fn park_target(
}
/// `movies-main`, `tv-kids` (§7.1). Distinct from Radarr's own labels, so
/// both stacks can run against one Transmission.
/// both stacks can run against one qBittorrent.
fn label(loaded: &MoviePolicy) -> String {
label_for_root(&loaded.root_kind, &loaded.root_audience)
}
@@ -1519,7 +1533,7 @@ pub(crate) fn label_for_root(kind: &str, audience: &str) -> String {
format!("{kind}-{audience}")
}
/// A resolved download, in the shape Transmission takes it.
/// A resolved download, in the shape qBittorrent takes it.
fn torrent_source(download: Download) -> TorrentSource {
match download {
Download::Magnet(uri) => TorrentSource::Magnet(uri),
@@ -1572,6 +1586,18 @@ pub(crate) mod test_downloads {
feed.replace(links, &format!("{}{PREFIX}", server.uri()))
}
/// The infohash the mock indexer's magnet for `name` carries.
///
/// Deterministic and distinct per release, so duplicates still collapse
/// and a test can blacklist a hash before the tick that would grab it.
pub(crate) fn infohash_for(name: &str) -> String {
let mut hash: u64 = 5381;
for byte in name.as_bytes() {
hash = hash.wrapping_mul(33) ^ u64::from(*byte);
}
format!("{hash:040x}")
}
/// Prowlarr's live behaviour: the download endpoint 302s to a magnet.
pub(crate) async fn mount(server: &MockServer) {
Mock::given(method("GET"))
@@ -1592,16 +1618,9 @@ pub(crate) mod test_downloads {
.next()
.unwrap_or_default()
.to_owned();
// Deterministic stand-in for the infohash the tracker would give,
// and distinct per release so duplicates still collapse.
let mut hash: u64 = 5381;
for byte in name.as_bytes() {
hash = hash.wrapping_mul(33) ^ u64::from(*byte);
}
ResponseTemplate::new(302).insert_header(
"location",
format!("magnet:?xt=urn:btih:{hash:040x}&dn={name}"),
)
let hash = super::test_downloads::infohash_for(&name);
ResponseTemplate::new(302)
.insert_header("location", format!("magnet:?xt=urn:btih:{hash}&dn={name}"))
}
}
}
@@ -1609,127 +1628,16 @@ pub(crate) mod test_downloads {
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use base64::Engine as _;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use serde_json::{json, Value};
use serde_json::json;
use wiremock::matchers::{method, path, query_param};
use wiremock::{Mock, MockServer, Request, Respond, ResponseTemplate};
use wiremock::{Mock, MockServer, ResponseTemplate};
use crate::qbit_fake::FakeQbit;
use super::*;
/// A Transmission that dedupes on the infohash, like the real one: the
/// same source added twice is one torrent and a `torrent-duplicate`
/// response. That is the property the restart case leans on.
#[derive(Clone, Debug, Default)]
struct FakeTransmission {
torrents: Arc<Mutex<Vec<FakeTorrent>>>,
}
#[derive(Clone, Debug)]
struct FakeTorrent {
id: i64,
hash: String,
source: String,
labels: Vec<String>,
progress: f64,
}
impl FakeTransmission {
fn torrents(&self) -> Vec<FakeTorrent> {
self.torrents.lock().unwrap().clone()
}
fn complete_all(&self) {
for torrent in self.torrents.lock().unwrap().iter_mut() {
torrent.progress = 1.0;
}
}
fn add(&self, arguments: &Value) -> ResponseTemplate {
// A magnet arrives as `filename`, a torrent body as base64
// `metainfo` — either identifies the torrent for the fake.
let source = arguments["filename"]
.as_str()
.or_else(|| arguments["metainfo"].as_str())
.unwrap_or_default()
.to_owned();
let labels: Vec<String> = arguments["labels"]
.as_array()
.map(|values| {
values
.iter()
.filter_map(|value| value.as_str().map(ToOwned::to_owned))
.collect()
})
.unwrap_or_default();
let mut torrents = self.torrents.lock().unwrap();
if let Some(existing) = torrents.iter().find(|torrent| torrent.source == source) {
return success(&json!({"torrent-duplicate": {
"id": existing.id, "name": existing.source, "hashString": existing.hash
}}));
}
let id = i64::try_from(torrents.len()).unwrap() + 1;
// Deterministic stand-in for the real infohash, which likewise
// comes out the same for the same torrent.
let hash = format!("{:040x}", id * 7);
torrents.push(FakeTorrent {
id,
hash: hash.clone(),
source: source.clone(),
labels,
progress: 0.0,
});
success(&json!({"torrent-added": {"id": id, "name": source, "hashString": hash}}))
}
}
impl Respond for FakeTransmission {
fn respond(&self, request: &Request) -> ResponseTemplate {
let body: Value = serde_json::from_slice(&request.body).unwrap();
let arguments = &body["arguments"];
match body["method"].as_str().unwrap_or_default() {
"torrent-add" => self.add(arguments),
"torrent-get" => {
let torrents: Vec<Value> = self
.torrents()
.into_iter()
.map(|torrent| {
json!({
"id": torrent.id, "name": torrent.source,
"hashString": torrent.hash, "status": 4,
"percentDone": torrent.progress,
"downloadDir": "/downloads", "labels": torrent.labels
})
})
.collect();
success(&json!({"torrents": torrents}))
}
"torrent-remove" => {
let removed: Vec<i64> = arguments["ids"]
.as_array()
.map(|ids| ids.iter().filter_map(serde_json::Value::as_i64).collect())
.unwrap_or_default();
self.torrents
.lock()
.unwrap()
.retain(|torrent| !removed.contains(&torrent.id));
success(&json!({}))
}
_ => success(&json!({})),
}
}
}
fn success(arguments: &Value) -> ResponseTemplate {
ResponseTemplate::new(200).set_body_json(json!({
"result": "success", "arguments": arguments
}))
}
const RSS: &str = r#"<rss><channel>
<item>
<title>Dune.Part.Two.2024.2160p.WEB-DL.DDP5.1.Atmos</title>
@@ -1846,14 +1754,8 @@ mod tests {
server
}
async fn transmission() -> (MockServer, FakeTransmission) {
let server = MockServer::start().await;
let fake = FakeTransmission::default();
Mock::given(method("POST"))
.respond_with(fake.clone())
.mount(&server)
.await;
(server, fake)
async fn qbit() -> (MockServer, FakeQbit) {
FakeQbit::start().await
}
async fn wanted_movie() -> (tempfile::TempDir, Db) {
@@ -1871,11 +1773,11 @@ mod tests {
(dir, database)
}
fn action(prowlarr: &MockServer, transmission: &MockServer) -> GrabAction {
fn action(prowlarr: &MockServer, qbit: &MockServer) -> GrabAction {
GrabAction::new(
ProwlarrClient::new(prowlarr.uri(), "key").unwrap(),
TransmissionClient::new(&transmission.uri()).unwrap(),
PathBuf::from("/mnt/media/transmission/complete"),
QbitClient::new(&qbit.uri()).unwrap(),
PathBuf::from("/mnt/media/qbittorrent/complete"),
SeedingRules::new(
SeedingLimits {
ratio: 1.5,
@@ -1888,10 +1790,10 @@ mod tests {
fn action_with_tmdb(
prowlarr: &MockServer,
transmission: &MockServer,
qbit: &MockServer,
metadata: &MockServer,
) -> GrabAction {
action(prowlarr, transmission).with_tmdb(Arc::new(
action(prowlarr, qbit).with_tmdb(Arc::new(
TmdbClient::builder("key")
.base_url(format!("{}/3/", metadata.uri()))
.build()
@@ -1945,7 +1847,7 @@ mod tests {
async fn a_wanted_movie_ends_with_one_torrent_and_one_grab() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1971,10 +1873,10 @@ mod tests {
async fn a_restart_mid_flight_does_not_grab_twice() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
// The crash: Transmission has the torrent, the database does not know.
// The crash: qBittorrent has the torrent, the database does not know.
sqlx::query("DELETE FROM grabs")
.execute(database.pool())
.await
@@ -1996,7 +1898,7 @@ mod tests {
async fn a_second_tick_grabs_nothing_new() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
@@ -2013,26 +1915,17 @@ mod tests {
async fn the_label_and_both_seed_limits_are_set_at_add_time() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
assert_eq!(fake.torrents()[0].labels, vec!["movies-main".to_owned()]);
let add = downloader
.received_requests()
.await
.unwrap()
.into_iter()
.filter_map(|request| serde_json::from_slice::<Value>(&request.body).ok())
.find(|body| body["method"] == "torrent-add")
.expect("torrent-add");
assert_eq!(add["arguments"]["seedRatioLimit"], json!(1.5));
assert_eq!(add["arguments"]["seedIdleLimit"], json!(60));
assert_eq!(add["arguments"]["seedRatioMode"], json!(1));
assert_eq!(add["arguments"]["seedIdleMode"], json!(1));
let added = &fake.torrents()[0];
assert_eq!(added.labels, vec!["movies-main".to_owned()]);
assert!((added.ratio_limit - 1.5).abs() < f64::EPSILON);
assert_eq!(added.idle_limit_minutes, 60);
assert_eq!(
add["arguments"]["download-dir"],
json!("/mnt/media/transmission/complete")
added.save_path,
String::from("/mnt/media/qbittorrent/complete")
);
}
@@ -2058,7 +1951,7 @@ mod tests {
async fn every_candidate_is_recorded_with_its_verdict() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -2081,13 +1974,13 @@ mod tests {
assert_eq!(attempts, 0);
}
/// Transmission is authoritative (§8): a completed torrent moves its grab
/// qBittorrent is authoritative (§8): a completed torrent moves its grab
/// out of `sent` without the process having watched it happen.
#[tokio::test]
async fn a_completed_torrent_moves_its_grab_to_downloaded() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
@@ -2099,14 +1992,14 @@ mod tests {
}
/// #108, overriding §86: a torrent removed by hand — gone from
/// Transmission before it finished — parks the movie instead of
/// qBittorrent before it finished — parks the movie instead of
/// reopening the gap, and is marked `vanished` rather than `failed` so
/// it never counts toward the `needs_decision` queue (attention.rs).
#[tokio::test]
async fn a_torrent_removed_by_hand_parks_the_movie() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
fake.torrents.lock().unwrap().clear();
@@ -2131,7 +2024,7 @@ mod tests {
async fn a_parked_movie_is_not_regrabbed() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
@@ -2154,7 +2047,7 @@ mod tests {
async fn a_rewanted_movie_can_regrab_the_same_infohash() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
@@ -2179,6 +2072,83 @@ mod tests {
assert_eq!(state, "downloading");
}
/// A grab row left behind by a deleted title holds the release's unique
/// infohash. Re-adding the title and grabbing the same release used to
/// record nothing at all: the conflict hit a row in `sent`, which is only
/// reclaimed when it is `vanished`, so the movie flipped to `downloading`
/// while the one row still pointed at the title that no longer existed —
/// and nothing ever imported for the new one.
#[tokio::test]
async fn an_orphaned_grab_is_reclaimed_by_the_title_that_regrabs_it() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
assert_eq!(grabs(&database).await.len(), 1);
// The title is deleted the way an old build left it: row gone, grab
// behind, still reading `sent`.
sqlx::query("UPDATE grabs SET target_id = 99, state = 'sent'")
.execute(database.pool())
.await
.unwrap();
sqlx::query(
"UPDATE movies SET state = 'missing', wanted = 1, search_attempts = 0,
last_searched_at = NULL WHERE id = 1",
)
.execute(database.pool())
.await
.unwrap();
let outcomes = action.tick(&database).await.unwrap();
assert_eq!(outcomes.len(), 1, "the regrab records a grab");
assert_eq!(fake.torrents().len(), 1, "same release, same torrent");
let grabs = grabs(&database).await;
assert_eq!(grabs.len(), 1, "the orphan is reclaimed, not duplicated");
assert_eq!(grabs[0].0, 1, "and it now belongs to the live movie");
}
/// The other half: a live title's row is still off limits, so a restart
/// mid-flight does not let one title steal another's torrent.
#[tokio::test]
async fn a_grab_belonging_to_a_live_title_is_not_stolen() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, _fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
// A second live movie, and the grab reassigned to it.
sqlx::query(
"INSERT INTO movies (id, tmdb_id, title, year, original_language, root_id, state)
SELECT 2, 999, 'Other', 2020, 'en', id, 'downloading'
FROM roots WHERE kind = 'movie' AND audience = 'main'",
)
.execute(database.pool())
.await
.unwrap();
sqlx::query("UPDATE grabs SET target_id = 2, state = 'sent'")
.execute(database.pool())
.await
.unwrap();
sqlx::query(
"UPDATE movies SET state = 'missing', wanted = 1, search_attempts = 0,
last_searched_at = NULL WHERE id = 1",
)
.execute(database.pool())
.await
.unwrap();
action.tick(&database).await.unwrap();
let grabs = grabs(&database).await;
assert_eq!(grabs.len(), 1);
assert_eq!(grabs[0].0, 2, "movie 2 keeps the torrent it is downloading");
}
/// §5.2: the language rules are expressed against the title's original
/// language, and guessing it is worse than waiting for it.
#[tokio::test]
@@ -2189,7 +2159,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -2212,7 +2182,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -2240,7 +2210,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -2253,23 +2223,24 @@ mod tests {
assert_eq!(rule.as_deref(), Some("blacklisted"));
}
/// §6.3's second key. A `.torrent` link hides its infohash until
/// Transmission has fetched it, so the blacklisted torrent is only
/// recognised after the add — and must not leave a grab behind.
/// §6.3's second key. The infohash is only known once the indexer link
/// has been resolved, which happens at grab time, so a blacklisted
/// torrent under a new release name is recognised after the add — and
/// must not leave a grab behind.
#[tokio::test]
async fn a_blacklisted_infohash_never_becomes_a_grab() {
let (_dir, database) = wanted_movie().await;
// What the fake hands back for the first torrent it accepts.
// The infohash the winning release's magnet carries.
blacklist::add(
database.pool(),
Some(&format!("{:040x}", 7)),
Some(&test_downloads::infohash_for("good.torrent")),
"Some.Older.Name.Of.The.Same.Torrent",
"required_audio",
)
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -2279,7 +2250,7 @@ mod tests {
"a torrent added this tick and then found blacklisted is removed"
);
// Recorded under its new name, so the next tick stops before paying
// Transmission again.
// qBittorrent again.
let blacklist = Blacklist::load(database.pool()).await.unwrap();
assert!(blacklist.blocks_name("Dune.Part.Two.2024.2160p.WEB-DL.DDP5.1.Atmos"));
// And the cached row stops reading eligible straight away, so §9.3's
@@ -2329,7 +2300,7 @@ mod tests {
async fn a_manual_search_on_an_available_movie_refreshes_the_deck() {
let (_dir, database) = available_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcome = action(&indexer, &downloader)
.search_now(&database, 1)
@@ -2351,7 +2322,7 @@ mod tests {
let (_dir, database) = available_movie().await;
let indexer = prowlarr().await;
let metadata = tmdb(UNRELEASED_METADATA).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action_with_tmdb(&indexer, &downloader, &metadata)
.search_now(&database, 1)
@@ -2367,7 +2338,7 @@ mod tests {
async fn a_manual_search_with_a_grab_in_flight_does_not_grab_again() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
assert_eq!(grabs(&database).await.len(), 1);
@@ -2384,7 +2355,7 @@ mod tests {
async fn a_manual_search_on_a_wanted_movie_still_grabs() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcome = action(&indexer, &downloader)
.search_now(&database, 1)
@@ -2405,7 +2376,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcome = action(&indexer, &downloader)
.search_now(&database, 1)
@@ -2427,7 +2398,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -2441,7 +2412,7 @@ mod tests {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let metadata = tmdb(UNRELEASED_METADATA).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
let action = action_with_tmdb(&indexer, &downloader, &metadata);
for _ in 0..3 {
@@ -2461,7 +2432,7 @@ mod tests {
let (_dir, database) = wanted_movie().await;
let indexer = empty_prowlarr().await;
let metadata = tmdb(RELEASED_METADATA).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
let action = action_with_tmdb(&indexer, &downloader, &metadata);
action.tick(&database).await.unwrap();
@@ -2513,7 +2484,7 @@ mod tests {
let (_dir, database) = wanted_movie().await;
let indexer = empty_prowlarr().await;
let metadata = tmdb(RELEASED_METADATA).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action_with_tmdb(&indexer, &downloader, &metadata)
.tick(&database)
@@ -2568,7 +2539,7 @@ mod tests {
"vote_average": 8.152,"#,
);
let metadata = tmdb(&artwork_metadata).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action_with_tmdb(&indexer, &downloader, &metadata)
.tick(&database)
@@ -2629,7 +2600,7 @@ mod tests {
"vote_average": 8.152,"#,
)
};
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action_with_tmdb(&indexer, &downloader, &tmdb(&unrated()).await)
.tick(&database)
@@ -2690,7 +2661,7 @@ mod tests {
.unwrap();
let indexer = empty_prowlarr().await;
let metadata = tmdb(&RELEASED_METADATA.replace("Dune Part Two", "Dune: Part Two")).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action_with_tmdb(&indexer, &downloader, &metadata)
.tick(&database)
@@ -2713,7 +2684,7 @@ mod tests {
async fn indexer_discovery_is_cached_across_ticks() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
@@ -2744,7 +2715,7 @@ mod tests {
)
.mount(&indexer)
.await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let mut action = action(&indexer, &downloader);
action.indexers.discovery_timeout = Duration::from_millis(50);
@@ -2761,7 +2732,7 @@ mod tests {
assert!(fake.torrents().is_empty());
}
/// §100: Transmission has no route to the indexer, so a download link
/// §100: qBittorrent has no route to the indexer, so a download link
/// that answers with the torrent itself is forwarded as inline metainfo
/// and the link never leaves arr.
#[tokio::test]
@@ -2779,13 +2750,14 @@ mod tests {
)
.mount(&indexer)
.await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
let sent = base64::engine::general_purpose::STANDARD
.decode(&fake.torrents()[0].source)
.expect("the torrent is sent as base64 metainfo");
let sent = fake.torrents()[0]
.metainfo
.clone()
.expect("the torrent file is sent inline");
assert_eq!(sent, TORRENT);
let fetched_with_key = indexer
.received_requests()
@@ -2798,12 +2770,12 @@ mod tests {
}
/// A link that redirects to a magnet — Prowlarr's usual answer — reaches
/// Transmission as the magnet, which it can act on without the indexer.
/// qBittorrent as the magnet, which it can act on without the indexer.
#[tokio::test]
async fn a_link_that_redirects_to_a_magnet_is_sent_as_the_magnet() {
let (_dir, database) = wanted_movie().await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
+176 -78
View File
@@ -5,7 +5,7 @@
//! The torrent's own files are never moved, renamed or deleted — the torrent
//! and the library entry are separate lifecycles (§7.3). A hard-failed
//! release is blacklisted and its grab marked failed, but the torrent keeps
//! seeding until Transmission's own limits clear it.
//! seeding until qBittorrent's own limits clear it.
//!
//! Everything here is idempotent from domain rows (§8): a grab in
//! `downloaded` with no import recorded is the gap, and re-running any prefix
@@ -22,8 +22,8 @@ use arr_core::layout;
use arr_core::policy::{evaluate, Candidate};
use arr_core::{ProbedMedia, Rule, Source, Verdict};
use arr_db::Db;
use arr_dl::TransmissionClient;
use arr_probe::Prober;
use arr_dl::QbitClient;
use arr_probe::{Prober, Skeletons};
use crate::notify::Notifier;
use crate::reconcile::{Action, ActionFuture, Outcome};
@@ -36,8 +36,8 @@ pub enum ImportError {
Database(#[from] sqlx::Error),
#[error("policy: {0}")]
Policy(#[from] arr_db::PolicyError),
#[error("transmission: {0}")]
Transmission(#[from] arr_dl::Error),
#[error("qbit: {0}")]
Qbit(#[from] arr_dl::Error),
#[error("probe: {0}")]
Probe(#[from] arr_probe::Error),
#[error("blocking task: {0}")]
@@ -73,8 +73,13 @@ enum ProbeOutcome {
/// the §7.4 layout.
#[derive(Debug)]
pub struct ImportAction {
transmission: TransmissionClient,
qbit: QbitClient,
prober: Prober,
/// §15, #268. Reads the packet timings of image-format subtitle tracks
/// into a cue skeleton. Here rather than in the subtitle lane because
/// deriving one costs a full demux, and import is where the file is
/// being read and hardlinked anyway.
skeletons: Skeletons,
jellyfin: JellyfinClient,
notifier: Notifier,
/// The operator's ntfy topic (DESIGN.md §9.5), for the *broken*
@@ -96,7 +101,7 @@ pub struct ImportAction {
probed: std::sync::Arc<tokio::sync::Mutex<HashMap<PathBuf, ProbeOutcome>>>,
}
/// A movie grab Transmission finished downloading, not yet imported.
/// A movie grab qBittorrent finished downloading, not yet imported.
#[derive(Debug, Clone)]
struct PendingImport {
grab_id: i64,
@@ -112,15 +117,16 @@ struct PendingImport {
impl ImportAction {
#[must_use]
pub fn new(
transmission: TransmissionClient,
qbit: QbitClient,
prober: Prober,
jellyfin: JellyfinClient,
notifier: Notifier,
operator_topic: Option<String>,
) -> Self {
Self {
transmission,
qbit,
prober,
skeletons: Skeletons::new(),
jellyfin,
notifier,
operator_topic,
@@ -176,6 +182,94 @@ impl ImportAction {
Ok(files)
}
/// Derive the cue skeleton of every non-forced image-format subtitle
/// track in a file that has just been placed (§15, #268).
///
/// Detached, like the probes above: one derivation is a full demux and
/// the reconcile lane's tick budget is 25 s. Nothing downstream waits on
/// it — a skeleton that never lands, because the process died or because
/// the packets did not pair, reads exactly like a track that has none,
/// and the subtitle lane aligns against the video instead.
async fn spawn_skeletons(&self, database: &Db, placed: &Path, feature: &arr_probe::ProbedFile) {
let tracks: Vec<(usize, String)> = feature
.media
.subtitle_tracks
.iter()
.enumerate()
.filter(|(_, track)| !track.forced && track.codec.is_image())
.map(|(index, track)| (index, track.language.to_string()))
.collect();
if tracks.is_empty() {
return;
}
let path_text = placed.to_string_lossy().into_owned();
let media_file_id = match sqlx::query_scalar!(
r#"SELECT id AS "id!: i64" FROM media_files WHERE path = ?"#,
path_text
)
.fetch_optional(database.pool())
.await
{
Ok(Some(id)) => id,
Ok(None) => return,
Err(error) => {
tracing::warn!(path = %placed.display(), %error, "no media file row to hang a cue skeleton on");
return;
}
};
let skeletons = self.skeletons.clone();
let database = database.clone();
let video = placed.to_path_buf();
let runtime = feature.duration;
tokio::spawn(async move {
for (stream_index, language) in tracks {
let outcome = skeletons.derive(&video, stream_index, Some(runtime)).await;
let index = i64::try_from(stream_index).unwrap_or(i64::MAX);
match outcome {
Ok(arr_probe::SkeletonOutcome::Derived(skeleton)) => {
let cues: Vec<(i64, i64)> = skeleton
.cues
.iter()
.map(|cue| (millis(cue.start), millis(cue.end)))
.collect();
if let Err(error) = arr_db::skeletons::record(
database.pool(),
media_file_id,
index,
&language,
&cues,
)
.await
{
tracing::warn!(%error, media_file_id, stream_index, "cue skeleton not recorded");
} else {
tracing::info!(
media_file_id,
stream_index,
cues = cues.len(),
"cue skeleton derived"
);
}
}
Ok(arr_probe::SkeletonOutcome::Implausible(reason)) => tracing::info!(
media_file_id,
stream_index,
%reason,
"cue skeleton refused, alignment falls back to the video"
),
Err(error) => tracing::warn!(
media_file_id,
stream_index,
%error,
"cue skeleton could not be read"
),
}
}
});
}
/// Drop a settled grab's probe results — imported or blacklisted, they
/// will not be needed again.
async fn forget_probes(&self, paths: &[PathBuf]) {
@@ -345,6 +439,7 @@ impl ImportAction {
tokio::task::spawn_blocking(move || place(&source_path, &link_target)).await??;
record_import(database, pending, &feature, waiver.as_ref(), &destination).await?;
self.spawn_skeletons(database, &destination, &feature).await;
self.forget_probes(&paths).await;
self.refresh_jellyfin().await;
let path_text = destination.to_string_lossy().into_owned();
@@ -388,7 +483,7 @@ impl ImportAction {
}
}
/// The torrent's files as safe local paths, or `None` when Transmission
/// The torrent's files as safe local paths, or `None` when qBittorrent
/// no longer has the torrent.
///
/// Torrent-declared names are untrusted input: an absolute or
@@ -399,13 +494,13 @@ impl ImportAction {
grab_id: i64,
infohash: &str,
) -> Result<Option<Vec<PathBuf>>, ImportError> {
let Some(content) = self.transmission.torrent_content(infohash).await? else {
// Gone from Transmission — the caller marks the grab vanished
let Some(content) = self.qbit.torrent_content(infohash).await? else {
// Gone from qBittorrent — the caller marks the grab vanished
// and parks the target (#108).
tracing::warn!(
grab_id,
infohash,
"downloaded grab has no torrent in Transmission; not importing"
"downloaded grab has no torrent in qBittorrent; not importing"
);
return Ok(None);
};
@@ -591,6 +686,7 @@ impl ImportAction {
record_episode_import(database, episode.id, feature, waiver.as_ref(), &destination)
.await?;
self.spawn_skeletons(database, &destination, feature).await;
imported += 1;
tracing::info!(
grab_id = pending.grab_id,
@@ -671,7 +767,7 @@ impl ImportAction {
))
}
/// §86/#108: a `downloaded` grab whose torrent Transmission no longer
/// §86/#108: a `downloaded` grab whose torrent qBittorrent no longer
/// reports — removed by hand, not a policy failure. Marked `vanished`
/// rather than `failed` so it does not feed the `needs_decision` queue
/// (attention.rs). Nothing is blacklisted, since the release itself
@@ -692,11 +788,11 @@ impl ImportAction {
movie_id = pending.movie_id,
title = pending.title,
release = pending.release_name,
"torrent vanished from Transmission; movie parked"
"torrent vanished from qBittorrent; movie parked"
);
Ok(Outcome::new(
format!(
"grab {} downloaded, torrent vanished from Transmission",
"grab {} downloaded, torrent vanished from qBittorrent",
pending.grab_id
),
format!("parked movie {}", pending.movie_id),
@@ -726,11 +822,11 @@ impl ImportAction {
grab_id = pending.grab_id,
series = pending.series_title,
release = pending.release_name,
"torrent vanished from Transmission; target parked"
"torrent vanished from qBittorrent; target parked"
);
Ok(Outcome::new(
format!(
"grab {} downloaded, torrent vanished from Transmission",
"grab {} downloaded, torrent vanished from qBittorrent",
pending.grab_id
),
format!("parked {target_kind} {target_id}"),
@@ -897,7 +993,7 @@ async fn record_import(
Ok(())
}
/// The gap, straight out of the domain rows (§8): a movie grab Transmission
/// The gap, straight out of the domain rows (§8): a movie grab qBittorrent
/// finished that no import has settled.
async fn pending_imports(database: &Db) -> Result<Vec<PendingImport>, ImportError> {
let rows = sqlx::query!(
@@ -935,7 +1031,7 @@ async fn pending_imports(database: &Db) -> Result<Vec<PendingImport>, ImportErro
.collect())
}
/// A TV grab Transmission finished downloading — one episode or a season
/// A TV grab qBittorrent finished downloading — one episode or a season
/// pack — not yet imported.
#[derive(Debug, Clone)]
struct PendingTvImport {
@@ -1200,6 +1296,11 @@ async fn record_episode_import(
Ok(())
}
/// Milliseconds, saturating. A cue past 292 million years does not exist.
fn millis(value: std::time::Duration) -> i64 {
i64::try_from(value.as_millis()).unwrap_or(i64::MAX)
}
/// The `probed` column (§4, §5.6): what `ffprobe` found, in the spellings the
/// policy columns use.
fn probed_json(media: &ProbedMedia) -> serde_json::Value {
@@ -1397,18 +1498,39 @@ mod tests {
path
}
async fn harness(media_json: &str) -> Harness {
harness_with(
media_json,
json!([
{"name": "Dune/Dune.mkv", "length": 13, "bytesCompleted": 13},
{"name": "Dune/Dune.nfo", "length": 10, "bytesCompleted": 10}
]),
)
.await
/// A qBittorrent holding exactly one torrent, `INFOHASH`, with `files`
/// under `save_path`.
async fn qbit(save_path: &str, files: &[(&str, u64)]) -> MockServer {
let server = MockServer::start().await;
let torrent = json!({
"hash": INFOHASH, "name": RELEASE_NAME, "state": "uploading",
"progress": 1.0, "save_path": save_path, "tags": "movies-main",
"ratio": 0.1, "ratio_limit": 1.5, "seeding_time": 1,
"seeding_time_limit": -1, "inactive_seeding_time_limit": -1,
"last_activity": 0
});
Mock::given(method("GET"))
.and(path("/api/v2/torrents/info"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!([torrent])))
.mount(&server)
.await;
let files: Vec<serde_json::Value> = files
.iter()
.map(|(name, size)| json!({"name": name, "size": size}))
.collect();
Mock::given(method("GET"))
.and(path("/api/v2/torrents/files"))
.respond_with(ResponseTemplate::new(200).set_body_json(files))
.mount(&server)
.await;
server
}
async fn harness_with(media_json: &str, files: serde_json::Value) -> Harness {
async fn harness(media_json: &str) -> Harness {
harness_with(media_json, &[("Dune/Dune.mkv", 13), ("Dune/Dune.nfo", 10)]).await
}
async fn harness_with(media_json: &str, files: &[(&str, u64)]) -> Harness {
let dir = tempfile::tempdir().unwrap();
let downloads = dir.path().join("downloads");
let library = dir.path().join("library");
@@ -1453,18 +1575,7 @@ mod tests {
.unwrap();
insert_owner(&database, "movie", 1, "Alice", "alice-topic").await;
let server = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"result": "success",
"arguments": {"torrents": [{
"hashString": INFOHASH,
"downloadDir": downloads.to_string_lossy(),
"files": files
}]}
})))
.mount(&server)
.await;
let server = qbit(&downloads.to_string_lossy(), files).await;
let jellyfin_server = start_jellyfin_server().await;
let ntfy_server = start_ntfy_server().await;
@@ -1473,7 +1584,7 @@ mod tests {
let jellyfin = JellyfinClient::new(jellyfin_server.uri(), None).unwrap();
let notifier = Notifier::new(ntfy_server.uri()).unwrap();
let action = ImportAction::new(
TransmissionClient::new(&server.uri()).unwrap(),
QbitClient::new(&server.uri()).unwrap(),
prober,
jellyfin,
notifier,
@@ -1727,7 +1838,7 @@ mod tests {
assert!(outcomes.is_empty());
}
/// #108, overriding §86: a `downloaded` grab whose torrent Transmission
/// #108, overriding §86: a `downloaded` grab whose torrent qBittorrent
/// no longer reports — removed by hand, not a policy failure — is marked
/// `vanished` and parks the movie (`wanted` cleared) instead of
/// reopening it as a gap, without touching the blacklist.
@@ -1764,17 +1875,15 @@ mod tests {
.unwrap();
let server = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"result": "success",
"arguments": {"torrents": []}
})))
Mock::given(method("GET"))
.and(path("/api/v2/torrents/info"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!([])))
.mount(&server)
.await;
let jellyfin_server = start_jellyfin_server().await;
let ntfy_server = start_ntfy_server().await;
let action = ImportAction::new(
TransmissionClient::new(&server.uri()).unwrap(),
QbitClient::new(&server.uri()).unwrap(),
Prober::new().with_binary(fake_ffprobe(dir.path(), HDR10_PROBE)),
JellyfinClient::new(jellyfin_server.uri(), None).unwrap(),
Notifier::new(ntfy_server.uri()).unwrap(),
@@ -1809,11 +1918,11 @@ mod tests {
async fn hostile_torrent_paths_never_leave_the_download_root() {
let h = harness_with(
HDR10_PROBE,
json!([
{"name": "../outside.mkv", "length": 13, "bytesCompleted": 13},
{"name": "/tmp/absolute.mkv", "length": 13, "bytesCompleted": 13},
{"name": "Dune/Dune.mkv", "length": 13, "bytesCompleted": 13}
]),
&[
("../outside.mkv", 13),
("/tmp/absolute.mkv", 13),
("Dune/Dune.mkv", 13),
],
)
.await;
@@ -1832,11 +1941,7 @@ mod tests {
/// hard fail, not an escape.
#[tokio::test]
async fn a_torrent_of_only_hostile_paths_hard_fails() {
let h = harness_with(
HDR10_PROBE,
json!([{"name": "../../etc/passwd", "length": 13, "bytesCompleted": 13}]),
)
.await;
let h = harness_with(HDR10_PROBE, &[("../../etc/passwd", 13)]).await;
let outcomes = h.action.tick(&h.database).await.unwrap();
@@ -1885,7 +1990,7 @@ mod tests {
std::fs::set_permissions(&script, permissions).unwrap();
let action = ImportAction::new(
TransmissionClient::new("http://127.0.0.1:1").unwrap(),
QbitClient::new("http://127.0.0.1:1").unwrap(),
Prober::new().with_binary(&script),
JellyfinClient::new("http://127.0.0.1:1", None).unwrap(),
Notifier::new("http://127.0.0.1:1").unwrap(),
@@ -1932,7 +2037,7 @@ mod tests {
permissions.set_mode(0o755);
std::fs::set_permissions(&script, permissions).unwrap();
let action = ImportAction::new(
TransmissionClient::new("http://127.0.0.1:1").unwrap(),
QbitClient::new("http://127.0.0.1:1").unwrap(),
Prober::new().with_binary(&script),
JellyfinClient::new("http://127.0.0.1:1", None).unwrap(),
Notifier::new("http://127.0.0.1:1").unwrap(),
@@ -2070,21 +2175,14 @@ mod tests {
.unwrap();
insert_owner(&database, "series", 1, "Bob", "bob-topic").await;
let server = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"result": "success",
"arguments": {"torrents": [{
"hashString": INFOHASH,
"downloadDir": downloads.to_string_lossy(),
"files": [
{"name": "Fallout.S01/Fallout.S01E01.mkv", "length": 2, "bytesCompleted": 2},
{"name": "Fallout.S01/Fallout.S01E02.mkv", "length": 2, "bytesCompleted": 2}
]
}]}
})))
.mount(&server)
.await;
let server = qbit(
&downloads.to_string_lossy(),
&[
("Fallout.S01/Fallout.S01E01.mkv", 2),
("Fallout.S01/Fallout.S01E02.mkv", 2),
],
)
.await;
let jellyfin_server = start_jellyfin_server().await;
let ntfy_server = start_ntfy_server().await;
@@ -2092,7 +2190,7 @@ mod tests {
let jellyfin = JellyfinClient::new(jellyfin_server.uri(), None).unwrap();
let notifier = Notifier::new(ntfy_server.uri()).unwrap();
let action = ImportAction::new(
TransmissionClient::new(&server.uri()).unwrap(),
QbitClient::new(&server.uri()).unwrap(),
prober,
jellyfin,
notifier,
+35 -50
View File
@@ -9,6 +9,8 @@ mod indexers;
mod manual;
mod metadata;
mod notify;
#[cfg(test)]
mod qbit_fake;
mod reaper;
pub mod reconcile;
mod rss;
@@ -92,8 +94,8 @@ enum Error {
Tmdb(#[from] arr_meta::Error),
#[error("prowlarr client: {0}")]
Prowlarr(#[from] arr_indexer::Error),
#[error("transmission client: {0}")]
Transmission(#[from] arr_dl::Error),
#[error("qbit client: {0}")]
Qbit(#[from] arr_dl::Error),
#[error("jellyfin client: {0}")]
Jellyfin(#[from] arr_api::jellyfin::Error),
#[error("ntfy client: {0}")]
@@ -118,19 +120,18 @@ fn api_state(
database: &Db,
jellyfin: arr_api::jellyfin::JellyfinClient,
translators: &Translators,
transmission: &arr_dl::TransmissionClient,
qbit: &arr_dl::QbitClient,
) -> Result<AppState, Error> {
let mut upstreams =
Upstreams::new(config.prowlarr_url.clone(), config.transmission_url.clone())
.with_prowlarr_api_key(config.prowlarr_api_key.clone())
.with_tmdb_api_key(config.tmdb_api_key.clone());
let mut upstreams = Upstreams::new(config.prowlarr_url.clone(), config.qbittorrent_url.clone())
.with_prowlarr_api_key(config.prowlarr_api_key.clone())
.with_tmdb_api_key(config.tmdb_api_key.clone());
if let Some(tmdb_url) = config.tmdb_url.clone() {
upstreams = upstreams.with_tmdb_url(tmdb_url);
}
let mut state = AppState::new(upstreams)?
.with_database(database.clone())
.with_transmission(transmission.clone())
.with_qbit(qbit.clone())
.with_subtitle_providers(subtitle_providers(
config.opensubtitles_api_key.clone(),
config.opensubtitles_username.clone(),
@@ -158,7 +159,14 @@ async fn run() -> Result<(), Error> {
let database = Db::connect(&config.database_path).await?;
database.migrate().await?;
let transmission = arr_dl::TransmissionClient::new(&config.transmission_url)?;
let qbit = match (&config.qbittorrent_username, &config.qbittorrent_password) {
(Some(username), Some(password)) => arr_dl::QbitClient::with_credentials(
&config.qbittorrent_url,
username.clone(),
password.clone(),
),
_ => arr_dl::QbitClient::new(&config.qbittorrent_url),
}?;
let tmdb = if let Some(key) = &config.tmdb_api_key {
let mut client = TmdbClient::builder(key.clone());
if let Some(url) = &config.tmdb_url {
@@ -178,7 +186,7 @@ async fn run() -> Result<(), Error> {
let (reconcile, manual_grab, manual_tv) = reconcile_loop(
&database,
&config,
&transmission,
&qbit,
tmdb.as_ref(),
&notifier,
&translators,
@@ -192,13 +200,7 @@ async fn run() -> Result<(), Error> {
compat = compat.with_tmdb(tmdb);
}
let state = api_state(
&config,
&database,
api_jellyfin,
&translators,
&transmission,
)?;
let state = api_state(&config, &database, api_jellyfin, &translators, &qbit)?;
let app = arr_api::router(state.clone())
.merge(arr_compat::router(compat))
.fallback(web::serve)
@@ -329,7 +331,7 @@ async fn seed_translator_settings(database: &Db, translators: &Translators) -> R
fn reconcile_loop(
database: &Db,
config: &Config,
transmission: &arr_dl::TransmissionClient,
qbit: &arr_dl::QbitClient,
tmdb: Option<&Arc<TmdbClient>>,
notifier: &Notifier,
translators: &Translators,
@@ -360,27 +362,16 @@ fn reconcile_loop(
.map(|key| arr_indexer::ProwlarrClient::new(config.prowlarr_url.clone(), key))
.transpose()?;
let (reconcile, manual_grab) = register_movie_grab(
reconcile,
prowlarr.as_ref(),
transmission,
config,
&seeding,
tmdb,
);
let (reconcile, manual_grab) =
register_movie_grab(reconcile, prowlarr.as_ref(), qbit, config, &seeding, tmdb);
let (mut reconcile, manual_tv) =
register_tv_grab(reconcile, prowlarr.as_ref(), transmission, config, &seeding);
register_tv_grab(reconcile, prowlarr.as_ref(), qbit, config, &seeding);
// RSS needs no TMDB: it matches what the feeds already carry against the
// wanted list (§6.2).
if let Some(prowlarr) = prowlarr {
reconcile = reconcile.register(
Tick::Rss,
RssAction::new(
prowlarr,
transmission.clone(),
config.download_dir.clone(),
seeding,
),
RssAction::new(prowlarr, qbit.clone(), config.download_dir.clone(), seeding),
);
}
// §8: metadata refresh is its own daily lane, staggered against the
@@ -396,7 +387,7 @@ fn reconcile_loop(
reconcile = reconcile.register(
Tick::Reconcile,
ImportAction::new(
transmission.clone(),
qbit.clone(),
arr_probe::Prober::new(),
jellyfin,
notifier.clone(),
@@ -430,11 +421,11 @@ fn reconcile_loop(
);
}
let reconcile = reconcile.register(Tick::Reaper, ReaperAction::new(transmission.clone()));
let reconcile = reconcile.register(Tick::Reaper, ReaperAction::new(qbit.clone()));
Ok((reconcile, manual_grab, manual_tv))
}
/// The *broken* lane (#200 included): Prowlarr, Transmission and TMDB, plus
/// The *broken* lane (#200 included): Prowlarr, qBittorrent and TMDB, plus
/// the subtitle lamps — an enabled provider, the selected engine, or a
/// missing `alass`/`ffmpeg` all fold into the same operator message.
fn register_broken(
@@ -447,7 +438,7 @@ fn register_broken(
let broken_upstreams = broken::Upstreams {
prowlarr_url: config.prowlarr_url.clone(),
prowlarr_api_key: config.prowlarr_api_key.clone(),
transmission_url: config.transmission_url.clone(),
qbittorrent_url: config.qbittorrent_url.clone(),
tmdb_url: config
.tmdb_url
.clone()
@@ -483,7 +474,7 @@ fn register_broken(
fn register_tv_grab(
mut reconcile: ReconcileLoop,
prowlarr: Option<&arr_indexer::ProwlarrClient>,
transmission: &arr_dl::TransmissionClient,
qbit: &arr_dl::QbitClient,
config: &Config,
seeding: &SeedingRules,
) -> (ReconcileLoop, Option<TvGrabAction>) {
@@ -493,7 +484,7 @@ fn register_tv_grab(
let tv_grab_action = || {
TvGrabAction::new(
prowlarr.clone(),
transmission.clone(),
qbit.clone(),
config.download_dir.clone(),
seeding.clone(),
)
@@ -511,7 +502,7 @@ fn register_tv_grab(
fn register_movie_grab(
mut reconcile: ReconcileLoop,
prowlarr: Option<&arr_indexer::ProwlarrClient>,
transmission: &arr_dl::TransmissionClient,
qbit: &arr_dl::QbitClient,
config: &Config,
seeding: &SeedingRules,
tmdb: Option<&Arc<TmdbClient>>,
@@ -522,29 +513,23 @@ fn register_movie_grab(
};
reconcile = reconcile.register(
Tick::Reconcile,
movie_grab_action(prowlarr, transmission, config, seeding, tmdb),
movie_grab_action(prowlarr, qbit, config, seeding, tmdb),
);
let manual_grab = Some(movie_grab_action(
prowlarr,
transmission,
config,
seeding,
tmdb,
));
let manual_grab = Some(movie_grab_action(prowlarr, qbit, config, seeding, tmdb));
(reconcile, manual_grab)
}
/// The movie grab lane, built fresh for each caller.
fn movie_grab_action(
prowlarr: &arr_indexer::ProwlarrClient,
transmission: &arr_dl::TransmissionClient,
qbit: &arr_dl::QbitClient,
config: &Config,
seeding: &SeedingRules,
tmdb: &Arc<TmdbClient>,
) -> GrabAction {
GrabAction::new(
prowlarr.clone(),
transmission.clone(),
qbit.clone(),
config.download_dir.clone(),
seeding.clone(),
)
+12 -20
View File
@@ -1,7 +1,7 @@
//! Drains `AppState`'s three command channels — the daemon-side consumer
//! DESIGN.md §6.2 and §9.3 assume exists (issues #107 and #132). A `Search`
//! resets backoff and re-runs the targeted-search + grab lane immediately;
//! `Grab` sends an already-chosen release straight to Transmission, skipping
//! `Grab` sends an already-chosen release straight to qBittorrent, skipping
//! search. Movies, episodes and seasons share one lane because they share
//! one operator waiting on a 202.
@@ -149,7 +149,7 @@ mod tests {
use std::path::PathBuf;
use arr_db::Db;
use arr_dl::TransmissionClient;
use arr_dl::QbitClient;
use arr_indexer::ProwlarrClient;
use wiremock::matchers::{method, path, path_regex, query_param};
use wiremock::{Mock, MockServer, ResponseTemplate};
@@ -172,23 +172,15 @@ mod tests {
(dir, database)
}
async fn transmission() -> MockServer {
let server = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({
"result": "success",
"arguments": {"torrent-added": {"id": 1, "name": "x", "hashString": "aaaa"}}
})))
.mount(&server)
.await;
server
async fn qbit() -> MockServer {
crate::qbit_fake::FakeQbit::start().await.0
}
fn grab_action(indexer: &MockServer, transmission: &MockServer) -> GrabAction {
fn grab_action(indexer: &MockServer, qbit: &MockServer) -> GrabAction {
GrabAction::new(
ProwlarrClient::new(indexer.uri(), "key").unwrap(),
TransmissionClient::new(&transmission.uri()).unwrap(),
PathBuf::from("/mnt/media/transmission/complete"),
QbitClient::new(&qbit.uri()).unwrap(),
PathBuf::from("/mnt/media/qbittorrent/complete"),
SeedingRules::new(
SeedingLimits {
ratio: 1.5,
@@ -239,8 +231,8 @@ mod tests {
)
.mount(&indexer)
.await;
let transmission = transmission().await;
let grab = grab_action(&indexer, &transmission);
let qbit = qbit().await;
let grab = grab_action(&indexer, &qbit);
handle_movie(&grab, &database, MovieCommand::Search { movie_id: 1 })
.await
@@ -269,7 +261,7 @@ mod tests {
}
/// Issue #107: a `Grab` command sends the already-chosen release straight
/// to Transmission — no indexer search at all.
/// to qBittorrent — no indexer search at all.
#[tokio::test]
async fn a_grab_command_sends_the_chosen_release_without_searching() {
let (_dir, database) = database_with_wanted_movie().await;
@@ -282,8 +274,8 @@ mod tests {
))
.mount(&indexer)
.await;
let transmission = transmission().await;
let grab = grab_action(&indexer, &transmission);
let qbit = qbit().await;
let grab = grab_action(&indexer, &qbit);
let release_id = sqlx::query(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, score, verdict)
+342
View File
@@ -0,0 +1,342 @@
//! An in-memory qBittorrent for the daemon's tests.
//!
//! Shared rather than per-module because every lane that grabs needs the same
//! two behaviours to be right, and both are easy to fake wrongly: `torrents/add`
//! answers `Ok.` with no hash, so the fake has to derive the same infohash the
//! client did, and adding the same source twice has to collapse to one torrent
//! or the restart and re-grab tests prove nothing.
#![allow(clippy::unwrap_used, dead_code)]
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use arr_dl::TorrentSource;
use serde_json::{json, Value};
use wiremock::matchers::any;
use wiremock::{Mock, MockServer, Request, Respond, ResponseTemplate};
#[derive(Clone, Debug)]
pub(crate) struct FakeTorrent {
pub hash: String,
pub name: String,
/// The magnet URI, or `<inline .torrent>` for a torrent sent as bytes.
pub source: String,
/// The bytes of a torrent sent inline, so a test can prove arr forwarded
/// the file rather than the indexer link.
pub metainfo: Option<Vec<u8>>,
pub labels: Vec<String>,
pub save_path: String,
pub progress: f64,
pub state: String,
pub ratio_limit: f64,
pub idle_limit_minutes: i64,
/// `(path, size)`, relative to `save_path`.
pub files: Vec<(String, u64)>,
}
#[derive(Clone, Debug, Default)]
pub(crate) struct FakeQbit {
pub torrents: Arc<Mutex<Vec<FakeTorrent>>>,
/// Every source ever added, in order, kept across removals.
pub added: Arc<Mutex<Vec<String>>>,
}
impl FakeQbit {
/// Start a mock server backed by a fresh fake.
pub(crate) async fn start() -> (MockServer, Self) {
let server = MockServer::start().await;
let fake = Self::default();
Mock::given(any())
.respond_with(fake.clone())
.mount(&server)
.await;
(server, fake)
}
pub(crate) fn torrents(&self) -> Vec<FakeTorrent> {
self.torrents.lock().unwrap().clone()
}
pub(crate) fn complete_all(&self) {
for torrent in self.torrents.lock().unwrap().iter_mut() {
torrent.progress = 1.0;
torrent.state = "uploading".into();
}
}
/// Give every torrent the same file list and download directory, for the
/// import lane.
pub(crate) fn set_contents(&self, save_path: &str, files: &[(&str, u64)]) {
for torrent in self.torrents.lock().unwrap().iter_mut() {
torrent.save_path = save_path.to_owned();
torrent.files = files
.iter()
.map(|(path, size)| ((*path).to_owned(), *size))
.collect();
}
}
fn add(&self, request: &Request) -> ResponseTemplate {
let parts = multipart(request);
let (source, name, metainfo) = if let Some(urls) = parts.get("urls") {
let uri = String::from_utf8_lossy(urls).trim().to_owned();
let name = uri.split_once("dn=").map_or_else(
|| uri.clone(),
|(_, rest)| rest.split('&').next().unwrap_or_default().to_owned(),
);
(TorrentSource::Magnet(uri), name, None)
} else {
let bytes = parts.get("torrents").cloned().unwrap_or_default();
(
TorrentSource::Metainfo(bytes.clone()),
"<inline .torrent>".to_owned(),
Some(bytes),
)
};
let Ok(hash) = arr_dl::infohash(&source) else {
// The real client refuses these before the call, so reaching here
// means the fixture is malformed.
return ResponseTemplate::new(415).set_body_string("invalid torrent");
};
let display = match &source {
TorrentSource::Magnet(uri) => uri.clone(),
TorrentSource::Metainfo(_) => "<inline .torrent>".to_owned(),
};
self.added.lock().unwrap().push(display.clone());
let mut torrents = self.torrents.lock().unwrap();
if !torrents.iter().any(|torrent| torrent.hash == hash) {
torrents.push(FakeTorrent {
hash,
name,
source: display,
metainfo,
labels: text(&parts, "tags")
.split(',')
.filter(|tag| !tag.is_empty())
.map(ToOwned::to_owned)
.collect(),
save_path: text(&parts, "savepath"),
progress: 0.0,
state: "downloading".into(),
ratio_limit: -1.0,
idle_limit_minutes: -1,
files: Vec::new(),
});
}
ResponseTemplate::new(200).set_body_string("Ok.")
}
fn info(&self, request: &Request) -> ResponseTemplate {
let wanted = query(request, "hashes");
let torrents: Vec<Value> = self
.torrents()
.into_iter()
.filter(|torrent| {
wanted
.as_ref()
.is_none_or(|hash| torrent.hash.eq_ignore_ascii_case(hash))
})
.map(|torrent| {
json!({
"hash": torrent.hash,
"name": torrent.name,
"state": torrent.state,
"progress": torrent.progress,
"dlspeed": 0,
"save_path": torrent.save_path,
"tags": torrent.labels.join(","),
"ratio": 0.0,
"ratio_limit": torrent.ratio_limit,
"seeding_time": 0,
"seeding_time_limit": -1,
"inactive_seeding_time_limit": torrent.idle_limit_minutes,
"last_activity": 0
})
})
.collect();
ResponseTemplate::new(200).set_body_json(torrents)
}
fn files(&self, request: &Request) -> ResponseTemplate {
let hash = query(request, "hash").unwrap_or_default();
let files: Vec<Value> = self
.torrents()
.into_iter()
.find(|torrent| torrent.hash.eq_ignore_ascii_case(&hash))
.map(|torrent| {
torrent
.files
.into_iter()
.map(|(path, size)| json!({"name": path, "size": size}))
.collect()
})
.unwrap_or_default();
ResponseTemplate::new(200).set_body_json(files)
}
fn mutate(&self, path: &str, request: &Request) -> ResponseTemplate {
let form = urlencoded(request);
let hashes = form.get("hashes").cloned().unwrap_or_default();
for torrent in self.torrents.lock().unwrap().iter_mut() {
if !hashes
.split('|')
.any(|hash| hash.eq_ignore_ascii_case(&torrent.hash))
{
continue;
}
match path {
"addTags" => {
for tag in form.get("tags").cloned().unwrap_or_default().split(',') {
if !tag.is_empty() && !torrent.labels.iter().any(|held| held == tag) {
torrent.labels.push(tag.to_owned());
}
}
}
"setLocation" => {
torrent.save_path = form.get("location").cloned().unwrap_or_default();
}
"setShareLimits" => {
torrent.ratio_limit = form
.get("ratioLimit")
.and_then(|value| value.parse().ok())
.unwrap_or(-1.0);
torrent.idle_limit_minutes = form
.get("inactiveSeedingTimeLimit")
.and_then(|value| value.parse().ok())
.unwrap_or(-1);
}
_ => {}
}
}
ResponseTemplate::new(200).set_body_string("")
}
fn delete(&self, request: &Request) -> ResponseTemplate {
let form = urlencoded(request);
let hashes = form.get("hashes").cloned().unwrap_or_default();
self.torrents.lock().unwrap().retain(|torrent| {
!hashes
.split('|')
.any(|hash| hash.eq_ignore_ascii_case(&torrent.hash))
});
ResponseTemplate::new(200).set_body_string("")
}
}
impl Respond for FakeQbit {
fn respond(&self, request: &Request) -> ResponseTemplate {
let path = request.url.path().to_owned();
match path.rsplit('/').next().unwrap_or_default() {
"add" => self.add(request),
"info" => self.info(request),
"files" => self.files(request),
"delete" => self.delete(request),
endpoint @ ("addTags" | "setLocation" | "setShareLimits") => {
self.mutate(endpoint, request)
}
_ => ResponseTemplate::new(200).set_body_string("Ok."),
}
}
}
fn query(request: &Request, key: &str) -> Option<String> {
request
.url
.query_pairs()
.find(|(name, _)| name == key)
.map(|(_, value)| value.into_owned())
}
fn urlencoded(request: &Request) -> HashMap<String, String> {
String::from_utf8_lossy(&request.body)
.split('&')
.filter_map(|pair| pair.split_once('='))
.map(|(key, value)| (decode(key), decode(value)))
.collect()
}
fn decode(input: &str) -> String {
let bytes = input.replace('+', " ").into_bytes();
let mut out = Vec::with_capacity(bytes.len());
let mut at = 0;
while at < bytes.len() {
if bytes[at] == b'%' && at + 2 < bytes.len() {
if let Some(byte) = std::str::from_utf8(&bytes[at + 1..at + 3])
.ok()
.and_then(|pair| u8::from_str_radix(pair, 16).ok())
{
out.push(byte);
at += 3;
continue;
}
}
out.push(bytes[at]);
at += 1;
}
String::from_utf8_lossy(&out).into_owned()
}
fn text(parts: &HashMap<String, Vec<u8>>, key: &str) -> String {
parts
.get(key)
.map(|bytes| String::from_utf8_lossy(bytes).into_owned())
.unwrap_or_default()
}
/// The named parts of a `multipart/form-data` body.
fn multipart(request: &Request) -> HashMap<String, Vec<u8>> {
let Some(boundary) = request
.headers
.get("content-type")
.and_then(|value| value.to_str().ok())
.and_then(|value| value.split("boundary=").nth(1))
.map(|value| format!("--{}", value.trim_matches('"')))
else {
return HashMap::new();
};
let mut parts = HashMap::new();
for section in split(&request.body, boundary.as_bytes()).skip(1) {
let Some(header_end) = find(section, b"\r\n\r\n") else {
continue;
};
let headers = String::from_utf8_lossy(&section[..header_end]);
let Some(name) = headers
.split("name=\"")
.nth(1)
.and_then(|rest| rest.split('"').next())
else {
continue;
};
let body = &section[header_end + 4..];
let body = body.strip_suffix(b"\r\n").unwrap_or(body);
parts.insert(name.to_owned(), body.to_vec());
}
parts
}
fn split<'a>(haystack: &'a [u8], needle: &'a [u8]) -> impl Iterator<Item = &'a [u8]> {
let mut rest = haystack;
std::iter::from_fn(move || {
if rest.is_empty() {
return None;
}
if let Some(at) = find(rest, needle) {
let (section, tail) = rest.split_at(at);
rest = &tail[needle.len()..];
Some(section)
} else {
let section = rest;
rest = &[];
Some(section)
}
})
}
fn find(haystack: &[u8], needle: &[u8]) -> Option<usize> {
haystack
.windows(needle.len())
.position(|window| window == needle)
}
+84 -55
View File
@@ -1,36 +1,36 @@
//! Removes arr torrents only after Transmission says their seeding obligation
//! Removes arr torrents only after qBittorrent says their seeding obligation
//! is complete. The library/import state is deliberately not consulted (§7.3).
use std::collections::HashSet;
use arr_db::Db;
use arr_dl::TransmissionClient;
use arr_dl::QbitClient;
use crate::grab::label_for_root;
use crate::reconcile::{Action, ActionFuture, Outcome};
#[derive(Debug)]
pub struct ReaperAction {
transmission: TransmissionClient,
qbit: QbitClient,
}
impl ReaperAction {
#[must_use]
pub fn new(transmission: TransmissionClient) -> Self {
Self { transmission }
pub fn new(qbit: QbitClient) -> Self {
Self { qbit }
}
async fn tick(&self, labels: &HashSet<String>) -> Result<Vec<Outcome>, arr_dl::Error> {
let torrents = self.transmission.list_torrents().await?;
let torrents = self.qbit.list_torrents().await?;
let mut outcomes = Vec::new();
for torrent in torrents {
if !torrent.is_finished || !torrent.labels.iter().any(|label| labels.contains(label)) {
continue;
}
self.transmission.remove_torrent(torrent.id, true).await?;
self.qbit.remove_torrent(&torrent.hash, true).await?;
outcomes.push(Outcome::new(
format!("torrent {} finished seeding", torrent.hash),
format!("removed torrent {} and its download data", torrent.id),
format!("removed torrent {} and its download data", torrent.name),
));
}
Ok(outcomes)
@@ -64,85 +64,114 @@ mod tests {
use std::sync::{Arc, Mutex};
use serde_json::{json, Value};
use wiremock::matchers::any;
use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, Request, Respond, ResponseTemplate};
use super::*;
/// Records what was asked to be deleted, so a test can assert the reaper
/// touched one torrent and not its neighbours.
#[derive(Clone)]
struct Transmission {
torrents: Value,
removed: Arc<Mutex<Vec<Value>>>,
}
struct Deletions(Arc<Mutex<Vec<String>>>);
impl Respond for Transmission {
impl Respond for Deletions {
fn respond(&self, request: &Request) -> ResponseTemplate {
let body: Value = serde_json::from_slice(&request.body).unwrap();
let arguments = match body["method"].as_str().unwrap() {
"torrent-get" => json!({"torrents": self.torrents}),
"torrent-remove" => {
self.removed.lock().unwrap().push(body["arguments"].clone());
json!({})
}
method => panic!("unexpected method {method}"),
};
ResponseTemplate::new(200)
.insert_header("x-transmission-session-id", "session")
.set_body_json(json!({"result": "success", "arguments": arguments}))
let body = String::from_utf8_lossy(&request.body).into_owned();
self.0.lock().unwrap().push(body);
ResponseTemplate::new(200).set_body_string("")
}
}
/// A torrent qBittorrent stopped on the ratio limit arr set on it, which
/// is what "finished seeding" means to the reaper.
fn torrent(id: i64, label: &str, finished: bool) -> Value {
json!({
"id": id, "name": format!("torrent-{id}"), "hashString": format!("hash-{id}"),
"status": if finished { 0 } else { 6 }, "percentDone": 1.0,
"downloadDir": "/downloads", "labels": [label], "isFinished": finished
"hash": format!("hash-{id}"), "name": format!("torrent-{id}"),
"state": if finished { "stoppedUP" } else { "uploading" },
"progress": 1.0, "save_path": "/downloads", "tags": label,
"ratio": 2.0, "ratio_limit": 1.5,
"seeding_time": 10, "seeding_time_limit": -1,
"inactive_seeding_time_limit": -1, "last_activity": 0
})
}
async fn qbit(server: &MockServer, torrents: Value) -> Arc<Mutex<Vec<String>>> {
let deleted = Arc::new(Mutex::new(Vec::new()));
Mock::given(method("GET"))
.and(path("/api/v2/torrents/info"))
.respond_with(ResponseTemplate::new(200).set_body_json(torrents))
.mount(server)
.await;
Mock::given(method("POST"))
.and(path("/api/v2/torrents/delete"))
.respond_with(Deletions(Arc::clone(&deleted)))
.mount(server)
.await;
deleted
}
#[tokio::test]
async fn removes_only_finished_arr_torrents_with_data() {
let server = MockServer::start().await;
let removed = Arc::new(Mutex::new(Vec::new()));
Mock::given(any())
.respond_with(Transmission {
torrents: json!([
torrent(1, "movies-main", false),
torrent(2, "movies-main", true),
torrent(3, "radarr", true)
]),
removed: Arc::clone(&removed),
})
.mount(&server)
.await;
let action = ReaperAction::new(TransmissionClient::new(&server.uri()).unwrap());
let deleted = qbit(
&server,
json!([
torrent(1, "movies-main", false),
torrent(2, "movies-main", true),
torrent(3, "radarr", true)
]),
)
.await;
let action = ReaperAction::new(QbitClient::new(&server.uri()).unwrap());
let labels = HashSet::from(["movies-main".to_owned(), "movies-kids".to_owned()]);
let outcomes = action.tick(&labels).await.unwrap();
assert_eq!(outcomes.len(), 1);
assert_eq!(
*removed.lock().unwrap(),
[json!({"ids": [2], "delete-local-data": true})]
let deleted = deleted.lock().unwrap();
assert_eq!(deleted.len(), 1);
assert!(
deleted[0].contains("hashes=hash-2"),
"asked: {}",
deleted[0]
);
assert!(
deleted[0].contains("deleteFiles=true"),
"asked: {}",
deleted[0]
);
}
#[tokio::test]
async fn finished_torrent_does_not_need_a_grab_or_import_row() {
let server = MockServer::start().await;
let removed = Arc::new(Mutex::new(Vec::new()));
Mock::given(any())
.respond_with(Transmission {
torrents: json!([torrent(9, "movies-kids", true)]),
removed: Arc::clone(&removed),
})
.mount(&server)
.await;
let action = ReaperAction::new(TransmissionClient::new(&server.uri()).unwrap());
let deleted = qbit(&server, json!([torrent(9, "movies-kids", true)])).await;
let action = ReaperAction::new(QbitClient::new(&server.uri()).unwrap());
let labels = HashSet::from(["movies-kids".to_owned()]);
action.tick(&labels).await.unwrap();
assert_eq!(removed.lock().unwrap().len(), 1);
assert_eq!(deleted.lock().unwrap().len(), 1);
}
#[tokio::test]
async fn a_torrent_stopped_by_hand_is_left_alone() {
let server = MockServer::start().await;
let deleted = qbit(
&server,
json!([{
"hash": "hash-4", "name": "paused by the operator", "state": "stoppedUP",
"progress": 1.0, "save_path": "/downloads", "tags": "movies-main",
"ratio": 0.2, "ratio_limit": 1.5,
"seeding_time": 10, "seeding_time_limit": -1,
"inactive_seeding_time_limit": -1, "last_activity": 0
}]),
)
.await;
let action = ReaperAction::new(QbitClient::new(&server.uri()).unwrap());
let labels = HashSet::from(["movies-main".to_owned()]);
assert!(action.tick(&labels).await.unwrap().is_empty());
assert!(deleted.lock().unwrap().is_empty());
}
}
+10 -10
View File
@@ -287,11 +287,11 @@ mod tests {
use super::*;
#[derive(Clone, Debug, Default)]
struct FakeTransmission {
struct FakeQbit {
torrents: Arc<Mutex<HashMap<i64, String>>>,
}
impl FakeTransmission {
impl FakeQbit {
async fn find_or_add(&self, movie_id: i64) -> (String, bool) {
let mut torrents = self.torrents.lock().await;
if let Some(hash) = torrents.get(&movie_id) {
@@ -310,7 +310,7 @@ mod tests {
#[derive(Debug)]
struct FakeGrabAction {
transmission: FakeTransmission,
qbit: FakeQbit,
release_id: i64,
fail_after_add: bool,
}
@@ -356,11 +356,11 @@ mod tests {
return Ok(Vec::new());
};
let (infohash, added) = self.transmission.find_or_add(movie_id).await;
let (infohash, added) = self.qbit.find_or_add(movie_id).await;
if added && self.fail_after_add {
return Err(Box::new(io::Error::new(
io::ErrorKind::Interrupted,
"simulated process death after Transmission accepted the torrent",
"simulated process death after qBittorrent accepted the torrent",
)) as ActionError);
}
@@ -448,25 +448,25 @@ mod tests {
#[tokio::test]
async fn restart_converges_without_duplicate_external_work() {
let (_directory, database, release_id) = seeded_database().await;
let transmission = FakeTransmission::default();
let qbit = FakeQbit::default();
let interrupted = ReconcileLoop::new(database.clone()).register(
Tick::Reconcile,
FakeGrabAction {
transmission: transmission.clone(),
qbit: qbit.clone(),
release_id,
fail_after_add: true,
},
);
let first = interrupted.run_tick(Tick::Reconcile).await;
assert_eq!(first.failures, 1);
assert_eq!(transmission.len().await, 1);
assert_eq!(qbit.len().await, 1);
drop(interrupted);
let restarted = ReconcileLoop::new(database.clone()).register(
Tick::Reconcile,
FakeGrabAction {
transmission: transmission.clone(),
qbit: qbit.clone(),
release_id,
fail_after_add: false,
},
@@ -476,7 +476,7 @@ mod tests {
assert_eq!(recovered.actions_taken, 1);
assert_eq!(stable.actions_taken, 0);
assert_eq!(transmission.len().await, 1);
assert_eq!(qbit.len().await, 1);
let grabs: i64 = sqlx::query_scalar("SELECT count(*) FROM grabs")
.fetch_one(database.pool())
.await
+22 -62
View File
@@ -28,7 +28,7 @@ use arr_core::matching::{
};
use arr_core::{EpisodeId, Language, MovieId};
use arr_db::{Blacklist, Db, MoviePolicy, TitlePolicy};
use arr_dl::TransmissionClient;
use arr_dl::QbitClient;
use arr_indexer::{ProwlarrClient, SearchRelease, SearchRequest};
use crate::grab::{
@@ -50,13 +50,13 @@ impl RssAction {
#[must_use]
pub fn new(
prowlarr: ProwlarrClient,
transmission: TransmissionClient,
qbit: QbitClient,
download_dir: PathBuf,
seeding: SeedingRules,
) -> Self {
Self {
indexers: IndexerDirectory::new(prowlarr.clone()),
grabber: Grabber::new(prowlarr.clone(), transmission, download_dir, seeding),
grabber: Grabber::new(prowlarr.clone(), qbit, download_dir, seeding),
prowlarr,
}
}
@@ -714,15 +714,15 @@ async fn load_grabbable(
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use std::sync::{Arc, Mutex};
use serde_json::{json, Value};
use serde_json::json;
use wiremock::matchers::{method, path, query_param, query_param_is_missing};
use wiremock::{Mock, MockServer, Request, Respond, ResponseTemplate};
use wiremock::{Mock, MockServer, ResponseTemplate};
use super::*;
use crate::grab::{test_downloads, SeedingLimits};
use arr_dl::TransmissionClient;
use crate::qbit_fake::FakeQbit;
use arr_dl::QbitClient;
/// The recorded feed: two releases of one wanted title, one that only a
/// TMDB id identifies, one near miss, one TV item and one film nobody
@@ -773,40 +773,6 @@ mod tests {
const INDEXERS: [i64; 2] = [7, 9];
/// Enough of Transmission to add a torrent and list nothing back.
#[derive(Clone, Debug, Default)]
struct FakeTransmission {
added: Arc<Mutex<Vec<String>>>,
}
impl Respond for FakeTransmission {
fn respond(&self, request: &Request) -> ResponseTemplate {
let body: Value = serde_json::from_slice(&request.body).unwrap();
match body["method"].as_str().unwrap_or_default() {
"torrent-add" => {
let source = body["arguments"]["filename"]
.as_str()
.unwrap_or_default()
.to_owned();
let mut added = self.added.lock().unwrap();
added.push(source.clone());
let id = i64::try_from(added.len()).unwrap();
success(&json!({"torrent-added": {
"id": id, "name": source, "hashString": format!("{id:040x}")
}}))
}
"torrent-get" => success(&json!({"torrents": []})),
_ => success(&json!({})),
}
}
}
fn success(arguments: &Value) -> ResponseTemplate {
ResponseTemplate::new(200).set_body_json(json!({
"result": "success", "arguments": arguments
}))
}
/// Two indexers, both advertising a text search, both serving the same
/// feed to an empty query.
async fn prowlarr(feed: &str) -> MockServer {
@@ -842,14 +808,8 @@ mod tests {
server
}
async fn transmission() -> (MockServer, FakeTransmission) {
let server = MockServer::start().await;
let fake = FakeTransmission::default();
Mock::given(method("POST"))
.respond_with(fake.clone())
.mount(&server)
.await;
(server, fake)
async fn qbit() -> (MockServer, FakeQbit) {
FakeQbit::start().await
}
/// `(tmdb_id, title, year, blocked)`.
@@ -874,11 +834,11 @@ mod tests {
(dir, database)
}
fn action(prowlarr: &MockServer, transmission: &MockServer) -> RssAction {
fn action(prowlarr: &MockServer, qbit: &MockServer) -> RssAction {
RssAction::new(
ProwlarrClient::new(prowlarr.uri(), "key").unwrap(),
TransmissionClient::new(&transmission.uri()).unwrap(),
PathBuf::from("/mnt/media/transmission/complete"),
QbitClient::new(&qbit.uri()).unwrap(),
PathBuf::from("/mnt/media/qbittorrent/complete"),
SeedingRules::new(
SeedingLimits {
ratio: 1.0,
@@ -975,7 +935,7 @@ mod tests {
])
.await;
let indexer = prowlarr(FEED).await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1019,7 +979,7 @@ mod tests {
async fn a_near_miss_is_not_grabbed() {
let (_dir, database) = wanted(&[(9_999_999, "Dune: Part Three", 2024, false)]).await;
let indexer = prowlarr(FEED).await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1038,7 +998,7 @@ mod tests {
])
.await;
let indexer = prowlarr(FEED).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1055,7 +1015,7 @@ mod tests {
async fn a_blocked_title_still_matches_rss() {
let (_dir, database) = wanted(&[(693_134, "Dune: Part Two", 2024, true)]).await;
let indexer = prowlarr(FEED).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1073,7 +1033,7 @@ mod tests {
])
.await;
let indexer = prowlarr(FEED).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1103,7 +1063,7 @@ mod tests {
let (_dir, database) = wanted(&[]).await;
let (_season_id, episodes) = wanted_series(&database, &["2024-04-11", "2099-01-01"]).await;
let indexer = prowlarr(TV_FEED).await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1149,7 +1109,7 @@ mod tests {
"Unrelated.S01E03.2160p.WEB-DL-GROUP</title>",
);
let indexer = prowlarr(&pack_only).await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1197,7 +1157,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr(TV_FEED).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1237,7 +1197,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr(TV_FEED).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1269,7 +1229,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr(TV_FEED).await;
let (downloader, _fake) = transmission().await;
let (downloader, _fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
+586 -85
View File
@@ -151,6 +151,14 @@ impl EmbeddedTrack {
fn is_text(&self) -> bool {
matches!(self.codec.as_deref(), Some("subrip" | "ass" | "mov_text"))
}
/// §15's bitmap tracks. They feed no translator, but their timings are
/// exact for the release, which is what a cue skeleton is made of (#268).
fn is_image(&self) -> bool {
self.codec
.as_deref()
.is_some_and(|codec| arr_core::SubtitleCodec::from_probe_name(codec).is_image())
}
}
fn embedded_tracks(probed: &str) -> Vec<EmbeddedTrack> {
@@ -403,6 +411,73 @@ struct Worker {
in_flight: Arc<Mutex<BTreeSet<(i64, String)>>>,
}
/// A finished translation on its way to disk.
struct Translation<'a> {
engine: String,
cues: &'a [arr_subs::Cue],
/// Its source came off a cue skeleton, so §15's post-translation `alass`
/// pass is skipped (#268).
skeleton_aligned: bool,
}
/// A subtitle to translate from (§15).
struct Source {
path: String,
language: Language,
/// Its timings came off a cue skeleton, so they are exact for this
/// release and no further `alass` pass should touch them (#268).
skeleton_aligned: bool,
}
/// Whether a translation source could be had, and at whose expense.
///
/// Distinct from `Option` because looking for one may now spend a provider
/// download (#268), so "none" and "not today" are different answers and land
/// the gap in different states.
enum SourceOutcome {
Found(Source),
/// Nothing on this file can feed a translator.
None,
/// A source exists but the provider is at its daily allowance.
Capped,
/// The fetch was tried and broke.
Failed(String),
}
impl SourceOutcome {
/// The source, or the state and sentence the attempt row records instead.
fn settled(self, wanted_tag: &str) -> Result<Source, (SubtitleState, String)> {
match self {
Self::Found(source) => Ok(source),
Self::None => Err((
SubtitleState::Unavailable,
format!(
"no provider has {wanted_tag} and there is no text source to translate from"
),
)),
Self::Capped => Err((
SubtitleState::Capped,
"provider daily budget exhausted fetching a translation source".to_owned(),
)),
Self::Failed(reason) => Err((SubtitleState::Failed, reason)),
}
}
}
/// Read a translation source off disk. The error is the sentence the attempt
/// row records.
async fn read_cues(path: &str) -> Result<Vec<arr_subs::Cue>, String> {
let raw = tokio::fs::read_to_string(path)
.await
.map_err(|error| format!("{path}: {error}"))?;
arr_subs::srt::parse(&raw).map_err(|error| format!("{path}: not SRT: {error}"))
}
/// Milliseconds as stored in a cue skeleton.
fn duration_from_millis(value: i64) -> std::time::Duration {
std::time::Duration::from_millis(u64::try_from(value).unwrap_or(0))
}
/// What one close settled to, for the log.
enum Closed {
Fetched {
@@ -756,17 +831,19 @@ impl Worker {
Ok(Closed::Recorded { state, reason })
};
let Some((source_path, source_language)) =
self.translation_source(database, target).await?
else {
return record(
SubtitleState::Unavailable,
format!(
"no provider has {wanted_tag} and there is no text source to translate from"
),
)
.await;
let source = match self
.translation_source(database, target, wanted, settings)
.await?
.settled(wanted_tag)
{
Ok(source) => source,
Err((state, reason)) => return record(state, reason).await,
};
let Source {
path: source_path,
language: source_language,
skeleton_aligned,
} = source;
let Some(engine) = settings.translation_engine.clone() else {
return record(
@@ -788,41 +865,14 @@ impl Worker {
.await;
};
let raw = match tokio::fs::read_to_string(&source_path).await {
Ok(raw) => raw,
Err(error) => {
return record(SubtitleState::Failed, format!("{source_path}: {error}")).await
}
};
let cues = match arr_subs::srt::parse(&raw) {
let cues = match read_cues(&source_path).await {
Ok(cues) => cues,
Err(error) => {
return record(
SubtitleState::Failed,
format!("{source_path}: not SRT: {error}"),
)
.await
}
Err(reason) => return record(SubtitleState::Failed, reason).await,
};
// Translators bill per character, not per call (§15), and the text
// going out is known before any of it is sent — no need to ask the
// backend afterwards (#197).
let characters = i64::try_from(
cues.iter()
.map(|cue| cue.text.chars().count())
.sum::<usize>(),
)
.unwrap_or(i64::MAX);
let allowance = settings.translator_allowance(&engine);
if !budget::try_spend(
database.pool(),
BudgetKind::Translator,
&engine,
characters,
allowance,
)
.await?
if !self
.claim_translator_budget(database, &engine, &cues, settings)
.await?
{
return record(
SubtitleState::Capped,
@@ -846,8 +896,45 @@ impl Worker {
};
self.clear_broken(&broken_name).await;
self.write_translation(database, target, wanted, wanted_tag, engine, &translated)
.await
self.write_translation(
database,
target,
wanted,
wanted_tag,
Translation {
engine,
cues: &translated,
skeleton_aligned,
},
)
.await
}
/// Claim the translator's daily character allowance for `cues` (§15,
/// #197). Translators bill per character, not per call, and the text
/// going out is known before any of it is sent — no need to ask the
/// backend afterwards.
async fn claim_translator_budget(
&self,
database: &Db,
engine: &str,
cues: &[arr_subs::Cue],
settings: &Settings,
) -> Result<bool, SubtitleError> {
let characters = i64::try_from(
cues.iter()
.map(|cue| cue.text.chars().count())
.sum::<usize>(),
)
.unwrap_or(i64::MAX);
Ok(budget::try_spend(
database.pool(),
BudgetKind::Translator,
engine,
characters,
settings.translator_allowance(engine),
)
.await?)
}
/// Sync and write a finished translation, record it, settle the want.
@@ -857,9 +944,13 @@ impl Worker {
target: &Target,
wanted: &Language,
wanted_tag: &str,
engine: String,
translated: &[arr_subs::Cue],
translation: Translation<'_>,
) -> Result<Closed, SubtitleError> {
let Translation {
engine,
cues: translated,
skeleton_aligned,
} = translation;
let media_file_id = target.media_file_id;
let record = |state: SubtitleState, reason: String| async move {
db::record_attempt(
@@ -884,37 +975,58 @@ impl Worker {
return record(SubtitleState::Failed, error).await;
}
let sync = self.syncer.settle(&target.path, &destination).await;
// §15 as amended by #268: a source aligned against a cue skeleton
// already carries the disc's own timings, and translation copies them
// over unchanged. A second `alass` pass has nothing left to find and
// can only move them off, so it is skipped.
let sync = if skeleton_aligned {
arr_subs::Settled {
content: None,
state: arr_subs::SyncState::NotRun,
}
} else {
self.syncer.settle(&target.path, &destination).await
};
if let Some(synced) = &sync.content {
if let Err(error) = write_sidecar(&destination, synced).await {
return record(SubtitleState::Failed, error).await;
}
}
db::record_file(
database.pool(),
&arr_db::NewSubtitleFile::translated(
media_file_id,
wanted_tag,
&engine,
&destination.to_string_lossy(),
)
.sync(db_sync_state(sync.state)),
let mut record_row = arr_db::NewSubtitleFile::translated(
media_file_id,
wanted_tag,
&engine,
&destination.to_string_lossy(),
)
.await?;
.sync(db_sync_state(sync.state));
if skeleton_aligned {
record_row = record_row.skeleton_aligned();
}
db::record_file(database.pool(), &record_row).await?;
db::mark_satisfied(database.pool(), media_file_id, wanted_tag).await?;
self.refresh_jellyfin().await;
Ok(Closed::Translated { engine })
}
/// The subtitle to translate from: any non-forced sidecar arr wrote — a
/// real one before a machine translation — or, failing that, a
/// text-format embedded track extracted now (§15).
/// real one before a machine translation — then a text-format embedded
/// track extracted now, and failing both a text subtitle downloaded for
/// the purpose (§15, #268).
///
/// That last step is the one §15 had to be amended for. A release whose
/// only subtitle is an image track reads as *satisfied* in that language
/// and so is never fetched again, while the track itself can never feed a
/// translator — the file is stuck both ways. The carve-out is narrow: a
/// fetch made to obtain a source, in a language the file already
/// satisfies, which settles no want of its own.
async fn translation_source(
&self,
database: &Db,
target: &Target,
) -> Result<Option<(String, Language)>, SubtitleError> {
wanted: &Language,
settings: &Settings,
) -> Result<SourceOutcome, SubtitleError> {
let files = db::files_for(database.pool(), target.media_file_id).await?;
let sidecar = files
.iter()
@@ -922,7 +1034,11 @@ impl Worker {
.min_by_key(|file| matches!(file.origin, SubtitleOrigin::Translated));
if let Some(file) = sidecar {
if let Some(path) = file.path.clone() {
return Ok(Some((path, arr_db::policy::language(&file.language))));
return Ok(SourceOutcome::Found(Source {
path,
language: arr_db::policy::language(&file.language),
skeleton_aligned: file.skeleton_aligned,
}));
}
}
@@ -934,42 +1050,209 @@ impl Worker {
.await?
.flatten();
let Some(probed) = probed else {
return Ok(None);
return Ok(SourceOutcome::None);
};
let Some(track) = embedded_tracks(&probed)
.into_iter()
.find(|track| !track.forced && track.is_text())
else {
return Ok(None);
let tracks = embedded_tracks(&probed);
if let Some(track) = tracks.iter().find(|track| !track.forced && track.is_text()) {
let language = arr_db::policy::language(&track.language);
let Some(destination) = target.sidecar(&language, false) else {
return Ok(SourceOutcome::None);
};
if let Err(error) = self
.extractor
.extract_srt(&target.path, track.index, &destination)
.await
{
tracing::warn!(
path = %target.path.display(),
stream = track.index,
%error,
"embedded track extraction failed"
);
return Ok(SourceOutcome::None);
}
let mut record = arr_db::NewSubtitleFile::extracted(
target.media_file_id,
&track.language,
&destination.to_string_lossy(),
);
if track.sdh {
record = record.sdh();
}
db::record_file(database.pool(), &record).await?;
return Ok(SourceOutcome::Found(Source {
path: destination.to_string_lossy().into_owned(),
language,
skeleton_aligned: false,
}));
}
self.fetch_source(database, target, wanted, &tracks, settings)
.await
}
/// Every candidate the enabled providers offer in `languages`.
///
/// Best-effort, unlike the search a gap opens with: this one runs after
/// that one has already reported, so a provider that errors here is
/// logged and skipped rather than backing the gap off a second time.
async fn offered_in(
&self,
target: &Target,
languages: &[Language],
settings: &Settings,
) -> Vec<Candidate> {
let request = target.search_request(languages.to_vec());
let mut offered: Vec<Candidate> = Vec::new();
for provider in self
.providers
.iter()
.filter(|provider| settings.providers_enabled.contains(provider.id().as_str()))
{
match provider.search(&request).await {
Ok(candidates) => offered.extend(
candidates
.into_iter()
.filter(|candidate| languages.contains(&candidate.language)),
),
Err(error) => tracing::debug!(
%error,
provider = %provider.id(),
"no translation source from this provider"
),
}
}
offered
}
/// Download a text subtitle to translate from, in a language an image
/// track already satisfies (§15 as amended, #268).
///
/// The image track decides the language: a skeleton derived from it is
/// the alignment reference, so the source has to be the subtitle that
/// track is a bitmap rendering of. Where no skeleton was derived — the
/// packets did not pair, or the file predates #268 — the fetch still
/// happens and `alass` aligns against the video as it always has.
async fn fetch_source(
&self,
database: &Db,
target: &Target,
wanted: &Language,
tracks: &[EmbeddedTrack],
settings: &Settings,
) -> Result<SourceOutcome, SubtitleError> {
let skeletons =
arr_db::skeletons::for_media_file(database.pool(), target.media_file_id).await?;
// A forced track covers signs alone, and one whose language already
// answers the want is the wrong direction to translate in.
let wanted_tag = wanted.to_string();
let Some(track) = tracks.iter().find(|track| {
!track.forced && track.is_image() && !satisfies(&wanted_tag, &track.language)
}) else {
return Ok(SourceOutcome::None);
};
let language = arr_db::policy::language(&track.language);
let languages = vec![language.clone()];
let Some(destination) = target.sidecar(&language, false) else {
return Ok(None);
return Ok(SourceOutcome::None);
};
if let Err(error) = self
.extractor
.extract_srt(&target.path, track.index, &destination)
.await
{
tracing::warn!(
path = %target.path.display(),
stream = track.index,
%error,
"embedded track extraction failed"
);
return Ok(None);
let offered = self.offered_in(target, &languages, settings).await;
let hash = moviehash(&target.path, target.size).await;
let (winner, budget_capped) = self
.claim_within_budget(
database,
target,
hash.as_deref(),
offered,
&languages,
settings,
)
.await?;
let Some(winner) = winner else {
return Ok(if budget_capped {
SourceOutcome::Capped
} else {
SourceOutcome::None
});
};
let provider_name = winner.provider.to_string();
let Some(provider) = self.provider(&provider_name) else {
return Ok(SourceOutcome::Failed(format!(
"provider {provider_name} vanished mid-close"
)));
};
let fetched = match provider.download(&winner.id).await {
Ok(fetched) => fetched,
Err(error) => return Ok(SourceOutcome::Failed(error.to_string())),
};
let text = match fetched.to_srt() {
Ok(text) => text,
Err(error) => return Ok(SourceOutcome::Failed(error.to_string())),
};
if let Err(error) = write_sidecar(&destination, &text).await {
return Ok(SourceOutcome::Failed(error));
}
let mut record = arr_db::NewSubtitleFile::extracted(
let skeleton = skeletons
.iter()
.find(|skeleton| usize::try_from(skeleton.stream_index) == Ok(track.index));
let sync = match skeleton {
Some(skeleton) => {
let spans: Vec<(std::time::Duration, std::time::Duration)> = skeleton
.cues
.iter()
.map(|(start, end)| (duration_from_millis(*start), duration_from_millis(*end)))
.collect();
self.syncer.settle_against_cues(&spans, &destination).await
}
None => self.syncer.settle(&target.path, &destination).await,
};
if let Some(synced) = &sync.content {
if let Err(error) = write_sidecar(&destination, synced).await {
return Ok(SourceOutcome::Failed(error));
}
}
// Only an accepted alignment against the skeleton leaves the file on
// disc-exact timings. A rejected or unrun one leaves the provider's
// own, which the post-translation pass should still get a look at.
let skeleton_aligned =
skeleton.is_some() && matches!(sync.state, arr_subs::SyncState::Synced);
let mut record = arr_db::NewSubtitleFile::fetched(
target.media_file_id,
&track.language,
&winner.language.to_string(),
&provider_name,
winner.id.as_str(),
&destination.to_string_lossy(),
);
if track.sdh {
)
.sync(db_sync_state(sync.state));
if winner.sdh {
record = record.sdh();
}
if skeleton_aligned {
record = record.skeleton_aligned();
}
db::record_file(database.pool(), &record).await?;
Ok(Some((destination.to_string_lossy().into_owned(), language)))
self.refresh_jellyfin().await;
tracing::info!(
media_file_id = target.media_file_id,
provider = %provider_name,
language = %winner.language,
skeleton_aligned,
"translation source fetched for a language an image track satisfies"
);
Ok(SourceOutcome::Found(Source {
path: destination.to_string_lossy().into_owned(),
language: winner.language,
skeleton_aligned,
}))
}
/// The same single rescan import makes (§7.5): Jellyfin's watcher misses
@@ -1439,10 +1722,228 @@ mod tests {
.inline()
}
fn action_with_syncer(
providers: Vec<Arc<dyn Provider>>,
backends: Vec<Arc<dyn Backend>>,
syncer: Syncer,
) -> SubtitleAction {
SubtitleAction::new(
providers,
backends,
syncer,
Extractor::new().with_binary("ffmpeg-not-installed"),
JellyfinClient::new("http://127.0.0.1:1", None).unwrap(),
)
.inline()
}
fn no_tracks() -> serde_json::Value {
serde_json::json!({ "sub_tracks": [] })
}
/// One non-forced PGS track, the shape #268 was found on: it satisfies
/// English for viewing and can never feed a translator.
fn one_image_track() -> serde_json::Value {
serde_json::json!({ "sub_tracks": [
{ "language": "en", "codec": "hdmv_pgs_subtitle", "forced": false, "sdh": false },
] })
}
/// A fake `alass` that copies its input to its output — every run
/// accepted — and appends a line to `log` so the runs can be counted.
async fn counting_alass(directory: &std::path::Path, log: &std::path::Path) -> PathBuf {
use std::os::unix::fs::PermissionsExt;
let binary = directory.join("alass");
tokio::fs::write(
&binary,
format!(
"#!/bin/sh\necho run >> {}\ncp \"$2\" \"$3\"\n",
log.display()
),
)
.await
.unwrap();
tokio::fs::set_permissions(&binary, std::fs::Permissions::from_mode(0o755))
.await
.unwrap();
binary
}
async fn alass_runs(log: &std::path::Path) -> usize {
tokio::fs::read_to_string(log)
.await
.map_or(0, |text| text.lines().count())
}
/// #268's deadlock. The release carries one English PGS track, so English
/// reads as satisfied and is never fetched, while the track itself is
/// bitmaps and can never be translated from. §15 as amended lets the
/// translation lane fetch a text English subtitle anyway.
#[tokio::test]
async fn an_image_track_does_not_block_the_language_it_satisfies_from_being_fetched() {
let fixture = Fixture::new(&one_image_track()).await;
fixture.configure(r#"["pt-PT"]"#, Some("openai")).await;
let provider = StubProvider::new(
"opensubtitles",
vec![candidate(
"opensubtitles",
"en-1",
Language::Other("en".to_owned()),
)],
);
let action = action(vec![Arc::new(provider)], vec![Arc::new(StubBackend)]);
action.run(&fixture.database).await.unwrap();
let translated = fixture
.directory
.path()
.join("Movie (2024) - [1080p].pt-PT.mt.srt");
assert!(
tokio::fs::read_to_string(&translated)
.await
.unwrap()
.contains("HELLO THERE"),
"the gap closed by translating the fetched English source"
);
let files = fixture.files().await;
let source = files
.iter()
.find(|file| file.origin == SubtitleOrigin::Provider)
.expect("the English source was fetched despite the image track satisfying English");
assert_eq!(source.language, "en");
assert_eq!(
fixture.attempt("pt-PT").await.unwrap().state,
SubtitleState::Satisfied
);
}
/// The point of a skeleton: the fetched source is aligned against the
/// disc's own cue structure, and the translation made from it is then
/// left alone. A second `alass` pass can only move disc-exact timings
/// off.
#[tokio::test]
async fn a_source_aligned_to_a_skeleton_skips_the_post_translation_pass() {
let fixture = Fixture::new(&one_image_track()).await;
fixture.configure(r#"["pt-PT"]"#, Some("openai")).await;
arr_db::skeletons::record(
fixture.database.pool(),
fixture.media_file_id,
0,
"en",
&[(1_000, 2_500), (3_000, 4_000)],
)
.await
.unwrap();
let log = fixture.directory.path().join("alass.log");
let binary = counting_alass(fixture.directory.path(), &log).await;
let action = action_with_syncer(
vec![Arc::new(StubProvider::new(
"opensubtitles",
vec![candidate(
"opensubtitles",
"en-1",
Language::Other("en".to_owned()),
)],
))],
vec![Arc::new(StubBackend)],
Syncer::new().with_binary(&binary),
);
action.run(&fixture.database).await.unwrap();
assert_eq!(
alass_runs(&log).await,
1,
"alass aligned the source against the skeleton and was not run again"
);
let files = fixture.files().await;
assert!(
files
.iter()
.find(|file| file.origin == SubtitleOrigin::Provider)
.unwrap()
.skeleton_aligned,
"the fetched source is recorded as disc-exact"
);
assert!(
files
.iter()
.find(|file| file.origin == SubtitleOrigin::Translated)
.unwrap()
.skeleton_aligned,
"and so is what was translated from it"
);
}
/// Without a skeleton — the packets did not pair, or the file predates
/// #268 — the fetch still happens and both passes align against the
/// video, exactly as they always did.
#[tokio::test]
async fn without_a_skeleton_both_passes_still_align_against_the_video() {
let fixture = Fixture::new(&one_image_track()).await;
fixture.configure(r#"["pt-PT"]"#, Some("openai")).await;
let log = fixture.directory.path().join("alass.log");
let binary = counting_alass(fixture.directory.path(), &log).await;
let action = action_with_syncer(
vec![Arc::new(StubProvider::new(
"opensubtitles",
vec![candidate(
"opensubtitles",
"en-1",
Language::Other("en".to_owned()),
)],
))],
vec![Arc::new(StubBackend)],
Syncer::new().with_binary(&binary),
);
action.run(&fixture.database).await.unwrap();
assert_eq!(
alass_runs(&log).await,
2,
"source and translation both synced"
);
assert!(fixture
.files()
.await
.iter()
.all(|file| !file.skeleton_aligned));
}
/// A forced image track covers signs alone (§15), so it is not a reason
/// to go looking for a source subtitle either.
#[tokio::test]
async fn a_forced_image_track_is_not_a_translation_source_to_fetch() {
let fixture = Fixture::new(&serde_json::json!({ "sub_tracks": [
{ "language": "en", "codec": "hdmv_pgs_subtitle", "forced": true, "sdh": false },
] }))
.await;
fixture.configure(r#"["pt-PT"]"#, Some("openai")).await;
let action = action(
vec![Arc::new(StubProvider::new(
"opensubtitles",
vec![candidate(
"opensubtitles",
"en-1",
Language::Other("en".to_owned()),
)],
))],
vec![Arc::new(StubBackend)],
);
action.run(&fixture.database).await.unwrap();
let attempt = fixture.attempt("pt-PT").await.unwrap();
assert_eq!(attempt.state, SubtitleState::Unavailable);
assert!(fixture.files().await.iter().all(|file| file.path.is_none()));
}
#[tokio::test]
async fn an_embedded_track_satisfies_without_asking_any_provider() {
let fixture = Fixture::new(&serde_json::json!({ "sub_tracks": [
+31 -110
View File
@@ -16,7 +16,7 @@ use std::path::PathBuf;
use arr_core::grabbing::{season_grab_mode, SeasonGrabFacts, SeasonGrabMode};
use arr_core::Language;
use arr_db::{Blacklist, Db, TitlePolicy};
use arr_dl::TransmissionClient;
use arr_dl::QbitClient;
use arr_indexer::{ProwlarrClient, SearchRelease, TvSelector, TvTarget};
use arr_parse::EpisodeClaim;
@@ -47,13 +47,13 @@ impl TvGrabAction {
#[must_use]
pub fn new(
prowlarr: ProwlarrClient,
transmission: TransmissionClient,
qbit: QbitClient,
download_dir: PathBuf,
seeding: SeedingRules,
) -> Self {
Self {
indexers: IndexerDirectory::new(prowlarr.clone()),
grabber: Grabber::new(prowlarr.clone(), transmission, download_dir, seeding),
grabber: Grabber::new(prowlarr.clone(), qbit, download_dir, seeding),
prowlarr,
}
}
@@ -544,7 +544,7 @@ impl TvGrabAction {
/// The manual one-click episode grab (§9.3, issue #132): the release is
/// already chosen off the episode deck, so this skips search and scoring
/// and sends it straight to Transmission. A manual grab may take a
/// and sends it straight to qBittorrent. A manual grab may take a
/// `waived` release (§9.3), never a `rejected` one.
pub(crate) async fn grab_episode_release_now(
&self,
@@ -645,7 +645,7 @@ impl TvGrabAction {
}
/// The manual one-click season grab (§9.3, issue #125): a chosen pack
/// goes straight to Transmission against the season's still-open gaps.
/// goes straight to qBittorrent against the season's still-open gaps.
pub(crate) async fn grab_season_release_now(
&self,
database: &Db,
@@ -1223,89 +1223,16 @@ fn air_date_time(value: Option<&str>) -> Option<std::time::SystemTime> {
#[allow(clippy::unwrap_used)]
mod tests {
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use serde_json::{json, Value};
use serde_json::json;
use wiremock::matchers::{method, path, query_param};
use wiremock::{Mock, MockServer, Request, Respond, ResponseTemplate};
use wiremock::{Mock, MockServer, ResponseTemplate};
use crate::grab::{test_downloads, SeedingLimits};
use crate::qbit_fake::FakeQbit;
use super::*;
/// A Transmission that dedupes on the infohash, like the real one.
#[derive(Clone, Debug, Default)]
struct FakeTransmission {
torrents: Arc<Mutex<Vec<FakeTorrent>>>,
}
#[derive(Clone, Debug)]
struct FakeTorrent {
id: i64,
hash: String,
source: String,
progress: f64,
}
impl FakeTransmission {
fn torrents(&self) -> Vec<FakeTorrent> {
self.torrents.lock().unwrap().clone()
}
fn add(&self, arguments: &Value) -> ResponseTemplate {
let source = arguments["filename"]
.as_str()
.unwrap_or_default()
.to_owned();
let mut torrents = self.torrents.lock().unwrap();
if let Some(existing) = torrents.iter().find(|torrent| torrent.source == source) {
return success(&json!({"torrent-duplicate": {
"id": existing.id, "name": existing.source, "hashString": existing.hash
}}));
}
let id = i64::try_from(torrents.len()).unwrap() + 1;
let hash = format!("{:040x}", id * 7);
torrents.push(FakeTorrent {
id,
hash: hash.clone(),
source: source.clone(),
progress: 0.0,
});
success(&json!({"torrent-added": {"id": id, "name": source, "hashString": hash}}))
}
}
impl Respond for FakeTransmission {
fn respond(&self, request: &Request) -> ResponseTemplate {
let body: Value = serde_json::from_slice(&request.body).unwrap();
match body["method"].as_str().unwrap_or_default() {
"torrent-add" => self.add(&body["arguments"]),
"torrent-get" => {
let torrents: Vec<Value> = self
.torrents()
.into_iter()
.map(|torrent| {
json!({
"id": torrent.id, "name": torrent.source,
"hashString": torrent.hash, "status": 4,
"percentDone": torrent.progress,
"downloadDir": "/downloads", "labels": []
})
})
.collect();
success(&json!({"torrents": torrents}))
}
_ => success(&json!({})),
}
}
}
fn success(arguments: &Value) -> ResponseTemplate {
ResponseTemplate::new(200).set_body_json(json!({
"result": "success", "arguments": arguments
}))
}
/// A season pack and its three episodes, all eligible under the seeded
/// TV main policy (§5.5 bands: 2160p floor 3 GiB, per #95).
const TV_RSS: &str = r#"<rss><channel>
@@ -1364,14 +1291,8 @@ mod tests {
server
}
async fn transmission() -> (MockServer, FakeTransmission) {
let server = MockServer::start().await;
let fake = FakeTransmission::default();
Mock::given(method("POST"))
.respond_with(fake.clone())
.mount(&server)
.await;
(server, fake)
async fn qbit() -> (MockServer, FakeQbit) {
FakeQbit::start().await
}
/// Fallout S01 with three wanted episodes at the given air dates.
@@ -1409,11 +1330,11 @@ mod tests {
(dir, database, season_id)
}
fn action(prowlarr: &MockServer, transmission: &MockServer) -> TvGrabAction {
fn action(prowlarr: &MockServer, qbit: &MockServer) -> TvGrabAction {
TvGrabAction::new(
ProwlarrClient::new(prowlarr.uri(), "key").unwrap(),
TransmissionClient::new(&transmission.uri()).unwrap(),
PathBuf::from("/mnt/media/transmission/complete"),
QbitClient::new(&qbit.uri()).unwrap(),
PathBuf::from("/mnt/media/qbittorrent/complete"),
SeedingRules::new(
SeedingLimits {
ratio: 1.5,
@@ -1448,7 +1369,7 @@ mod tests {
let (_dir, database, season_id) =
wanted_season(&["2024-04-11", "2024-04-11", "2024-04-11"]).await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let outcomes = action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1473,7 +1394,7 @@ mod tests {
let (_dir, database, _season_id) =
wanted_season(&["2024-04-11", "2024-04-18", "2999-01-01"]).await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1547,7 +1468,7 @@ mod tests {
wanted_season(&["2024-04-11", "2024-04-11", "2024-04-11"]).await;
failed_packs(&database, season_id, &["-10 minutes"]).await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1579,7 +1500,7 @@ mod tests {
wanted_season(&["2024-04-11", "2024-04-11", "2024-04-11"]).await;
failed_packs(&database, season_id, &["-2 hours"]).await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1601,7 +1522,7 @@ mod tests {
wanted_season(&["2024-04-11", "2024-04-11", "2024-04-11"]).await;
stalled_failed_packs(&database, season_id, &[("-35 days", "-10 minutes")]).await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1631,7 +1552,7 @@ mod tests {
)
.await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&inside.1).await.unwrap();
assert!(
fake.torrents()
@@ -1647,7 +1568,7 @@ mod tests {
&["-30 days", "-25 days", "-20 days", "-12 days", "-8 days"],
)
.await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader)
.tick(&elapsed.1)
.await
@@ -1664,7 +1585,7 @@ mod tests {
wanted_season(&["2024-04-11", "2024-04-11", "2024-04-11"]).await;
failed_packs(&database, season_id, &["-10 minutes"]).await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader)
.search_season_now(&database, season_id)
@@ -1701,7 +1622,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader).tick(&database).await.unwrap();
@@ -1724,7 +1645,7 @@ mod tests {
let (_dir, database, season_id) =
wanted_season(&["2024-04-11", "2024-04-11", "2024-04-11"]).await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let stamp = |database: Db| async move {
sqlx::query_scalar::<_, Option<String>>(
@@ -1788,7 +1709,7 @@ mod tests {
)
.mount(&indexer)
.await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let action = action(&indexer, &downloader);
action.tick(&database).await.unwrap();
@@ -1823,7 +1744,7 @@ mod tests {
/// be accepted onto the channel. `search_attempts` starts at 5 (a 7-day
/// backoff, nowhere near elapsed) so a tick would skip this episode;
/// ending at 1 rather than 6 proves the reset happened, and a torrent in
/// Transmission proves the grab did.
/// qBittorrent proves the grab did.
#[tokio::test]
async fn a_manual_episode_search_searches_and_grabs_now() {
let (_dir, database, _season_id) =
@@ -1841,7 +1762,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader)
.search_episode_now(&database, episode_id)
@@ -1871,7 +1792,7 @@ mod tests {
}
/// Issue #132: an `EpisodeCommand::Grab` sends the already-chosen release
/// straight to Transmission — no indexer search at all.
/// straight to qBittorrent — no indexer search at all.
#[tokio::test]
async fn a_manual_episode_grab_sends_the_chosen_release_without_searching() {
let (_dir, database, _season_id) = wanted_season(&["2024-04-11", "2024-04-18"]).await;
@@ -1880,7 +1801,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let release_id: i64 = sqlx::query_scalar(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, score, verdict)
VALUES (7, 'chosen', 'Fallout.S01E01.2160p.WEB-DL.DDP5.1', 10737418240,
@@ -1936,7 +1857,7 @@ mod tests {
.await
.unwrap();
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
action(&indexer, &downloader)
.search_season_now(&database, season_id)
@@ -1973,13 +1894,13 @@ mod tests {
}
/// Issue #132: a `SeasonCommand::Grab` sends the already-chosen pack
/// straight to Transmission — no indexer search at all.
/// straight to qBittorrent — no indexer search at all.
#[tokio::test]
async fn a_manual_season_grab_sends_the_chosen_pack_without_searching() {
let (_dir, database, season_id) =
wanted_season(&["2024-04-11", "2024-04-11", "2024-04-11"]).await;
let indexer = prowlarr().await;
let (downloader, fake) = transmission().await;
let (downloader, fake) = qbit().await;
let release_id: i64 = sqlx::query_scalar(
"INSERT INTO releases (indexer_id, guid, name, size, download_url, parsed, score, verdict)
VALUES (7, 'chosen-pack', 'Fallout.S01.2160p.WEB-DL.DDP5.1.Atmos', 85899345920,
+1
View File
@@ -13,6 +13,7 @@ serde = { workspace = true }
serde_json = { workspace = true }
sqlx = { workspace = true }
thiserror = { workspace = true }
tracing = { workspace = true }
[dev-dependencies]
tempfile = { workspace = true }
@@ -14,7 +14,7 @@ CREATE TABLE subtitle_settings (
-- Provider ids in search order (DESIGN.md §15's ranking runs per
-- provider before it runs per candidate). Absent from the array means
-- disabled.
providers_enabled TEXT NOT NULL DEFAULT '["opensubtitles"]'
providers_enabled TEXT NOT NULL DEFAULT '["opensubtitles","podnapisi"]'
CHECK (json_valid(providers_enabled)),
-- The compiled-in backend in use, or NULL when translation is off. Not
-- checked against the running binary's features here — the API does
@@ -0,0 +1,34 @@
-- #268. The cue structure of an image-format subtitle track (DESIGN.md §15).
--
-- A PGS or VobSub track carries bitmaps, so it can never feed a translator.
-- Its packet timings are still exact for the release it came off, and that is
-- what `alass` matches on: aligning a downloaded English subtitle against this
-- skeleton lands it on the disc's own cue structure instead of on a heuristic
-- read of the video's audio.
--
-- Derived once at import, where the file is being read and hardlinked anyway.
-- A track whose packets do not pair cleanly gets no row at all — the absence
-- is the fallback signal, and the lane aligns against the video instead.
CREATE TABLE subtitle_skeletons (
media_file_id INTEGER NOT NULL REFERENCES media_files (id) ON DELETE CASCADE,
-- The track's position among the file's subtitle streams, the same index
-- `ffmpeg -map 0:s:N` takes.
stream_index INTEGER NOT NULL CHECK (stream_index >= 0),
-- As `arr_core::Language` spells it. The language a source subtitle is
-- fetched in when this skeleton is the alignment reference.
language TEXT NOT NULL,
-- The cues, as a JSON array of `[start_ms, end_ms]` pairs in file order.
-- No text: there is none to read, and `alass` does not want any.
cues TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
PRIMARY KEY (media_file_id, stream_index)
) STRICT;
-- §15, as amended by #268: a subtitle aligned against a cue skeleton already
-- carries disc-exact timings, so the post-translation `alass` pass is skipped
-- for it and for anything translated from it. A second pass can only move
-- those timings off. This says which rows that applies to, durably — a crash
-- between the fetch and the translation must not lose the fact.
ALTER TABLE subtitle_files
ADD COLUMN skeleton_aligned INTEGER NOT NULL DEFAULT 0
CHECK (skeleton_aligned IN (0, 1));
+2 -2
View File
@@ -81,7 +81,7 @@ impl Blacklist {
}
/// Whether this infohash has been blacklisted. Case-insensitive:
/// Transmission and Torznab disagree on the hex casing.
/// qBittorrent and Torznab disagree on the hex casing.
#[must_use]
pub fn blocks_infohash(&self, infohash: &str) -> bool {
self.infohashes.contains_key(&infohash.to_ascii_lowercase())
@@ -176,7 +176,7 @@ pub async fn add(
/// The infohash a magnet link declares, lowercased.
///
/// Only the 40-character hex form is recognised. Base32 `btih` values exist
/// in the wild but Transmission normalises them away, and guessing wrong here
/// in the wild but qBittorrent normalises them away, and guessing wrong here
/// would blacklist an unrelated release.
#[must_use]
pub fn magnet_infohash(download_url: &str) -> Option<String> {
+2
View File
@@ -7,11 +7,13 @@ use std::path::Path;
pub mod blacklist;
pub mod policy;
pub mod skeletons;
pub mod subtitle_budget;
pub mod subtitles;
pub use blacklist::Blacklist;
pub use policy::{MoviePolicy, PolicyColumns, PolicyError, TitlePolicy};
pub use skeletons::Skeleton;
pub use subtitle_budget::BudgetKind;
pub use subtitles::{
NewSubtitleFile, PendingSubtitle, SubtitleAttempt, SubtitleFile, SubtitleOrigin, SubtitleState,
+1 -1
View File
@@ -29,7 +29,7 @@ pub enum PolicyError {
/// The effective policy for one title, plus the root it is attached to.
///
/// The root's `kind` and `audience` are the Transmission label and the
/// The root's `kind` and `audience` are the qBittorrent label and the
/// on-disk layout (§7.1, §7.4), and they only exist together with the policy,
/// so they are returned together.
#[derive(Debug, Clone, PartialEq, Eq)]
+104
View File
@@ -0,0 +1,104 @@
//! Cue skeletons of image-format subtitle tracks (`DESIGN.md` §15, #268).
//!
//! A skeleton is the cue structure of a PGS or `VobSub` track, derived from
//! its packet timings at import. It carries no text — there is none to read —
//! and exists for one purpose: to be `alass`'s alignment reference instead of
//! the video, so a downloaded subtitle lands on the disc's own timings.
//!
//! A row means the track's packets paired cleanly. Its absence means they did
//! not, or that nothing has looked: both cases fall back to aligning against
//! the video, so the reader never has to tell them apart.
use sqlx::SqlitePool;
/// The cue structure of one image-format track.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Skeleton {
/// The track's position among the file's subtitle streams.
pub stream_index: i64,
/// As `arr_core::Language` spells it: the language a source subtitle is
/// worth fetching in when this skeleton is the reference.
pub language: String,
/// `(start, end)` in milliseconds, in file order.
pub cues: Vec<(i64, i64)>,
}
/// Store the skeleton of one track, replacing whatever a previous import
/// derived for it.
///
/// A re-import of the same file re-derives from the same packets, so the
/// replace is a convergence rather than an upgrade.
///
/// # Errors
///
/// If the insert fails, or the cues cannot be serialised.
pub async fn record(
pool: &SqlitePool,
media_file_id: i64,
stream_index: i64,
language: &str,
cues: &[(i64, i64)],
) -> Result<(), sqlx::Error> {
let encoded = serde_json::to_string(cues).map_err(|error| sqlx::Error::Encode(error.into()))?;
sqlx::query!(
"INSERT INTO subtitle_skeletons (media_file_id, stream_index, language, cues)
VALUES (?, ?, ?, ?)
ON CONFLICT (media_file_id, stream_index) DO UPDATE SET
language = excluded.language,
cues = excluded.cues",
media_file_id,
stream_index,
language,
encoded
)
.execute(pool)
.await?;
Ok(())
}
/// Every skeleton known for one media file, by stream index.
///
/// A row whose cues do not decode is dropped rather than returned: the caller
/// treats a missing skeleton as "align against the video", which is the right
/// answer for an unreadable one too.
///
/// # Errors
///
/// If the query fails.
pub async fn for_media_file(
pool: &SqlitePool,
media_file_id: i64,
) -> Result<Vec<Skeleton>, sqlx::Error> {
let rows = sqlx::query!(
r#"SELECT stream_index AS "stream_index!: i64",
language AS "language!: String",
cues AS "cues!: String"
FROM subtitle_skeletons
WHERE media_file_id = ?
ORDER BY stream_index"#,
media_file_id
)
.fetch_all(pool)
.await?;
Ok(rows
.into_iter()
.filter_map(|row| {
let cues: Vec<(i64, i64)> = serde_json::from_str(&row.cues)
.inspect_err(|error| {
tracing::warn!(
media_file_id,
stream_index = row.stream_index,
%error,
"subtitle skeleton does not decode, ignoring it"
);
})
.ok()?;
Some(Skeleton {
stream_index: row.stream_index,
language: row.language,
cues,
})
})
.collect())
}
+20 -2
View File
@@ -101,6 +101,10 @@ pub struct SubtitleFile {
pub forced: bool,
pub sdh: bool,
pub sync: SubtitleSync,
/// Aligned against a cue skeleton rather than the video (§15, #268), so
/// its timings are exact for this release and a second `alass` pass —
/// here or on anything translated from it — can only move them off.
pub skeleton_aligned: bool,
/// `None` only for [`SubtitleOrigin::Embedded`].
pub path: Option<String>,
}
@@ -122,6 +126,7 @@ pub struct NewSubtitleFile {
forced: bool,
sdh: bool,
sync: SubtitleSync,
skeleton_aligned: bool,
path: Option<String>,
}
@@ -137,6 +142,7 @@ impl NewSubtitleFile {
forced: false,
sdh: false,
sync: SubtitleSync::NotRun,
skeleton_aligned: false,
path: None,
}
}
@@ -206,6 +212,15 @@ impl NewSubtitleFile {
self.sync = sync;
self
}
/// Aligned against a cue skeleton rather than the video (§15, #268).
/// Timings are disc-exact from here on, so the post-translation `alass`
/// pass is skipped for this file and for what is translated from it.
#[must_use]
pub fn skeleton_aligned(mut self) -> Self {
self.skeleton_aligned = true;
self
}
}
/// Record a subtitle, returning its row id.
@@ -235,8 +250,8 @@ pub async fn record_file(
let inserted = sqlx::query_scalar!(
r#"INSERT INTO subtitle_files
(media_file_id, language, origin, provider, candidate_id, engine,
forced, sdh, synced, sync_rejected, path)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
forced, sdh, synced, sync_rejected, skeleton_aligned, path)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (path) DO NOTHING
ON CONFLICT (media_file_id, language, forced, sdh)
WHERE origin = 'embedded' DO NOTHING
@@ -251,6 +266,7 @@ pub async fn record_file(
subtitle.sdh,
synced,
sync_rejected,
subtitle.skeleton_aligned,
subtitle.path,
)
.fetch_optional(pool)
@@ -306,6 +322,7 @@ pub async fn files_for(
sdh AS "sdh!: bool",
synced AS "synced!: bool",
sync_rejected AS "sync_rejected!: bool",
skeleton_aligned AS "skeleton_aligned!: bool",
path
FROM subtitle_files
WHERE media_file_id = ?
@@ -328,6 +345,7 @@ pub async fn files_for(
forced: row.forced,
sdh: row.sdh,
sync: SubtitleSync::from_columns(row.synced, row.sync_rejected),
skeleton_aligned: row.skeleton_aligned,
path: row.path,
})
.collect())
+1 -1
View File
@@ -7,10 +7,10 @@ repository.workspace = true
publish = false
[dependencies]
base64 = { workspace = true }
reqwest = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
sha1 = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true }
url = { workspace = true }
+266
View File
@@ -0,0 +1,266 @@
//! Deriving a torrent's v1 infohash without the client's help.
//!
//! qBittorrent's `torrents/add` answers `Ok.` and nothing else: no hash, no
//! name, and no signal that the torrent was already there. arr needs the hash
//! before it can do anything with the torrent — it is the identity in `grabs`
//! and §6.3's second blacklist key — so it computes the hash from what it is
//! about to send, and looks the torrent up by that.
use sha1::{Digest as _, Sha1};
/// A source arr cannot turn into an infohash.
#[derive(Debug, thiserror::Error)]
pub enum InfohashError {
#[error("magnet link carries no v1 infohash (xt=urn:btih:)")]
NoBtih,
#[error("magnet infohash is neither 40 hex nor 32 base32 characters: {0:?}")]
MalformedBtih(String),
#[error("torrent file is not valid bencode: {0}")]
Bencode(&'static str),
#[error("torrent file has no info dictionary")]
NoInfoDict,
}
/// The v1 infohash named by a magnet link, lowercase hex.
///
/// v2-only magnets (`urn:btmh:`) are rejected rather than guessed at: arr
/// keys everything on the v1 hash.
///
/// # Errors
///
/// Returns an error when the link carries no `urn:btih:` topic, or carries
/// one that is not 40 hex or 32 base32 characters.
pub fn from_magnet(uri: &str) -> Result<String, InfohashError> {
let query = uri.split_once('?').map_or(uri, |(_, query)| query);
let raw = query
.split('&')
.filter_map(|pair| pair.split_once('='))
.filter(|(key, _)| percent_decode(key).eq_ignore_ascii_case("xt"))
.map(|(_, value)| percent_decode(value))
.find_map(|value| {
let rest = value
.get(..9)?
.eq_ignore_ascii_case("urn:btih:")
.then(|| value[9..].to_owned())?;
Some(rest)
})
.ok_or(InfohashError::NoBtih)?;
if raw.len() == 40 && raw.bytes().all(|byte| byte.is_ascii_hexdigit()) {
return Ok(raw.to_ascii_lowercase());
}
if raw.len() == 32 {
if let Some(bytes) = base32_decode(&raw) {
return Ok(hex(&bytes));
}
}
Err(InfohashError::MalformedBtih(raw))
}
/// The v1 infohash of a `.torrent` file: SHA-1 over the raw bytes of its
/// `info` dictionary, exactly as they appear in the file.
///
/// The bytes are hashed as-is rather than re-encoded, because a torrent whose
/// bencode is technically non-canonical still has the infohash its tracker and
/// peers agreed on, and re-encoding would invent a different one.
///
/// # Errors
///
/// Returns an error when the file is not a bencoded dictionary, or has no
/// `info` key.
pub fn from_metainfo(bytes: &[u8]) -> Result<String, InfohashError> {
let mut at = 0;
if bytes.first() != Some(&b'd') {
return Err(InfohashError::Bencode("top level is not a dictionary"));
}
at += 1;
while bytes.get(at) != Some(&b'e') {
if at >= bytes.len() {
return Err(InfohashError::Bencode("dictionary is not terminated"));
}
let (key, after_key) = scan_string(bytes, at)?;
let after_value = scan(bytes, after_key)?;
if key == b"info" {
return Ok(hex(&Sha1::digest(&bytes[after_key..after_value])));
}
at = after_value;
}
Err(InfohashError::NoInfoDict)
}
/// The index just past the bencoded value starting at `at`.
fn scan(bytes: &[u8], at: usize) -> Result<usize, InfohashError> {
match bytes.get(at) {
None => Err(InfohashError::Bencode("value ends early")),
Some(b'i') => bytes[at..]
.iter()
.position(|byte| *byte == b'e')
.map(|offset| at + offset + 1)
.ok_or(InfohashError::Bencode("integer is not terminated")),
Some(b'l' | b'd') => {
let container = bytes[at];
let mut at = at + 1;
while bytes.get(at) != Some(&b'e') {
if at >= bytes.len() {
return Err(InfohashError::Bencode("container is not terminated"));
}
// A dictionary's keys are bencoded strings like any other
// value, so both containers scan the same way.
let _ = container;
at = scan(bytes, at)?;
}
Ok(at + 1)
}
Some(byte) if byte.is_ascii_digit() => Ok(scan_string(bytes, at)?.1),
Some(_) => Err(InfohashError::Bencode("unexpected type marker")),
}
}
/// The bytes of the bencoded string at `at`, and the index just past it.
fn scan_string(bytes: &[u8], at: usize) -> Result<(&[u8], usize), InfohashError> {
let colon = bytes[at..]
.iter()
.position(|byte| *byte == b':')
.map(|offset| at + offset)
.ok_or(InfohashError::Bencode("string has no length separator"))?;
let length: usize = std::str::from_utf8(&bytes[at..colon])
.ok()
.and_then(|digits| digits.parse().ok())
.ok_or(InfohashError::Bencode("string length is not a number"))?;
let start = colon + 1;
let end = start
.checked_add(length)
.filter(|end| *end <= bytes.len())
.ok_or(InfohashError::Bencode(
"string runs past the end of the file",
))?;
Ok((&bytes[start..end], end))
}
/// RFC 4648 base32, the 32-character form some trackers still hand out.
fn base32_decode(input: &str) -> Option<Vec<u8>> {
let mut accumulator: u16 = 0;
let mut bits = 0_u32;
let mut out = Vec::with_capacity(20);
for character in input.chars() {
let value = match character.to_ascii_uppercase() {
letter @ 'A'..='Z' => letter as u16 - 'A' as u16,
digit @ '2'..='7' => digit as u16 - '2' as u16 + 26,
_ => return None,
};
accumulator = (accumulator << 5) | value;
bits += 5;
if bits >= 8 {
bits -= 8;
#[allow(clippy::cast_possible_truncation)]
out.push((accumulator >> bits) as u8);
}
}
(out.len() == 20).then_some(out)
}
fn hex(bytes: &[u8]) -> String {
use std::fmt::Write as _;
bytes
.iter()
.fold(String::with_capacity(40), |mut out, byte| {
let _ = write!(out, "{byte:02x}");
out
})
}
/// Enough percent-decoding for a magnet query parameter. Non-UTF-8 escapes
/// are left alone rather than dropped: the caller only ever compares the
/// result against ASCII.
fn percent_decode(input: &str) -> String {
let bytes = input.as_bytes();
let mut out = Vec::with_capacity(bytes.len());
let mut at = 0;
while at < bytes.len() {
if bytes[at] == b'%' && at + 2 < bytes.len() {
if let Some(byte) = std::str::from_utf8(&bytes[at + 1..at + 3])
.ok()
.and_then(|pair| u8::from_str_radix(pair, 16).ok())
{
out.push(byte);
at += 3;
continue;
}
}
out.push(bytes[at]);
at += 1;
}
String::from_utf8_lossy(&out).into_owned()
}
#[cfg(test)]
mod tests {
use super::{from_magnet, from_metainfo, InfohashError};
#[test]
fn reads_a_hex_magnet_and_lowercases_it() {
let hash =
from_magnet("magnet:?xt=urn:btih:C12FE1C06BBA254A9DC9F519B335AA7C1367A88A&dn=example")
.expect("hex magnet");
assert_eq!(hash, "c12fe1c06bba254a9dc9f519b335aa7c1367a88a");
}
#[test]
fn reads_a_base32_magnet_as_the_same_hash() {
// The base32 form of the hex hash above.
let hash = from_magnet("magnet:?xt=urn:btih:YEX6DQDLXISUVHOJ6UM3GNNKPQJWPKEK")
.expect("base32 magnet");
assert_eq!(hash, "c12fe1c06bba254a9dc9f519b335aa7c1367a88a");
}
#[test]
fn reads_a_percent_encoded_topic() {
let hash = from_magnet("magnet:?xt=urn%3Abtih%3Ac12fe1c06bba254a9dc9f519b335aa7c1367a88a")
.expect("encoded magnet");
assert_eq!(hash, "c12fe1c06bba254a9dc9f519b335aa7c1367a88a");
}
#[test]
fn a_v2_only_magnet_is_rejected_not_guessed() {
let error = from_magnet(
"magnet:?xt=urn:btmh:1220caf1e1c30e81cb361b8f26e5d34c7f7b0f1b4f0d2c3a4b5c6d7e8f9a0b1c2d3",
)
.expect_err("no v1 hash");
assert!(matches!(error, InfohashError::NoBtih));
}
#[test]
fn hashes_the_info_dictionary_of_a_torrent_file() {
// d8:announce5:x:aaa4:infod6:lengthi3e4:name1:ae4:zzzzi1ee
let file = b"d8:announce5:x:aaa4:infod6:lengthi3e4:name1:ae4:zzzzi1ee";
let hash = from_metainfo(file).expect("torrent file");
// SHA-1 of `d6:lengthi3e4:name1:ae`.
assert_eq!(hash, sha1_hex(b"d6:lengthi3e4:name1:ae"));
}
#[test]
fn skips_a_list_valued_key_before_info() {
let file = b"d13:announce-listll1:al1:beee4:infod6:lengthi3e4:name1:aee";
let hash = from_metainfo(file).expect("torrent file");
assert_eq!(hash, sha1_hex(b"d6:lengthi3e4:name1:ae"));
}
#[test]
fn a_truncated_file_is_an_error_not_a_panic() {
let error = from_metainfo(b"d4:infod6:length").expect_err("truncated");
assert!(matches!(error, InfohashError::Bencode(_)));
}
#[test]
fn a_file_without_an_info_key_is_rejected() {
let error = from_metainfo(b"d8:announce5:x:aaae").expect_err("no info");
assert!(matches!(error, InfohashError::NoInfoDict));
}
fn sha1_hex(bytes: &[u8]) -> String {
use sha1::{Digest as _, Sha1};
super::hex(&Sha1::digest(bytes))
}
}
+794 -348
View File
File diff suppressed because it is too large Load Diff
+10 -11
View File
@@ -1,6 +1,6 @@
//! arr-e2e — cross-process integration test harness. See DESIGN.md §12.
//!
//! Tests built on this harness talk to a real Transmission container and to
//! Tests built on this harness talk to a real qBittorrent container and to
//! `wiremock` fakes serving recorded responses for Prowlarr and TMDB. Never a
//! live tracker: trackers rate-limit and it would leak credentials into CI.
//!
@@ -55,13 +55,12 @@ pub const INDEXER_ID: i64 = 3;
/// The movie in [`fixtures::TMDB_MOVIE_DUNE`].
pub const TMDB_MOVIE_ID: u32 = 693_134;
/// The Transmission RPC endpoint tests should use: `TRANSMISSION_RPC_URL`
/// when set (CI points it at the service container), a local container's
/// default port otherwise. Never the production LXC.
/// The qBittorrent `WebUI` base URL tests should use: `QBITTORRENT_URL` when
/// set (CI points it at the service container), a local container's default
/// port otherwise. Never the production instance.
#[must_use]
pub fn transmission_url() -> String {
std::env::var("TRANSMISSION_RPC_URL")
.unwrap_or_else(|_| "http://127.0.0.1:9091/transmission/rpc".into())
pub fn qbittorrent_url() -> String {
std::env::var("QBITTORRENT_URL").unwrap_or_else(|_| "http://127.0.0.1:8080".into())
}
/// A `wiremock` Prowlarr: the REST indexer enumeration plus one enabled
@@ -197,8 +196,8 @@ impl Daemon {
///
/// Panics when the binary cannot be built or spawned, or when the health
/// endpoint does not answer within the boot timeout.
pub async fn spawn(prowlarr_url: &str, tmdb_url: &str, transmission_url: &str) -> Self {
Self::spawn_with_env(prowlarr_url, tmdb_url, transmission_url, &[]).await
pub async fn spawn(prowlarr_url: &str, tmdb_url: &str, qbittorrent_url: &str) -> Self {
Self::spawn_with_env(prowlarr_url, tmdb_url, qbittorrent_url, &[]).await
}
/// Same as [`Self::spawn`], plus extra environment variables for the
@@ -212,7 +211,7 @@ impl Daemon {
pub async fn spawn_with_env(
prowlarr_url: &str,
tmdb_url: &str,
transmission_url: &str,
qbittorrent_url: &str,
extra_env: &[(&str, &str)],
) -> Self {
let dir = tempfile::tempdir().expect("create daemon tempdir");
@@ -232,7 +231,7 @@ impl Daemon {
.env("ARR_PROWLARR_API_KEY", API_KEY)
.env("ARR_TMDB_URL", tmdb_url)
.env("ARR_TMDB_API_KEY", API_KEY)
.env("ARR_TRANSMISSION_URL", transmission_url)
.env("ARR_QBITTORRENT_URL", qbittorrent_url)
.stdin(Stdio::null());
for (key, value) in extra_env {
command.env(key, value);
+26 -21
View File
@@ -1,5 +1,5 @@
//! Cross-process end-to-end tests over the DESIGN.md §12 seams: the real
//! `arr` binary, a real Transmission container, and `wiremock` fakes serving
//! `arr` binary, a real qBittorrent container, and `wiremock` fakes serving
//! recorded Prowlarr and TMDB responses. Never a live tracker.
//!
//! The full add-to-imported run lands here once the reconcile loop (#21),
@@ -12,28 +12,28 @@ use {arr_db as _, sqlx as _, tempfile as _, wiremock as _};
use std::path::PathBuf;
use arr_dl::{AddTorrent, TorrentSource, TransmissionClient};
use arr_dl::{AddTorrent, QbitClient, TorrentSource};
use arr_e2e::{
transmission_url, Daemon, FakeProwlarr, FakeTmdb, API_KEY, INDEXER_ID, TMDB_MOVIE_ID,
qbittorrent_url, Daemon, FakeProwlarr, FakeTmdb, API_KEY, INDEXER_ID, TMDB_MOVIE_ID,
};
use arr_indexer::{ProwlarrClient, SearchRequest};
use arr_meta::TmdbClient;
use chrono::NaiveDate;
/// The daemon boots as its own process against faked Prowlarr and TMDB plus a
/// real Transmission, reports every upstream healthy, and serves the movie
/// real qBittorrent, reports every upstream healthy, and serves the movie
/// API over the wire.
#[tokio::test]
async fn daemon_runs_against_fakes_and_a_real_transmission() {
async fn daemon_runs_against_fakes_and_a_real_qbittorrent() {
let prowlarr = FakeProwlarr::start().await;
let tmdb = FakeTmdb::start().await;
let daemon = Daemon::spawn(&prowlarr.url(), &tmdb.url(), &transmission_url()).await;
let daemon = Daemon::spawn(&prowlarr.url(), &tmdb.url(), &qbittorrent_url()).await;
let base = daemon.base_url();
let health = daemon.health().await;
assert_eq!(health["status"], "ok", "health report: {health}");
assert_eq!(health["prowlarr"]["status"], "ok");
assert_eq!(health["transmission"]["status"], "ok");
assert_eq!(health["qbit"]["status"], "ok");
assert_eq!(health["tmdb"]["status"], "ok");
// Movie CRUD across the process boundary, against the migrated seed data.
@@ -134,11 +134,13 @@ async fn recorded_fixtures_satisfy_the_real_clients() {
assert!(movie.is_digitally_released(after_digital));
}
/// Torrent lifecycle against the real Transmission container: RPC semantics
/// are the seam most likely to surprise (DESIGN.md §12).
/// Torrent lifecycle against the real qBittorrent container: the `WebUI` API's
/// semantics are the seam most likely to surprise (DESIGN.md §12). In
/// particular `torrents/add` answers `Ok.` and nothing else, so the infohash
/// arr derived locally has to be the one qBittorrent went on to list.
#[tokio::test]
async fn transmission_add_list_and_remove() {
let client = TransmissionClient::new(&transmission_url()).expect("valid endpoint");
async fn qbittorrent_add_list_and_remove() {
let client = QbitClient::new(&qbittorrent_url()).expect("valid endpoint");
let name = format!("arr-e2e-{}", uuid::Uuid::new_v4());
let metainfo = torrent_with_name(&name);
let download_dir = PathBuf::from("/tmp/arr-e2e");
@@ -156,34 +158,37 @@ async fn transmission_add_list_and_remove() {
let torrents = client.list_torrents().await.expect("list torrents");
let listed = torrents
.iter()
.find(|torrent| torrent.id == first.id)
.find(|torrent| torrent.hash == first.hash)
.expect("added torrent is authoritative in list");
assert_eq!(listed.name, name);
assert_eq!(listed.hash, first.hash);
assert_eq!(listed.download_dir, download_dir);
assert_eq!(listed.labels, ["movies-main"]);
assert!((0.0..=1.0).contains(&listed.progress));
client
.remove_torrent(first.id, false)
.remove_torrent(&first.hash, false)
.await
.expect("remove without data");
let second = client.add_torrent(request()).await.expect("add again");
assert_eq!(
second.hash, first.hash,
"the same file is the same infohash"
);
client
.remove_torrent(second.id, true)
.remove_torrent(&second.hash, true)
.await
.expect("remove with data");
let torrents = client.list_torrents().await.expect("list after remove");
assert!(torrents.iter().all(|torrent| torrent.id != second.id));
assert!(torrents.iter().all(|torrent| torrent.hash != second.hash));
}
/// Transmission, not arr's import state, owns the done-seeding boundary. The
/// qBittorrent, not arr's import state, owns the done-seeding boundary. The
/// real service must report an unfinished torrent before its seed limit clears.
#[tokio::test]
async fn transmission_reports_done_only_after_its_seed_limit() {
let client = TransmissionClient::new(&transmission_url()).expect("valid endpoint");
async fn qbittorrent_reports_done_only_after_its_seed_limit() {
let client = QbitClient::new(&qbittorrent_url()).expect("valid endpoint");
let name = format!("arr-e2e-reaper-{}", uuid::Uuid::new_v4());
let download_dir = PathBuf::from("/tmp/arr-e2e");
let metainfo = torrent_with_name(&name);
@@ -204,7 +209,7 @@ async fn transmission_reports_done_only_after_its_seed_limit() {
.await
.expect("list before limit")
.into_iter()
.find(|torrent| torrent.id == added.id)
.find(|torrent| torrent.hash == added.hash)
.expect("torrent before limit");
assert!(
!before.is_finished,
@@ -212,7 +217,7 @@ async fn transmission_reports_done_only_after_its_seed_limit() {
);
client
.remove_torrent(added.id, true)
.remove_torrent(&added.hash, true)
.await
.expect("cleanup");
}
+1 -1
View File
@@ -164,7 +164,7 @@ impl ProwlarrClient {
/// Resolves an indexer download link into something the download client
/// can take without reaching the indexer itself.
///
/// Handing Transmission the Prowlarr link fails twice over: Transmission
/// Handing qBittorrent the Prowlarr link fails twice over: qBittorrent
/// has no route to Prowlarr, and Prowlarr answers a `.torrent` link with a
/// redirect to a magnet, which a plain file fetch cannot follow. arr has
/// the route and the API key, so it resolves the link here and passes on
+5
View File
@@ -27,6 +27,7 @@ mod extract;
mod ffprobe;
mod language;
mod model;
mod skeleton;
// `unused_crate_dependencies` is a per-target lint and the library's own test
// target links the dev-dependencies without using them. The real uses are in
@@ -37,6 +38,10 @@ use tempfile as _;
pub use error::{Error, Result};
pub use extract::{Extractor, DEFAULT_EXTRACTION_TIMEOUT};
pub use model::{FeatureSelection, ProbedFile, RuntimeMatch};
pub use skeleton::{
Implausible, Outcome as SkeletonOutcome, Skeleton, SkeletonCue, Skeletons,
DEFAULT_SKELETON_TIMEOUT,
};
/// The binary invoked when nothing else is configured.
pub const DEFAULT_BINARY: &str = "ffprobe";
+448
View File
@@ -0,0 +1,448 @@
//! Deriving a cue skeleton from an image-format subtitle track (§15, #268).
//!
//! A PGS or `VobSub` track carries bitmaps, so nothing here reads a pixel.
//! What it reads is packet metadata: `ffprobe -show_packets` returns the
//! presentation timestamps of a subtitle track directly, and on a retail disc
//! those alternate show, clear, show, clear. Pairing them yields the cue
//! structure of the release — the disc's own timings, exact for that file.
//!
//! That structure is what `alass` matches on. It compares interval shapes and
//! not words, so a skeleton with no text in it is still a strong alignment
//! reference for a downloaded subtitle whose cues never line up one-for-one
//! with the disc's.
//!
//! The pairing is load-bearing. Treating every packet as a cue start instead
//! of pairing them puts the whole reference out by roughly half a cue —
//! better than no alignment, and visibly wrong. So a track that does not pair
//! cleanly is rejected here rather than handed on: PGS permits several
//! composition segments per subtitle, and the caller must fall back to
//! aligning against the video rather than trust a skeleton that is quietly
//! half a second out.
use std::{ffi::OsString, path::Path, process::Stdio, time::Duration};
use serde::Deserialize;
use tokio::process::Command;
use crate::error::{Error, Result};
/// The binary invoked when nothing else is configured.
pub const DEFAULT_BINARY: &str = "ffprobe";
/// How long one derivation may take. Generous: reading every subtitle packet
/// out of a 60 GB remux is a full demux of the container.
pub const DEFAULT_SKELETON_TIMEOUT: Duration = Duration::from_secs(300);
/// The shortest a cue may last and still be a cue.
pub const MIN_CUE: Duration = Duration::from_millis(300);
/// The longest a cue may last and still be a cue. A pairing that has slipped
/// spans the gap between two subtitles, which on any real track is longer
/// than this.
pub const MAX_CUE: Duration = Duration::from_secs(10);
/// The sparsest a real subtitle track gets, in cues per minute of runtime.
pub const MIN_DENSITY: f64 = 0.2;
/// The densest a real subtitle track gets, in cues per minute of runtime.
pub const MAX_DENSITY: f64 = 60.0;
/// How much of `ffprobe`'s stderr is kept in an error.
const STDERR_LIMIT: usize = 512;
/// One cue of a skeleton: when the bitmap appeared, and when it was cleared.
/// No text — there is none to read, and `alass` does not want any.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct SkeletonCue {
/// When the subtitle appears.
pub start: Duration,
/// When it disappears.
pub end: Duration,
}
impl SkeletonCue {
/// How long the subtitle is on screen.
#[must_use]
pub fn duration(&self) -> Duration {
self.end.saturating_sub(self.start)
}
}
/// The cue structure of one image-format track.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Skeleton {
/// The track's position among the file's subtitle streams — the same
/// index [`crate::Extractor::extract_srt`] maps.
pub stream_index: usize,
/// The cues, in file order.
pub cues: Vec<SkeletonCue>,
}
/// Why a track's packets could not be trusted as a cue structure (§15).
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum Implausible {
/// Fewer than two packets: nothing to pair.
TooFewPackets {
/// How many there were.
packets: usize,
},
/// An odd number of packets, so at least one show has no clear. The
/// track does not follow the show/clear alternation pairing assumes.
OddPacketCount {
/// How many there were.
packets: usize,
},
/// A paired cue lasts an implausible length of time, which means the
/// pairing slipped somewhere at or before it.
CueDuration {
/// The cue's position in the track, 0-based.
index: usize,
/// How long it came out.
duration: Duration,
},
/// The cue count does not fit the runtime — too sparse to be a subtitle
/// track, or too dense to be one cue per subtitle.
Density {
/// How many cues were paired.
cues: usize,
/// Per minute of runtime.
per_minute: f64,
},
}
impl std::fmt::Display for Implausible {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::TooFewPackets { packets } => {
write!(formatter, "{packets} packets is too few to pair")
}
Self::OddPacketCount { packets } => {
write!(formatter, "{packets} packets do not pair show to clear")
}
Self::CueDuration { index, duration } => {
write!(formatter, "cue {index} lasts {duration:.1?}")
}
Self::Density { cues, per_minute } => {
write!(formatter, "{cues} cues is {per_minute:.1} per minute")
}
}
}
}
/// What one derivation decided.
#[derive(Clone, Debug, PartialEq)]
pub enum Outcome {
/// The packets paired into a trustworthy cue structure.
Derived(Skeleton),
/// They did not. The caller aligns against the video instead.
Implausible(Implausible),
}
/// Reads the packet timings of image-format subtitle tracks.
///
/// The same shape as [`crate::Prober`] and [`crate::Extractor`]: one small
/// binary, invoked and discarded, with a configurable path and its own
/// timeout. It is a separate handle rather than a method on `Prober` because
/// the timeout is a different order of magnitude — a metadata read against a
/// full demux.
#[derive(Clone, Debug)]
pub struct Skeletons {
binary: OsString,
timeout: Duration,
}
impl Default for Skeletons {
fn default() -> Self {
Self {
binary: DEFAULT_BINARY.into(),
timeout: DEFAULT_SKELETON_TIMEOUT,
}
}
}
impl Skeletons {
/// A reader that runs `ffprobe` from `PATH`.
#[must_use]
pub fn new() -> Self {
Self::default()
}
/// Run a specific binary instead of whatever `PATH` resolves to.
#[must_use]
pub fn with_binary(mut self, binary: impl Into<OsString>) -> Self {
self.binary = binary.into();
self
}
/// Change how long one derivation may take.
#[must_use]
pub fn with_timeout(mut self, timeout: Duration) -> Self {
self.timeout = timeout;
self
}
/// Derive the cue skeleton of one subtitle track.
///
/// `stream_index` is the track's position among the file's subtitle
/// streams, not the container's absolute stream id — the same index
/// [`crate::Extractor::extract_srt`] takes. `runtime` is the file's
/// playing time, which the density check needs; without it that check is
/// skipped and the rest still applies.
///
/// # Errors
///
/// [`Error::Spawn`] when `ffprobe` is not installed, [`Error::Timeout`],
/// [`Error::Rejected`] when it exits non-zero — which includes selecting
/// a stream that does not exist — and [`Error::Decode`] on output that is
/// not the JSON this asked for.
pub async fn derive(
&self,
video: &Path,
stream_index: usize,
runtime: Option<Duration>,
) -> Result<Outcome> {
let output = self.run(video, stream_index).await?;
let document: PacketList =
serde_json::from_slice(&output).map_err(|source| Error::Decode {
path: video.to_path_buf(),
source,
})?;
let stamps: Vec<Duration> = document
.packets
.iter()
.filter_map(Packet::timestamp)
.collect();
Ok(pair(stream_index, &stamps, runtime))
}
/// Spawn `ffprobe` and collect its stdout.
///
/// Only `pts_time` is asked for: a feature-length track is a few thousand
/// packets, and the rest of what `-show_packets` prints per packet would
/// be megabytes of JSON for facts nothing here reads.
async fn run(&self, video: &Path, stream_index: usize) -> Result<Vec<u8>> {
let select = format!("s:{stream_index}");
let mut command = Command::new(&self.binary);
command
.args([
"-v",
"error",
"-hide_banner",
"-print_format",
"json",
"-select_streams",
&select,
"-show_entries",
"packet=pts_time",
"-i",
])
.arg(video)
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.kill_on_drop(true);
let child = command.spawn().map_err(|source| Error::Spawn {
binary: self.binary.to_string_lossy().into_owned(),
source,
})?;
let output = match tokio::time::timeout(self.timeout, child.wait_with_output()).await {
Ok(output) => output.map_err(|source| Error::Io {
path: video.to_path_buf(),
source,
})?,
Err(_) => {
return Err(Error::Timeout {
path: video.to_path_buf(),
})
}
};
if output.status.success() {
return Ok(output.stdout);
}
let stderr = String::from_utf8_lossy(&output.stderr)
.trim()
.chars()
.take(STDERR_LIMIT)
.collect();
Err(Error::Rejected {
path: video.to_path_buf(),
status: output.status.code(),
stderr,
})
}
}
/// Pair packet timestamps into cues, and judge the result.
///
/// Split out from the spawn so the rule is testable without `ffprobe`: this
/// is the whole of what makes a skeleton trustworthy.
fn pair(stream_index: usize, stamps: &[Duration], runtime: Option<Duration>) -> Outcome {
if stamps.len() < 2 {
return Outcome::Implausible(Implausible::TooFewPackets {
packets: stamps.len(),
});
}
if !stamps.len().is_multiple_of(2) {
return Outcome::Implausible(Implausible::OddPacketCount {
packets: stamps.len(),
});
}
let mut cues = Vec::with_capacity(stamps.len() / 2);
for (index, [start, end]) in stamps.as_chunks::<2>().0.iter().enumerate() {
let cue = SkeletonCue {
start: *start,
end: *end,
};
let duration = cue.duration();
if duration < MIN_CUE || duration > MAX_CUE {
return Outcome::Implausible(Implausible::CueDuration { index, duration });
}
cues.push(cue);
}
if let Some(runtime) = runtime {
let minutes = runtime.as_secs_f64() / 60.0;
if minutes > 0.0 {
#[allow(clippy::cast_precision_loss)]
let per_minute = cues.len() as f64 / minutes;
if per_minute < MIN_DENSITY || per_minute > MAX_DENSITY {
return Outcome::Implausible(Implausible::Density {
cues: cues.len(),
per_minute,
});
}
}
}
Outcome::Derived(Skeleton { stream_index, cues })
}
/// The slice of `ffprobe -show_entries packet=pts_time` this reads.
#[derive(Debug, Deserialize)]
struct PacketList {
#[serde(default)]
packets: Vec<Packet>,
}
#[derive(Debug, Deserialize)]
struct Packet {
/// `"12.345000"`, or absent — `ffprobe` omits a field rather than
/// nulling it, and prints the string `"N/A"` where it has no value.
pts_time: Option<String>,
}
impl Packet {
fn timestamp(&self) -> Option<Duration> {
let seconds: f64 = self.pts_time.as_deref()?.trim().parse().ok()?;
if seconds.is_finite() && seconds >= 0.0 {
Duration::try_from_secs_f64(seconds).ok()
} else {
None
}
}
}
#[cfg(test)]
mod tests {
use std::time::Duration;
use super::{pair, Implausible, Outcome, SkeletonCue};
fn secs(value: f64) -> Duration {
Duration::from_secs_f64(value)
}
/// The real shape: packets strictly alternate show and clear, so every
/// other one closes the cue the one before it opened.
#[test]
fn packets_pair_show_to_clear() {
let stamps = [secs(1.0), secs(3.0), secs(5.0), secs(7.5)];
let Outcome::Derived(skeleton) = pair(0, &stamps, Some(Duration::from_secs(600))) else {
panic!("the pairing must be derived");
};
assert_eq!(
skeleton.cues,
vec![
SkeletonCue {
start: secs(1.0),
end: secs(3.0)
},
SkeletonCue {
start: secs(5.0),
end: secs(7.5)
},
]
);
}
/// #268's measurement: reading every packet as a cue start is out by
/// roughly half a cue over the whole film. An odd count is the visible
/// symptom of a track that does not alternate, and it is refused rather
/// than paired anyway.
#[test]
fn an_odd_packet_count_is_refused() {
let stamps = [secs(1.0), secs(3.0), secs(5.0)];
assert_eq!(
pair(0, &stamps, None),
Outcome::Implausible(Implausible::OddPacketCount { packets: 3 })
);
}
#[test]
fn fewer_than_two_packets_pair_into_nothing() {
assert_eq!(
pair(0, &[secs(1.0)], None),
Outcome::Implausible(Implausible::TooFewPackets { packets: 1 })
);
}
/// PGS permits several composition segments per subtitle. A track built
/// that way still has an even packet count, and pairing it spans the gap
/// between two subtitles — which is what the duration bound catches.
#[test]
fn a_slipped_pairing_shows_up_as_an_impossible_duration() {
let stamps = [secs(1.0), secs(2.0), secs(3.0), secs(40.0)];
assert_eq!(
pair(0, &stamps, None),
Outcome::Implausible(Implausible::CueDuration {
index: 1,
duration: secs(37.0)
})
);
}
#[test]
fn a_cue_shorter_than_a_glance_is_refused() {
let stamps = [secs(1.0), secs(1.1)];
assert!(matches!(
pair(0, &stamps, None),
Outcome::Implausible(Implausible::CueDuration { index: 0, .. })
));
}
/// Two cues over a two-hour film is a signs-only oddity, not the
/// structure of the film's dialogue.
#[test]
fn a_track_too_sparse_for_the_runtime_is_refused() {
let stamps = [secs(1.0), secs(3.0), secs(5.0), secs(7.0)];
assert!(matches!(
pair(0, &stamps, Some(Duration::from_secs(7200))),
Outcome::Implausible(Implausible::Density { cues: 2, .. })
));
}
/// Without a runtime there is nothing to judge density against, and the
/// rest of the checks still stand.
#[test]
fn density_is_skipped_when_the_runtime_is_unknown() {
let stamps = [secs(1.0), secs(3.0)];
assert!(matches!(pair(0, &stamps, None), Outcome::Derived(_)));
}
}
+128 -58
View File
@@ -117,7 +117,6 @@ impl OpenSubtitlesConfig {
#[derive(Clone, Debug)]
struct CandidateMeta {
language: Language,
format: SubtitleFormat,
}
/// The OpenSubtitles.com client behind the [`Provider`] trait.
@@ -251,7 +250,12 @@ impl OpenSubtitles {
.header("Api-Key", &self.config.api_key)
.header("User-Agent", USER_AGENT);
if let Some(query) = query {
request = request.query(query);
// The API answers a 301 when the parameters are not in
// alphabetical order, so sort before sending rather than trust a
// redirect to carry the request intact (#270).
let mut query = query.to_vec();
query.sort_by(|(left, _), (right, _)| left.cmp(right));
request = request.query(&query);
}
if let Some(json) = json {
request = request.json(json);
@@ -282,12 +286,20 @@ impl OpenSubtitles {
provider: self.id.clone(),
detail: format!("bad login path: {err}"),
})?;
// The credentials travel as a JSON body, not as HTTP basic auth
// (#271). Basic auth is answered 401, which reads as "your username
// and password are wrong" and is what this reported for every
// download.
let credentials = serde_json::json!({
"username": username,
"password": password,
});
let response = self
.http
.post(url)
.basic_auth(username, Some(password))
.header("Api-Key", &self.config.api_key)
.header("User-Agent", USER_AGENT)
.json(&credentials)
.send()
.await
.map_err(|err| Error::Transport {
@@ -343,25 +355,7 @@ impl OpenSubtitles {
source,
})?;
let mut params: Vec<(String, String)> =
vec![("languages".to_owned(), languages_param(&request.languages))];
match request.file.media {
MediaRef::Movie { tmdb_id } => {
params.push(("tmdb_movie_id".to_owned(), tmdb_id.to_string()));
}
MediaRef::Episode {
tmdb_id,
season,
episode,
} => {
params.push(("tmdb_series_id".to_owned(), tmdb_id.to_string()));
params.push(("season_number".to_owned(), season.to_string()));
params.push(("episode_number".to_owned(), episode.to_string()));
}
}
if let Some(hash) = hash.as_deref() {
params.push(("moviehash".to_owned(), hash.to_owned()));
}
let params = search_params(request, hash.as_deref());
let response = self
.send(Method::GET, "subtitles", Some(&params), None, None)
@@ -385,7 +379,6 @@ impl OpenSubtitles {
offered.candidate.id.clone(),
CandidateMeta {
language: offered.candidate.language.clone(),
format: offered.format.clone(),
},
)
}));
@@ -416,7 +409,11 @@ impl OpenSubtitles {
})?;
let token = self.current_token().await?;
let body = serde_json::json!({ "file_id": file_id });
// Ask for SRT rather than take whatever the uploader posted (#272).
// A search entry often carries no `format` at all, which left the
// download labelled `Other("")` and the conversion with no parser to
// pick. The API converts on its side, so the answer is always SRT.
let body = serde_json::json!({ "file_id": file_id, "sub_format": "srt" });
let response = self
.send(Method::POST, "download", None, Some(&body), Some(&token))
@@ -436,7 +433,7 @@ impl OpenSubtitles {
Ok(Fetched {
id: id.clone(),
language: meta.language,
format: meta.format,
format: SubtitleFormat::Srt,
content,
})
}
@@ -501,7 +498,6 @@ fn truncate(body: &str) -> String {
/// One API entry turned into arr's terms, plus the facts download will need.
struct Offered {
candidate: Candidate,
format: SubtitleFormat,
}
/// The wire shape of `POST /login`: the user token everything that touches
@@ -536,8 +532,6 @@ impl SubtitleEntry {
let file_id = attributes.files.first()?.file_id;
let language = language_of(attributes.language.as_deref()?)?;
let release_name = attributes.release.filter(|release| !release.is_empty());
let format = format_of(attributes.format.as_deref());
Some(Offered {
candidate: Candidate {
provider: ProviderId::new(PROVIDER_NAME),
@@ -552,7 +546,6 @@ impl SubtitleEntry {
sdh: attributes.hearing_impaired,
release_name,
},
format,
})
}
}
@@ -570,7 +563,6 @@ struct SubtitleAttributes {
hearing_impaired: bool,
#[serde(default)]
foreign_parts_only: bool,
format: Option<String>,
files: Vec<SubtitleFileRef>,
}
@@ -579,6 +571,39 @@ struct SubtitleFileRef {
file_id: u64,
}
/// The query a search sends, in the parameter names the API documents.
///
/// `tmdb_movie_id` and `tmdb_series_id` — what this sent until #270 — are not
/// parameters of this API. Unknown ones are ignored rather than refused, so
/// the id lane contributed nothing and every search came back as whatever the
/// `moviehash` alone matched: one result for a popular release, none at all
/// for anything the hash database has never seen.
fn search_params(request: &SearchRequest, hash: Option<&str>) -> Vec<(String, String)> {
let mut params: Vec<(String, String)> =
vec![("languages".to_owned(), languages_param(&request.languages))];
match request.file.media {
MediaRef::Movie { tmdb_id } => {
params.push(("tmdb_id".to_owned(), tmdb_id.to_string()));
}
MediaRef::Episode {
tmdb_id,
season,
episode,
} => {
params.push(("parent_tmdb_id".to_owned(), tmdb_id.to_string()));
params.push(("season_number".to_owned(), season.to_string()));
params.push(("episode_number".to_owned(), episode.to_string()));
}
}
// Sent alongside the id rather than instead of it: the API answers the id
// search and flags the entries the hash also matched, which is what
// ranking reads.
if let Some(hash) = hash {
params.push(("moviehash".to_owned(), hash.to_owned()));
}
params
}
/// The API spells languages `pt-PT`, `pt-BR`, `en`; accept `_` too, since
/// tooling around this API uses it.
fn language_of(code: &str) -> Option<Language> {
@@ -594,16 +619,21 @@ fn language_of(code: &str) -> Option<Language> {
/// The `languages` query parameter: comma-separated codes, pt-PT and pt-BR
/// kept apart — keeping them apart is why this is the primary provider.
fn languages_param(languages: &[Language]) -> String {
languages
// Lowercase and sorted, both of which the API requires to answer rather
// than redirect (#270). Case is not what keeps pt-PT and pt-BR apart --
// the region subtag is -- so lowercasing costs nothing.
let mut codes: Vec<String> = languages
.iter()
.map(|language| match language {
Language::PortuguesePortugal => "pt-PT",
Language::PortugueseBrazil => "pt-BR",
Language::PortugueseUnverified => "pt",
Language::Other(tag) => tag.as_str(),
Language::PortuguesePortugal => "pt-pt".to_owned(),
Language::PortugueseBrazil => "pt-br".to_owned(),
Language::PortugueseUnverified => "pt".to_owned(),
Language::Other(tag) => tag.to_ascii_lowercase(),
})
.collect::<Vec<_>>()
.join(",")
.collect();
codes.sort();
codes.dedup();
codes.join(",")
}
/// The release group of the release a subtitle was timed against, read off
@@ -616,15 +646,6 @@ fn source_of_release(release: &str) -> Option<arr_core::Source> {
arr_parse::parse(release).source.map(Into::into)
}
fn format_of(format: Option<&str>) -> SubtitleFormat {
match format.unwrap_or_default().to_ascii_lowercase().as_str() {
"srt" | "subrip" => SubtitleFormat::Srt,
"ass" | "ssa" => SubtitleFormat::Ass,
"vtt" | "webvtt" => SubtitleFormat::Vtt,
other => SubtitleFormat::Other(other.to_owned()),
}
}
/// Order the candidates best first through the pure ranker (#185), building
/// the target from the same facts the search ran on: the file's own hash and
/// whatever the release name claims about group and source.
@@ -658,8 +679,9 @@ fn rank_candidates(
#[cfg(test)]
mod tests {
use super::{format_of, language_of, moviehash, rank_candidates, PROVIDER_NAME};
use crate::{Candidate, CandidateId, MediaFile, MediaRef, ProviderId, SubtitleFormat};
use super::{language_of, moviehash, rank_candidates, search_params, PROVIDER_NAME};
use crate::SearchRequest;
use crate::{Candidate, CandidateId, MediaFile, MediaRef, ProviderId};
use arr_core::Language;
#[test]
@@ -718,21 +740,69 @@ mod tests {
Language::PortugueseUnverified,
Language::Other("en".to_owned()),
]),
"pt-PT,pt-BR,pt,en"
"en,pt,pt-br,pt-pt"
);
}
fn request(media: MediaRef, languages: Vec<Language>) -> SearchRequest {
SearchRequest {
file: MediaFile {
path: std::path::PathBuf::from("/media/x.mkv"),
size: 1,
release_name: None,
media,
},
languages,
}
}
/// #270: the id lane sent parameter names this API does not have. It
/// ignores unknown ones, so every search silently degraded to a
/// `moviehash` lookup and answered nothing for a release no one has
/// hashed.
#[test]
fn a_movie_search_asks_by_tmdb_id() {
let params = search_params(
&request(
MediaRef::Movie { tmdb_id: 272 },
vec![Language::Other("en".to_owned())],
),
Some("abc123"),
);
assert_eq!(
params,
vec![
("languages".to_owned(), "en".to_owned()),
("tmdb_id".to_owned(), "272".to_owned()),
("moviehash".to_owned(), "abc123".to_owned()),
]
);
}
#[test]
fn formats_spell_the_way_providers_do() {
assert_eq!(format_of(Some("srt")), SubtitleFormat::Srt);
assert_eq!(format_of(Some("subrip")), SubtitleFormat::Srt);
assert_eq!(format_of(Some("ass")), SubtitleFormat::Ass);
assert_eq!(format_of(Some("vtt")), SubtitleFormat::Vtt);
assert_eq!(
format_of(Some("idx")),
SubtitleFormat::Other("idx".to_owned())
fn an_episode_search_asks_by_the_series_tmdb_id_and_the_numbers() {
let params = search_params(
&request(
MediaRef::Episode {
tmdb_id: 60625,
season: 2,
episode: 4,
},
vec![Language::PortuguesePortugal],
),
None,
);
assert_eq!(
params,
vec![
("languages".to_owned(), "pt-pt".to_owned()),
("parent_tmdb_id".to_owned(), "60625".to_owned()),
("season_number".to_owned(), "2".to_owned()),
("episode_number".to_owned(), "4".to_owned()),
]
);
assert_eq!(format_of(None), SubtitleFormat::Other(String::new()));
}
fn candidate(id: u64, hash_match: bool, downloads: u64) -> Candidate {
+55 -2
View File
@@ -11,8 +11,9 @@
//! went missing between input and output, make the whole result implausible —
//! and an implausible result is not an error but a [`Outcome::Rejected`], so
//! the caller keeps the unsynced original and flags the row (#186). The
//! reference is always the media file itself; subtitle and video are both on
//! disk by the time this runs.
//! reference is the media file itself, or — for a release whose only text
//! source has to be downloaded (#268) — the cue skeleton of one of its image
//! subtitle tracks, which carries the disc's own timings.
use std::{ffi::OsStr, ffi::OsString, path::Path, process::Stdio, time::Duration};
@@ -37,6 +38,10 @@ pub const MAX_SHIFT: Duration = Duration::from_secs(60);
/// How much of `alass`'s stderr is kept in an error.
const STDERR_LIMIT: usize = 512;
/// What a cue skeleton's cues say. `alass` reads only their timings, and SRT
/// has no way to spell a cue with no text at all.
const SKELETON_TEXT: &str = ".";
/// Whether a configured external binary resolves to an executable.
///
/// A name with any path component (`/usr/local/bin/alass`, `./alass`) must
@@ -175,6 +180,54 @@ impl Syncer {
})
}
/// Settle `subtitle` against a cue skeleton instead of the video (§15,
/// #268).
///
/// `reference` is the `(start, end)` spans of an image-format embedded
/// track, whose timings are exact for the release the file came off.
/// `alass` matches on interval structure and not on words, so the
/// placeholder text written here is never read — a sparse skeleton whose
/// cues do not correspond one-for-one with the subtitle's is still a
/// strong signal.
///
/// Degrades the same way [`Self::settle`] does: a reference that cannot
/// be written is a [`SyncState::NotRun`], not a failure of the caller.
pub async fn settle_against_cues(
&self,
reference: &[(Duration, Duration)],
subtitle: impl AsRef<Path>,
) -> Settled {
let cues: Vec<srt::Cue> = reference
.iter()
.map(|(start, end)| srt::Cue {
start: *start,
end: *end,
text: SKELETON_TEXT.to_owned(),
})
.collect();
let directory = match tempfile::tempdir() {
Ok(directory) => directory,
Err(error) => {
tracing::warn!(%error, "no temporary directory for the cue skeleton");
return Settled {
content: None,
state: SyncState::NotRun,
};
}
};
let path = directory.path().join("skeleton.srt");
if let Err(error) = tokio::fs::write(&path, srt::render(&cues)).await {
tracing::warn!(%error, "the cue skeleton could not be written");
return Settled {
content: None,
state: SyncState::NotRun,
};
}
self.settle(&path, subtitle).await
}
/// Run [`Self::sync`] and settle the result into a form neither caller
/// has to branch on twice.
///
+16 -5
View File
@@ -92,9 +92,9 @@ async fn movie_search_sends_both_lanes_and_ranks_hash_match_first() {
Mock::given(method("GET"))
.and(path("/api/v1/subtitles"))
.and(header("Api-Key", "test-api-key"))
.and(query_param("tmdb_movie_id", "693134"))
.and(query_param("tmdb_id", "693134"))
.and(query_param("moviehash", "0000000000040000"))
.and(query_param("languages", "pt-PT,pt-BR,en"))
.and(query_param("languages", "en,pt-br,pt-pt"))
.respond_with(ResponseTemplate::new(200).set_body_string(SEARCH_MOVIE))
.mount(&server)
.await;
@@ -139,7 +139,7 @@ async fn episode_search_addresses_the_series_not_the_movie_lane() {
Mock::given(method("GET"))
.and(path("/api/v1/subtitles"))
.and(query_param("tmdb_series_id", "94605"))
.and(query_param("parent_tmdb_id", "94605"))
.and(query_param("season_number", "3"))
.and(query_param("episode_number", "7"))
.respond_with(ResponseTemplate::new(200).set_body_string(SEARCH_EPISODE))
@@ -182,7 +182,7 @@ async fn a_file_too_small_to_hash_searches_without_the_hash_lane() {
Mock::given(method("GET"))
.and(path("/api/v1/subtitles"))
.and(query_param("tmdb_movie_id", "42"))
.and(query_param("tmdb_id", "42"))
.and(QueryParamMissing("moviehash"))
.respond_with(ResponseTemplate::new(200).set_body_string(SEARCH_EPISODE))
.mount(&server)
@@ -250,6 +250,12 @@ async fn an_expired_key_is_unauthorized_and_a_cap_is_rate_limited() {
async fn mount_login(server: &MockServer, token: &'static str) {
Mock::given(method("POST"))
.and(path("/api/v1/login"))
// The credentials are a JSON body. Sent as HTTP basic auth — what
// this did until #271 — the API answers 401 and every download
// reports the username and password as refused.
.and(body_partial_json(
serde_json::json!({ "username": "user", "password": "pass" }),
))
.respond_with(
ResponseTemplate::new(200).set_body_json(serde_json::json!({ "token": token })),
)
@@ -261,7 +267,12 @@ async fn mount_download_link(server: &MockServer, token: &'static str, file_id:
Mock::given(method("POST"))
.and(path("/api/v1/download"))
.and(header("Authorization", format!("Bearer {token}")))
.and(body_partial_json(serde_json::json!({ "file_id": file_id })))
// `sub_format` is what makes the answer SRT (#272). A search entry
// often carries no format at all, so without it the download was
// labelled `Other("")` and no parser matched.
.and(body_partial_json(
serde_json::json!({ "file_id": file_id, "sub_format": "srt" }),
))
.respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({
"link": format!("{}/file/{file_id}.srt", server.uri()),
"file_name": "subtitle.srt",
+5 -5
View File
@@ -9,7 +9,7 @@
</head>
<body>
<!--
THESIS: arr is a signal chain — TMDB and Prowlarr feed in, Transmission is
THESIS: arr is a signal chain — TMDB and Prowlarr feed in, qBittorrent is
driven out — and health is lamps on that chain, read in one glance. Refuses
the admin-dashboard rut: no sidebar, no stat cards, no hero metric.
OWN-WORLD: broadcast master-control panel. Near-black charcoal ground,
@@ -20,7 +20,7 @@
second whether arr can do its job — and which upstream is down, by name.
FIRST VIEWPORT: a top rail (wordmark + master lamp + version),
then the chain: TMDB and PROWLARR modules on the left feeding the central
arr module, TRANSMISSION driven on the right, joined by bus hairlines with
arr module, QBITTORRENT driven on the right, joined by bus hairlines with
direction arrows. On a phone the chain stacks vertically.
FORM: broadcast master-control panel, #2 of 7 on the ordered list; external
roll (shell RANDOM → 2, witness 57910; concept-seed.mjs degraded to empty
@@ -453,14 +453,14 @@
</svg>
</div>
<article class="module area-transmission" data-check="transmission">
<article class="module area-qbittorrent" data-check="qbit">
<header class="module-head">
<span class="lamp" data-state="probing" aria-hidden="true"></span>
<h2 class="module-name">transmission</h2>
<h2 class="module-name">qbittorrent</h2>
<span class="module-status readout" data-role="status">probing</span>
</header>
<p class="module-role">torrents out</p>
<p class="module-detail readout" data-role="detail">rpc · grab and seed</p>
<p class="module-detail readout" data-role="detail">webui · grab and seed</p>
</article>
</section>
+1 -1
View File
@@ -17,7 +17,7 @@ export interface Download {
}
/** §9.8: the live snapshot, degrading to nothing rather than breaking the
* page it decorates Transmission or the daemon being down is a fact the
* page it decorates qBittorrent or the daemon being down is a fact the
* signal chain already reports, not something this poll repeats. */
export async function fetchDownloads(): Promise<Download[]> {
try {
+1 -1
View File
@@ -28,7 +28,7 @@ export interface HealthReport {
status: "ok" | "degraded";
version: string;
prowlarr: Check;
transmission: Check;
qbit: Check;
tmdb: Check;
subtitles: SubtitleHealth;
}
+4 -4
View File
@@ -271,7 +271,7 @@ function main() {
const checks = {
tmdb: moduleRefs("tmdb"),
prowlarr: moduleRefs("prowlarr"),
transmission: moduleRefs("transmission"),
qbit: moduleRefs("qbit"),
} as const;
const traces = {
@@ -293,7 +293,7 @@ function main() {
}
// stagger order for the one power-up animation, upstream to downstream
const strikeOrder = [checks.tmdb, checks.prowlarr, master, checks.transmission];
const strikeOrder = [checks.tmdb, checks.prowlarr, master, checks.qbit];
strikeOrder.forEach((refs, index) => {
refs.lamp.style.setProperty("--strike", String(index));
});
@@ -331,7 +331,7 @@ function main() {
setModule(master, degraded ? "degraded" : "ok", degraded ? "warn" : "ok", report.status);
setTrace("master", degraded ? "warn" : "ok");
for (const name of ["tmdb", "prowlarr", "transmission"] as const) {
for (const name of ["tmdb", "prowlarr", "qbit"] as const) {
const check = report[name];
setModule(
checks[name],
@@ -340,7 +340,7 @@ function main() {
check.status,
check.detail,
);
if (name !== "transmission") {
if (name !== "qbit") {
setTrace(name, TONE_BY_STATUS[check.status]);
}
}
+1 -1
View File
@@ -129,7 +129,7 @@ export interface SeasonPackState {
* the release is blacklisted and every episode it covered reopens as a gap.
* Nothing on screen joined those facts, so the season read `0/10` as though
* no grab had ever been tried and the operator found out by opening
* Transmission.
* qBittorrent.
*/
export interface ImportFailure {
release: string;
+1 -1
View File
@@ -308,7 +308,7 @@ body {
grid-area: out;
}
.area-transmission {
.area-qbittorrent {
grid-area: sink;
}