Cover the subtitle path in arr-e2e #205

Closed
opened 2026-08-24 21:28:40 +01:00 by naps62-yolo · 1 comment
Owner

Cover the subtitle path end to end in arr-e2e, cross-process like the rest of
that crate.

At least: a media file with no subtitles gets one fetched, synced and written
with the right name; a file whose embedded English text track is the only
source gets it extracted and translated into a .mt sidecar; a file whose
subtitle sync is implausible keeps the unsynced original and shows as
sync-rejected; a language once satisfied is never re-searched.

Providers and translators are stubbed at the HTTP boundary and the
remote-command backend by a stub script — the rule that tests never touch a
live tracker (CLAUDE.md) applies to subtitle providers exactly the same way.
ffmpeg and alass are real, as tmux and git are real in this repo's
other integration tests.

Depends on: #195, #196

Cover the subtitle path end to end in `arr-e2e`, cross-process like the rest of that crate. At least: a media file with no subtitles gets one fetched, synced and written with the right name; a file whose embedded English text track is the only source gets it extracted and translated into a `.mt` sidecar; a file whose subtitle sync is implausible keeps the unsynced original and shows as sync-rejected; a language once satisfied is never re-searched. Providers and translators are stubbed at the HTTP boundary and the remote-command backend by a stub script — the rule that tests never touch a live tracker (`CLAUDE.md`) applies to subtitle providers exactly the same way. `ffmpeg` and `alass` are real, as `tmux` and `git` are real in this repo's other integration tests. Depends on: #195, #196
naps62-yolo added this to the Subtitles milestone 2026-08-24 21:28:40 +01:00
naps62-yolo added the area/infraphase/9-subtitlestype/testdifficulty/moderate labels 2026-08-24 21:28:40 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:28 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:28 +01:00
Author
Owner

Implemented three cross-process arr-e2e tests covering the subtitle path (DESIGN.md §15), against the real arr binary, real ffmpeg and real alass, with Podnapisi and the remote-command translation backend stubbed at the HTTP/process boundary:

  • a_missing_subtitle_is_fetched_synced_and_named — a media file with no subtitles gets a pt-PT candidate fetched from Podnapisi, synced by real alass, written as <name>.pt-PT.srt, and confirms the provider is never re-searched once satisfied (waits past a second reconcile tick).
  • an_embedded_track_is_extracted_and_translated — a file whose only English source is an embedded subrip track gets it extracted by real ffmpeg and translated (via a stub translate-command script) into <name>.pt-PT.mt.srt.
  • an_implausible_sync_keeps_the_original_and_is_flagged — a fetched subtitle timed five minutes off a three-second clip is rejected by real alass (300s shift), the unsynced original is kept verbatim, and it surfaces in /api/queues/subtitles as sync_rejected.

There is no HTTP route to adopt an already-imported file (grab/download/import is a separate seam), so each test seeds a movie, an imported grab and a media_files row directly against the daemon's own SQLite file — the harness now exposes Daemon::database_path() and Daemon::media_root() for this, plus Daemon::spawn_with_env for the extra child env vars the scenarios need.

Two small seams were added because there was otherwise no way to write this test at all:

  • ARR_PODNAPISI_URL, an env-only seam mirroring the existing tmdb_url one, so Podnapisi can point at a wiremock fake.
  • The arr-e2e harness now builds the daemon with --features translate-command so a stub script can stand in for a translation backend.

Branch: subtitles/205-subs-e2e. just ci passed locally (686 tests). The new arr-e2e tests are outside that gate per the repo's own convention (.gitea/workflows/e2e.yml) and were run manually against a locally installed alass-cli (symlinked as alass) — all three pass.

Findings

  • Bug, fixed as part of this issue (not adjacent scope — it blocked writing this test at all): Syncer::run in crates/arr-subs/src/sync.rs passed alass its arguments as [subtitle, video, output]. The real alass binary's signature is <reference-file> <incorrect-sub-file> <output-file-path>, where the reference must be the video. With the old order, every real alass invocation failed immediately (unknown subtitle format for file <video>), and Syncer::settle degraded this to SyncState::NotRun silently — meaning every subtitle sync in production was silently a no-op since #194 landed. Fixed the argument order and the one unit test whose fake script depended on it; all arr-subs tests still pass.
  • CI gap: .gitea/workflows/e2e.yml never installed alass (only ffmpeg), and never built the daemon with any translate-* feature, so the e2e suite could never have exercised the sync or translation paths for real. Added an install step (cargo install alass-cli, symlinked to alass) and --features translate-command to the daemon build step. Also added crates/arr-subs/** to the workflow's trigger paths — it wasn't there even though the daemon's subtitle behavior depends on it directly.
  • Not fixed, out of scope: no HTTP endpoint exists to adopt an already-imported media file for testing purposes; every subtitle-focused test (including the daemon's own in-process ones) has to insert media_files/grabs/releases rows by hand. Worth a dedicated test-support seam if more of this kind of test gets written.
Implemented three cross-process `arr-e2e` tests covering the subtitle path (DESIGN.md §15), against the real `arr` binary, real `ffmpeg` and real `alass`, with Podnapisi and the remote-command translation backend stubbed at the HTTP/process boundary: - `a_missing_subtitle_is_fetched_synced_and_named` — a media file with no subtitles gets a pt-PT candidate fetched from Podnapisi, synced by real `alass`, written as `<name>.pt-PT.srt`, and confirms the provider is never re-searched once satisfied (waits past a second reconcile tick). - `an_embedded_track_is_extracted_and_translated` — a file whose only English source is an embedded `subrip` track gets it extracted by real `ffmpeg` and translated (via a stub `translate-command` script) into `<name>.pt-PT.mt.srt`. - `an_implausible_sync_keeps_the_original_and_is_flagged` — a fetched subtitle timed five minutes off a three-second clip is rejected by real `alass` (300s shift), the unsynced original is kept verbatim, and it surfaces in `/api/queues/subtitles` as `sync_rejected`. There is no HTTP route to adopt an already-imported file (grab/download/import is a separate seam), so each test seeds a movie, an imported grab and a `media_files` row directly against the daemon's own SQLite file — the harness now exposes `Daemon::database_path()` and `Daemon::media_root()` for this, plus `Daemon::spawn_with_env` for the extra child env vars the scenarios need. Two small seams were added because there was otherwise no way to write this test at all: - `ARR_PODNAPISI_URL`, an env-only seam mirroring the existing `tmdb_url` one, so Podnapisi can point at a `wiremock` fake. - The `arr-e2e` harness now builds the daemon with `--features translate-command` so a stub script can stand in for a translation backend. Branch: `subtitles/205-subs-e2e`. `just ci` passed locally (686 tests). The new `arr-e2e` tests are outside that gate per the repo's own convention (`.gitea/workflows/e2e.yml`) and were run manually against a locally installed `alass-cli` (symlinked as `alass`) — all three pass. ## Findings - **Bug, fixed as part of this issue (not adjacent scope — it blocked writing this test at all):** `Syncer::run` in `crates/arr-subs/src/sync.rs` passed `alass` its arguments as `[subtitle, video, output]`. The real `alass` binary's signature is `<reference-file> <incorrect-sub-file> <output-file-path>`, where the reference must be the video. With the old order, every real `alass` invocation failed immediately (`unknown subtitle format for file <video>`), and `Syncer::settle` degraded this to `SyncState::NotRun` silently — meaning **every subtitle sync in production was silently a no-op** since #194 landed. Fixed the argument order and the one unit test whose fake script depended on it; all `arr-subs` tests still pass. - **CI gap:** `.gitea/workflows/e2e.yml` never installed `alass` (only `ffmpeg`), and never built the daemon with any `translate-*` feature, so the e2e suite could never have exercised the sync or translation paths for real. Added an install step (`cargo install alass-cli`, symlinked to `alass`) and `--features translate-command` to the daemon build step. Also added `crates/arr-subs/**` to the workflow's trigger paths — it wasn't there even though the daemon's subtitle behavior depends on it directly. - Not fixed, out of scope: no HTTP endpoint exists to adopt an already-imported media file for testing purposes; every subtitle-focused test (including the daemon's own in-process ones) has to insert `media_files`/`grabs`/`releases` rows by hand. Worth a dedicated test-support seam if more of this kind of test gets written.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Reference: yolo/arr#205