Fetch subtitles from OpenSubtitles.com #187

Closed
opened 2026-08-24 21:28:32 +01:00 by naps62-yolo · 2 comments
Owner

Implement the OpenSubtitles.com provider against the Provider trait from
#184. It is the primary provider: it has an official REST API, it distinguishes
pt-PT from pt-BR properly, and it supports moviehash matching, which is the
signal that wins ranking outright (DESIGN.md §15).

  • Search by moviehash computed from the media file, and by TMDB id with the
    season and episode numbers for TV, so both lanes feed the ranker.
  • Return candidates carrying everything #185 ranks on: hash match, release name,
    group, source, uploader rating, download count, forced flag, SDH flag.
  • Download a chosen candidate to a temporary path; writing it into the library
    is #195's job, not this crate's.
  • Credentials come from config or environment per DESIGN.md §10, never from the
    database.
  • The API caps downloads per day and answers 429 when the cap is hit. Surface
    that as a distinct error the loop can recognise, rather than a generic
    transport failure — #197 turns it into a visible queue state.

Never point tests at the live service. Fixture responses, as everywhere else in
this repo.

Depends on: #184

Implement the OpenSubtitles.com provider against the `Provider` trait from #184. It is the primary provider: it has an official REST API, it distinguishes pt-PT from pt-BR properly, and it supports `moviehash` matching, which is the signal that wins ranking outright (DESIGN.md §15). - Search by `moviehash` computed from the media file, and by TMDB id with the season and episode numbers for TV, so both lanes feed the ranker. - Return candidates carrying everything #185 ranks on: hash match, release name, group, source, uploader rating, download count, forced flag, SDH flag. - Download a chosen candidate to a temporary path; writing it into the library is #195's job, not this crate's. - Credentials come from config or environment per DESIGN.md §10, never from the database. - The API caps downloads per day and answers `429` when the cap is hit. Surface that as a distinct error the loop can recognise, rather than a generic transport failure — #197 turns it into a visible queue state. Never point tests at the live service. Fixture responses, as everywhere else in this repo. Depends on: #184
naps62-yolo added this to the Subtitles milestone 2026-08-24 21:28:32 +01:00
naps62-yolo added the area/subsphase/9-subtitlestype/featuredifficulty/moderate labels 2026-08-24 21:28:32 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:27 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:27 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:28 +01:00
Author
Owner

Integration note from the milestone driver, before you start.

This issue says "download a chosen candidate to a temporary path". The Provider trait merged in #184 does not do that — download returns Fetched { id, language, format, content: Vec<u8> }, the bytes in memory. That was a deliberate call and DESIGN.md §15 backs it: alass runs over the subtitle (#194) before it gets a name and a place in the title folder (#195), so a provider that wrote temp files would also own cleaning them up.

Implement against the merged trait. The issue wording is the stale half.

Also: rank the candidates you return via arr_core::subs::rank, and use Candidate::to_core for the conversion rather than building SubtitleCandidate yourself — that mapping is decided in one place now (cdd6133).

Integration note from the milestone driver, before you start. This issue says "download a chosen candidate to a temporary path". The `Provider` trait merged in #184 does not do that — `download` returns `Fetched { id, language, format, content: Vec<u8> }`, the bytes in memory. That was a deliberate call and DESIGN.md §15 backs it: `alass` runs over the subtitle (#194) before it gets a name and a place in the title folder (#195), so a provider that wrote temp files would also own cleaning them up. Implement against the merged trait. The issue wording is the stale half. Also: rank the candidates you return via `arr_core::subs::rank`, and use `Candidate::to_core` for the conversion rather than building `SubtitleCandidate` yourself — that mapping is decided in one place now (cdd6133).
Author
Owner

Implemented the OpenSubtitles.com provider in crates/arr-subs/src/opensubtitles.rs against the Provider trait from #184. Search runs both lanes: moviehash computed from the media file's own bytes (64 KiB end chunks per the OpenSubtitles algorithm, dropped for files too small to have two distinct ends) and TMDB id with season/episode for TV. Candidates carry hash match, release name, group and source (read off the release name with arr_parse::parse, so ranking shares one parser), uploader rating, download count, forced (foreign_parts_only) and SDH (hearing_impaired) — and are returned ordered best-first through arr_core::subs::rank via Candidate::to_core. Download logs in on demand (/login basic auth, token cached, one refresh on mid-flight 401), posts /download for a file id, then fetches the link bytes into Fetched; download needs user credentials and answers Error::Unauthorized without them. The daily cap (429, and 406 as documented) surfaces as Error::RateLimited { retry_after } for #197. Credentials arrive as OpenSubtitlesConfig — config/env shaped, never database; Debug impls are hand-written so keys and tokens never render. Tests are wiremock fixtures plus hand-derived hash vectors; nothing touches the live service.

Branch: subtitles/187-opensubtitles (based on blitz/subtitles).

just ci: passed — 542 tests green through the gate (exit 0).

Findings

  • The issue's "download to a temporary path" wording conflicts with the merged Provider trait, which returns bytes in memory. The driver's integration note already resolves this in favour of the trait; implemented accordingly.
  • Error::Io is raised when the media file cannot be opened for hashing, but files under 128 KiB silently search without the hash lane. That matches the API's own behaviour, but #196's loop may want to surface "too small to hash" somewhere visible.
  • The provider records candidate metadata (language, format) at search time so download can fill Fetched from an opaque id. If the loop ever holds a candidate id across a restart, it must re-search before downloading or it gets NotFound.
Implemented the OpenSubtitles.com provider in `crates/arr-subs/src/opensubtitles.rs` against the `Provider` trait from #184. Search runs both lanes: `moviehash` computed from the media file's own bytes (64 KiB end chunks per the OpenSubtitles algorithm, dropped for files too small to have two distinct ends) and TMDB id with season/episode for TV. Candidates carry hash match, release name, group and source (read off the release name with `arr_parse::parse`, so ranking shares one parser), uploader rating, download count, forced (`foreign_parts_only`) and SDH (`hearing_impaired`) — and are returned ordered best-first through `arr_core::subs::rank` via `Candidate::to_core`. Download logs in on demand (`/login` basic auth, token cached, one refresh on mid-flight 401), posts `/download` for a file id, then fetches the link bytes into `Fetched`; download needs user credentials and answers `Error::Unauthorized` without them. The daily cap (429, and 406 as documented) surfaces as `Error::RateLimited { retry_after }` for #197. Credentials arrive as `OpenSubtitlesConfig` — config/env shaped, never database; Debug impls are hand-written so keys and tokens never render. Tests are wiremock fixtures plus hand-derived hash vectors; nothing touches the live service. Branch: `subtitles/187-opensubtitles` (based on `blitz/subtitles`). `just ci`: passed — 542 tests green through the gate (exit 0). ## Findings - The issue's "download to a temporary path" wording conflicts with the merged `Provider` trait, which returns bytes in memory. The driver's integration note already resolves this in favour of the trait; implemented accordingly. - `Error::Io` is raised when the media file cannot be opened for hashing, but files under 128 KiB silently search without the hash lane. That matches the API's own behaviour, but #196's loop may want to surface "too small to hash" somewhere visible. - The provider records candidate metadata (language, format) at search time so `download` can fill `Fetched` from an opaque id. If the loop ever holds a candidate id across a restart, it must re-search before downloading or it gets `NotFound`.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Reference: yolo/arr#187