Close the subtitle gap in the reconcile loop #196

Closed
opened 2026-08-24 21:28:39 +01:00 by naps62-yolo · 2 comments
Owner

Teach the reconcile loop to close subtitle gaps. This is where the milestone's
pieces become behaviour; everything it calls already exists by the time this
lands.

The loop's rule (§8) applies unchanged: the rows are the work list. For each
media file, the desired state is the wanted set from settings — Portuguese and
English, per DESIGN.md §15 — and the actual state is the subtitle rows from
#186. The gap is a language with no satisfying subtitle.

Per gap, in order:

  1. An embedded track already satisfies it (#189) — no work.
  2. Search the enabled providers (#187, #188), rank the candidates (#185), take the
    winner, sync it (#194), write it (#195).
  3. No candidate in that language: translate immediately, with no waiting
    window. The source is an existing subtitle for the file — a fetched one, or
    one extracted from a text-format embedded track. Sync and write the result
    with the .mt marker.
  4. Nothing to translate from either: record the reason on the attempt row and
    leave it for the missing-subtitles queue (#202).

Also:

  • Back off per attempt using the attempt counters from #186, the same way movie
    and episode searches already do.
  • Work newest imports first, so enabling this on an existing library serves
    today's viewing before the back catalogue.
  • Once a language is satisfied, stop. A machine translation satisfies it. There
    is no background re-search and no replacement — §5.4's rule, applied here.
  • Provider or translator unreachable folds into the existing "Broken"
    notification to the operator (§9.5). No new notification classes, and a
    subtitle fetch never notifies anyone.

Depends on: #185, #187, #188, #189, #190, #194, #195, #212, #213

Teach the reconcile loop to close subtitle gaps. This is where the milestone's pieces become behaviour; everything it calls already exists by the time this lands. The loop's rule (§8) applies unchanged: the rows are the work list. For each media file, the desired state is the wanted set from settings — Portuguese and English, per DESIGN.md §15 — and the actual state is the subtitle rows from #186. The gap is a language with no satisfying subtitle. Per gap, in order: 1. An embedded track already satisfies it (#189) — no work. 2. Search the enabled providers (#187, #188), rank the candidates (#185), take the winner, sync it (#194), write it (#195). 3. No candidate in that language: translate immediately, with no waiting window. The source is an existing subtitle for the file — a fetched one, or one extracted from a text-format embedded track. Sync and write the result with the `.mt` marker. 4. Nothing to translate from either: record the reason on the attempt row and leave it for the missing-subtitles queue (#202). Also: - Back off per attempt using the attempt counters from #186, the same way movie and episode searches already do. - Work newest imports first, so enabling this on an existing library serves today's viewing before the back catalogue. - Once a language is satisfied, stop. A machine translation satisfies it. There is no background re-search and no replacement — §5.4's rule, applied here. - Provider or translator unreachable folds into the existing "Broken" notification to the operator (§9.5). No new notification classes, and a subtitle fetch never notifies anyone. Depends on: #185, #187, #188, #189, #190, #194, #195, #212, #213
naps62-yolo added this to the Subtitles milestone 2026-08-24 21:28:39 +01:00
naps62-yolo added the area/daemondifficulty/hardphase/9-subtitlestype/feature labels 2026-08-24 21:28:39 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:27 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:27 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:27 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:27 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:27 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:28 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:28 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:28 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:28 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:28 +01:00
naps62-yolo added a new dependency 2026-08-24 22:37:11 +01:00
naps62-yolo added a new dependency 2026-08-24 22:37:27 +01:00
Author
Owner

Integration note from the milestone driver — carried forward from #187, which is now merged.

Provider candidate ids are not durable across a restart. The OpenSubtitles provider records a candidate's language and format at search time so download can fill Fetched from an opaque id. Podnapisi encodes the language into the id itself ({pid}|{language}), so the two providers differ in how much a bare id carries.

Practical consequence: if a candidate id is held anywhere that outlives the search — a queue row, a UI selection the operator comes back to, anything surviving a daemon restart — it must be re-searched before download, or OpenSubtitles answers Error::NotFound. Do not persist a candidate id and treat it as a durable handle.

Also worth knowing: Candidate.rating and Candidate.download_count are always None for Podnapisi (no documented field), so ranking's tiebreakers do nothing for that provider. That is expected, not a bug to chase.

Integration note from the milestone driver — carried forward from #187, which is now merged. **Provider candidate ids are not durable across a restart.** The OpenSubtitles provider records a candidate's language and format at search time so `download` can fill `Fetched` from an opaque id. Podnapisi encodes the language into the id itself (`{pid}|{language}`), so the two providers differ in how much a bare id carries. Practical consequence: if a candidate id is held anywhere that outlives the search — a queue row, a UI selection the operator comes back to, anything surviving a daemon restart — it must be re-searched before download, or OpenSubtitles answers `Error::NotFound`. Do not persist a candidate id and treat it as a durable handle. Also worth knowing: `Candidate.rating` and `Candidate.download_count` are always `None` for Podnapisi (no documented field), so ranking's tiebreakers do nothing for that provider. That is expected, not a bug to chase.
Author
Owner

Implemented #196 on branch subtitles/196-reconcile-subs (one commit, based off blitz/subtitles). A new SubtitleAction on the reconcile tick seeds the wanted set from settings over every probed media file, then per gap: records embedded tracks from the probe data (the half #189 explicitly left here) and satisfies from them for free; searches the enabled providers, ranks per §15 (pt-PT bucket before pt-BR, moviehash computed per file), downloads the winner, syncs with alass, writes the sidecar and records it; with a definitively empty answer it translates immediately — extracting a text-format embedded track when that is the only source — writing the .mt sidecar; with nothing to translate from it records unavailable for the queue (#202). Failures back off on the same §6.2 curve as movie searches; provider errors block the translate fallback (a candidate may exist where nothing could be asked); rate limits land as the capped queue state; unreachable providers/translators fold edge-triggered into the existing §9.5 broken notification. Closes run as detached tasks because alass (180 s) and translation outlive the 25 s reconcile action budget — every outcome is written to domain rows, so a crash converges next tick. 11 new tests, including a real-ffmpeg extraction round trip against the arr-probe fixture.

just ci: passed (gate exit 0; 664 tests green). Not merged, no PR opened.

Findings

  • No translation backend is constructed at startup. arr-daemon never builds the #191–#193 backends (its arr-subs dependency doesn't even enable the translate-* features, and OpenAI needs a model-name config knob that doesn't exist), and AppState.with_translation_backends has no production caller either. Both the manual translate endpoint and this loop's translate step therefore report "engine not compiled" in a real deployment. Needs its own wiring issue: feature forwarding in arr-daemon/Cargo.toml, backend construction from config, and passing the list to both AppState and SubtitleAction.
  • Nothing ever resets a capped attempt row. pending() excludes capped, and the daily-allowance reset belongs to #197, so until #197 lands a provider rate limit parks that (file, language) permanently rather than for a day.
  • The manual translate endpoint (arr-api::subtitles::translate) never runs alass on its output, though §15 says sync runs on every translated subtitle; the loop's translate path does sync. Worth a small fix on the API side.
  • main.rs's doc comment on subtitle_providers still says "Translation backends are not wired: none is implemented yet (#191-#193)" — stale since #193 merged; I left it for the wiring issue to rewrite.
Implemented #196 on branch `subtitles/196-reconcile-subs` (one commit, based off `blitz/subtitles`). A new `SubtitleAction` on the reconcile tick seeds the wanted set from settings over every probed media file, then per gap: records embedded tracks from the probe data (the half #189 explicitly left here) and satisfies from them for free; searches the enabled providers, ranks per §15 (pt-PT bucket before pt-BR, moviehash computed per file), downloads the winner, syncs with alass, writes the sidecar and records it; with a definitively empty answer it translates immediately — extracting a text-format embedded track when that is the only source — writing the `.mt` sidecar; with nothing to translate from it records `unavailable` for the queue (#202). Failures back off on the same §6.2 curve as movie searches; provider errors block the translate fallback (a candidate may exist where nothing could be asked); rate limits land as the `capped` queue state; unreachable providers/translators fold edge-triggered into the existing §9.5 broken notification. Closes run as detached tasks because alass (180 s) and translation outlive the 25 s reconcile action budget — every outcome is written to domain rows, so a crash converges next tick. 11 new tests, including a real-ffmpeg extraction round trip against the arr-probe fixture. `just ci`: **passed** (gate exit 0; 664 tests green). Not merged, no PR opened. ## Findings - **No translation backend is constructed at startup.** `arr-daemon` never builds the #191–#193 backends (its `arr-subs` dependency doesn't even enable the `translate-*` features, and OpenAI needs a model-name config knob that doesn't exist), and `AppState.with_translation_backends` has no production caller either. Both the manual translate endpoint and this loop's translate step therefore report "engine not compiled" in a real deployment. Needs its own wiring issue: feature forwarding in `arr-daemon/Cargo.toml`, backend construction from config, and passing the list to both `AppState` and `SubtitleAction`. - **Nothing ever resets a `capped` attempt row.** `pending()` excludes `capped`, and the daily-allowance reset belongs to #197, so until #197 lands a provider rate limit parks that (file, language) permanently rather than for a day. - The manual translate endpoint (`arr-api::subtitles::translate`) never runs alass on its output, though §15 says sync runs on every translated subtitle; the loop's translate path does sync. Worth a small fix on the API side. - `main.rs`'s doc comment on `subtitle_providers` still says "Translation backends are not wired: none is implemented yet (#191-#193)" — stale since #193 merged; I left it for the wiring issue to rewrite.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Reference: yolo/arr#196