yolo/arr
3
0
Fork 0
Code Issues 12 Pull Requests Actions Packages Projects Releases Wiki Activity

Budget provider and translator calls with a token bucket #197

New Issue
Closed
opened 2026-08-24 21:28:39 +01:00 by naps62-yolo · 2 comments
naps62-yolo commented 2026-08-24 21:28:39 +01:00
Owner
Copy Link
Copy Source

Bound how fast arr spends other people's APIs. Enabling subtitles on an
existing library means every media file wants two languages at once, and the
OpenSubtitles free tier is a single-digit number of downloads per day.

Per DESIGN.md §15 (added by #183):

  • A token bucket per provider and per translator, refilled on a configured
    daily allowance. The allowance is a database setting (#198).
  • The reconcile loop (#196) spends the budget newest-import-first, so the
    backlog drains over days without starving current viewing.
  • Being at the cap is a state, not an error: the affected files show in the
    missing-subtitles queue (#202) as waiting on budget, and nothing notifies.
  • Translators bill per character, so their bucket counts characters, not calls,
    using the usage the backends report (#192).
  • A provider's own 429 still backs off independently — the bucket is arr
    being polite, not a guarantee about the other end.

Depends on: #196

Bound how fast arr spends other people's APIs. Enabling subtitles on an existing library means every media file wants two languages at once, and the OpenSubtitles free tier is a single-digit number of downloads per day. Per DESIGN.md §15 (added by #183): - A token bucket per provider and per translator, refilled on a configured daily allowance. The allowance is a database setting (#198). - The reconcile loop (#196) spends the budget newest-import-first, so the backlog drains over days without starving current viewing. - Being at the cap is a state, not an error: the affected files show in the missing-subtitles queue (#202) as waiting on budget, and nothing notifies. - Translators bill per character, so their bucket counts characters, not calls, using the usage the backends report (#192). - A provider's own `429` still backs off independently — the bucket is arr being polite, not a guarantee about the other end. Depends on: #196
naps62-yolo added this to the Subtitles milestone 2026-08-24 21:28:39 +01:00
naps62-yolo added the area/daemonphase/9-subtitlestype/featuredifficulty/moderate labels 2026-08-24 21:28:39 +01:00
naps62-yolo added a new dependency 2026-08-24 21:29:28 +01:00
#196 Close the subtitle gap in the reconcile loop
naps62-yolo commented 2026-08-25 01:05:59 +01:00
Author
Owner
Copy Link
Copy Source

Integration note from the milestone driver, carried forward from #192.

Backend::characters_billed() (added by #192, default zero so it is additive for the other backends) is cumulative since the backend was constructed, not per-call. #192 flagged that this issue may want a per-call number instead, since polling a counter on a dyn Backend is awkward for a token bucket that has to attribute spend to one translation.

The trait change is small if you decide per-call is the right shape — but decide it here rather than working around the counter, because this is the only issue that consumes it.

Also relevant: just ci now runs --all-features (see blitz/subtitles). Before that, no translation backend was compiled by the gate at all, so treat any pre-existing green CI on backend code as meaningless.

Integration note from the milestone driver, carried forward from #192. `Backend::characters_billed()` (added by #192, default zero so it is additive for the other backends) is **cumulative since the backend was constructed**, not per-call. #192 flagged that this issue may want a per-call number instead, since polling a counter on a `dyn Backend` is awkward for a token bucket that has to attribute spend to one translation. The trait change is small if you decide per-call is the right shape — but decide it here rather than working around the counter, because this is the only issue that consumes it. Also relevant: `just ci` now runs `--all-features` (see `blitz/subtitles`). Before that, no translation backend was compiled by the gate at all, so treat any pre-existing green CI on backend code as meaningless.
naps62-yolo commented 2026-08-25 02:40:58 +01:00
Author
Owner
Copy Link
Copy Source

Implemented DESIGN.md §15's token bucket, on branch subtitles/197-subs-budget (based on blitz/subtitles).

A new subtitle_budget_spend table (migration 0027) tracks one row per (kind, name, day). arr_db::subtitle_budget::try_spend is a single atomic upsert-with-cap query, so concurrent reconcile closes (up to 4 in flight) can't both slip a spend past the daily allowance. No allowance configured for a name reads as unlimited, per the existing subtitle_settings comment from #198.

Providers are charged one unit per download, claimed right before Provider::download is called (search itself is not budgeted — the scarce resource DESIGN.md names is downloads, e.g. OpenSubtitles' free-tier daily cap). If the ranked winner's provider is over budget, the reconcile loop drops that provider's candidates and re-ranks rather than giving up on the gap, so a cheaper/lower-ranked candidate from another provider still gets downloaded when affordable. Only when every remaining candidate is budget-blocked does the gap record capped — it deliberately does not fall through to translation, since that would permanently settle the language with a worse (machine-translated) subtitle instead of waiting for the cap to refill.

Translators are charged the source character count, computed from the parsed cues before the network call, and spent atomically before arr_subs::translate::translate runs.

Design decision on Backend::characters_billed(): the milestone driver's comment on this issue flagged that the trait's cumulative counter (added by #192) can't attribute cost to one call under concurrent use. I removed it — DeepL and Google's AtomicU64 billing counters and their trait impl are gone — and moved billing to a simple upfront character count of the source text, computed once in the daemon before spending. This is the same math DeepL/Google already used internally (sum of characters sent), just relocated so it's available before the call instead of polled after, and it now also applies uniformly to openai/command, which previously always reported zero and were never budgeted at all.

just ci passed through the gate (fmt, clippy --all-features, cargo machete, full test suite, web checks).

Findings

  • openai and command translation backends were never subject to any character budget before this issue, since characters_billed() defaulted to 0 for both and nothing else computed usage. Not a regression I introduced — just noting it as a pre-existing gap this issue happens to close as a side effect.
  • The missing-subtitles queue (#202) will want to surface budget-capped rows distinctly from provider-429-capped ones; both currently map to the same capped state with a last_failure string distinguishing them ("provider daily budget exhausted" / "translator daily budget exhausted" vs a provider's own error text). No queue UI exists yet to consume that, so nothing to fix here.
Implemented DESIGN.md §15's token bucket, on branch `subtitles/197-subs-budget` (based on `blitz/subtitles`). A new `subtitle_budget_spend` table (migration `0027`) tracks one row per (kind, name, day). `arr_db::subtitle_budget::try_spend` is a single atomic upsert-with-cap query, so concurrent reconcile closes (up to 4 in flight) can't both slip a spend past the daily allowance. No allowance configured for a name reads as unlimited, per the existing `subtitle_settings` comment from #198. Providers are charged one unit per download, claimed right before `Provider::download` is called (search itself is not budgeted — the scarce resource DESIGN.md names is downloads, e.g. OpenSubtitles' free-tier daily cap). If the ranked winner's provider is over budget, the reconcile loop drops that provider's candidates and re-ranks rather than giving up on the gap, so a cheaper/lower-ranked candidate from another provider still gets downloaded when affordable. Only when every remaining candidate is budget-blocked does the gap record `capped` — it deliberately does not fall through to translation, since that would permanently settle the language with a worse (machine-translated) subtitle instead of waiting for the cap to refill. Translators are charged the source character count, computed from the parsed cues before the network call, and spent atomically before `arr_subs::translate::translate` runs. **Design decision on `Backend::characters_billed()`:** the milestone driver's comment on this issue flagged that the trait's cumulative counter (added by #192) can't attribute cost to one call under concurrent use. I removed it — DeepL and Google's `AtomicU64` billing counters and their trait impl are gone — and moved billing to a simple upfront character count of the source text, computed once in the daemon before spending. This is the same math DeepL/Google already used internally (sum of characters sent), just relocated so it's available before the call instead of polled after, and it now also applies uniformly to `openai`/`command`, which previously always reported zero and were never budgeted at all. `just ci` passed through the gate (fmt, clippy `--all-features`, `cargo machete`, full test suite, web checks). ## Findings - `openai` and `command` translation backends were never subject to any character budget before this issue, since `characters_billed()` defaulted to 0 for both and nothing else computed usage. Not a regression I introduced — just noting it as a pre-existing gap this issue happens to close as a side effect. - The missing-subtitles queue (#202) will want to surface budget-capped rows distinctly from provider-429-capped ones; both currently map to the same `capped` state with a `last_failure` string distinguishing them ("provider daily budget exhausted" / "translator daily budget exhausted" vs a provider's own error text). No queue UI exists yet to consume that, so nothing to fix here.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area/api
arr-api
 area/ci
CI and test harness
 area/compat
arr-compat
 area/core
arr-core
 area/daemon
arr-daemon
 area/db
arr-db
 area/dl
arr-dl
 area/indexer
arr-indexer
 area/infra
workspace and tooling
 area/meta
arr-meta
 area/parse
arr-parse
 area/probe
arr-probe
 area/subs
arr-subs crate: providers, extraction, translation, sync
 area/web
web/
 difficulty/easy
bounded, obvious approach, few files
 difficulty/hard
subtle correctness or cross-cutting
 difficulty/moderate
multiple files, judgement, an interface to design
 difficulty/trivial
one file, mechanical, no design decisions
 phase/1-skeleton
workspace, CI, config, database, empty API and SPA
 phase/2-logic
parsing and policy engine, pure, no network
 phase/3-sourcing
TMDB and Prowlarr, read-only
 phase/4-movies
movies end to end
 phase/5-ui
search, buckets, library views, queues
 phase/6-tv
seasons, episodes, tracking, derived status
 phase/7-people
owner tags and notifications
 phase/8-compat
Jellyseerr shim
 phase/9-subtitles
build order: subtitles, DESIGN.md §15
 type/bug
type/chore
type/feature
type/test
No labels area/daemon difficulty/moderate phase/9-subtitles type/feature
Milestone
No items
No Milestone Subtitles
Projects
Clear projects
No projects
Assignees
Clear assignees
hermes (Hermes (agent)) naps62 (Miguel Palhas) naps62-yolo (naps62-yolo (agent))
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Depends on
#196 Close the subtitle gap in the reconcile loop
yolo/arr
Reference: yolo/arr#197