Files
arr/DESIGN.md
T
2026-08-23 16:23:28 +01:00

27 KiB

arr — design

One service that replaces Radarr and Sonarr, and later Bazarr, for a single household. Rust workspace, API-first, SQLite, no authentication layer.

This document is the contract. Issues reference it by section rather than restating it.


1. Why

Five .NET services idle at 600-900 MB to do a job with one loop in it: decide what you want, find it, fetch it, put it somewhere Jellyfin can read. Each has a configuration surface an order of magnitude larger than the subset actually used, and none of them share a model, so the same show is configured three times in three dialects.

The replacement targets 20-40 MB resident, one binary, one database, one policy language.

This is not a general-purpose Radarr replacement. Every decision below is allowed to be narrow.

2. Non-goals

Explicitly out of scope, permanently unless stated:

  • Authentication. The perimeter is a VPN plus Authelia at the proxy. The service binds without auth, same trust model as the existing Transmission RPC.
  • Library migration or filesystem scan. The service knows only what it put on disk. Adopting the pre-existing library, if ever wanted, is a one-off script against both APIs, not a feature.
  • Automatic quality upgrades. See §5.4.
  • Multiple quality profiles. Policy attaches to a root folder. See §5.1.
  • Trakt. Jellyfin and Stremio already write watch state there and nothing in this pipeline reads it.
  • Calendar view, actor/director following, per-indexer UIs. Prowlarr owns indexer configuration and keeps doing so.
  • Books and music. shelfmark and calibre-web-automated already cover books. If they ever land here they reuse maybe three of the ten crates, so nothing is generalised in advance. See §11.

3. System context

                    ┌──────────────┐
        TMDB ──────▶│              │
                    │     arr      │────▶ Transmission RPC   10.6.10.45:9091
    Prowlarr ──────▶│              │
   (Torznab)        │   (this)     │────▶ ffprobe            local subprocess
                    │              │
   Jellyseerr ─────▶│              │────▶ Jellyfin refresh   10.6.10.18:8096
   (Radarr-compat)  └──────┬───────┘
                           │              ntfy                per-person topics
                           ▼
                    SQLite + media tree on /mnt/media

Everything already exists except arr. Prowlarr keeps owning tracker auth, Cloudflare bypass via FlareSolverr, rate limiting and the Cardigann definitions — replacing it buys nothing.

Transmission runs natively in its own LXC (VMID 130, 10.6.10.45), RPC unauthenticated, download dir /mnt/media/transmission/complete.

4. Domain model

Movies and series are separate aggregates. They share machinery, not a table.

Root
  id, kind (movie|tv), audience (main|kids), path, policy_id

Policy
  id, name
  required_audio          rule set, see §5.2
  dub_blacklist           [pt-BR]
  hdr_rules               see §5.3
  size_bands              per resolution: floor, target, penalty curve
  resolution_pref         [2160p, 1080p]
  source_weights          small tiebreaker

Movie
  id, tmdb_id, title, year, original_language, root_id
  wanted (bool), overrides (json), state, blocked (bool)
  search_attempts, last_searched_at

Series
  id, tmdb_id, title, year, original_language, root_id
  auto_track (bool), overrides (json), blocked (bool)

Season
  id, series_id, number, tracked (bool)

Episode
  id, season_id, number, title, air_date
  wanted (bool), state
  search_attempts, last_searched_at

MediaFile
  id, owner_kind (movie|episode), owner_id, path, size
  probed (json: resolution, source, hdr, audio_tracks, sub_tracks)
  waiver (json|null)   which rule was relaxed to allow this import

Release
  id, indexer_id, guid, name, size, seeders, publish_date, download_url
  parsed (json)        what the name claims
  score, verdict       eligible | waived | rejected(rule)

Grab
  id, release_id, target_kind, target_id, infohash
  state, grabbed_at, imported_at

Blacklist
  infohash, normalised_name, reason, created_at

Owner
  id, name, ntfy_topic

TitleOwner
  title_kind, title_id, owner_id      many-to-many

4.1 Intent lives at the leaf

Sonarr's monitored conflates "do I want this show" with "is anything happening right now". It is user-maintained, so it drifts and stops meaning anything.

Here, intent is Episode.wanted and Movie.wanted only. Series.auto_track is not intent — it is a rule one level up: when metadata reveals a new season, that season becomes tracked. An untracked series where you manually marked S02 needs no special case: three wanted episodes, nothing else.

Season.tracked is also a rule, not intent. Turning tracking on marks every already-revealed episode in the season wanted, and keeps marking episodes wanted as metadata reveals them. Turning it off withdraws nothing: leaf intent is never removed implicitly, so anything already marked wanted stays wanted until cleared explicitly.

Both rules skip season 0. Specials are dozens of undated shorts and recaps that indexers do not carry, so auto_track never marks season 0 tracked. See §4.2 for how season 0 stays out of derived status.

4.2 Status is derived, never stored as intent

Recomputed on metadata refresh and on file change:

Status Condition
airing an episode aired or airs within ±14 days
incomplete wanted episodes missing, nothing currently airing
waiting season finished, next unannounced or future-dated
complete everything wanted is on disk
ended series finished upstream and complete

Default list view shows airing and incomplete. Everything else collapses behind one toggle. A one-off season grab therefore disappears from the default view by itself once satisfied, and a tracked show reappears by itself when a new season is announced. Nothing to remember to flip.

Season 0 is invisible to all of it: derived status ignores season 0 episodes entirely, so a manually wanted special cannot pin a series at incomplete or hold back ended. The accepted consequence is that specials are visible and grabbable in the series detail view, but never affect status.

4.3 People are tags, not paths

Owner is a many-to-many tag on titles. It drives UI filtering, the default list, and notification routing. It never appears in a path. A file lives in one place; audiences overlap.

5. Policy engine

The core of the app and the part worth testing hardest. Lives in arr-core, pure, no IO.

5.1 Policy attaches to a root

Two roots per media kind (main, kids), each with one policy. Jellyseerr's "pick a quality profile per request" model is satisfied by exposing exactly one fake profile per root and ignoring whatever it sends.

Per-title overrides relax the root policy for one title. Same mechanism in both directions — only_4k tightens, allow_english_audio loosens.

5.2 Language

Requires a concept the release name does not carry: the original language of the title, from TMDB, distinct from a track's language.

reject audio track T if  T.lang ∈ dub_blacklist  and  T.lang ≠ title.original_language

dub_blacklist = [pt-BR] globally. That single expression covers both roots:

  • main — require a track matching original_language. Others may ride along. A Brazilian film has original_language = pt-BR, so its own soundtrack passes; a pt-BR dub of an English film does not.
  • kids — require pt-PT, or original_language when that is already Portuguese. pt-BR never satisfies the requirement, because the child does not read and a Brazilian dub is not what he should be watching.
  • subtitles — no blacklist. pt-BR subtitles are always fine.

The pt-PT / pt-BR detection problem. ISO-639-2 has one code, por, for both. ffprobe reports por either way. Three signals in order:

  1. Release name markers — Brazilian releases self-identify loudly: PT-BR, Dublado, Nacional, Dual Áudio.
  2. ffprobe stream title and handler_name strings, not the language code — often literally Portuguese (Brazil).
  3. Container-level BCP-47 tags where present (Matroska can carry pt-BR).

When none resolve it, tag the track por-unverified and surface it. Do not guess. Guessing toward kids gives a child a Brazilian dub; guessing toward main rejects a Brazilian film's own audio.

Sourcing consequence for kids. European Portuguese dubs live almost entirely in streaming WEB-DLs (Disney+, Netflix, Max) and essentially never in BluRay encodes or remuxes. That root biases hard toward WEB-DL sources, and unlike audio in general it is often pre-grab visible, since multi-audio releases advertise themselves (MULTi, DUAL, explicit language lists).

When nothing qualifies, the title sits in a visible no-PT-source queue rather than being grabbed in English. That queue is acted on manually, sometimes for months. A per-title allow_english_audio override empties one item from it with one click, and the resulting file is recorded with a waiver so the UI shows it as "English, no dub" rather than a clean match.

5.3 HDR and Dolby Vision

The naive rule "blacklist Dolby Vision" is wrong and would discard a large share of good 4K releases.

Profile Base layer Behaviour without DV support Verdict
5 none (ICtCp) green/purple, unwatchable reject
7 dual-layer BL+EL depends on player, unreliable reject
8.1 HDR10 plays as HDR10 accept

The existing library is already almost entirely Profile 8.1 with no Profile 5, and Jellyfin tonemaps it correctly on the current GPU path.

Profile is knowable only from ffprobe, never from a release name. A DV chip in search results means "the name claims DV" and its absence means nothing.

5.4 Quality, and why there is no upgrade loop

Default: take 4K if it exists now, otherwise 1080p. Per-title only_4k rejects 1080p outright and keeps searching indefinitely — the escape hatch for the handful of titles worth waiting on.

No grab delay. Deliberately rejected. Radarr's delay profile exists to stop first-match-wins from always yielding 1080p; here the only_4k flag solves the same problem upfront and without making every grab slower.

No automatic upgrade loop. An automatic upgrade re-downloads 40-80 GB and restarts a seeding obligation for something already watched. needs_upgrade is a marker in the UI and a manual "search again" button, not a background job.

5.5 Scoring: distance from a target size

Source tier is not the dominant term. A remux should win only when nothing smaller exists.

Per resolution, a floor, a target and a growing penalty above target. For 4K: floor ~8 GB, target ~22 GB. A 20 GB WEB-DL sits at target and wins; a 60 GB remux scores badly but stays eligible, so it is picked when it is the only option. The floor matters — unbounded "smaller is better" selects a 3 GB 4K encode that looks like mud.

Source tier (Remux > BluRay > WEB-DL > WEBRip > HDTV) survives as a small tiebreaker. Seeders are log-scaled and small: enough to complete, past that it does not matter. Telesync, CAM and screener are hard filters, not low scores.

How resolutions rank against each other. Size is scored against the band for the release's own resolution, so on its own it says nothing across resolutions: an at-target 4K and an at-target 1080p both score the top of the size term, and a 4K a couple of gigabytes over target loses to a 1080p that is merely on target. That is wrong — resolution_pref is an ordered list, and the order is a preference, not just an eligibility filter.

So each step up resolution_pref is worth a fixed number of points. The last entry is worth nothing and every earlier one a step more. A resolution the list does not carry scores nothing rather than being penalised, the same as an unclaimed resolution: no ranking is no opinion.

The step is sized against the size term, not chosen in isolation. With the seeded 4K band — target 22 GB, 60 points per gigabyte over — a step of 300 is five gigabytes of overshoot: a 4K up to about 27 GB beats an at-target 1080p, and a bloated 40 GB 4K does not. Below target the same arithmetic asks a 4K to be within about four gigabytes of its target to win, which is what keeps an 8 GB 4K that looks like mud from beating a good 1080p.

Exact numbers are policy rows, tuned by hand. The model is the decision.

5.6 Two phases of truth

Release names lie or omit. Files do not.

  • Pre-grab — parse the name. It is all there is. Use it for hard filters (source type, resolution, explicit language markers) and for scoring.
  • Post-downloadffprobe before import. Real audio languages, real HDR format and DV profile, real codec, real duration.

Store both, parsed on the release and probed on the file. The gap between them is also data: it tells you which release groups mislabel.

5.7 Hard fail versus soft fail

A policy violation found by ffprobe is not one thing.

  • Hard fail — useless. No original-language audio track at all, DV Profile 5, wrong title, corrupt. Do not import, blacklist the release, grab the next candidate.
  • Soft fail — watchable but not what was asked. 1080p while only_4k was set, PT audio missing on a kids title. Import it, record a waiver, surface it.

Neither deletes the torrent. See §7.3.

6. Sourcing

6.1 Prowlarr, per-indexer Torznab

GET /api/v1/indexer enumerates. Each indexer is then addressed at its own Torznab endpoint:

GET /{indexerId}/api?apikey=…&t=caps
GET /{indexerId}/api?apikey=…&t=search&q=…
GET /{indexerId}/api?apikey=…&t=movie&imdbid=…
GET /{indexerId}/api?apikey=…&t=tvsearch&tvdbid=…&season=&ep=

Prowlarr's aggregate /api/v1/search is deliberately not used: Torznab is the stable standard rather than an internal UI API, it gives real RSS semantics, and per-indexer addressing keeps indexer identity in hand — which the per-tracker seeding rules need anyway.

t=caps per indexer records which search modes each tracker supports. Not all do ID-based search; some are text-only.

6.2 Three triggers, three cost profiles

Trigger Cost Cadence
RSS sync one call per indexer, independent of library size ~10 min, never backs off
Targeted search one call per wanted item per attempt exponential backoff
Manual one call, user-initiated on demand

RSS is an empty-query Torznab call whose results are matched against the wanted list locally. Because its cost does not scale with the wanted list, every wanted item is matched against every RSS result forever — with one guard: the RSS lane skips a season pack for a season that already has episodes on disk, the same guard §14 applies to re-grabs. Both lanes agree, so a season does not behave differently depending on which lane sees a release first. A pack for a season with nothing on disk stays eligible. When a single episode cannot be found on its own, the escape hatch is the season release deck, not a lane exception.

Targeted search backs off 1h → 6h → 1d → 3d, capped at 7d, reset when the title's metadata changes. It never gives up entirely, it goes quiet.

Do not search before the release exists. TMDB carries release dates; a movie with no digital release date gets zero targeted searches. This is the single largest source of wasted queries in Radarr and it is free to avoid.

6.3 Blacklist

Keyed on infohash and normalised release name. Anything that hard-failed post-ffprobe is never grabbed again, including by RSS.

A manual blocked flag stops targeted search for a title entirely while leaving RSS matching on.

7. Download and import

7.1 Transmission

RPC at 10.6.10.45:9091, no credentials. Labels are movies-main, tv-kids and so on — enough to find things in Transmission's own UI, and distinct from Radarr's existing radarr/sonarr labels so both stacks can run side by side.

Verified: Radarr's only media bind is /mnt/media-v2:/mnt/media, one mount, one dataset, and the existing arrs hardlink-import today. So link() succeeds.

Implementation still falls back to copy on EXDEV rather than trusting that forever — no configuration flag, no way to get it wrong.

Hardlinking gives the library file and the seeding file independent names at zero extra disk. Copy would double storage for the entire seeding window, which at 4K is 40-80 GB per title.

7.3 Two lifecycles, deliberately decoupled

The torrent and the library entry are separate state machines.

Seeding obligation is per tracker, configured locally because Prowlarr does not expose tracker rules — ratio and min_seed_time. Set seedRatioLimit and seedIdleLimit on the torrent at add time and let Transmission enforce them. A reaper deletes torrents Transmission reports as done seeding.

Consequently a hard-failed release is blacklisted and never imported, but its torrent keeps seeding until the obligation clears. Nothing is deleted early to satisfy the library.

7.4 Layout

/mnt/media/movies/main/Dune Part Two (2024) [tmdbid-693134]/
    Dune Part Two (2024) [tmdbid-693134] - [2160p][WEB-DL][HDR10].mkv

/mnt/media/tv/kids/Bluey (2018) [tmdbid-82728]/
    Season 01/
        Bluey (2018) - S01E02 - Hospital [1080p][WEB-DL][pt-PT].mkv

Media kind first, hard audience boundary second, people nowhere.

  • Provider ID in the folder name turns Jellyfin matching from fuzzy string guessing into exact lookup.
  • Attribute tags come from ffprobe, not from the release name, so they are true. The filename doubles as an audit surface: ls shows which files are DV or which of the kids' files are still English-only, with the app stopped.
  • One folder per title even for a single file, so sidecar subtitles and artwork stay contained and deletes are atomic.
  • Release group is deliberately absent. It is not a selection criterion and it makes filenames long enough to break a terminal.

During transition, write into the existing roots so Jellyfin needs no reconfiguration and new content appears immediately. Radarr will not touch a folder it has no record of.

7.5 Jellyfin

On successful import, call Jellyfin's refresh endpoint for the affected library. Its filesystem watcher misses things. One HTTP call at the end of import.

8. The reconcile loop

There is no job queue.

The database rows are the work list. A wanted episode with no file is a pending grab. A torrent past its seeding rule is a pending delete. A downloaded file not yet probed is a pending import. Every tick, compare desired state to actual state and act on the gap.

This is idempotent and crash-safe by construction. Kill the process mid-grab and the next tick recomputes the same gap and continues. A job table would need retry counts, dead-lettering and reconciliation against Transmission anyway, because Transmission is an external system that changes underneath the app.

Transient state — a search in flight, download progress — is in memory and rebuilt from Transmission on startup.

Ticks are staggered: reconcile every 30 s, RSS every 10 min, metadata refresh daily, reaper every 5 min.

9. API and UI

9.1 API-first

The HTTP API is the product surface; the web UI is a client of it with no privileged path. OpenAPI spec generated from handler annotations, served alongside the app, and used to generate the TypeScript client.

Own schema first. Radarr-compatible endpoints are a later, separate, lower-priority crate (§9.4), not the primary shape.

One box. Two grouped result sets: in library first (title match, and for TV also episode title, so bluey hospital finds the episode), on TMDB below. Enter on a TMDB result opens the add flow with root and policy pre-filled.

The same box accepts a raw TMDB or IMDb ID, and a pasted magnet or .torrent, which skips to the manual-grab flow.

There is never a moment where the user has to know whether they are searching or adding.

9.3 Manual search results

Radarr's manual search is unusable because the raw release name is the dominant column, pushing everything that actually decides the choice off-screen.

Invert it. The policy engine has already classified every candidate:

  • Three buckets. eligible shown by default, sorted by score. waived (fails a soft rule, grabbable with one click that writes an override) and rejected collapse to a count, expandable.
  • Columns are parsed attributes as chips — score, resolution, source, HDR, audio languages, size, seeders. Fixed width, no horizontal scroll.
  • Release name is secondary, truncated, full string on expand. It is evidence for when you disagree with the parse, not the primary key.
  • Every rejected row names the rule that killed it, so over-strict filters are visible without reading names.

Pre-grab, HDR and audio chips are best-effort from the name (§5.6).

9.4 Jellyseerr compatibility

Jellyseerr stays. It already does Jellyfin user auth, discovery and request approval — none of which is the problem being solved, and rebuilding it doubles the project.

A thin arr-compat crate exposes the slice Jellyseerr actually calls:

GET  /api/v3/system/status
GET  /api/v3/rootfolder            → the real roots
GET  /api/v3/qualityprofile        → one fake profile per root
GET  /api/v3/tag
GET  /api/v3/movie                 existence check
POST /api/v3/movie                 add
GET  /api/v3/movie/lookup

plus the series equivalents. Thin mappings onto the real domain, isolated in one crate that never leaks into arr-core.

Lower priority than everything in §5-8.

9.5 Notifications

ntfy, one topic per Owner. Three events only — Radarr's failure mode is notifying on everything and being muted within a week.

  • Imported → to the title's owners. The only good-news notification.
  • Needs a decision → to the operator alone. Entered the no-PT-source queue, or hard-failed twice on different releases.
  • Broken → to the operator alone. Prowlarr, Transmission or TMDB unreachable, disk full.

Not notified: grabs, searches, downloads starting or finishing, soft fails.

10. Persistence

SQLite via sqlx, compile-time-checked queries, migrations in arr-db.

A few thousand rows, single writer. Postgres would buy nothing and cost a service.

Policy lives in the database, not a config file — size targets and DV rules get tuned by hand during testing and a restart-to-reload loop gets old immediately. Only bootstrap settings (bind address, Prowlarr URL, Transmission URL, TMDB key, media root) come from config/env.

Backup is sqlite3 .backup on a timer.

11. Crate layout

Cargo workspace, members = ["crates/*"], versions pinned once in [workspace.dependencies], following ~/tea/maestro.

arr-core      domain types, policy engine, scoring     no IO, no heavy deps
arr-parse     release name parsing                     no IO
arr-meta      TMDB client
arr-indexer   Torznab via Prowlarr
arr-dl        Transmission RPC
arr-probe     ffprobe wrapper
arr-db        sqlx + migrations
arr-api       axum + OpenAPI
arr-compat    Radarr/Sonarr v3 shim for Jellyseerr
arr-daemon    reconcile loop, wires everything
arr-e2e       cross-process integration tests
web/          Vite + TypeScript SPA, embedded via include_dir

The rule: arr-core and arr-parse hold the logic worth testing constantly and must not depend on axum, sqlx or reqwest. Everything expensive is downstream of them.

Nothing is generic over media kind. Movies are built concretely, then TV concretely, and shared machinery is extracted only once both exist. An abstraction derived from one example fits one example.

12. CI

Lints declared once in the root manifest:

[workspace.lints.rust]
unused_crate_dependencies = "warn"
missing_debug_implementations = "warn"

[workspace.lints.clippy]
pedantic = { level = "warn", priority = -1 }
unwrap_used = "warn"

Member crates carry only [lints] workspace = true.

Per-push gate, target under 5 minutes:

Step Tool
format cargo fmt --check
lint cargo clippy --all-targets -- -D warnings
unused deps cargo machete (stable; cargo udeps needs nightly and a full rebuild)
test cargo nextest run
frontend biome ci web/ and tsc -b --noEmit

Off the gate, scheduled: cargo deny for advisories and licenses. Coverage, if ever, likewise. Neither blocks a push.

Total wall-clock on the self-hosted Gitea runner is an explicit constraint. The lever is caching, not step selection: cache ~/.cargo/registry, ~/.cargo/git and target/, keyed on Cargo.lock plus rust-toolchain.toml. Never build --release in the gate.

End-to-end tests, only at the seams that actually break:

  • Prowlarr and TMDB — wiremock with recorded real responses as fixtures. Never live: trackers rate-limit, and it would leak credentials into CI.
  • Transmission — a real container as a CI service. RPC semantics are the most likely source of surprise and it starts in a second.
  • ffprobe — tiny committed clips, a few KB each. The Jellyfin LXC already has the right fixtures at /srv/jellyfin-test, including a real DV Profile 5 clip. That is the test proving the policy engine rejects Profile 5 and accepts 8.1 — the subtlest rule in the system.

E2E runs on main and on pull requests touching those crates, not every push.

13. Build order

Each phase ends at something usable end to end. No phase is a refactor of the previous one.

  1. Skeleton — workspace, CI gate, config, SQLite migrations, health endpoint, embedded empty SPA.
  2. Parse and scorearr-parse and arr-core against fixture release names. Pure, fast, heavily tested. No network.
  3. Read-only sourcing — TMDB lookup, Prowlarr enumeration and search, classified results over the API. Still grabs nothing.
  4. Movies, end to end — add a movie, grab, download, ffprobe, hardlink, rename, Jellyfin refresh, seeding reaper. The first real cutover test is one movie Radarr does not know about.
  5. UI — unified search, manual search buckets, library views, the queues.
  6. TV — seasons, episodes, auto_track, per-episode versus season-pack grabbing, derived status.
  7. Owners and notifications — tags, per-person ntfy topics, filtered views.
  8. Jellyseerr compatarr-compat.
  9. Subtitles — replaces Bazarr. Out of scope for this document.

Movies before TV because TV adds season packs, air-date calendars and per-episode state on top of an otherwise identical pipeline. Doing it second means that pipeline is already proven.

14. Open questions

  • Remux playback. Whether a 4K remux streams cleanly to the Shield is untested. The usual failure is audio, not bitrate: BluRay remuxes carry TrueHD/DTS-HD MA, and if the Shield cannot bitstream that downstream, Jellyfin transcodes audio and playback stutters while video direct-plays. Test before fixing the 4K size ceiling in §5.5. The outcome changes the conclusion from "remuxes are bad" to "remux audio needs a downmix".
  • Size band numbers in §5.5 are placeholders pending that test.
  • Season-pack re-grab. When an airing season completes, the episodes are already present individually. Nothing re-grabs the pack, and per §6.2 the RSS lane obeys the same guard — it skips packs for any season with episodes on disk. Whether a re-grab is ever wanted remains unresolved and deliberately deferred.