Import pipeline: probe, hardlink, rename, layout #82

2026-08-22T22:56:00+01:00

naps62-yolo commented

2026-08-22 22:56:00 +01:00

Implements #23 (DESIGN.md §7.2–§7.4).

A downloaded grab is the gap: the daemon probes the torrent's files, picks the feature, re-evaluates the policy with real evidence, and hardlinks it into the library — copy only on EXDEV, staged through a rename so no partial file is ever visible. Attribute tags come from ffprobe; the source tag is the release name's claim, the one thing a file cannot know (§5.6). Hard fails blacklist the release, fail the grab and reopen the gap; the torrent is never moved or deleted (§7.3). Grab selection now skips blacklisted names (§6.3) so the next candidate wins.

New: arr_core::layout (naming, pure), arr_parse::normalise (blacklist key), TransmissionClient::torrent_content (file list), ImportAction on the reconcile lane.

Acceptance test produces the exact §7.4 path and asserts the seeding file keeps a link count of two; a DV Profile 5 probe hard-fails end to end. just ci green.

🤖 Generated with Claude Code

Implements #23 (DESIGN.md §7.2–§7.4). A `downloaded` grab is the gap: the daemon probes the torrent's files, picks the feature, re-evaluates the policy with real evidence, and hardlinks it into the library — copy only on `EXDEV`, staged through a rename so no partial file is ever visible. Attribute tags come from `ffprobe`; the source tag is the release name's claim, the one thing a file cannot know (§5.6). Hard fails blacklist the release, fail the grab and reopen the gap; the torrent is never moved or deleted (§7.3). Grab selection now skips blacklisted names (§6.3) so the next candidate wins. New: `arr_core::layout` (naming, pure), `arr_parse::normalise` (blacklist key), `TransmissionClient::torrent_content` (file list), `ImportAction` on the reconcile lane. Acceptance test produces the exact §7.4 path and asserts the seeding file keeps a link count of two; a DV Profile 5 probe hard-fails end to end. `just ci` green. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

naps62-yolo added 1 commit 2026-08-22 22:56:00 +01:00

feat(daemon): import pipeline for downloaded grabs

ci / web (pull_request) Successful in 32s

Details

ci / rust (pull_request) Successful in 2m1s

Details

e2e / e2e (pull_request) Successful in 2m29s

Details

ab2c927ea6

Probe with ffprobe, judge against the policy a second time with real
evidence (DESIGN.md 5.6-5.7), then hardlink into the 7.4 layout,
falling back to copy on EXDEV only. Hard fails blacklist the release
and reopen the gap; the torrent is never touched (7.3). Grab selection
now skips blacklisted names (6.3).

Closes #23

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

naps62-yolo reviewed

2026-08-22 23:00:05 +01:00

crates/arr-daemon/src/import.rs Outdated

						
				@@ -0,0 +139,4 @@

				        let paths: Vec<PathBuf> = content

				            .files

				            .iter()

				            .map(|file| content.download_dir.join(&file.path))

naps62-yolo commented

2026-08-22 23:00:05 +01:00

file.path comes from Transmission and can be absolute or contain ..; Path::join then probes and hardlinks outside download_dir. Reject rooted and parent components, and verify the resolved path stays below the download root before selecting a feature.

`file.path` comes from Transmission and can be absolute or contain `..`; `Path::join` then probes and hardlinks outside `download_dir`. Reject rooted and parent components, and verify the resolved path stays below the download root before selecting a feature.

naps62-yolo marked this conversation as resolved

crates/arr-daemon/src/import.rs Outdated

						
				@@ -0,0 +144,4 @@

				        // No expected runtime yet: the movies table carries no TMDB runtime,

				        // so feature selection is by size alone (largest readable video).

				        let selection = match self.prober.select_feature(paths, None).await {

naps62-yolo commented

2026-08-22 23:00:05 +01:00

select_feature probes every torrent file serially with a 60-second per-file timeout, but the Reconcile lane cancels its whole action after 25 seconds. A slow or multi-file torrent restarts from the first path every tick and can never import; give imports a compatible budget or persist/bound the work.

`select_feature` probes every torrent file serially with a 60-second per-file timeout, but the Reconcile lane cancels its whole action after 25 seconds. A slow or multi-file torrent restarts from the first path every tick and can never import; give imports a compatible budget or persist/bound the work.

naps62-yolo marked this conversation as resolved

naps62-yolo added 1 commit 2026-08-22 23:04:32 +01:00

fix(daemon): harden import against review findings

ci / rust (pull_request) Successful in 1m23s

Details

ci / web (pull_request) Successful in 39s

Details

e2e / e2e (pull_request) Successful in 1m11s

Details

5856dbc56c

Reject torrent file paths that escape the download root (absolute or
with parent components) before probing or linking. Cache probe results
across reconcile ticks: the lane's 25 s budget cancels the action while
one probe may take 60 s, so a multi-file torrent needs its settled
probes to survive into the next tick.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

naps62-yolo reviewed

2026-08-22 23:04:46 +01:00

crates/arr-daemon/src/import.rs

naps62-yolo commented

2026-08-22 23:04:46 +01:00

Fixed in 5856dbc: torrent-declared paths now go through safe_join, which refuses absolute paths, drive prefixes and .. components before anything is probed or linked. A torrent whose every entry escapes the root hard-fails as "no readable video file". Covered by hostile_torrent_paths_never_leave_the_download_root, a_torrent_of_only_hostile_paths_hard_fails and safe_join_refuses_escapes_and_keeps_normal_paths.

Fixed in 5856dbc: torrent-declared paths now go through `safe_join`, which refuses absolute paths, drive prefixes and `..` components before anything is probed or linked. A torrent whose every entry escapes the root hard-fails as "no readable video file". Covered by `hostile_torrent_paths_never_leave_the_download_root`, `a_torrent_of_only_hostile_paths_hard_fails` and `safe_join_refuses_escapes_and_keeps_normal_paths`.

naps62-yolo commented

2026-08-22 23:04:46 +01:00

Fixed in 5856dbc: probe results are now cached per path across ticks (ImportAction.probed), so a cancelled tick resumes where it stopped instead of restarting from the first file — each tick makes forward progress as long as a single probe fits the 25 s budget. Entries are dropped once the grab settles. Covered by probe_results_are_reused_across_calls. The remaining single-probe-over-25s case is a local-disk ffprobe, which stays well under it.

Fixed in 5856dbc: probe results are now cached per path across ticks (`ImportAction.probed`), so a cancelled tick resumes where it stopped instead of restarting from the first file — each tick makes forward progress as long as a single probe fits the 25 s budget. Entries are dropped once the grab settles. Covered by `probe_results_are_reused_across_calls`. The remaining single-probe-over-25s case is a local-disk ffprobe, which stays well under it.

naps62-yolo reviewed

2026-08-22 23:05:57 +01:00

crates/arr-daemon/src/import.rs

						
				@@ -0,0 +120,4 @@

				            let outcome = if let Some(outcome) = cached {

				                outcome

				            } else {

				                let outcome = match self.prober.probe(path.clone()).await {

naps62-yolo commented

2026-08-22 23:05:57 +01:00

This cache is populated only after probe() completes. The reconcile timeout cancels a 25–60 second single-file probe before that insertion, so it still restarts forever; use a compatible action/probe budget or retain work that survives cancellation.

This cache is populated only after `probe()` completes. The reconcile timeout cancels a 25–60 second single-file probe before that insertion, so it still restarts forever; use a compatible action/probe budget or retain work that survives cancellation.

naps62-yolo marked this conversation as resolved

naps62-yolo added 1 commit 2026-08-22 23:08:58 +01:00

fix(daemon): probes survive a cancelled reconcile tick

ci / rust (pull_request) Successful in 1m35s

Details

ci / web (pull_request) Successful in 30s

Details

e2e / e2e (pull_request) Successful in 1m32s

Details

c6f94aab74

The cache was written only after probe() returned, so a single probe
longer than the lane's 25 s budget was cancelled before depositing and
restarted forever. Each probe now runs as a detached task that writes
the cache itself; a cancelled tick loses nothing and the next one reads
the result.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

naps62-yolo reviewed

2026-08-22 23:09:08 +01:00

crates/arr-daemon/src/import.rs

naps62-yolo commented

2026-08-22 23:09:08 +01:00

Fixed in c6f94aa: each probe now runs as a tokio::spawned detached task that writes the cache itself before returning. A tick cancelled by the lane's 25 s budget drops only the await — the probe keeps running (its own 60 s limit intact, kill_on_drop no longer triggered since the task owns the child) and deposits the result for the next tick. Covered by a_cancelled_probe_still_deposits_its_result, which cancels probe_all mid-probe and asserts the next call reuses the deposited result with exactly one ffprobe invocation.

Fixed in c6f94aa: each probe now runs as a `tokio::spawn`ed detached task that writes the cache itself before returning. A tick cancelled by the lane's 25 s budget drops only the await — the probe keeps running (its own 60 s limit intact, `kill_on_drop` no longer triggered since the task owns the child) and deposits the result for the next tick. Covered by `a_cancelled_probe_still_deposits_its_result`, which cancels `probe_all` mid-probe and asserts the next call reuses the deposited result with exactly one ffprobe invocation.

naps62-yolo merged commit ffb8485939 into main

2026-08-22 23:21:31 +01:00

naps62-yolo deleted branch issue/23-import-pipeline

2026-08-22 23:21:31 +01:00

naps62-yolo referenced this issue from a commit

2026-08-22 23:23:58 +01:00

fix(daemon): use canonical movie states in import

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: yolo/arr#82