Files

T

dtourolle 62873cab3d feat(search): answer search from a local index; tier downloads by lifetime

Search's instant leg read only downloaded items, so with no downloads it
returned nothing and every keystroke fell through to a full Recursive=true
server query. It now reads the whole synced catalog through the same
availability CTE get_items uses, gated on the same include_catalog_browse
flag so search and browse cannot diverge. (UR-065, DR-108)

Also fixes three defects found while confirming that:

- items_fts grew by a full duplicate index every catalog pass. INSERT OR
  REPLACE fires no AFTER DELETE trigger without recursive_triggers, so the
  old index row was orphaned, and a TEXT PRIMARY KEY meant the replacement
  took a fresh rowid and inserted a second entry. Now a real upsert, with
  migration 021 rebuilding existing indexes. (DR-110)
- DELETE FROM items existed nowhere, so server-side deletions never
  propagated. Adds a post-crawl mark-and-sweep, scoped to crawled types,
  skipping downloaded items, and refusing to run after a partial crawl
  because items.parent_id cascades. (DR-110)
- The index omitted MusicArtist, Playlist and People, which search groups
  results by. Adds them plus people_fts (migration 022). (DR-111)

Re-indexing moves from a frontend startup call to a Rust background task
with a 6h TTL, so a long session no longer searches a stale catalog and a
restart no longer forces a crawl regardless of freshness. (DR-109, IR-030)

Downloads gain a lifetime tier. Eviction selected every completed row by
age with no download_source filter, so hitting the storage limit deleted
the oldest download -- typically one saved deliberately for offline -- to
make room for a precached track. It now reclaims only 'auto' rows, and
expired ones are reclaimed first, before live cache is evicted.
(DR-126, DR-127)

Downloaded video and audio-only handoffs now play from disk instead of
streaming; the video path had never consulted downloads at all. No
transcode is involved: MPV runs video=no and ExoPlayer has no surface for
an Audio item. (DR-123 in part, DR-128)

FTS queries are built as quoted phrases so apostrophes, hyphens and
slashes are data rather than operator syntax, and the item-type filter is
bound rather than interpolated.

Specs: docs/specs/catalog-index-search.md,
docs/specs/read-through-media-cache.md

Includes concurrently-developed favourites browsing and background-audio
stream-end handling; the two workstreams share offline.rs, lib.rs and
online.rs, so no subset of files builds independently.

2026-08-04 17:35:17 +02:00

11 KiB

Raw Blame History

Spec: Two-path media — selectable playback bitrate, independent whole-file download

Status: Proposed Requirements: UR-070, UR-071 → DR-121, DR-122, DR-123, DR-124, DR-125; IR-032 UX spec: player quality selector — needs a ux-flows.md section before build Related: catalog-index-search.md, downloads-as-offline-library.md

Summary

Two things that are today tangled become explicitly separate:

The playback path streams at a bitrate the viewer can change from the player. It is ephemeral and its rendition is volatile.
The download path fetches the whole file at one canonical quality, in the background, independently of whatever playback is doing.

Bytes fetched for playback are kept only when the playback rendition happens to be the same artifact the download path would produce — i.e. direct play. Otherwise playback bytes are discarded and the download path does its own fetch.

Motivation

The appealing version of this — "stream and download at once, switch when enough has arrived" — breaks the moment the viewer can change bitrate. A capture taken while the rendition changes underneath it is a splice of two encodings: not a playable file, and not something that can be honestly recorded as a download. Once bitrate is selectable, one stream cannot serve both jobs.

Separating the paths also removes the thing that made the original idea expensive: there is no mid-playback source swap to engineer, because the download never has to take over the live session. It lands on disk and is used at the next natural boundary — next episode, or next time the item is played.

What exists already and is not this: SmartCache predictively downloads other items, player_preload_upcoming warms the next one, and refresh_queue_local_sources swaps queue entries to local at boundaries. All of it concerns items you are not currently playing.

Layer assignment

Logic / responsibility	Layer	Why it belongs there
Available bitrate options for an item	Rust	Derived from Jellyfin's media sources and playback-info negotiation; changes with the API.
Mapping a chosen bitrate to transcode parameters	Rust	Domain vocabulary. `get_video_download_url` already owns the quality→params mapping; playback must reuse it, not restate it.
Deciding whether playback bytes are keepable (direct play vs transcode)	Rust	Depends on the negotiated session.
Canonical download quality	Rust	Policy over domain data.
Cache eviction, storage budget, sparse-range bookkeeping	Rust	Storage policy.
Promotion to a `downloads` row, and what invalidates a cache entry	Rust	Domain state.
Rendering the quality selector; remembering the last choice	Frontend	Presentation and a view preference. The list comes from Rust.
WiFi-only / opt-in toggles	Frontend collects, Rust enforces	The control is UI; the gate must hold even if the UI never calls.

Borderline, recorded: the default playback bitrate could look like a user preference (frontend). It goes to Rust because it must be reconcilable with what the server can actually produce for a given media source — a preference the backend has to validate is not a preference the frontend can own alone. The frontend stores the user's choice; Rust decides what that choice resolves to.

Design

DR-121 — Bitrate selection in the player

The player exposes the qualities Rust reports for the current item. Changing it re-negotiates the stream URL at the new quality and resumes at the current position. This is a deliberate, user-initiated interruption — a brief rebuffer is expected and acceptable, unlike the involuntary swap the earlier design would have needed.

Constraints that must not be broken:

On Linux, video playback must keep using the HLS master.m3u8 URL. CLAUDE.md records that returning stream.mp4 means transcoded playback never starts. A quality change re-negotiates within HLS.
The quality→transcode-parameter mapping already exists in get_video_download_url (online.rs:1702-1717). Playback must call into the same mapping. Two copies of that table will drift.
Track selection (audio/subtitle) already survives a stream re-negotiation elsewhere in the player; a quality change must preserve it too.

DR-122 — The playback path is ephemeral

Playback bytes are not persisted unless DR-124 says they are keepable. No partial capture is ever retained across a quality change: on change, any in-flight capture for that session is abandoned and its partial file deleted.

DR-123 — The download path is independent

Downloading the whole file is a separate operation through the existing download manager, at one canonical quality (default original, the direct static copy), using /Videos/{id}/stream.mp4 — progressive and Range-capable, which is what the resumable download worker relies on. It is unaffected by what playback is doing, and playback is unaffected by it.

Once complete it becomes an ordinary download row, so everything already built on top of downloads — offline browsing, refresh_queue_local_sources, the Downloads page — picks it up with no further work.

Prerequisite: downloaded video is currently never played locally. repository_get_video_stream_url goes straight to the online repo and player/[id]/+page.svelte:316 calls it with no local check — so a completed video download is still streamed. This must be fixed or the whole feature is invisible for video.

DR-124 — Keep playback bytes only when they are the download

Capture is enabled only where the played bytes and the canonical download artifact are the same thing — a direct-play session. Then:

Path	Mechanism
Android / ExoPlayer	`SimpleCache` + `CacheDataSource`, keyed by item id and media-source id so renditions never collide. LRU evictor sharing the existing smart-cache budget — not a second budget over the same disk.
Linux audio / MPV	`stream-record`, set through the existing `set_property` plumbing.
Linux video (HLS transcode)	Not captured. Segments are not a file; assembling one needs ffmpeg, which is not a dependency and which CI is forbidden from installing at job time. The download path (DR-123) covers this case instead.

Two abandonment rules, both of which must delete the partial rather than promote it:

Seek during an mpv capture. stream-record is documented as intended for linear streams; seeking breaks the recording. Straight-through listening captures, scrubbing does not.
Any quality change (DR-122).

DR-125 — Promotion, rendition, and invalidation

A capture is promoted to a downloads row (status = 'completed') only when it covers the whole resource. Partial captures stay cache and remain evictable.

A new downloads.source_rendition column records the negotiated quality/container/codec of whatever produced the bytes; NULL for rows fetched by the existing paths, which are always original. This is what makes an "upgrade to original" action possible later, and what stops a 720p capture and a 4K download being indistinguishable rows.

Invalidation. A quality change never touches a file that already exists — neither a permanent download nor a completed temporary one. Both remain valid copies of the rendition they hold, and deleting either would throw away bytes already paid for.

What a quality change does invalidate is an in-flight capture or background download of cached media: it is abandoned and restarted at the newly chosen quality, because a capture spanning a rendition change is a splice of two encodings rather than a playable file (DR-122).

So the rule is about ongoing work, not stored files. Nothing in this spec deletes user data.

Gating

Capture and background download obey the existing WiFi-only gate and storage budget, and are off unless opted in. Enforcement is in Rust.

Out of scope

Mid-playback switch onto a completing download. Two independent paths make it unnecessary; the download is used from the next boundary.
Backfilling the unplayed remainder of a capture. Watch 40 minutes and you have 40 minutes; completing it needs sparse-range bookkeeping and a resumable tail fetch. The DR-123 download path already produces a complete file, which is the reason this can wait.
Bundling ffmpeg to make transcoded video capturable. Real option, large packaging decision, its own proposal.
Routing Linux video playback through stream.mp4. Regresses a documented, hard-won fix.

Acceptance criteria

The player offers the qualities Rust reports, and changing one resumes at the same position with audio/subtitle selection preserved.
A quality change abandons any in-flight capture and leaves no partial file.
A quality change never deletes a downloads row.
A completed background download of a video is played from disk on the next play (the DR-123 prerequisite).
A direct-play session played start-to-finish leaves a complete local file with no second fetch; replaying it fetches no media bytes.
Seeking during an mpv capture abandons it; no truncated file is promoted.
A transcoded Linux video session is never captured, and never partially promoted.
Promoted rows record their rendition; existing paths still record NULL/original.
Gates hold with the setting off and with the frontend never sending it.
Eviction cannot delete bytes backing a promoted download row.
bun run check, bun run test, cargo fmt, cargo clippy, bun run test:rust, bun run check:boundary pass; bindings.ts regenerated if Rust types changed.

Testing

Rust, table-driven and pure where possible: quality→params resolution shared with the download path; keepability (direct play vs transcode vs gate off); promotion (complete → promoted, partial → not, seek-abandoned → not, quality-changed → not); invalidation (evicts cache, never a download row); rendition round-trip.

Android: instrumented — a played direct-play item yields cache entries, and a replay issues no media network request.

Frontend: the quality list renders from backend data with no item-type or codec taxonomy in src/; the selector's remembered choice is a view preference.

TRACES

Piece	Tag
Quality selector + re-negotiation	`// TRACES: UR-070 \| DR-121`
Ephemeral playback / capture abandonment	`// TRACES: UR-070 \| DR-122`
Independent whole-file download + local video playback fix	`// TRACES: UR-071 \| DR-123, IR-032`
ExoPlayer cache / mpv stream-record / keepability	`// TRACES: UR-071 \| DR-124`
Promotion, `source_rendition`, invalidation	`// TRACES: UR-071 \| DR-125`

Notes for the implementer

A parallel Claude session is active in this repo. git diff before "repairing" anything you did not write.
Do not duplicate the quality→transcode-parameter table. Call the existing one.
Reuse the smart-cache storage budget; two budgets over one disk is how devices fill up.
The downloads FK to items is relaxed (migration 005) — exercise promotion for an item that was never cached.
Build DR-123's local-playback fix first. Without it nothing in this spec is observable for video.

11 KiB Raw Blame History