dtourolle
109700b949
Playing a video meant asking the server to re-encode it, always. That
decision was made nowhere and written down nowhere, so whoever needed it
re-derived it downstream — the player worked out whether it had been handed
a playlist by looking for ".m3u8" in the URL, in two places. A viewer paid
for a transcode of a file their device could have played untouched, and the
app could not tell them which it was.
One negotiation now produces one self-describing StreamSelection — direct
play, remux or transcode; over a playlist, a plain HTTP file, or a local one
— and every renderer consumes that same answer.
Measured against the development server (Jellyfin 10.11.5), 400 items
sampled for codec mix and 40 put through a real PlaybackInfo negotiation
per profile:
Linux / WebKitGTK (h264 only, 2ch) 3/40 — 7% direct play
Android / ExoPlayer (hevc, ac3/eac3, 6ch) 34/40 — 85% direct play
The library is ~80% hevc, which is why the two diverge so hard. The payoff
is overwhelmingly Android, where 85% of plays were starting a transcode
nobody needed. Linux stays near 7% until libmpv decodes the picture — the
h264-only profile is a WebKitGTK constraint, not a JellyTau choice.
DR-219 StreamSelection: url + tagged Transport (hls/progressive/localFile)
+ PlaybackKind (directPlay/directStream/transcode) + the negotiated
rendition + this source's ladder + a needs_transcoding flag derived
in Rust so the rule is answered once. Both enums are serde-tagged
so the frontend matches a discriminant, not a substring. The paths
that never negotiate get the same shape from Rust rather than
assembling one — media_local_selection for a downloaded file,
LiveStreamInfo.transport for a live channel — so there is no second
place where a transport is decided.
DR-220 The ceiling becomes two levels: a durable device default (Settings,
persisted) and a per-playback override the in-player picker sets.
The picker had called itself a "this film, this connection" control
since it was written but wrote the process-wide default, so dropping
one awkward film to 2 Mbps silently capped every video played
afterwards for the rest of the process, with Settings still showing
the old value. The override is cleared whenever playback moves to a
new item, which stops it surviving into an autoplayed next episode.
effective_streaming_quality() is the single resolution point.
DR-221 The quality picker is filled from what this media source can offer.
Rust marks a rung exceeds_source when its ceiling is at or above the
source's own bitrate — such a rung is another way to spell Original
— and the frontend does not draw those. Original is never marked; a
source whose bitrate the server does not report marks nothing, which
keeps every rung offered.
DR-222 Direct play and direct stream are negotiated, with two client-side
overrides on top because the server's answer is right about the file
and wrong about what this app will do with it: undecodable audio
(Jellyfin 10.11.5 honours a DirectPlayProfile's container and video
codec but ignores its audio codec, so it offers direct play for an
E-AC-3 track the webview renders in silence) and a viewer-pinned
audio track the file does not default to. A direct stream is a remux
and is deliberately not counted as transcoding.
DR-223 Dropped on measurement, not deferred. A master playlist from this
server carries exactly one EXT-X-STREAM-INF: Jellyfin builds it from
the single rendition the request asked for rather than publishing a
ladder. So there is no adaptation for hls.js to be preserving and
none mpv would lose — the claim that there was, in
playback-backend-unification.md, does not hold. Recorded rather than
deleted because it is a measurement: a server that does publish a
ladder would change the answer.
DR-224 Every backend consumes the same selection. The queue item carries
the transport, so player_seek_video picks its seek strategy from the
backend's decision instead of the last stream_url.contains(".m3u8")
in the codebase. Items queued by a path that never negotiated carry
None and fall back to needs_transcoding, which is exact rather than
a guess because every transcode this app requests is HLS (DR-140).
The frontend loader decision moves to streamTransport.ts so it can be
tested: the two cases that pin it are the ones that failed against the old
implementation — a progressive stream whose URL contains ".m3u8" must not
get an HLS loader, and an HLS stream whose URL contains none must.
Also verified the URL the direct-play branch builds actually serves playable
bytes: 206, video/mp4, valid ISO-BMFF, and a mid-file range works, so
seeking a direct play works.
The spec is folded into docs/architecture/{01,02,03} and deleted, per the
rule that docs/specs holds only work that has not shipped. DR-121 leaves
read-through-media-cache.md with a pointer; that spec keeps its capture half.
Not verified: real playback on a device. Direct play changes what actually
gets played, and neither fixtures nor curl prove the WebKitGTK and ExoPlayer
paths render it.
12 KiB
Spec: Two-path media — selectable playback bitrate, independent whole-file download
Status: Partially implemented. Landed: the cache/download unification
(DR-126, DR-127 — a cache entry is a downloads row with a shorter life, and
eviction only reclaims the temporary tier), local playback of downloaded media
(DR-128), and the one-path/one-row invariants that followed (DR-133 … DR-138).
DR-123 is in progress. Still open: the read-through capture itself — DR-122,
DR-124, DR-125.
DR-121 has shipped and left this spec. The player quality selector, the per-playback bitrate ceiling, and the backend-owned stream decision it needed were built as backend-owned stream selection (DR-224 … DR-227) and are described in 01-rust-backend.md and 03-data-flow.md. The settings-level ceiling (DR-162) is the same section. What remains here is the capture half only — this spec no longer specifies anything about choosing a bitrate.
Requirements: UR-070, UR-071 → DR-122, DR-123, DR-124, DR-125; IR-032 Related: the locally-indexed search and downloaded-browse work, both shipped — see 03-data-flow.md and 06-downloads-and-offline.md
Summary
Two things that are today tangled become explicitly separate:
- The playback path streams at a bitrate the viewer can change from the player. It is ephemeral and its rendition is volatile.
- The download path fetches the whole file at one canonical quality, in the background, independently of whatever playback is doing.
Bytes fetched for playback are kept only when the playback rendition happens to be the same artifact the download path would produce — i.e. direct play. Otherwise playback bytes are discarded and the download path does its own fetch.
Motivation
The appealing version of this — "stream and download at once, switch when enough has arrived" — breaks the moment the viewer can change bitrate. A capture taken while the rendition changes underneath it is a splice of two encodings: not a playable file, and not something that can be honestly recorded as a download. Once bitrate is selectable, one stream cannot serve both jobs.
Separating the paths also removes the thing that made the original idea expensive: there is no mid-playback source swap to engineer, because the download never has to take over the live session. It lands on disk and is used at the next natural boundary — next episode, or next time the item is played.
What exists already and is not this: SmartCache predictively downloads other
items, player_preload_upcoming warms the next one, and
refresh_queue_local_sources swaps queue entries to local at boundaries. All of
it concerns items you are not currently playing.
Layer assignment
| Logic / responsibility | Layer | Why it belongs there |
|---|---|---|
| Available bitrate options for an item | Rust | Derived from Jellyfin's media sources and playback-info negotiation; changes with the API. |
| Mapping a chosen bitrate to transcode parameters | Rust | Domain vocabulary. get_video_download_url already owns the quality→params mapping; playback must reuse it, not restate it. |
| Deciding whether playback bytes are keepable (direct play vs transcode) | Rust | Depends on the negotiated session. |
| Canonical download quality | Rust | Policy over domain data. |
| Cache eviction, storage budget, sparse-range bookkeeping | Rust | Storage policy. |
Promotion to a downloads row, and what invalidates a cache entry |
Rust | Domain state. |
| Rendering the quality selector; remembering the last choice | Frontend | Presentation and a view preference. The list comes from Rust. |
| WiFi-only / opt-in toggles | Frontend collects, Rust enforces | The control is UI; the gate must hold even if the UI never calls. |
Borderline, recorded: the default playback bitrate could look like a user preference (frontend). It goes to Rust because it must be reconcilable with what the server can actually produce for a given media source — a preference the backend has to validate is not a preference the frontend can own alone. The frontend stores the user's choice; Rust decides what that choice resolves to.
Design
DR-121 — moved out (shipped)
Bitrate selection in the player shipped as DR-224 … DR-227; see 01-rust-backend.md.
The one constraint here that the capture work still has to respect: a quality
change re-negotiates within HLS. Returning a progressive stream.mp4 for a
transcode means playback never starts, because the server encodes the whole file
before serving a byte (DR-140). That is why DR-122 below abandons a capture on a
quality change rather than trying to splice one.
DR-122 — The playback path is ephemeral
Playback bytes are not persisted unless DR-124 says they are keepable. No partial capture is ever retained across a quality change: on change, any in-flight capture for that session is abandoned and its partial file deleted.
DR-123 — The download path is independent
Downloading the whole file is a separate operation through the existing download
manager, at one canonical quality (default original, the direct static copy),
using /Videos/{id}/stream.mp4 — progressive and Range-capable, which is what
the resumable download worker relies on. It is unaffected by what playback is
doing, and playback is unaffected by it.
Once complete it becomes an ordinary download row, so everything already built on
top of downloads — offline browsing, refresh_queue_local_sources, the Downloads
page — picks it up with no further work.
Prerequisite: downloaded video is currently never played locally.
repository_get_video_stream_url goes straight to the online repo and
player/[id]/+page.svelte:316
calls it with no local check — so a completed video download is still streamed.
This must be fixed or the whole feature is invisible for video.
DR-124 — Keep playback bytes only when they are the download
Capture is enabled only where the played bytes and the canonical download artifact are the same thing — a direct-play session. Then:
| Path | Mechanism |
|---|---|
| Android / ExoPlayer | SimpleCache + CacheDataSource, keyed by item id and media-source id so renditions never collide. LRU evictor sharing the existing smart-cache budget — not a second budget over the same disk. |
| Linux audio / MPV | stream-record, set through the existing set_property plumbing. |
| Linux video (HLS transcode) | Not captured. Segments are not a file; assembling one needs ffmpeg, which is not a dependency and which CI is forbidden from installing at job time. The download path (DR-123) covers this case instead. |
Two abandonment rules, both of which must delete the partial rather than promote it:
- Seek during an mpv capture.
stream-recordis documented as intended for linear streams; seeking breaks the recording. Straight-through listening captures, scrubbing does not. - Any quality change (DR-122).
DR-125 — Promotion, rendition, and invalidation
A capture is promoted to a downloads row (status = 'completed') only when it
covers the whole resource. Partial captures stay cache and remain evictable.
A new downloads.source_rendition column records the negotiated
quality/container/codec of whatever produced the bytes; NULL for rows fetched by
the existing paths, which are always original. This is what makes an "upgrade to
original" action possible later, and what stops a 720p capture and a 4K download
being indistinguishable rows.
Invalidation. A quality change never touches a file that already exists — neither a permanent download nor a completed temporary one. Both remain valid copies of the rendition they hold, and deleting either would throw away bytes already paid for.
What a quality change does invalidate is an in-flight capture or background download of cached media: it is abandoned and restarted at the newly chosen quality, because a capture spanning a rendition change is a splice of two encodings rather than a playable file (DR-122).
So the rule is about ongoing work, not stored files. Nothing in this spec deletes user data.
Gating
Capture and background download obey the existing WiFi-only gate and storage budget, and are off unless opted in. Enforcement is in Rust.
Out of scope
- Mid-playback switch onto a completing download. Two independent paths make it unnecessary; the download is used from the next boundary.
- Backfilling the unplayed remainder of a capture. Watch 40 minutes and you have 40 minutes; completing it needs sparse-range bookkeeping and a resumable tail fetch. The DR-123 download path already produces a complete file, which is the reason this can wait.
- Bundling ffmpeg to make transcoded video capturable. Real option, large packaging decision, its own proposal.
- Routing Linux video playback through
stream.mp4. Regresses a documented, hard-won fix.
Acceptance criteria
- The player offers the qualities Rust reports, and changing one resumes at the same position with audio/subtitle selection preserved.
- A quality change abandons any in-flight capture and leaves no partial file.
- A quality change never deletes a
downloadsrow. - A completed background download of a video is played from disk on the next play (the DR-123 prerequisite).
- A direct-play session played start-to-finish leaves a complete local file with no second fetch; replaying it fetches no media bytes.
- Seeking during an mpv capture abandons it; no truncated file is promoted.
- A transcoded Linux video session is never captured, and never partially promoted.
- Promoted rows record their rendition; existing paths still record
NULL/original. - Gates hold with the setting off and with the frontend never sending it.
- Eviction cannot delete bytes backing a promoted download row.
bun run check,bun run test,cargo fmt,cargo clippy,bun run test:rust,bun run check:boundarypass;bindings.tsregenerated if Rust types changed.
Testing
Rust, table-driven and pure where possible: quality→params resolution shared with the download path; keepability (direct play vs transcode vs gate off); promotion (complete → promoted, partial → not, seek-abandoned → not, quality-changed → not); invalidation (evicts cache, never a download row); rendition round-trip.
Android: instrumented — a played direct-play item yields cache entries, and a replay issues no media network request.
Frontend: the quality list renders from backend data with no item-type or
codec taxonomy in src/; the selector's remembered choice is a view preference.
TRACES
| Piece | Tag |
|---|---|
| Ephemeral playback / capture abandonment | // TRACES: UR-070 | DR-122 |
| Independent whole-file download + local video playback fix | // TRACES: UR-071 | DR-123, IR-032 |
| ExoPlayer cache / mpv stream-record / keepability | // TRACES: UR-071 | DR-124 |
Promotion, source_rendition, invalidation |
// TRACES: UR-071 | DR-125 |
Notes for the implementer
- A parallel Claude session is active in this repo.
git diffbefore "repairing" anything you did not write. - Do not duplicate the quality→transcode-parameter table. Call the existing one.
- Reuse the smart-cache storage budget; two budgets over one disk is how devices fill up.
- The
downloadsFK toitemsis relaxed (migration 005) — exercise promotion for an item that was never cached. - Build DR-123's local-playback fix first. Without it nothing in this spec is observable for video.