dtourolle
83dc8c7028
Playing a video meant asking the server to re-encode it, always. That
decision was made nowhere and written down nowhere, so whoever needed it
re-derived it downstream — the player worked out whether it had been handed
a playlist by looking for ".m3u8" in the URL, in two places. A viewer paid
for a transcode of a file their device could have played untouched, and the
app could not tell them which it was.
One negotiation now produces one self-describing StreamSelection — direct
play, remux or transcode; over a playlist, a plain HTTP file, or a local one
— and every renderer consumes that same answer.
Measured against the development server (Jellyfin 10.11.5), 400 items
sampled for codec mix and 40 put through a real PlaybackInfo negotiation
per profile:
Linux / WebKitGTK (h264 only, 2ch) 3/40 — 7% direct play
Android / ExoPlayer (hevc, ac3/eac3, 6ch) 34/40 — 85% direct play
The library is ~80% hevc, which is why the two diverge so hard. The payoff
is overwhelmingly Android, where 85% of plays were starting a transcode
nobody needed. Linux stays near 7% until libmpv decodes the picture — the
h264-only profile is a WebKitGTK constraint, not a JellyTau choice.
DR-219 StreamSelection: url + tagged Transport (hls/progressive/localFile)
+ PlaybackKind (directPlay/directStream/transcode) + the negotiated
rendition + this source's ladder + a needs_transcoding flag derived
in Rust so the rule is answered once. Both enums are serde-tagged
so the frontend matches a discriminant, not a substring. The paths
that never negotiate get the same shape from Rust rather than
assembling one — media_local_selection for a downloaded file,
LiveStreamInfo.transport for a live channel — so there is no second
place where a transport is decided.
DR-220 The ceiling becomes two levels: a durable device default (Settings,
persisted) and a per-playback override the in-player picker sets.
The picker had called itself a "this film, this connection" control
since it was written but wrote the process-wide default, so dropping
one awkward film to 2 Mbps silently capped every video played
afterwards for the rest of the process, with Settings still showing
the old value. The override is cleared whenever playback moves to a
new item, which stops it surviving into an autoplayed next episode.
effective_streaming_quality() is the single resolution point.
DR-221 The quality picker is filled from what this media source can offer.
Rust marks a rung exceeds_source when its ceiling is at or above the
source's own bitrate — such a rung is another way to spell Original
— and the frontend does not draw those. Original is never marked; a
source whose bitrate the server does not report marks nothing, which
keeps every rung offered.
DR-222 Direct play and direct stream are negotiated, with two client-side
overrides on top because the server's answer is right about the file
and wrong about what this app will do with it: undecodable audio
(Jellyfin 10.11.5 honours a DirectPlayProfile's container and video
codec but ignores its audio codec, so it offers direct play for an
E-AC-3 track the webview renders in silence) and a viewer-pinned
audio track the file does not default to. A direct stream is a remux
and is deliberately not counted as transcoding.
DR-223 Dropped on measurement, not deferred. A master playlist from this
server carries exactly one EXT-X-STREAM-INF: Jellyfin builds it from
the single rendition the request asked for rather than publishing a
ladder. So there is no adaptation for hls.js to be preserving and
none mpv would lose — the claim that there was, in
playback-backend-unification.md, does not hold. Recorded rather than
deleted because it is a measurement: a server that does publish a
ladder would change the answer.
DR-224 Every backend consumes the same selection. The queue item carries
the transport, so player_seek_video picks its seek strategy from the
backend's decision instead of the last stream_url.contains(".m3u8")
in the codebase. Items queued by a path that never negotiated carry
None and fall back to needs_transcoding, which is exact rather than
a guess because every transcode this app requests is HLS (DR-140).
The frontend loader decision moves to streamTransport.ts so it can be
tested: the two cases that pin it are the ones that failed against the old
implementation — a progressive stream whose URL contains ".m3u8" must not
get an HLS loader, and an HLS stream whose URL contains none must.
Also verified the URL the direct-play branch builds actually serves playable
bytes: 206, video/mp4, valid ISO-BMFF, and a mid-file range works, so
seeking a direct play works.
The spec is folded into docs/architecture/{01,02,03} and deleted, per the
rule that docs/specs holds only work that has not shipped. DR-121 leaves
read-through-media-cache.md with a pointer; that spec keeps its capture half.
Not verified: real playback on a device. Direct play changes what actually
gets played, and neither fixtures nor curl prove the WebKitGTK and ExoPlayer
paths render it.
228 lines
12 KiB
Markdown
228 lines
12 KiB
Markdown
# Spec: Two-path media — selectable playback bitrate, independent whole-file download
|
|
|
|
**Status:** Partially implemented. Landed: the cache/download unification
|
|
(DR-126, DR-127 — a cache entry *is* a `downloads` row with a shorter life, and
|
|
eviction only reclaims the temporary tier), local playback of downloaded media
|
|
(DR-128), and the one-path/one-row invariants that followed (DR-133 … DR-138).
|
|
DR-123 is in progress. Still open: the read-through capture itself — DR-122,
|
|
DR-124, DR-125.
|
|
|
|
**DR-121 has shipped and left this spec.** The player quality selector, the
|
|
per-playback bitrate ceiling, and the backend-owned stream decision it needed
|
|
were built as *backend-owned stream selection* (DR-224 … DR-227) and are
|
|
described in
|
|
[01-rust-backend.md](../architecture/01-rust-backend.md#stream-selection) and
|
|
[03-data-flow.md](../architecture/03-data-flow.md#video-stream-selection-flow).
|
|
The settings-level ceiling (DR-162) is the same section. What remains here is the
|
|
*capture* half only — this spec no longer specifies anything about choosing a
|
|
bitrate.
|
|
|
|
**Requirements:** UR-070, UR-071 → DR-122, DR-123, DR-124, DR-125; IR-032
|
|
**Related:** the locally-indexed search and downloaded-browse work, both
|
|
shipped — see
|
|
[03-data-flow.md](../architecture/03-data-flow.md) and
|
|
[06-downloads-and-offline.md](../architecture/06-downloads-and-offline.md)
|
|
|
|
## Summary
|
|
|
|
Two things that are today tangled become explicitly separate:
|
|
|
|
- **The playback path** streams at a bitrate the viewer can change from the
|
|
player. It is ephemeral and its rendition is volatile.
|
|
- **The download path** fetches the whole file at one canonical quality, in the
|
|
background, independently of whatever playback is doing.
|
|
|
|
Bytes fetched for playback are kept **only** when the playback rendition happens
|
|
to be the same artifact the download path would produce — i.e. direct play.
|
|
Otherwise playback bytes are discarded and the download path does its own fetch.
|
|
|
|
## Motivation
|
|
|
|
The appealing version of this — "stream and download at once, switch when enough
|
|
has arrived" — breaks the moment the viewer can change bitrate. A capture taken
|
|
while the rendition changes underneath it is a splice of two encodings: not a
|
|
playable file, and not something that can be honestly recorded as a download.
|
|
Once bitrate is selectable, one stream cannot serve both jobs.
|
|
|
|
Separating the paths also removes the thing that made the original idea
|
|
expensive: there is no mid-playback source swap to engineer, because the download
|
|
never has to take over the live session. It lands on disk and is used at the next
|
|
natural boundary — next episode, or next time the item is played.
|
|
|
|
What exists already and is *not* this: `SmartCache` predictively downloads *other*
|
|
items, `player_preload_upcoming` warms the next one, and
|
|
`refresh_queue_local_sources` swaps queue entries to local at boundaries. All of
|
|
it concerns items you are not currently playing.
|
|
|
|
## Layer assignment
|
|
|
|
| Logic / responsibility | Layer | Why it belongs there |
|
|
|------------------------|-------|----------------------|
|
|
| Available bitrate options for an item | **Rust** | Derived from Jellyfin's media sources and playback-info negotiation; changes with the API. |
|
|
| Mapping a chosen bitrate to transcode parameters | **Rust** | Domain vocabulary. `get_video_download_url` already owns the quality→params mapping; playback must reuse it, not restate it. |
|
|
| Deciding whether playback bytes are keepable (direct play vs transcode) | **Rust** | Depends on the negotiated session. |
|
|
| Canonical download quality | **Rust** | Policy over domain data. |
|
|
| Cache eviction, storage budget, sparse-range bookkeeping | **Rust** | Storage policy. |
|
|
| Promotion to a `downloads` row, and what invalidates a cache entry | **Rust** | Domain state. |
|
|
| Rendering the quality selector; remembering the last choice | **Frontend** | Presentation and a view preference. The *list* comes from Rust. |
|
|
| WiFi-only / opt-in toggles | **Frontend collects, Rust enforces** | The control is UI; the gate must hold even if the UI never calls. |
|
|
|
|
Borderline, recorded: the **default** playback bitrate could look like a user
|
|
preference (frontend). It goes to Rust because it must be reconcilable with what
|
|
the server can actually produce for a given media source — a preference the
|
|
backend has to validate is not a preference the frontend can own alone. The
|
|
frontend stores the user's *choice*; Rust decides what that choice resolves to.
|
|
|
|
## Design
|
|
|
|
### DR-121 — moved out (shipped)
|
|
|
|
Bitrate selection in the player shipped as DR-224 … DR-227; see
|
|
[01-rust-backend.md](../architecture/01-rust-backend.md#stream-selection).
|
|
|
|
The one constraint here that the capture work still has to respect: a quality
|
|
change re-negotiates **within HLS**. Returning a progressive `stream.mp4` for a
|
|
transcode means playback never starts, because the server encodes the whole file
|
|
before serving a byte (DR-140). That is why DR-122 below abandons a capture on a
|
|
quality change rather than trying to splice one.
|
|
|
|
### DR-122 — The playback path is ephemeral
|
|
|
|
Playback bytes are not persisted unless DR-124 says they are keepable. No partial
|
|
capture is ever retained across a quality change: on change, any in-flight capture
|
|
for that session is abandoned and its partial file deleted.
|
|
|
|
### DR-123 — The download path is independent
|
|
|
|
Downloading the whole file is a separate operation through the existing download
|
|
manager, at one canonical quality (default `original`, the direct static copy),
|
|
using `/Videos/{id}/stream.mp4` — progressive and Range-capable, which is what
|
|
the resumable download worker relies on. It is unaffected by what playback is
|
|
doing, and playback is unaffected by it.
|
|
|
|
Once complete it becomes an ordinary download row, so everything already built on
|
|
top of downloads — offline browsing, `refresh_queue_local_sources`, the Downloads
|
|
page — picks it up with no further work.
|
|
|
|
**Prerequisite:** downloaded *video* is currently never played locally.
|
|
`repository_get_video_stream_url` goes straight to the online repo and
|
|
[player/[id]/+page.svelte:316](../../src/routes/player/[id]/+page.svelte#L316)
|
|
calls it with no local check — so a completed video download is still streamed.
|
|
This must be fixed or the whole feature is invisible for video.
|
|
|
|
### DR-124 — Keep playback bytes only when they *are* the download
|
|
|
|
Capture is enabled only where the played bytes and the canonical download artifact
|
|
are the same thing — a **direct-play** session. Then:
|
|
|
|
| Path | Mechanism |
|
|
|---|---|
|
|
| Android / ExoPlayer | `SimpleCache` + `CacheDataSource`, keyed by item id **and** media-source id so renditions never collide. LRU evictor sharing the existing smart-cache budget — not a second budget over the same disk. |
|
|
| Linux audio / MPV | `stream-record`, set through the existing `set_property` plumbing. |
|
|
| Linux video (HLS transcode) | **Not captured.** Segments are not a file; assembling one needs ffmpeg, which is not a dependency and which CI is forbidden from installing at job time. The download path (DR-123) covers this case instead. |
|
|
|
|
Two abandonment rules, both of which must delete the partial rather than promote
|
|
it:
|
|
|
|
- **Seek during an mpv capture.** `stream-record` is documented as intended for
|
|
linear streams; seeking breaks the recording. Straight-through listening
|
|
captures, scrubbing does not.
|
|
- **Any quality change** (DR-122).
|
|
|
|
### DR-125 — Promotion, rendition, and invalidation
|
|
|
|
A capture is promoted to a `downloads` row (`status = 'completed'`) only when it
|
|
covers the whole resource. Partial captures stay cache and remain evictable.
|
|
|
|
A new `downloads.source_rendition` column records the negotiated
|
|
quality/container/codec of whatever produced the bytes; `NULL` for rows fetched by
|
|
the existing paths, which are always `original`. This is what makes an "upgrade to
|
|
original" action possible later, and what stops a 720p capture and a 4K download
|
|
being indistinguishable rows.
|
|
|
|
**Invalidation.** A quality change never touches a file that already exists —
|
|
neither a permanent download nor a completed temporary one. Both remain valid
|
|
copies of the rendition they hold, and deleting either would throw away bytes
|
|
already paid for.
|
|
|
|
What a quality change *does* invalidate is an **in-flight** capture or background
|
|
download of cached media: it is abandoned and restarted at the newly chosen
|
|
quality, because a capture spanning a rendition change is a splice of two
|
|
encodings rather than a playable file (DR-122).
|
|
|
|
So the rule is about *ongoing* work, not stored files. Nothing in this spec
|
|
deletes user data.
|
|
|
|
### Gating
|
|
|
|
Capture and background download obey the existing WiFi-only gate and storage
|
|
budget, and are off unless opted in. Enforcement is in Rust.
|
|
|
|
## Out of scope
|
|
|
|
- **Mid-playback switch onto a completing download.** Two independent paths make
|
|
it unnecessary; the download is used from the next boundary.
|
|
- **Backfilling the unplayed remainder of a capture.** Watch 40 minutes and you
|
|
have 40 minutes; completing it needs sparse-range bookkeeping and a resumable
|
|
tail fetch. The DR-123 download path already produces a complete file, which is
|
|
the reason this can wait.
|
|
- **Bundling ffmpeg** to make transcoded video capturable. Real option, large
|
|
packaging decision, its own proposal.
|
|
- **Routing Linux video playback through `stream.mp4`.** Regresses a documented,
|
|
hard-won fix.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] The player offers the qualities Rust reports, and changing one resumes at
|
|
the same position with audio/subtitle selection preserved.
|
|
- [ ] A quality change abandons any in-flight capture and leaves no partial file.
|
|
- [ ] A quality change never deletes a `downloads` row.
|
|
- [ ] A completed background download of a video is *played from disk* on the next
|
|
play (the DR-123 prerequisite).
|
|
- [ ] A direct-play session played start-to-finish leaves a complete local file
|
|
with no second fetch; replaying it fetches no media bytes.
|
|
- [ ] Seeking during an mpv capture abandons it; no truncated file is promoted.
|
|
- [ ] A transcoded Linux video session is never captured, and never partially
|
|
promoted.
|
|
- [ ] Promoted rows record their rendition; existing paths still record
|
|
`NULL`/`original`.
|
|
- [ ] Gates hold with the setting off *and* with the frontend never sending it.
|
|
- [ ] Eviction cannot delete bytes backing a promoted download row.
|
|
- [ ] `bun run check`, `bun run test`, `cargo fmt`, `cargo clippy`,
|
|
`bun run test:rust`, `bun run check:boundary` pass; `bindings.ts`
|
|
regenerated if Rust types changed.
|
|
|
|
## Testing
|
|
|
|
Rust, table-driven and pure where possible: quality→params resolution shared with
|
|
the download path; keepability (direct play vs transcode vs gate off); promotion
|
|
(complete → promoted, partial → not, seek-abandoned → not, quality-changed → not);
|
|
invalidation (evicts cache, never a download row); rendition round-trip.
|
|
|
|
Android: instrumented — a played direct-play item yields cache entries, and a
|
|
replay issues no media network request.
|
|
|
|
Frontend: the quality list renders from backend data with no item-type or
|
|
codec taxonomy in `src/`; the selector's remembered choice is a view preference.
|
|
|
|
## TRACES
|
|
|
|
| Piece | Tag |
|
|
|---|---|
|
|
| Ephemeral playback / capture abandonment | `// TRACES: UR-070 \| DR-122` |
|
|
| Independent whole-file download + local video playback fix | `// TRACES: UR-071 \| DR-123, IR-032` |
|
|
| ExoPlayer cache / mpv stream-record / keepability | `// TRACES: UR-071 \| DR-124` |
|
|
| Promotion, `source_rendition`, invalidation | `// TRACES: UR-071 \| DR-125` |
|
|
|
|
## Notes for the implementer
|
|
|
|
- **A parallel Claude session is active in this repo.** `git diff` before
|
|
"repairing" anything you did not write.
|
|
- Do not duplicate the quality→transcode-parameter table. Call the existing one.
|
|
- Reuse the smart-cache storage budget; two budgets over one disk is how devices
|
|
fill up.
|
|
- The `downloads` FK to `items` is relaxed (migration 005) — exercise promotion
|
|
for an item that was never cached.
|
|
- Build DR-123's local-playback fix first. Without it nothing in this spec is
|
|
observable for video.
|