From 1d56517f07426a9f6691f531a6b43b8574e43f63 Mon Sep 17 00:00:00 2001
From: Duncan Tourolle <duncan@tourolle.paris>
Date: Fri, 21 Aug 2026 19:41:32 +0200
Subject: [PATCH] docs(specs): add backend-owned stream selection
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Rust becomes the single owner of which stream to play — direct play or
transcode, at what ceiling, over what transport — and hands every player
backend a self-describing StreamSelection instead of a bare URL. mpv,
ExoPlayer and the HTML5/hls.js path all consume one decision rather than
three places re-deriving it.

The motivating leak is concrete. VideoPlayer.svelte determines transport with
`currentStreamUrl.includes(".m3u8")`, in two places, for a URL Rust
constructed and therefore already knows the shape of. That is the boundary
rule in miniature: not item-type taxonomy, but the same error of
reconstructing a domain fact in the presentation layer because the wire shape
did not carry it. A tagged Transport enum deletes it.

The design line, which ExoPlayer forces: Rust decides *what stream*, the
player decides *how to deliver it*. ExoPlayer has genuine adaptive track
selection; this spec must not reimplement or fight it. Rust only adapts where
the player cannot (mpv) and the server actually offers a ladder.

Six phases, and phase 1 stands alone as pure ownership movement with no
behaviour change. Phase 4 (direct-play negotiation) is what removes the
transcode and unblocks the Linux native-video work. Phase 5 (adaptation) is
gated on counting EXT-X-STREAM-INF entries in a real playlist — the
acceptance criteria require that count be recorded before it is either
started or dropped.

Takes DR-121 from read-through-media-cache.md, which specced Rust-owned
quality reporting but never built it; that spec keeps its capture half.
---
 docs-site/SUMMARY.md                         |   1 +
 docs/specs/README.md                         |   1 +
 docs/specs/backend-owned-stream-selection.md | 242 +++++++++++++++++++
 3 files changed, 244 insertions(+)
 create mode 100644 docs/specs/backend-owned-stream-selection.md

diff --git a/docs-site/SUMMARY.md b/docs-site/SUMMARY.md
index ce86b824..45a207be 100644
--- a/docs-site/SUMMARY.md
+++ b/docs-site/SUMMARY.md
@@ -33,6 +33,7 @@
 - [Spec Review Checklist](specs/SPEC-REVIEW-CHECKLIST.md)
 - [Playback Backend Unification](specs/playback-backend-unification.md)
 - [Linux Native Video Spike](specs/linux-native-video-spike.md)
+- [Backend-Owned Stream Selection](specs/backend-owned-stream-selection.md)
 - [Player Facade Enforcement](specs/player-facade-enforcement.md)
 - [Windows Native Audio Backend](specs/windows-native-audio-backend.md)
 - [libmpv2 Migration](specs/libmpv2-migration.md)
diff --git a/docs/specs/README.md b/docs/specs/README.md
index 4ac015a9..9c4eac56 100644
--- a/docs/specs/README.md
+++ b/docs/specs/README.md
@@ -44,6 +44,7 @@ taken by other work; each carries a ⚠️ note at the top.
 
 | Spec | Blocked on / note |
 |---|---|
+| [backend-owned-stream-selection.md](backend-owned-stream-selection.md) | Rust owns direct-play-vs-transcode, transport and quality; players consume one `StreamSelection`. Phase 1 (delete the `.m3u8` sniff) stands alone. Unblocks Linux native video. |
 | [build-provenance.md](build-provenance.md) | `build.rs` is still bare. ⚠️ suggested id DR-093 is taken. |
 | [player-facade-enforcement.md](player-facade-enforcement.md) | ~60 `commands.player*` sites still outside the facade; no lint rule. ⚠️ suggested id DR-095 is taken. |
 | [windows-native-audio-backend.md](windows-native-audio-backend.md) | Blocked on the libmpv2 swap. ⚠️ suggested id IR-030 is taken. |
diff --git a/docs/specs/backend-owned-stream-selection.md b/docs/specs/backend-owned-stream-selection.md
new file mode 100644
index 00000000..a32391d8
--- /dev/null
+++ b/docs/specs/backend-owned-stream-selection.md
@@ -0,0 +1,242 @@
+# Spec: Backend-owned stream selection
+
+**Status:** Proposed
+**Requirements:** UR-079 (new) → DR-219 … DR-224 (new); **implements and extends
+DR-121**, currently allocated to
+[read-through-media-cache.md](read-through-media-cache.md) and not started.
+Re-check `requirements.md` before allocating — the ids moved twice while this was
+being written (`DR` max was 215, then 218).
+**UX spec:** the quality selector in `VideoPlayer.svelte` already exists; this
+changes what fills it, not how it looks.
+**Supersedes / revises:** takes DR-121 out of
+[read-through-media-cache.md](read-through-media-cache.md), which should keep
+only its capture/eviction half. Unblocks
+[linux-native-video-spike.md](linux-native-video-spike.md).
+
+**Destination on completion:**
+[01-rust-backend.md](../architecture/01-rust-backend.md) — extends the
+"Streaming quality ladder" section; and
+[03-data-flow.md](../architecture/03-data-flow.md) — playback initiation. The
+durable half is the layer line and the `StreamSelection` contract; phases and
+acceptance criteria are disposable.
+
+## Summary
+
+Make Rust the single owner of *which stream to play* — direct play or transcode,
+at what ceiling, over what transport — and hand every player backend a
+self-describing selection instead of a bare URL. mpv, ExoPlayer and the HTML5
+`<video>`/hls.js path all become consumers of the same decision rather than three
+places that re-derive it.
+
+Nothing about how playback *looks* changes. What changes is that the frontend
+stops inferring transport from a URL string, and that direct play becomes
+possible at all.
+
+## Motivation
+
+Four concrete problems, all the same shape.
+
+**1. The frontend sniffs transport out of the URL.**
+[VideoPlayer.svelte:569](../../src/lib/components/player/VideoPlayer.svelte#L569):
+
+```ts
+const isHlsStream = currentStreamUrl.includes(".m3u8");
+```
+
+and again inline at line 2364. Rust *built* that URL and knows exactly what it
+is; the frontend re-derives it by substring match. Change the endpoint, add a DASH
+path, serve a progressive file, and this silently picks wrong. This is the
+boundary rule in miniature — not item-type taxonomy, but the same error: a
+domain fact reconstructed in the presentation layer because the wire shape did
+not carry it.
+
+**2. There is no direct-play path.** `get_video_stream_url` always builds an HLS
+transcode URL (`TranscodingProtocol=hls`, `VideoCodec=h264` first). Every video
+play burns server CPU, even when the file would play untouched. This is the cost
+the Linux native-video work exists to remove, and it cannot be removed without a
+decision that does not currently exist anywhere in the codebase.
+
+**3. Quality is a process-wide global.** `streaming_quality()` /
+`set_streaming_quality()` in `repository/online.rs` read and write a static.
+It is not per-session or per-item, so it cannot express "this 4K remux needs a
+ceiling, that podcast does not", and two concurrent playbacks would share one
+setting.
+
+**4. Rust cannot say what qualities *this* media source supports.** The selector
+is populated from a fixed enum rather than from what the source actually offers.
+DR-121 already names this; it has not been built.
+
+### The prior question
+
+Finding 3 of [playback-backend-unification.md](playback-backend-unification.md)
+holds that hls.js gives us real adaptive bitrate and mpv would lose it. Evidence
+in this repo suggests **there is no ABR today**: a single rendition is requested,
+no level-handling code exists anywhere in the frontend, and a quality switch is
+implemented by re-opening the stream.
+
+**Run this before sizing the adaptation work.** It needs a live server:
+
+```
+curl -s "https://<server>/Videos/<itemId>/master.m3u8?api_key=<key>&…" \
+  | grep -c EXT-X-STREAM-INF
+```
+
+`1` → there is no adaptation to preserve, and the adaptation half of this spec
+collapses to "pick well at open". `>1` → finding 3 stands and DR-223 applies.
+**Everything else in this spec is worth doing either way** — the ownership
+problems above are independent of the answer.
+
+## Layer assignment
+
+| Logic / responsibility | Layer | Why it belongs there |
+|---|---|---|
+| Direct play vs direct stream vs transcode | Rust | Depends on Jellyfin's `PlaybackInfo`, container/codec support and the device profile. Changes when Jellyfin's API or our profile changes → domain, by the litmus test. |
+| Transport of the chosen stream (HLS / progressive / local file) | Rust | Rust constructs the URL; it is the only place that *knows* rather than infers. Today the frontend guesses from `.m3u8`. |
+| Which qualities this media source can offer | Rust | Derived from the source's own streams and the quality→transcode-parameter mapping that `get_video_download_url` already holds. DR-121. |
+| The quality ceiling in force, per playback session | Rust | Domain state that outlives any one view and must survive a backend swap or a mode transfer. Currently a process-wide static. |
+| Deciding to re-negotiate mid-playback (if adaptation is needed) | Rust | It performs the HTTP and already derives reachability from real traffic via `ConnectivityMonitor`. Throughput estimation is the same pattern on the same data — a side-channel probe would repeat the mistake that principle exists to prevent. |
+| Frame-level delivery *within* the selected stream, including a player's own ABR | **Player** | ExoPlayer has genuine adaptive selection; if Rust hands it a multi-variant playlist it should use it. Rust chooses *what to request*, never how a player paces bytes. See "The line". |
+| Rendering the selector, showing the current quality, ordering the list | Frontend | Pure presentation over a backend-supplied list. |
+| Poster, letterbox, controls, overlay z-order | Frontend | Unchanged. |
+
+### The line
+
+**Rust decides *what stream*. The player decides *how to deliver it*.**
+
+This matters most for ExoPlayer, which already does real adaptive track selection
+over HLS. This spec must not reimplement that or fight it — if a multi-variant
+playlist reaches ExoPlayer, ExoPlayer adapts and Rust stays out of the way. The
+same restraint applies to any future backend that gains the capability. Rust only
+steps in where the player has no such ability (mpv) *and* the server actually
+offers a ladder.
+
+Borderline row, with its tie-breaker: "which media source of a multi-source item"
+looks like a user choice, and its *presentation* is. The default and the
+constraint set are domain → **Rust**, per the borderline-defaults-to-Rust rule.
+
+## Design
+
+### The contract
+
+One self-describing selection replaces the bare URL. Nested fields are
+camelCase over the wire (`#[serde(rename_all = "camelCase")]`); the enums are
+tagged so the frontend matches a tag instead of parsing a string.
+
+```rust
+#[derive(Serialize, Type)]
+#[serde(rename_all = "camelCase")]
+pub struct StreamSelection {
+    pub url: String,
+    pub transport: Transport,
+    pub playback_kind: PlaybackKind,
+    /// The negotiated rendition; None when direct-playing the source as-is.
+    pub rendition: Option<Rendition>,
+    /// What this media source can offer — fills the selector (DR-121).
+    pub available: Vec<QualityOption>,
+}
+
+#[derive(Serialize, Type)]
+#[serde(tag = "type", rename_all = "camelCase")]
+pub enum Transport { Hls, Progressive, LocalFile }
+
+#[derive(Serialize, Type)]
+#[serde(tag = "type", rename_all = "camelCase")]
+pub enum PlaybackKind { DirectPlay, DirectStream, Transcode }
+```
+
+`Transport` is the field that deletes the `.m3u8` sniff. The frontend picks
+hls.js on `Hls` and the element's own loader otherwise — a tag match, not a
+substring search.
+
+### Re-negotiation
+
+Rust emits `stream-selection-changed` (kebab-case, per convention) carrying a new
+`StreamSelection` plus the position to resume at. The existing
+`playerSetStreamQuality` response already has exactly the right shape — a tagged
+`strategy` that tells the caller who reloads, with the backend handling native
+itself and handing HTML5 a URL for `reloadSource`
+([index.ts:198](../../src/lib/player/index.ts#L198)). **Extend that; do not
+invent a second mechanism.** It is the one piece of this that is already right.
+
+Note the existing wart to preserve or fix deliberately, not accidentally:
+tauri-specta keeps those response fields snake_case (`new_url`), and the facade
+comments say so.
+
+### Phases
+
+1. **DR-219** `StreamSelection` + `Transport`; delete the `.m3u8` sniff. No
+   behaviour change — pure ownership move, and independently shippable.
+2. **DR-220** Per-session quality ceiling replacing the `online.rs` static.
+3. **DR-221** `available` populated from the media source (DR-121's substance).
+4. **DR-222** Direct-play/direct-stream negotiation via `PlaybackInfo`. This is
+   the phase that unlocks native video and removes the transcode.
+5. **DR-223** Adaptation, **only if the playlist check says a ladder exists**.
+   Cheapest sufficient design: re-negotiate on sustained throughput drop, reusing
+   the phase-1 re-negotiation path. A local proxy synthesizing a single-variant
+   playlist is a last resort, not a starting point.
+6. **DR-224** ExoPlayer and mpv consume `StreamSelection` unchanged, proving the
+   contract is player-agnostic rather than HTML5-shaped.
+
+Phases 1–4 stand on their own merits with no dependency on the ladder question.
+
+## Out of scope
+
+- Rendering, compositing, and the Linux native-video work itself. This spec
+  unblocks [linux-native-video-spike.md](linux-native-video-spike.md); it does
+  not contain it.
+- Replacing hls.js. It stays as the HLS loader for the webview path.
+- Reimplementing or overriding ExoPlayer's own adaptive selection. See "The line".
+- The download/capture half of [read-through-media-cache.md](read-through-media-cache.md)
+  (DR-122, DR-124, DR-125), which keeps its own spec.
+- Audio. The same argument applies, but video is where the transcode cost is.
+
+## Acceptance criteria
+
+- [ ] The `.m3u8` substring check is gone from `VideoPlayer.svelte` (both sites)
+      and transport comes from the tagged enum.
+- [ ] `bun run check`, `bun run test`, `bun run format:check`, `bun run lint` pass.
+- [ ] `cargo fmt` clean, `cargo clippy -D warnings` clean, `bun run test:rust` passes.
+- [ ] `bun run check:boundary` passes — and the reviewer confirms by reading that
+      no transport/kind decision was reconstructed in `src/`, since the tripwire
+      only catches item-type array literals.
+- [ ] `bindings.ts` regenerated from Rust, not hand-edited.
+- [ ] New code carries `// TRACES:` comments; `bun run traces:validate` passes and
+      coverage stays ≥ the CI ratchet.
+- [ ] The `EXT-X-STREAM-INF` count is recorded in this spec before DR-223 is
+      started or dropped.
+- [ ] DR-121 is removed from `read-through-media-cache.md` with a pointer here.
+
+## Testing
+
+- Rust: `PlaybackInfo` fixtures → expected `PlaybackKind`, one per branch
+  (supported container direct-plays; unsupported codec transcodes; a ceiling
+  below the source bitrate transcodes even when the codec is fine).
+- Rust: `Transport` round-trips through serde with the tag the frontend matches.
+- Frontend: adapter selection driven by `transport`, including the case a URL
+  ending `.m3u8` is served as `Progressive` — that test fails on today's code,
+  which is the point.
+- Extend `tauriIntegration.test.ts` for the new command params (camelCase rule).
+- No test asserts a URL substring.
+
+## TRACES
+
+| Piece | Tag |
+|---|---|
+| `StreamSelection` / `Transport` | `UR-079 \| DR-219` |
+| Per-session ceiling | `UR-074 \| DR-220` |
+| `available` from media source | `UR-079 \| DR-221, DR-121` |
+| Direct-play negotiation | `UR-079 \| DR-222` |
+| Adaptation, if built | `UR-079 \| DR-223` |
+| ExoPlayer/mpv consumers | `UR-003, UR-004 \| DR-224` |
+
+## Notes for the implementer
+
+- **Phase 1 is worth doing on its own**, even if everything after it is dropped.
+  It removes a real leak and costs almost nothing.
+- Do not frame any phase as "no Rust changes required" — that framing is what
+  produced the leak `scoped-search-boundary.md` records.
+- `ConnectivityMonitor` is the precedent for DR-223: derive network facts from
+  real traffic, never from a side-channel poller.
+- A parallel Claude session may be active in this repo — `git diff` before
+  "repairing" unexpected changes. Requirement ids in particular moved twice
+  during the writing of this spec.