docs(specs): add backend-owned stream selection
Traceability Validation / Check Requirement Traces (pull_request) Successful in 11s
Traceability Validation / Check Requirement Traces (pull_request) Successful in 11s
Rust becomes the single owner of which stream to play — direct play or
transcode, at what ceiling, over what transport — and hands every player
backend a self-describing StreamSelection instead of a bare URL. mpv,
ExoPlayer and the HTML5/hls.js path all consume one decision rather than
three places re-deriving it.
The motivating leak is concrete. VideoPlayer.svelte determines transport with
`currentStreamUrl.includes(".m3u8")`, in two places, for a URL Rust
constructed and therefore already knows the shape of. That is the boundary
rule in miniature: not item-type taxonomy, but the same error of
reconstructing a domain fact in the presentation layer because the wire shape
did not carry it. A tagged Transport enum deletes it.
The design line, which ExoPlayer forces: Rust decides *what stream*, the
player decides *how to deliver it*. ExoPlayer has genuine adaptive track
selection; this spec must not reimplement or fight it. Rust only adapts where
the player cannot (mpv) and the server actually offers a ladder.
Six phases, and phase 1 stands alone as pure ownership movement with no
behaviour change. Phase 4 (direct-play negotiation) is what removes the
transcode and unblocks the Linux native-video work. Phase 5 (adaptation) is
gated on counting EXT-X-STREAM-INF entries in a real playlist — the
acceptance criteria require that count be recorded before it is either
started or dropped.
Takes DR-121 from read-through-media-cache.md, which specced Rust-owned
quality reporting but never built it; that spec keeps its capture half.
This commit is contained in:
@@ -33,6 +33,7 @@
|
||||
- [Spec Review Checklist](specs/SPEC-REVIEW-CHECKLIST.md)
|
||||
- [Playback Backend Unification](specs/playback-backend-unification.md)
|
||||
- [Linux Native Video Spike](specs/linux-native-video-spike.md)
|
||||
- [Backend-Owned Stream Selection](specs/backend-owned-stream-selection.md)
|
||||
- [Player Facade Enforcement](specs/player-facade-enforcement.md)
|
||||
- [Windows Native Audio Backend](specs/windows-native-audio-backend.md)
|
||||
- [libmpv2 Migration](specs/libmpv2-migration.md)
|
||||
|
||||
@@ -44,6 +44,7 @@ taken by other work; each carries a ⚠️ note at the top.
|
||||
|
||||
| Spec | Blocked on / note |
|
||||
|---|---|
|
||||
| [backend-owned-stream-selection.md](backend-owned-stream-selection.md) | Rust owns direct-play-vs-transcode, transport and quality; players consume one `StreamSelection`. Phase 1 (delete the `.m3u8` sniff) stands alone. Unblocks Linux native video. |
|
||||
| [build-provenance.md](build-provenance.md) | `build.rs` is still bare. ⚠️ suggested id DR-093 is taken. |
|
||||
| [player-facade-enforcement.md](player-facade-enforcement.md) | ~60 `commands.player*` sites still outside the facade; no lint rule. ⚠️ suggested id DR-095 is taken. |
|
||||
| [windows-native-audio-backend.md](windows-native-audio-backend.md) | Blocked on the libmpv2 swap. ⚠️ suggested id IR-030 is taken. |
|
||||
|
||||
@@ -0,0 +1,242 @@
|
||||
# Spec: Backend-owned stream selection
|
||||
|
||||
**Status:** Proposed
|
||||
**Requirements:** UR-079 (new) → DR-219 … DR-224 (new); **implements and extends
|
||||
DR-121**, currently allocated to
|
||||
[read-through-media-cache.md](read-through-media-cache.md) and not started.
|
||||
Re-check `requirements.md` before allocating — the ids moved twice while this was
|
||||
being written (`DR` max was 215, then 218).
|
||||
**UX spec:** the quality selector in `VideoPlayer.svelte` already exists; this
|
||||
changes what fills it, not how it looks.
|
||||
**Supersedes / revises:** takes DR-121 out of
|
||||
[read-through-media-cache.md](read-through-media-cache.md), which should keep
|
||||
only its capture/eviction half. Unblocks
|
||||
[linux-native-video-spike.md](linux-native-video-spike.md).
|
||||
|
||||
**Destination on completion:**
|
||||
[01-rust-backend.md](../architecture/01-rust-backend.md) — extends the
|
||||
"Streaming quality ladder" section; and
|
||||
[03-data-flow.md](../architecture/03-data-flow.md) — playback initiation. The
|
||||
durable half is the layer line and the `StreamSelection` contract; phases and
|
||||
acceptance criteria are disposable.
|
||||
|
||||
## Summary
|
||||
|
||||
Make Rust the single owner of *which stream to play* — direct play or transcode,
|
||||
at what ceiling, over what transport — and hand every player backend a
|
||||
self-describing selection instead of a bare URL. mpv, ExoPlayer and the HTML5
|
||||
`<video>`/hls.js path all become consumers of the same decision rather than three
|
||||
places that re-derive it.
|
||||
|
||||
Nothing about how playback *looks* changes. What changes is that the frontend
|
||||
stops inferring transport from a URL string, and that direct play becomes
|
||||
possible at all.
|
||||
|
||||
## Motivation
|
||||
|
||||
Four concrete problems, all the same shape.
|
||||
|
||||
**1. The frontend sniffs transport out of the URL.**
|
||||
[VideoPlayer.svelte:569](../../src/lib/components/player/VideoPlayer.svelte#L569):
|
||||
|
||||
```ts
|
||||
const isHlsStream = currentStreamUrl.includes(".m3u8");
|
||||
```
|
||||
|
||||
and again inline at line 2364. Rust *built* that URL and knows exactly what it
|
||||
is; the frontend re-derives it by substring match. Change the endpoint, add a DASH
|
||||
path, serve a progressive file, and this silently picks wrong. This is the
|
||||
boundary rule in miniature — not item-type taxonomy, but the same error: a
|
||||
domain fact reconstructed in the presentation layer because the wire shape did
|
||||
not carry it.
|
||||
|
||||
**2. There is no direct-play path.** `get_video_stream_url` always builds an HLS
|
||||
transcode URL (`TranscodingProtocol=hls`, `VideoCodec=h264` first). Every video
|
||||
play burns server CPU, even when the file would play untouched. This is the cost
|
||||
the Linux native-video work exists to remove, and it cannot be removed without a
|
||||
decision that does not currently exist anywhere in the codebase.
|
||||
|
||||
**3. Quality is a process-wide global.** `streaming_quality()` /
|
||||
`set_streaming_quality()` in `repository/online.rs` read and write a static.
|
||||
It is not per-session or per-item, so it cannot express "this 4K remux needs a
|
||||
ceiling, that podcast does not", and two concurrent playbacks would share one
|
||||
setting.
|
||||
|
||||
**4. Rust cannot say what qualities *this* media source supports.** The selector
|
||||
is populated from a fixed enum rather than from what the source actually offers.
|
||||
DR-121 already names this; it has not been built.
|
||||
|
||||
### The prior question
|
||||
|
||||
Finding 3 of [playback-backend-unification.md](playback-backend-unification.md)
|
||||
holds that hls.js gives us real adaptive bitrate and mpv would lose it. Evidence
|
||||
in this repo suggests **there is no ABR today**: a single rendition is requested,
|
||||
no level-handling code exists anywhere in the frontend, and a quality switch is
|
||||
implemented by re-opening the stream.
|
||||
|
||||
**Run this before sizing the adaptation work.** It needs a live server:
|
||||
|
||||
```
|
||||
curl -s "https://<server>/Videos/<itemId>/master.m3u8?api_key=<key>&…" \
|
||||
| grep -c EXT-X-STREAM-INF
|
||||
```
|
||||
|
||||
`1` → there is no adaptation to preserve, and the adaptation half of this spec
|
||||
collapses to "pick well at open". `>1` → finding 3 stands and DR-223 applies.
|
||||
**Everything else in this spec is worth doing either way** — the ownership
|
||||
problems above are independent of the answer.
|
||||
|
||||
## Layer assignment
|
||||
|
||||
| Logic / responsibility | Layer | Why it belongs there |
|
||||
|---|---|---|
|
||||
| Direct play vs direct stream vs transcode | Rust | Depends on Jellyfin's `PlaybackInfo`, container/codec support and the device profile. Changes when Jellyfin's API or our profile changes → domain, by the litmus test. |
|
||||
| Transport of the chosen stream (HLS / progressive / local file) | Rust | Rust constructs the URL; it is the only place that *knows* rather than infers. Today the frontend guesses from `.m3u8`. |
|
||||
| Which qualities this media source can offer | Rust | Derived from the source's own streams and the quality→transcode-parameter mapping that `get_video_download_url` already holds. DR-121. |
|
||||
| The quality ceiling in force, per playback session | Rust | Domain state that outlives any one view and must survive a backend swap or a mode transfer. Currently a process-wide static. |
|
||||
| Deciding to re-negotiate mid-playback (if adaptation is needed) | Rust | It performs the HTTP and already derives reachability from real traffic via `ConnectivityMonitor`. Throughput estimation is the same pattern on the same data — a side-channel probe would repeat the mistake that principle exists to prevent. |
|
||||
| Frame-level delivery *within* the selected stream, including a player's own ABR | **Player** | ExoPlayer has genuine adaptive selection; if Rust hands it a multi-variant playlist it should use it. Rust chooses *what to request*, never how a player paces bytes. See "The line". |
|
||||
| Rendering the selector, showing the current quality, ordering the list | Frontend | Pure presentation over a backend-supplied list. |
|
||||
| Poster, letterbox, controls, overlay z-order | Frontend | Unchanged. |
|
||||
|
||||
### The line
|
||||
|
||||
**Rust decides *what stream*. The player decides *how to deliver it*.**
|
||||
|
||||
This matters most for ExoPlayer, which already does real adaptive track selection
|
||||
over HLS. This spec must not reimplement that or fight it — if a multi-variant
|
||||
playlist reaches ExoPlayer, ExoPlayer adapts and Rust stays out of the way. The
|
||||
same restraint applies to any future backend that gains the capability. Rust only
|
||||
steps in where the player has no such ability (mpv) *and* the server actually
|
||||
offers a ladder.
|
||||
|
||||
Borderline row, with its tie-breaker: "which media source of a multi-source item"
|
||||
looks like a user choice, and its *presentation* is. The default and the
|
||||
constraint set are domain → **Rust**, per the borderline-defaults-to-Rust rule.
|
||||
|
||||
## Design
|
||||
|
||||
### The contract
|
||||
|
||||
One self-describing selection replaces the bare URL. Nested fields are
|
||||
camelCase over the wire (`#[serde(rename_all = "camelCase")]`); the enums are
|
||||
tagged so the frontend matches a tag instead of parsing a string.
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Type)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct StreamSelection {
|
||||
pub url: String,
|
||||
pub transport: Transport,
|
||||
pub playback_kind: PlaybackKind,
|
||||
/// The negotiated rendition; None when direct-playing the source as-is.
|
||||
pub rendition: Option<Rendition>,
|
||||
/// What this media source can offer — fills the selector (DR-121).
|
||||
pub available: Vec<QualityOption>,
|
||||
}
|
||||
|
||||
#[derive(Serialize, Type)]
|
||||
#[serde(tag = "type", rename_all = "camelCase")]
|
||||
pub enum Transport { Hls, Progressive, LocalFile }
|
||||
|
||||
#[derive(Serialize, Type)]
|
||||
#[serde(tag = "type", rename_all = "camelCase")]
|
||||
pub enum PlaybackKind { DirectPlay, DirectStream, Transcode }
|
||||
```
|
||||
|
||||
`Transport` is the field that deletes the `.m3u8` sniff. The frontend picks
|
||||
hls.js on `Hls` and the element's own loader otherwise — a tag match, not a
|
||||
substring search.
|
||||
|
||||
### Re-negotiation
|
||||
|
||||
Rust emits `stream-selection-changed` (kebab-case, per convention) carrying a new
|
||||
`StreamSelection` plus the position to resume at. The existing
|
||||
`playerSetStreamQuality` response already has exactly the right shape — a tagged
|
||||
`strategy` that tells the caller who reloads, with the backend handling native
|
||||
itself and handing HTML5 a URL for `reloadSource`
|
||||
([index.ts:198](../../src/lib/player/index.ts#L198)). **Extend that; do not
|
||||
invent a second mechanism.** It is the one piece of this that is already right.
|
||||
|
||||
Note the existing wart to preserve or fix deliberately, not accidentally:
|
||||
tauri-specta keeps those response fields snake_case (`new_url`), and the facade
|
||||
comments say so.
|
||||
|
||||
### Phases
|
||||
|
||||
1. **DR-219** `StreamSelection` + `Transport`; delete the `.m3u8` sniff. No
|
||||
behaviour change — pure ownership move, and independently shippable.
|
||||
2. **DR-220** Per-session quality ceiling replacing the `online.rs` static.
|
||||
3. **DR-221** `available` populated from the media source (DR-121's substance).
|
||||
4. **DR-222** Direct-play/direct-stream negotiation via `PlaybackInfo`. This is
|
||||
the phase that unlocks native video and removes the transcode.
|
||||
5. **DR-223** Adaptation, **only if the playlist check says a ladder exists**.
|
||||
Cheapest sufficient design: re-negotiate on sustained throughput drop, reusing
|
||||
the phase-1 re-negotiation path. A local proxy synthesizing a single-variant
|
||||
playlist is a last resort, not a starting point.
|
||||
6. **DR-224** ExoPlayer and mpv consume `StreamSelection` unchanged, proving the
|
||||
contract is player-agnostic rather than HTML5-shaped.
|
||||
|
||||
Phases 1–4 stand on their own merits with no dependency on the ladder question.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Rendering, compositing, and the Linux native-video work itself. This spec
|
||||
unblocks [linux-native-video-spike.md](linux-native-video-spike.md); it does
|
||||
not contain it.
|
||||
- Replacing hls.js. It stays as the HLS loader for the webview path.
|
||||
- Reimplementing or overriding ExoPlayer's own adaptive selection. See "The line".
|
||||
- The download/capture half of [read-through-media-cache.md](read-through-media-cache.md)
|
||||
(DR-122, DR-124, DR-125), which keeps its own spec.
|
||||
- Audio. The same argument applies, but video is where the transcode cost is.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] The `.m3u8` substring check is gone from `VideoPlayer.svelte` (both sites)
|
||||
and transport comes from the tagged enum.
|
||||
- [ ] `bun run check`, `bun run test`, `bun run format:check`, `bun run lint` pass.
|
||||
- [ ] `cargo fmt` clean, `cargo clippy -D warnings` clean, `bun run test:rust` passes.
|
||||
- [ ] `bun run check:boundary` passes — and the reviewer confirms by reading that
|
||||
no transport/kind decision was reconstructed in `src/`, since the tripwire
|
||||
only catches item-type array literals.
|
||||
- [ ] `bindings.ts` regenerated from Rust, not hand-edited.
|
||||
- [ ] New code carries `// TRACES:` comments; `bun run traces:validate` passes and
|
||||
coverage stays ≥ the CI ratchet.
|
||||
- [ ] The `EXT-X-STREAM-INF` count is recorded in this spec before DR-223 is
|
||||
started or dropped.
|
||||
- [ ] DR-121 is removed from `read-through-media-cache.md` with a pointer here.
|
||||
|
||||
## Testing
|
||||
|
||||
- Rust: `PlaybackInfo` fixtures → expected `PlaybackKind`, one per branch
|
||||
(supported container direct-plays; unsupported codec transcodes; a ceiling
|
||||
below the source bitrate transcodes even when the codec is fine).
|
||||
- Rust: `Transport` round-trips through serde with the tag the frontend matches.
|
||||
- Frontend: adapter selection driven by `transport`, including the case a URL
|
||||
ending `.m3u8` is served as `Progressive` — that test fails on today's code,
|
||||
which is the point.
|
||||
- Extend `tauriIntegration.test.ts` for the new command params (camelCase rule).
|
||||
- No test asserts a URL substring.
|
||||
|
||||
## TRACES
|
||||
|
||||
| Piece | Tag |
|
||||
|---|---|
|
||||
| `StreamSelection` / `Transport` | `UR-079 \| DR-219` |
|
||||
| Per-session ceiling | `UR-074 \| DR-220` |
|
||||
| `available` from media source | `UR-079 \| DR-221, DR-121` |
|
||||
| Direct-play negotiation | `UR-079 \| DR-222` |
|
||||
| Adaptation, if built | `UR-079 \| DR-223` |
|
||||
| ExoPlayer/mpv consumers | `UR-003, UR-004 \| DR-224` |
|
||||
|
||||
## Notes for the implementer
|
||||
|
||||
- **Phase 1 is worth doing on its own**, even if everything after it is dropped.
|
||||
It removes a real leak and costs almost nothing.
|
||||
- Do not frame any phase as "no Rust changes required" — that framing is what
|
||||
produced the leak `scoped-search-boundary.md` records.
|
||||
- `ConnectivityMonitor` is the precedent for DR-223: derive network facts from
|
||||
real traffic, never from a side-channel poller.
|
||||
- A parallel Claude session may be active in this repo — `git diff` before
|
||||
"repairing" unexpected changes. Requirement ids in particular moved twice
|
||||
during the writing of this spec.
|
||||
Reference in New Issue
Block a user