Merge pull request 'Chore/infra hardening' (#15) from chore/infra-hardening into master
Publish Documentation / Build & publish docs to gitea-pages (push) Successful in 5m22s
Traceability Validation / Check Requirement Traces (push) Successful in 12s

Reviewed-on: #15
This commit was merged in pull request #15.
This commit is contained in:
2026-08-21 17:44:48 +00:00
5 changed files with 329 additions and 14 deletions
+1
View File
@@ -33,6 +33,7 @@
- [Spec Review Checklist](specs/SPEC-REVIEW-CHECKLIST.md) - [Spec Review Checklist](specs/SPEC-REVIEW-CHECKLIST.md)
- [Playback Backend Unification](specs/playback-backend-unification.md) - [Playback Backend Unification](specs/playback-backend-unification.md)
- [Linux Native Video Spike](specs/linux-native-video-spike.md) - [Linux Native Video Spike](specs/linux-native-video-spike.md)
- [Backend-Owned Stream Selection](specs/backend-owned-stream-selection.md)
- [Player Facade Enforcement](specs/player-facade-enforcement.md) - [Player Facade Enforcement](specs/player-facade-enforcement.md)
- [Windows Native Audio Backend](specs/windows-native-audio-backend.md) - [Windows Native Audio Backend](specs/windows-native-audio-backend.md)
- [libmpv2 Migration](specs/libmpv2-migration.md) - [libmpv2 Migration](specs/libmpv2-migration.md)
+1
View File
@@ -44,6 +44,7 @@ taken by other work; each carries a ⚠️ note at the top.
| Spec | Blocked on / note | | Spec | Blocked on / note |
|---|---| |---|---|
| [backend-owned-stream-selection.md](backend-owned-stream-selection.md) | Rust owns direct-play-vs-transcode, transport and quality; players consume one `StreamSelection`. Phase 1 (delete the `.m3u8` sniff) stands alone. Unblocks Linux native video. |
| [build-provenance.md](build-provenance.md) | `build.rs` is still bare. ⚠️ suggested id DR-093 is taken. | | [build-provenance.md](build-provenance.md) | `build.rs` is still bare. ⚠️ suggested id DR-093 is taken. |
| [player-facade-enforcement.md](player-facade-enforcement.md) | ~60 `commands.player*` sites still outside the facade; no lint rule. ⚠️ suggested id DR-095 is taken. | | [player-facade-enforcement.md](player-facade-enforcement.md) | ~60 `commands.player*` sites still outside the facade; no lint rule. ⚠️ suggested id DR-095 is taken. |
| [windows-native-audio-backend.md](windows-native-audio-backend.md) | Blocked on the libmpv2 swap. ⚠️ suggested id IR-030 is taken. | | [windows-native-audio-backend.md](windows-native-audio-backend.md) | Blocked on the libmpv2 swap. ⚠️ suggested id IR-030 is taken. |
@@ -0,0 +1,242 @@
# Spec: Backend-owned stream selection
**Status:** Proposed
**Requirements:** UR-079 (new) → DR-219 … DR-224 (new); **implements and extends
DR-121**, currently allocated to
[read-through-media-cache.md](read-through-media-cache.md) and not started.
Re-check `requirements.md` before allocating — the ids moved twice while this was
being written (`DR` max was 215, then 218).
**UX spec:** the quality selector in `VideoPlayer.svelte` already exists; this
changes what fills it, not how it looks.
**Supersedes / revises:** takes DR-121 out of
[read-through-media-cache.md](read-through-media-cache.md), which should keep
only its capture/eviction half. Unblocks
[linux-native-video-spike.md](linux-native-video-spike.md).
**Destination on completion:**
[01-rust-backend.md](../architecture/01-rust-backend.md) — extends the
"Streaming quality ladder" section; and
[03-data-flow.md](../architecture/03-data-flow.md) — playback initiation. The
durable half is the layer line and the `StreamSelection` contract; phases and
acceptance criteria are disposable.
## Summary
Make Rust the single owner of *which stream to play* — direct play or transcode,
at what ceiling, over what transport — and hand every player backend a
self-describing selection instead of a bare URL. mpv, ExoPlayer and the HTML5
`<video>`/hls.js path all become consumers of the same decision rather than three
places that re-derive it.
Nothing about how playback *looks* changes. What changes is that the frontend
stops inferring transport from a URL string, and that direct play becomes
possible at all.
## Motivation
Four concrete problems, all the same shape.
**1. The frontend sniffs transport out of the URL.**
[VideoPlayer.svelte:569](../../src/lib/components/player/VideoPlayer.svelte#L569):
```ts
const isHlsStream = currentStreamUrl.includes(".m3u8");
```
and again inline at line 2364. Rust *built* that URL and knows exactly what it
is; the frontend re-derives it by substring match. Change the endpoint, add a DASH
path, serve a progressive file, and this silently picks wrong. This is the
boundary rule in miniature — not item-type taxonomy, but the same error: a
domain fact reconstructed in the presentation layer because the wire shape did
not carry it.
**2. There is no direct-play path.** `get_video_stream_url` always builds an HLS
transcode URL (`TranscodingProtocol=hls`, `VideoCodec=h264` first). Every video
play burns server CPU, even when the file would play untouched. This is the cost
the Linux native-video work exists to remove, and it cannot be removed without a
decision that does not currently exist anywhere in the codebase.
**3. Quality is a process-wide global.** `streaming_quality()` /
`set_streaming_quality()` in `repository/online.rs` read and write a static.
It is not per-session or per-item, so it cannot express "this 4K remux needs a
ceiling, that podcast does not", and two concurrent playbacks would share one
setting.
**4. Rust cannot say what qualities *this* media source supports.** The selector
is populated from a fixed enum rather than from what the source actually offers.
DR-121 already names this; it has not been built.
### The prior question
Finding 3 of [playback-backend-unification.md](playback-backend-unification.md)
holds that hls.js gives us real adaptive bitrate and mpv would lose it. Evidence
in this repo suggests **there is no ABR today**: a single rendition is requested,
no level-handling code exists anywhere in the frontend, and a quality switch is
implemented by re-opening the stream.
**Run this before sizing the adaptation work.** It needs a live server:
```
curl -s "https://<server>/Videos/<itemId>/master.m3u8?api_key=<key>&…" \
| grep -c EXT-X-STREAM-INF
```
`1` → there is no adaptation to preserve, and the adaptation half of this spec
collapses to "pick well at open". `>1` → finding 3 stands and DR-223 applies.
**Everything else in this spec is worth doing either way** — the ownership
problems above are independent of the answer.
## Layer assignment
| Logic / responsibility | Layer | Why it belongs there |
|---|---|---|
| Direct play vs direct stream vs transcode | Rust | Depends on Jellyfin's `PlaybackInfo`, container/codec support and the device profile. Changes when Jellyfin's API or our profile changes → domain, by the litmus test. |
| Transport of the chosen stream (HLS / progressive / local file) | Rust | Rust constructs the URL; it is the only place that *knows* rather than infers. Today the frontend guesses from `.m3u8`. |
| Which qualities this media source can offer | Rust | Derived from the source's own streams and the quality→transcode-parameter mapping that `get_video_download_url` already holds. DR-121. |
| The quality ceiling in force, per playback session | Rust | Domain state that outlives any one view and must survive a backend swap or a mode transfer. Currently a process-wide static. |
| Deciding to re-negotiate mid-playback (if adaptation is needed) | Rust | It performs the HTTP and already derives reachability from real traffic via `ConnectivityMonitor`. Throughput estimation is the same pattern on the same data — a side-channel probe would repeat the mistake that principle exists to prevent. |
| Frame-level delivery *within* the selected stream, including a player's own ABR | **Player** | ExoPlayer has genuine adaptive selection; if Rust hands it a multi-variant playlist it should use it. Rust chooses *what to request*, never how a player paces bytes. See "The line". |
| Rendering the selector, showing the current quality, ordering the list | Frontend | Pure presentation over a backend-supplied list. |
| Poster, letterbox, controls, overlay z-order | Frontend | Unchanged. |
### The line
**Rust decides *what stream*. The player decides *how to deliver it*.**
This matters most for ExoPlayer, which already does real adaptive track selection
over HLS. This spec must not reimplement that or fight it — if a multi-variant
playlist reaches ExoPlayer, ExoPlayer adapts and Rust stays out of the way. The
same restraint applies to any future backend that gains the capability. Rust only
steps in where the player has no such ability (mpv) *and* the server actually
offers a ladder.
Borderline row, with its tie-breaker: "which media source of a multi-source item"
looks like a user choice, and its *presentation* is. The default and the
constraint set are domain → **Rust**, per the borderline-defaults-to-Rust rule.
## Design
### The contract
One self-describing selection replaces the bare URL. Nested fields are
camelCase over the wire (`#[serde(rename_all = "camelCase")]`); the enums are
tagged so the frontend matches a tag instead of parsing a string.
```rust
#[derive(Serialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct StreamSelection {
pub url: String,
pub transport: Transport,
pub playback_kind: PlaybackKind,
/// The negotiated rendition; None when direct-playing the source as-is.
pub rendition: Option<Rendition>,
/// What this media source can offer — fills the selector (DR-121).
pub available: Vec<QualityOption>,
}
#[derive(Serialize, Type)]
#[serde(tag = "type", rename_all = "camelCase")]
pub enum Transport { Hls, Progressive, LocalFile }
#[derive(Serialize, Type)]
#[serde(tag = "type", rename_all = "camelCase")]
pub enum PlaybackKind { DirectPlay, DirectStream, Transcode }
```
`Transport` is the field that deletes the `.m3u8` sniff. The frontend picks
hls.js on `Hls` and the element's own loader otherwise — a tag match, not a
substring search.
### Re-negotiation
Rust emits `stream-selection-changed` (kebab-case, per convention) carrying a new
`StreamSelection` plus the position to resume at. The existing
`playerSetStreamQuality` response already has exactly the right shape — a tagged
`strategy` that tells the caller who reloads, with the backend handling native
itself and handing HTML5 a URL for `reloadSource`
([index.ts:198](../../src/lib/player/index.ts#L198)). **Extend that; do not
invent a second mechanism.** It is the one piece of this that is already right.
Note the existing wart to preserve or fix deliberately, not accidentally:
tauri-specta keeps those response fields snake_case (`new_url`), and the facade
comments say so.
### Phases
1. **DR-219** `StreamSelection` + `Transport`; delete the `.m3u8` sniff. No
behaviour change — pure ownership move, and independently shippable.
2. **DR-220** Per-session quality ceiling replacing the `online.rs` static.
3. **DR-221** `available` populated from the media source (DR-121's substance).
4. **DR-222** Direct-play/direct-stream negotiation via `PlaybackInfo`. This is
the phase that unlocks native video and removes the transcode.
5. **DR-223** Adaptation, **only if the playlist check says a ladder exists**.
Cheapest sufficient design: re-negotiate on sustained throughput drop, reusing
the phase-1 re-negotiation path. A local proxy synthesizing a single-variant
playlist is a last resort, not a starting point.
6. **DR-224** ExoPlayer and mpv consume `StreamSelection` unchanged, proving the
contract is player-agnostic rather than HTML5-shaped.
Phases 14 stand on their own merits with no dependency on the ladder question.
## Out of scope
- Rendering, compositing, and the Linux native-video work itself. This spec
unblocks [linux-native-video-spike.md](linux-native-video-spike.md); it does
not contain it.
- Replacing hls.js. It stays as the HLS loader for the webview path.
- Reimplementing or overriding ExoPlayer's own adaptive selection. See "The line".
- The download/capture half of [read-through-media-cache.md](read-through-media-cache.md)
(DR-122, DR-124, DR-125), which keeps its own spec.
- Audio. The same argument applies, but video is where the transcode cost is.
## Acceptance criteria
- [ ] The `.m3u8` substring check is gone from `VideoPlayer.svelte` (both sites)
and transport comes from the tagged enum.
- [ ] `bun run check`, `bun run test`, `bun run format:check`, `bun run lint` pass.
- [ ] `cargo fmt` clean, `cargo clippy -D warnings` clean, `bun run test:rust` passes.
- [ ] `bun run check:boundary` passes — and the reviewer confirms by reading that
no transport/kind decision was reconstructed in `src/`, since the tripwire
only catches item-type array literals.
- [ ] `bindings.ts` regenerated from Rust, not hand-edited.
- [ ] New code carries `// TRACES:` comments; `bun run traces:validate` passes and
coverage stays ≥ the CI ratchet.
- [ ] The `EXT-X-STREAM-INF` count is recorded in this spec before DR-223 is
started or dropped.
- [ ] DR-121 is removed from `read-through-media-cache.md` with a pointer here.
## Testing
- Rust: `PlaybackInfo` fixtures → expected `PlaybackKind`, one per branch
(supported container direct-plays; unsupported codec transcodes; a ceiling
below the source bitrate transcodes even when the codec is fine).
- Rust: `Transport` round-trips through serde with the tag the frontend matches.
- Frontend: adapter selection driven by `transport`, including the case a URL
ending `.m3u8` is served as `Progressive` — that test fails on today's code,
which is the point.
- Extend `tauriIntegration.test.ts` for the new command params (camelCase rule).
- No test asserts a URL substring.
## TRACES
| Piece | Tag |
|---|---|
| `StreamSelection` / `Transport` | `UR-079 \| DR-219` |
| Per-session ceiling | `UR-074 \| DR-220` |
| `available` from media source | `UR-079 \| DR-221, DR-121` |
| Direct-play negotiation | `UR-079 \| DR-222` |
| Adaptation, if built | `UR-079 \| DR-223` |
| ExoPlayer/mpv consumers | `UR-003, UR-004 \| DR-224` |
## Notes for the implementer
- **Phase 1 is worth doing on its own**, even if everything after it is dropped.
It removes a real leak and costs almost nothing.
- Do not frame any phase as "no Rust changes required" — that framing is what
produced the leak `scoped-search-boundary.md` records.
- `ConnectivityMonitor` is the precedent for DR-223: derive network facts from
real traffic, never from a side-channel poller.
- A parallel Claude session may be active in this repo — `git diff` before
"repairing" unexpected changes. Requirement ids in particular moved twice
during the writing of this spec.
+74 -14
View File
@@ -1,6 +1,6 @@
# Spec: Linux native video — bounded compositing spike # Spec: Linux native video — bounded compositing spike
**Status:** **Run 2026-08-21 — G1-G6 green except the Tauri-tree half of G1.** **Status:** **Run 2026-08-21 — compositing works; G5 carries an open crash.**
The compositing claim it set out to test is falsified on Linux. See "Result". The compositing claim it set out to test is falsified on Linux. See "Result".
This file stays open until the implementation spec exists; ABR is unresolved. This file stays open until the implementation spec exists; ABR is unresolved.
**Requirements:** none allocated. This spike produces a decision record, not **Requirements:** none allocated. This spike produces a decision record, not
@@ -186,7 +186,7 @@ mpv's render API with an update callback, frame-gated repaints and
| G2 webview paints transparently over it | ✅ green | `with_transparent(true)` alone. No window-level transparency was used or needed. | | G2 webview paints transparently over it | ✅ green | `with_transparent(true)` alone. No window-level transparency was used or needed. |
| G3 mpv renders into our FBO | ✅ green | `vo=libmpv` + `mpv_render_context_create` with `MPV_RENDER_PARAM_OPENGL_FBO` into the FBO GTK binds. | | G3 mpv renders into our FBO | ✅ green | `vo=libmpv` + `mpv_render_context_create` with `MPV_RENDER_PARAM_OPENGL_FBO` into the FBO GTK binds. |
| G4 HTML over video | ✅ green | Opaque panel and a translucent control bar both drew over moving video. | | G4 HTML over video | ✅ green | Opaque panel and a translucent control bar both drew over moving video. |
| G5 resize / drag / fullscreen | green | No flicker, no gap, no misalignment. Fullscreen juddered until frame pacing was done properly — see trap 3; it is smooth with `report_swap` in place. | | G5 resize / drag / fullscreen | 🟡 **green on appearance, suspect underneath** | No flicker, gap or misalignment, and smooth once frame pacing was correct (trap 3). But the only crash observed came from the only session where fullscreen was exercised — see "What is still open". |
| G6 X11 **and** Wayland | ✅ green | Identical on both; `GDK_BACKEND` flipped between runs. | | G6 X11 **and** Wayland | ✅ green | Identical on both; `GDK_BACKEND` flipped between runs. |
**Finding 2 of [playback-backend-unification.md](playback-backend-unification.md) **Finding 2 of [playback-backend-unification.md](playback-backend-unification.md)
@@ -244,9 +244,12 @@ which is the least efficient hardware path. An implementation should evaluate
zero-copy VA-API on the iGPU (after confirming the driver is installed) before zero-copy VA-API on the iGPU (after confirming the driver is installed) before
accepting `auto`. accepting `auto`.
`hwdec=auto-safe`, the default this spike started with, probes Vulkan video `hwdec=auto-safe` probes Vulkan video decode, which this GPU does not support.
decode, which this GPU does not support. It failed per-frame and logged It logs two `Failed setup for format vulkan` / `no frame!` pairs at start-up and
`no frame!` on every frame. Do not ship `auto-safe` here without checking that. then settles on `nvdec-copy` — the same place `auto` lands. A first reading of
these logs mistook the start-up pair for a per-frame flood; **it is not**. Every
run, clean or crashed, contains exactly two. `auto-safe` is not implicated in
anything.
### What is still open ### What is still open
@@ -255,15 +258,72 @@ decode, which this GPU does not support. It failed per-frame and logged
Tauri's existing webview into an overlay. Low risk — the same widgets, one Tauri's existing webview into an overlay. Low risk — the same widgets, one
extra reparent — but unproven, and it is the only place Tauri-specific extra reparent — but unproven, and it is the only place Tauri-specific
behaviour could still bite. behaviour could still bite.
- **ABR — unchanged and still the blocker.** Nothing here addresses finding 3. - 🔴 **ABR — finding 3's premise is in doubt.** Finding 3 says mpv would regress
What has changed is that the direct-play/transcode split is now worth designing streaming quality because "the webview path already has real ABR via hls.js".
rather than moot. Three pieces of evidence in this repo suggest that is **not true of the URLs we
- **One unexplained SIGSEGV.** A ~180s run crashed in a *decoder* thread actually build**:
(libavcodec -> `av_log` -> libmpv's log handler -> libc), not in the GL or
compositing path, while `hwdec=auto-safe` was failing its Vulkan probe on every 1. `get_video_stream_url` (`repository/online.rs`) requests a *single*
frame. It did **not** reproduce across five subsequent runs (2x45s, 3x20s) on rendition — one `VideoBitrate`, one `MaxStreamingBitrate`, one `MaxHeight`.
`no`, `auto` and `vaapi`. Cause unconfirmed; recorded rather than dismissed. Jellyfin transcodes to what it is asked for; it does not build a ladder.
Anyone implementing this should run a multi-hour soak before trusting it. 2. The frontend contains **no level-handling code at all** — no `hls.levels`,
no `LEVEL_SWITCH`, no `currentLevel`. The `abrEwma*` options in
`VideoPlayer.svelte` are default tuning with nothing to act on. hls.js is
serving as an HLS *demuxer* (WebKitGTK cannot play HLS natively), not as an
adaptation engine.
3. That function's own comment describes a quality switch as **rebuilding the
URL** — "every path that re-opens a stream (quality switch, transcoded seek,
audio-track switch)". Manual selection by stream re-open is what you build
when there is no adaptation, and mpv can do the same thing.
**The decisive test has not been run** and needs a live server plus an API key:
count `#EXT-X-STREAM-INF` lines in a real `master.m3u8`. One line means there
is no ABR to lose and this blocker disappears. More than one means finding 3
stands and the work below applies.
If ABR does turn out to be real, it belongs in **Rust**, not in mpv, and there
are three designs in increasing cost: pick the variant at open; re-open at a
new bitrate on sustained throughput drops (this is the quality-switch path the
app already has, so it is nearly free); or run a local proxy serving mpv a
synthesized single-variant playlist while swapping renditions underneath. The
middle option is almost certainly sufficient.
Either way the **direct-play path still does not exist** — every video play
currently goes through the HLS transcode endpoint. Building it is the real
project; the compositing work proven above is the smaller half.
- 🔴 **One unexplained SIGSEGV.** A ~180s
run died in a *decoder* thread (libavcodec -> `av_log` -> libmpv's log handler
-> libc). No Tauri, wry, WebKitGTK, GTK or GL frame appears anywhere in the
stack, so the fault is on the mpv/ffmpeg side of the process rather than in the
compositing seam.
Three hypotheses were tested and **none reproduced it**:
| Hypothesis | Test | Result |
|---|---|---|
| `hwdec=auto-safe`'s Vulkan failures | 300s soak on `auto-safe` | Survived. Also based on a misreading — the failures are 2 per run at start-up, not per-frame. Dead. |
| Fullscreen transitions recreating the GL context under mpv's render context | 240s soak, ~120 automated transitions | Survived, no core dumped. |
| Continuous resize thrashing the GL framebuffer | 240s soak, ~2000 resizes | Survived, no core dumped. |
**The crash is therefore unexplained.** It was observed exactly once, in the
only session a human interacted with, and did not recur in ~13 minutes of
targeted stress across the three most plausible causes. It is recorded here
rather than dismissed precisely because nothing explains it: an intermittent
fault that nobody can reproduce is worse to inherit than a deterministic one,
not better.
The underlying concern stands regardless of which test eventually reproduces
it. A SIGSEGV in an unrelated thread is characteristic of memory corruption,
and this spike never calls `mpv_render_context_free` and never tears down on
`unrealize` — it has no defence against the GL context being recreated beneath
the render context. That is DR-184 on Android restated: a surface outliving its
player. An implementation must bind the two lifetimes together whether or not
this particular crash is ever explained.
**Therefore G5 is recorded green on appearance only**, and this crash is the
single largest piece of unfinished business in the spike. Do not read the green
gates above as "safe to build on" until it is explained or a long soak clears
it.
- Long-run stability, seeking, track switching, HDR, and multi-window were not - Long-run stability, seeking, track switching, HDR, and multi-window were not
exercised at all. exercised at all.
@@ -111,6 +111,17 @@ The webview path already has real ABR via hls.js. Moving video to mpv would be a
**downgrade** on every platform — no graceful degradation on weak networks, and **downgrade** on every platform — no graceful degradation on weak networks, and
quality changes requiring teardown and reload. quality changes requiring teardown and reload.
> **Premise in doubt (2026-08-21).** "The webview path already has real ABR"
> was not verified against the URLs this app actually builds.
> `get_video_stream_url` requests a *single* rendition (one `VideoBitrate`, one
> `MaxHeight`), the frontend has **no** level-handling code (`hls.levels`,
> `LEVEL_SWITCH`, `currentLevel` appear nowhere), and this repo implements a
> quality switch by *re-opening the stream* — all of which point to a
> single-variant playlist, i.e. no ABR to lose. The decisive test is counting
> `#EXT-X-STREAM-INF` lines in a real `master.m3u8`; it needs a live server and
> has not been run. See
> [linux-native-video-spike.md](linux-native-video-spike.md).
### 4. Crossfade is architecturally blocked on mpv ### 4. Crossfade is architecturally blocked on mpv
mpv's audio chain is single-stream. FFmpeg's `acrossfade` is an `N→A` filter mpv's audio chain is single-stream. FFmpeg's `acrossfade` is an `N→A` filter