From 99d96163d8a3f17093b435346dc79c07f4e732a0 Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Fri, 21 Aug 2026 19:41:21 +0200 Subject: [PATCH 1/2] docs(specs): correct the spike's crash and ABR findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three corrections to the Linux native-video spike, each of which reverses something recorded earlier in the same session: - The crash is unexplained. It was first blamed on hwdec=auto-safe's Vulkan failures, on a misreading of the logs — those are two per run at start-up, not per-frame, and every clean run has the same two. A 300s soak on auto-safe survived. So did 240s of automated fullscreen toggling (~120 transitions) and 240s of continuous resizing (~2000 reallocations). Three hypotheses, none reproduced. Recorded rather than dismissed: an intermittent fault nobody can reproduce is worse to inherit than a deterministic one. - G5 drops to amber. It looked and felt smooth, but the only SIGSEGV observed came from the only session in which fullscreen was exercised, and the spike has no lifecycle handling at all — it never frees the render context. An implementation must bind that to the GL context's lifetime regardless of what caused this crash, because Android already paid for that lesson as DR-184. - Finding 3's premise is in doubt. "The webview path already has real ABR via hls.js" was never checked against the URLs this app builds: get_video_stream_url requests a single rendition, the frontend has no level-handling code at all, and a quality switch is implemented by re-opening the stream. If the playlist is single-variant there is no adaptation to lose. The decisive test needs a live server and is recorded as unrun. Also records hardware decode working through the render API (nvdec-copy engaged), and that hwdec=vaapi silently fell back to software on this box. --- docs/specs/linux-native-video-spike.md | 88 ++++++++++++++++++---- docs/specs/playback-backend-unification.md | 11 +++ 2 files changed, 85 insertions(+), 14 deletions(-) diff --git a/docs/specs/linux-native-video-spike.md b/docs/specs/linux-native-video-spike.md index 9ce7bc1d..c7be86f2 100644 --- a/docs/specs/linux-native-video-spike.md +++ b/docs/specs/linux-native-video-spike.md @@ -1,6 +1,6 @@ # Spec: Linux native video — bounded compositing spike -**Status:** **Run 2026-08-21 — G1-G6 green except the Tauri-tree half of G1.** +**Status:** **Run 2026-08-21 — compositing works; G5 carries an open crash.** The compositing claim it set out to test is falsified on Linux. See "Result". This file stays open until the implementation spec exists; ABR is unresolved. **Requirements:** none allocated. This spike produces a decision record, not @@ -186,7 +186,7 @@ mpv's render API with an update callback, frame-gated repaints and | G2 webview paints transparently over it | ✅ green | `with_transparent(true)` alone. No window-level transparency was used or needed. | | G3 mpv renders into our FBO | ✅ green | `vo=libmpv` + `mpv_render_context_create` with `MPV_RENDER_PARAM_OPENGL_FBO` into the FBO GTK binds. | | G4 HTML over video | ✅ green | Opaque panel and a translucent control bar both drew over moving video. | -| G5 resize / drag / fullscreen | ✅ green | No flicker, no gap, no misalignment. Fullscreen juddered until frame pacing was done properly — see trap 3; it is smooth with `report_swap` in place. | +| G5 resize / drag / fullscreen | 🟡 **green on appearance, suspect underneath** | No flicker, gap or misalignment, and smooth once frame pacing was correct (trap 3). But the only crash observed came from the only session where fullscreen was exercised — see "What is still open". | | G6 X11 **and** Wayland | ✅ green | Identical on both; `GDK_BACKEND` flipped between runs. | **Finding 2 of [playback-backend-unification.md](playback-backend-unification.md) @@ -244,9 +244,12 @@ which is the least efficient hardware path. An implementation should evaluate zero-copy VA-API on the iGPU (after confirming the driver is installed) before accepting `auto`. -`hwdec=auto-safe`, the default this spike started with, probes Vulkan video -decode, which this GPU does not support. It failed per-frame and logged -`no frame!` on every frame. Do not ship `auto-safe` here without checking that. +`hwdec=auto-safe` probes Vulkan video decode, which this GPU does not support. +It logs two `Failed setup for format vulkan` / `no frame!` pairs at start-up and +then settles on `nvdec-copy` — the same place `auto` lands. A first reading of +these logs mistook the start-up pair for a per-frame flood; **it is not**. Every +run, clean or crashed, contains exactly two. `auto-safe` is not implicated in +anything. ### What is still open @@ -255,15 +258,72 @@ decode, which this GPU does not support. It failed per-frame and logged Tauri's existing webview into an overlay. Low risk — the same widgets, one extra reparent — but unproven, and it is the only place Tauri-specific behaviour could still bite. -- **ABR — unchanged and still the blocker.** Nothing here addresses finding 3. - What has changed is that the direct-play/transcode split is now worth designing - rather than moot. -- **One unexplained SIGSEGV.** A ~180s run crashed in a *decoder* thread - (libavcodec -> `av_log` -> libmpv's log handler -> libc), not in the GL or - compositing path, while `hwdec=auto-safe` was failing its Vulkan probe on every - frame. It did **not** reproduce across five subsequent runs (2x45s, 3x20s) on - `no`, `auto` and `vaapi`. Cause unconfirmed; recorded rather than dismissed. - Anyone implementing this should run a multi-hour soak before trusting it. +- 🔴 **ABR — finding 3's premise is in doubt.** Finding 3 says mpv would regress + streaming quality because "the webview path already has real ABR via hls.js". + Three pieces of evidence in this repo suggest that is **not true of the URLs we + actually build**: + + 1. `get_video_stream_url` (`repository/online.rs`) requests a *single* + rendition — one `VideoBitrate`, one `MaxStreamingBitrate`, one `MaxHeight`. + Jellyfin transcodes to what it is asked for; it does not build a ladder. + 2. The frontend contains **no level-handling code at all** — no `hls.levels`, + no `LEVEL_SWITCH`, no `currentLevel`. The `abrEwma*` options in + `VideoPlayer.svelte` are default tuning with nothing to act on. hls.js is + serving as an HLS *demuxer* (WebKitGTK cannot play HLS natively), not as an + adaptation engine. + 3. That function's own comment describes a quality switch as **rebuilding the + URL** — "every path that re-opens a stream (quality switch, transcoded seek, + audio-track switch)". Manual selection by stream re-open is what you build + when there is no adaptation, and mpv can do the same thing. + + **The decisive test has not been run** and needs a live server plus an API key: + count `#EXT-X-STREAM-INF` lines in a real `master.m3u8`. One line means there + is no ABR to lose and this blocker disappears. More than one means finding 3 + stands and the work below applies. + + If ABR does turn out to be real, it belongs in **Rust**, not in mpv, and there + are three designs in increasing cost: pick the variant at open; re-open at a + new bitrate on sustained throughput drops (this is the quality-switch path the + app already has, so it is nearly free); or run a local proxy serving mpv a + synthesized single-variant playlist while swapping renditions underneath. The + middle option is almost certainly sufficient. + + Either way the **direct-play path still does not exist** — every video play + currently goes through the HLS transcode endpoint. Building it is the real + project; the compositing work proven above is the smaller half. +- 🔴 **One unexplained SIGSEGV.** A ~180s + run died in a *decoder* thread (libavcodec -> `av_log` -> libmpv's log handler + -> libc). No Tauri, wry, WebKitGTK, GTK or GL frame appears anywhere in the + stack, so the fault is on the mpv/ffmpeg side of the process rather than in the + compositing seam. + + Three hypotheses were tested and **none reproduced it**: + + | Hypothesis | Test | Result | + |---|---|---| + | `hwdec=auto-safe`'s Vulkan failures | 300s soak on `auto-safe` | Survived. Also based on a misreading — the failures are 2 per run at start-up, not per-frame. Dead. | + | Fullscreen transitions recreating the GL context under mpv's render context | 240s soak, ~120 automated transitions | Survived, no core dumped. | + | Continuous resize thrashing the GL framebuffer | 240s soak, ~2000 resizes | Survived, no core dumped. | + + **The crash is therefore unexplained.** It was observed exactly once, in the + only session a human interacted with, and did not recur in ~13 minutes of + targeted stress across the three most plausible causes. It is recorded here + rather than dismissed precisely because nothing explains it: an intermittent + fault that nobody can reproduce is worse to inherit than a deterministic one, + not better. + + The underlying concern stands regardless of which test eventually reproduces + it. A SIGSEGV in an unrelated thread is characteristic of memory corruption, + and this spike never calls `mpv_render_context_free` and never tears down on + `unrealize` — it has no defence against the GL context being recreated beneath + the render context. That is DR-184 on Android restated: a surface outliving its + player. An implementation must bind the two lifetimes together whether or not + this particular crash is ever explained. + + **Therefore G5 is recorded green on appearance only**, and this crash is the + single largest piece of unfinished business in the spike. Do not read the green + gates above as "safe to build on" until it is explained or a long soak clears + it. - Long-run stability, seeking, track switching, HDR, and multi-window were not exercised at all. diff --git a/docs/specs/playback-backend-unification.md b/docs/specs/playback-backend-unification.md index 130179cd..8aafb5e3 100644 --- a/docs/specs/playback-backend-unification.md +++ b/docs/specs/playback-backend-unification.md @@ -111,6 +111,17 @@ The webview path already has real ABR via hls.js. Moving video to mpv would be a **downgrade** on every platform — no graceful degradation on weak networks, and quality changes requiring teardown and reload. +> **Premise in doubt (2026-08-21).** "The webview path already has real ABR" +> was not verified against the URLs this app actually builds. +> `get_video_stream_url` requests a *single* rendition (one `VideoBitrate`, one +> `MaxHeight`), the frontend has **no** level-handling code (`hls.levels`, +> `LEVEL_SWITCH`, `currentLevel` appear nowhere), and this repo implements a +> quality switch by *re-opening the stream* — all of which point to a +> single-variant playlist, i.e. no ABR to lose. The decisive test is counting +> `#EXT-X-STREAM-INF` lines in a real `master.m3u8`; it needs a live server and +> has not been run. See +> [linux-native-video-spike.md](linux-native-video-spike.md). + ### 4. Crossfade is architecturally blocked on mpv mpv's audio chain is single-stream. FFmpeg's `acrossfade` is an `N→A` filter -- 2.39.5 From 1d56517f07426a9f6691f531a6b43b8574e43f63 Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Fri, 21 Aug 2026 19:41:32 +0200 Subject: [PATCH 2/2] docs(specs): add backend-owned stream selection MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rust becomes the single owner of which stream to play — direct play or transcode, at what ceiling, over what transport — and hands every player backend a self-describing StreamSelection instead of a bare URL. mpv, ExoPlayer and the HTML5/hls.js path all consume one decision rather than three places re-deriving it. The motivating leak is concrete. VideoPlayer.svelte determines transport with `currentStreamUrl.includes(".m3u8")`, in two places, for a URL Rust constructed and therefore already knows the shape of. That is the boundary rule in miniature: not item-type taxonomy, but the same error of reconstructing a domain fact in the presentation layer because the wire shape did not carry it. A tagged Transport enum deletes it. The design line, which ExoPlayer forces: Rust decides *what stream*, the player decides *how to deliver it*. ExoPlayer has genuine adaptive track selection; this spec must not reimplement or fight it. Rust only adapts where the player cannot (mpv) and the server actually offers a ladder. Six phases, and phase 1 stands alone as pure ownership movement with no behaviour change. Phase 4 (direct-play negotiation) is what removes the transcode and unblocks the Linux native-video work. Phase 5 (adaptation) is gated on counting EXT-X-STREAM-INF entries in a real playlist — the acceptance criteria require that count be recorded before it is either started or dropped. Takes DR-121 from read-through-media-cache.md, which specced Rust-owned quality reporting but never built it; that spec keeps its capture half. --- docs-site/SUMMARY.md | 1 + docs/specs/README.md | 1 + docs/specs/backend-owned-stream-selection.md | 242 +++++++++++++++++++ 3 files changed, 244 insertions(+) create mode 100644 docs/specs/backend-owned-stream-selection.md diff --git a/docs-site/SUMMARY.md b/docs-site/SUMMARY.md index ce86b824..45a207be 100644 --- a/docs-site/SUMMARY.md +++ b/docs-site/SUMMARY.md @@ -33,6 +33,7 @@ - [Spec Review Checklist](specs/SPEC-REVIEW-CHECKLIST.md) - [Playback Backend Unification](specs/playback-backend-unification.md) - [Linux Native Video Spike](specs/linux-native-video-spike.md) +- [Backend-Owned Stream Selection](specs/backend-owned-stream-selection.md) - [Player Facade Enforcement](specs/player-facade-enforcement.md) - [Windows Native Audio Backend](specs/windows-native-audio-backend.md) - [libmpv2 Migration](specs/libmpv2-migration.md) diff --git a/docs/specs/README.md b/docs/specs/README.md index 4ac015a9..9c4eac56 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -44,6 +44,7 @@ taken by other work; each carries a ⚠️ note at the top. | Spec | Blocked on / note | |---|---| +| [backend-owned-stream-selection.md](backend-owned-stream-selection.md) | Rust owns direct-play-vs-transcode, transport and quality; players consume one `StreamSelection`. Phase 1 (delete the `.m3u8` sniff) stands alone. Unblocks Linux native video. | | [build-provenance.md](build-provenance.md) | `build.rs` is still bare. ⚠️ suggested id DR-093 is taken. | | [player-facade-enforcement.md](player-facade-enforcement.md) | ~60 `commands.player*` sites still outside the facade; no lint rule. ⚠️ suggested id DR-095 is taken. | | [windows-native-audio-backend.md](windows-native-audio-backend.md) | Blocked on the libmpv2 swap. ⚠️ suggested id IR-030 is taken. | diff --git a/docs/specs/backend-owned-stream-selection.md b/docs/specs/backend-owned-stream-selection.md new file mode 100644 index 00000000..a32391d8 --- /dev/null +++ b/docs/specs/backend-owned-stream-selection.md @@ -0,0 +1,242 @@ +# Spec: Backend-owned stream selection + +**Status:** Proposed +**Requirements:** UR-079 (new) → DR-219 … DR-224 (new); **implements and extends +DR-121**, currently allocated to +[read-through-media-cache.md](read-through-media-cache.md) and not started. +Re-check `requirements.md` before allocating — the ids moved twice while this was +being written (`DR` max was 215, then 218). +**UX spec:** the quality selector in `VideoPlayer.svelte` already exists; this +changes what fills it, not how it looks. +**Supersedes / revises:** takes DR-121 out of +[read-through-media-cache.md](read-through-media-cache.md), which should keep +only its capture/eviction half. Unblocks +[linux-native-video-spike.md](linux-native-video-spike.md). + +**Destination on completion:** +[01-rust-backend.md](../architecture/01-rust-backend.md) — extends the +"Streaming quality ladder" section; and +[03-data-flow.md](../architecture/03-data-flow.md) — playback initiation. The +durable half is the layer line and the `StreamSelection` contract; phases and +acceptance criteria are disposable. + +## Summary + +Make Rust the single owner of *which stream to play* — direct play or transcode, +at what ceiling, over what transport — and hand every player backend a +self-describing selection instead of a bare URL. mpv, ExoPlayer and the HTML5 +`