docs(specs): mpv draws desktop video, and the webview path goes
The spike proved compositing works on Linux, including Wayland, and left two blockers. One is now closed: DR-228 measured a single EXT-X-STREAM-INF in the server's master playlist, so there is no adaptive bitrate for mpv to lose and finding 3 of playback-backend-unification.md is false. The spike is updated to record that. The other — an unexplained SIGSEGV in a decoder thread — is carried into the spec as DR-231 rather than chased: the spike had no render-context teardown at all, which is DR-184 on Android restated, and removing the likeliest cause is worth doing whether or not it was the cause. The spec targets every desktop platform rather than Linux alone, because the maintenance argument runs the other way. Video has three renderers today. A Linux-only version makes it four, permanently — mpv on Linux, HTML5 on Windows, ExoPlayer on Android, hls.js underneath — and the webview path then survives indefinitely because something still needs it. Finishing the job leaves mpv on desktop and ExoPlayer on Android, and hls.js, html5Adapter.ts, videoLoaderFor and the <video> element are deleted in a phase that has its own acceptance criterion so it cannot quietly become "later". The load-bearing change is DR-233: the device profile stops being a compile-time platform constant and becomes a property of the renderer that will decode the stream. The measured 7% desktop direct-play rate and Android's 85% differ by nothing except which component decodes, so that one change is what converts the former toward the latter. It looks like configuration and is not — it decides whether the server re-encodes, and it fails silently when wrong. Windows is costed rather than waved at: the surface is genuinely different code (WebView2 in an HWND, not GTK), but everything else is shared, so nothing may be guarded on cfg!(target_os = "linux"). The real cost is build — libmpv is a Linux-only dependency while Windows cross-compiles via cargo-xwin, so a Windows libmpv must reach that build and ship in the NSIS bundle under the LGPL terms DR-216 already records. Allocates UR-080, DR-230..236, IR-033. No product code yet.
This commit is contained in:
@@ -44,6 +44,7 @@ taken by other work; each carries a ⚠️ note at the top.
|
||||
|
||||
| Spec | Blocked on / note |
|
||||
|---|---|
|
||||
| [desktop-native-video.md](desktop-native-video.md) | mpv draws video on every desktop platform, then the webview `<video>` path and hls.js are deleted. Converts a measured 7% direct-play rate toward Android's 85%. Stacked on backend-owned stream selection. |
|
||||
| [build-provenance.md](build-provenance.md) | `build.rs` is still bare. ⚠️ suggested id DR-093 is taken. |
|
||||
| [player-facade-enforcement.md](player-facade-enforcement.md) | ~60 `commands.player*` sites still outside the facade; no lint rule. ⚠️ suggested id DR-095 is taken. |
|
||||
| [windows-native-audio-backend.md](windows-native-audio-backend.md) | Blocked on the libmpv2 swap. ⚠️ suggested id IR-030 is taken. |
|
||||
|
||||
@@ -0,0 +1,389 @@
|
||||
# Spec: Desktop native video — mpv renders the picture, everywhere
|
||||
|
||||
**Status:** Proposed
|
||||
**Requirements:** UR-080 (new) → DR-230 … DR-236 (new); IR-033 (new)
|
||||
**UX spec:** n/a — nothing about the player's appearance changes. What changes is
|
||||
what is behind the controls.
|
||||
**Supersedes / revises:** consumes and closes
|
||||
[linux-native-video-spike.md](linux-native-video-spike.md), whose gates
|
||||
authorised exactly this spec and nothing more. Settles finding 2 of
|
||||
[playback-backend-unification.md](playback-backend-unification.md) on the
|
||||
desktop; finding 3 was already settled by DR-228. Absorbs the video half of what
|
||||
[windows-native-audio-backend.md](windows-native-audio-backend.md) leaves open.
|
||||
**Depends on:** backend-owned stream selection (DR-224 … DR-229), the branch
|
||||
below this one. mpv is a *consumer* of `StreamSelection`, never a second place to
|
||||
decide what to play.
|
||||
|
||||
**Destination on completion:**
|
||||
[05-platform-backends.md](../architecture/05-platform-backends.md) — a "Native
|
||||
Video Compositing (Desktop)" section beside the existing Android one, which this
|
||||
mirrors; and [01-rust-backend.md](../architecture/01-rust-backend.md) — the
|
||||
device profile becomes renderer-dependent, beside the stream-selection section.
|
||||
**The spike is deleted in the same commit**, its three traps and its
|
||||
hardware-decode table folded in; they are the durable half.
|
||||
|
||||
## Summary
|
||||
|
||||
mpv decodes and draws video on **every desktop platform**, composited beneath the
|
||||
transparent webview, exactly as Android already does with ExoPlayer. The HTML5
|
||||
`<video>` path and hls.js are then **deleted**, not merely bypassed.
|
||||
|
||||
The user-visible change is that most video stops being re-encoded by the server
|
||||
before it can be watched. The change for whoever maintains this is that video
|
||||
goes from three renderers to two.
|
||||
|
||||
## Motivation
|
||||
|
||||
### The transcode is a decoder constraint, not a rendering one
|
||||
|
||||
Desktop video goes through an h264 HLS transcode because the picture is drawn by
|
||||
a WebKitGTK `<video>` element, and that element decodes little else. The device
|
||||
profile therefore claims `h264` alone. That is not a statement about the machine
|
||||
— the same machine runs mpv, which decodes essentially everything in the library
|
||||
— it is a statement about which widget is holding the frame.
|
||||
|
||||
DR-227 made the cost measurable. Over 40 items negotiated against the development
|
||||
server:
|
||||
|
||||
| Profile | Direct play |
|
||||
|---|---|
|
||||
| Desktop / WebKitGTK — `h264` only, 2ch | **7%** |
|
||||
| Android / ExoPlayer — `h264,hevc,vp8,vp9,av1,mpeg4` + `ac3,eac3`, 6ch | **85%** |
|
||||
|
||||
The sampled library is ~80% hevc. **Those rows differ only by which component
|
||||
decodes.** Moving the picture to mpv is what lets the desktop row claim what the
|
||||
machine can actually do, and that — not the compositing — is the product.
|
||||
|
||||
### One desktop video path, not two
|
||||
|
||||
This is why the spec covers Windows rather than stopping at Linux.
|
||||
|
||||
Today video has **three** renderers: ExoPlayer, the WebKitGTK `<video>` element,
|
||||
and (on Android, via the opt-out) that same element again. A Linux-only version
|
||||
of this work would make it four, permanently: mpv on Linux, HTML5 on Windows,
|
||||
ExoPlayer on Android, plus hls.js underneath the HTML5 one. Every seek strategy,
|
||||
every track switch, every quality change, every lifecycle bug would then have one
|
||||
more place to be got right — and the HTML5 path would survive indefinitely
|
||||
because *something* would still need it.
|
||||
|
||||
Finishing the job removes that: **mpv on desktop, ExoPlayer on Android**, and
|
||||
`hls.js`, `html5Adapter.ts`, `videoLoaderFor` and the webview video element all
|
||||
go. The maintenance win is the reason Windows is in this spec and not in a
|
||||
follow-up that never gets written.
|
||||
|
||||
### Three blockers are gone
|
||||
|
||||
1. **Compositing works, including Wayland.** The spike ran all six gates; the
|
||||
2024 "not possible on Wayland at all" claim is out of date when the render API
|
||||
is used instead of foreign-window embedding.
|
||||
2. **There is no ABR to lose.** DR-228: the server's master playlist carries one
|
||||
`EXT-X-STREAM-INF`. hls.js was demuxing, not adapting.
|
||||
3. **A direct-play path exists.** It did not when the spike was written. DR-227
|
||||
built it; DR-229 proved the contract is player-agnostic.
|
||||
|
||||
And on Windows specifically, `tauri-plugin-libmpv` lists Windows as its **fully
|
||||
tested** platform — the inverse of the Linux situation the spike had to
|
||||
disprove. The embedding difficulty was always WebKitGTK-specific.
|
||||
|
||||
## Layer assignment
|
||||
|
||||
| Logic / responsibility | Layer | Why it belongs there |
|
||||
|---|---|---|
|
||||
| **Which codecs this device can decode** | **Rust** | Domain: it is the input to Jellyfin's `PlaybackInfo` negotiation. It stops being a property of the *platform* and becomes a property of *the renderer in use* — see "The structural change". |
|
||||
| Which backend renders video | **Rust** | Rust already owns this (`use_html5_element` / `VideoBackend`). It stops being a `cfg!` constant and becomes a runtime fact. |
|
||||
| What stream to play (direct / remux / transcode, transport, ceiling) | **Rust — already decided** | DR-224. mpv consumes `StreamSelection`. Re-deriving any of it in a new backend would be the defect DR-224 exists to remove, restated. |
|
||||
| Creating the GL surface, reparenting the webview, owning the render context | **Rust (platform layer)** | Native window and GL-context lifetime. Not presentation, and not expressible above the IPC boundary at all. |
|
||||
| Render-context ↔ GL-context lifetime binding | **Rust** | A correctness invariant over native resources. DR-231. |
|
||||
| Frame pacing (update callback, `report_swap`) | **Rust** | Timing against the compositor; mpv's own contract. |
|
||||
| Hardware-decode selection | **Rust** | A capability question about the machine, answered from what mpv reports it actually selected. |
|
||||
| Z-order of controls over video, overlay chrome, letterbox colour | **Frontend / mpv** | Presentation. Controls already draw over a transparent webview on Android; mpv paints its own letterbox bars (better than the Android equivalent, which shipped DR-194 as a defect). |
|
||||
| Whether the surface is visible right now | **Frontend** | `nativeVideoActive` already exists and toggles `data-native-video`. Unchanged. |
|
||||
|
||||
### The structural change
|
||||
|
||||
Everything above is routine except one row, and it carries the whole benefit.
|
||||
|
||||
`video_codecs` in `build_device_profile` is a **compile-time constant per
|
||||
platform**:
|
||||
|
||||
```rust
|
||||
#[cfg(all(not(target_os = "android"), target_os = "linux"))]
|
||||
let (video_codecs, audio_codecs) = ("h264".to_string(), "aac,mp3,opus,…");
|
||||
```
|
||||
|
||||
That is correct only while a build has exactly one video renderer. It must be
|
||||
derived from **which renderer will decode this stream**, which is runtime state.
|
||||
|
||||
It looks like configuration and is not: it is the input that decides whether the
|
||||
server re-encodes, it changes when Jellyfin's API or our renderer changes, and
|
||||
getting it wrong fails *silently* — a claimed codec the renderer cannot decode is
|
||||
a black picture or silence, which is DR-148 and DR-227's audio override already.
|
||||
|
||||
**Write this against "the active video renderer", never `cfg!(target_os)`.** It
|
||||
is the single piece that must not be Linux-shaped, because phase 2 reuses it
|
||||
unchanged.
|
||||
|
||||
## Design
|
||||
|
||||
### Backend and compositing (DR-230, IR-033)
|
||||
|
||||
An `MpvVideoBackend` beside the existing `MpvBackend` (audio). The mpv side —
|
||||
render context, FBO, update callback, hwdec — is **shared**; only the surface
|
||||
differs per platform:
|
||||
|
||||
| Platform | Surface | Status |
|
||||
|---|---|---|
|
||||
| Linux (X11 + Wayland) | `GtkOverlay`: `GtkGLArea` main child, Tauri's webview reparented as overlay child, via `WebviewWindow::gtk_window()` / `default_vbox()` | Proven by the spike, bar the reparent |
|
||||
| Windows | Native HWND child beneath a transparent WebView2 | Phase 2 |
|
||||
|
||||
`vo=libmpv` plus `mpv_render_context_create` with `MPV_RENDER_PARAM_OPENGL_FBO`.
|
||||
Webview transparency via `with_transparent(true)` — no window-level transparency;
|
||||
the spike showed it is neither used nor needed.
|
||||
|
||||
**The untested half of G1 is the reparent.** The spike built its own overlay; the
|
||||
app must adopt Tauri's. Same widgets, one extra step, but it is the only place
|
||||
Tauri-specific behaviour can still bite — so it is the first gate, proven before
|
||||
anything is built on it.
|
||||
|
||||
Three traps from the spike, each of which cost a debugging cycle and each of
|
||||
which looks like a platform limitation and is not:
|
||||
|
||||
1. **`LC_NUMERIC` must be reset *after* `gtk::init()`.** mpv refuses to start
|
||||
under a non-C numeric locale. `mpv_backend.rs` already handles this but has no
|
||||
GTK init in front of it; here `gtk::init()` applies the user's locale
|
||||
afterwards and `mpv_create` returns null.
|
||||
2. **libepoxy exports GL entry points as *data* symbols.** There is no `glFoo`
|
||||
function — there is `epoxy_glFoo`, a variable holding a lazily-resolving
|
||||
pointer. `get_proc_address` must return the pointer **stored at** that symbol;
|
||||
returning the symbol's own address makes mpv jump into non-executable data and
|
||||
take SIGSEGV on the first GL call. The `epoxy` crate does this correctly but is
|
||||
unusable — its `gl_generator` dependency pulls a yanked `xml-rs`.
|
||||
3. **Frame pacing is not optional and its symptom misleads.** See DR-232.
|
||||
|
||||
### Render-context lifetime (DR-231) — the crash defence
|
||||
|
||||
The spike's one unexplained SIGSEGV landed in a *decoder* thread with no Tauri,
|
||||
GTK or GL frame in the stack, and three plausible causes failed to reproduce it
|
||||
across ~13 minutes of targeted stress.
|
||||
|
||||
What is **not** unexplained is that the spike had no defence: it never calls
|
||||
`mpv_render_context_free` and never tears down on `unrealize`, so nothing stopped
|
||||
the GL context being recreated beneath the render context. That is DR-184 on
|
||||
Android restated — a surface outliving its player.
|
||||
|
||||
Built as a requirement in its own right, not as a fix for a crash we cannot yet
|
||||
reproduce:
|
||||
|
||||
- Render context created on `realize`, freed on `unrealize`, same thread, before
|
||||
the GL context goes away.
|
||||
- The update callback is unregistered **before** the context is freed, so a
|
||||
callback cannot land on a freed context.
|
||||
- Playback teardown and surface teardown are ordered, not racing.
|
||||
|
||||
If the crash recurs after this, it is a different bug and the likeliest cause is
|
||||
out of the search space. If it does not, we needed this anyway.
|
||||
|
||||
### Frame pacing (DR-232)
|
||||
|
||||
Register `mpv_render_context_set_update_callback`; redraw only when it reports a
|
||||
frame ready; call `mpv_render_context_report_swap` after each render.
|
||||
|
||||
Recorded because the failure mode is a trap: driving `queue_render()` off the
|
||||
frame clock every tick without reporting the swap leaves mpv nothing to time
|
||||
against. It looks fine in a window and **judders at fullscreen**, which reads as
|
||||
a compositing or GPU limit and is neither.
|
||||
|
||||
### Renderer-dependent device profile (DR-233)
|
||||
|
||||
`build_device_profile` takes the active video renderer and derives the codec
|
||||
lists from it:
|
||||
|
||||
| Renderer | Video codecs | Audio (video direct play) | Channels |
|
||||
|---|---|---|---|
|
||||
| mpv (desktop native) | `h264,hevc,vp8,vp9,av1,mpeg4` | platform list incl. `ac3,eac3` where the sink can voice it | from the audio route |
|
||||
| WebKitGTK `<video>` | `h264` | webview-decodable set only | 2 |
|
||||
| ExoPlayer (Android) | unchanged | unchanged | unchanged |
|
||||
|
||||
The existing `video_audio_codecs()` narrowing exists because *the webview decodes
|
||||
a narrower audio set than the platform*. With mpv decoding, that no longer
|
||||
applies to the video path — but the multichannel bound still does, since a 5.1
|
||||
track direct-played into a 2-channel sink is silence or inaudible dialogue. Both
|
||||
constraints stay, sourced from the renderer rather than assumed.
|
||||
|
||||
**This converts 7% into ~85%**, and it is also the change most able to break
|
||||
playback silently — so it lands after compositing is proven, covered by the
|
||||
DR-227 override tests.
|
||||
|
||||
### Deleting the webview video path (DR-234)
|
||||
|
||||
`get_player_status` stops reporting `use_html5_element: true` on desktop;
|
||||
`supports_native_video` becomes true there.
|
||||
|
||||
Deletion is staged, because a path cannot be removed while a shipped platform
|
||||
still needs it:
|
||||
|
||||
| Phase | Linux | Windows | HTML5 video path |
|
||||
|---|---|---|---|
|
||||
| 1 | mpv | HTML5 | alive — Windows needs it |
|
||||
| 2 | mpv | mpv | alive but unreached |
|
||||
| 3 | mpv | mpv | **deleted**, with hls.js |
|
||||
|
||||
Phase 3 is a real phase with its own acceptance criterion, not a "later". The
|
||||
whole maintenance argument for including Windows collapses if the fork survives.
|
||||
|
||||
Android keeps ExoPlayer and keeps the webview as its documented opt-out; the
|
||||
`<audio>` element and the background-audio handoff are untouched throughout.
|
||||
|
||||
**What happens when mpv fails to initialise.** With no HTML5 path there is no
|
||||
silent fallback, and inventing one resurrects what we deleted. The
|
||||
graceful-backend-init principle applies as written: fall back to the no-op
|
||||
backend, emit `backend-init-failed`, and surface a real error rather than a black
|
||||
rectangle. An honest failure beats a hidden downgrade to the transcode we are
|
||||
trying to stop paying for.
|
||||
|
||||
### Hardware decode (DR-235)
|
||||
|
||||
The spike established the load-bearing fact: **hardware decode works through the
|
||||
render API** (`hwdec-current` reported `nvdec-copy` on the discrete GPU), so the
|
||||
direct-play prize is not traded for software decoding.
|
||||
|
||||
Policy is decided from what mpv reports it *selected*, never from what it was
|
||||
asked for:
|
||||
|
||||
- Prefer zero-copy VA-API on the integrated GPU where the driver is present.
|
||||
- `auto` reached for the discrete GPU in **copy-back** mode on a hybrid
|
||||
Intel+NVIDIA laptop — the least efficient hardware path — so `auto` is a
|
||||
fallback, not the default.
|
||||
- `vaapi` silently fell back to software on the spike box because `vainfo` was
|
||||
absent. A missing driver must be detected and logged, not mistaken for a
|
||||
compositing limit.
|
||||
- Log `hwdec-current` at start-up; knowing what was actually chosen is the whole
|
||||
diagnostic value.
|
||||
|
||||
### Windows: what phase 2 actually costs (DR-236)
|
||||
|
||||
Not hidden, because it is the part most likely to be underestimated:
|
||||
|
||||
- **The surface is different code.** WebView2 in an HWND, not GTK. A transparent
|
||||
WebView2 over a native child window is a solved arrangement, but DR-230's
|
||||
Linux surface does not transfer. Everything else does.
|
||||
- **libmpv is currently a Linux-only dependency**, and Windows is
|
||||
**cross-compiled from Linux** via `x86_64-pc-windows-msvc` + `cargo-xwin`. Phase
|
||||
2 must source a Windows libmpv (DLL + import library) into that cross-build and
|
||||
ship the DLL in the NSIS bundle.
|
||||
- **LGPL obligations follow the DLL.** DR-216 already records them for Linux:
|
||||
keep the linkage dynamic, ship libmpv's licence text with any bundle carrying
|
||||
it. The Windows bundle inherits both.
|
||||
- **`bun run test:rust` and CI must still build.** Per the CI rule, any tool this
|
||||
needs goes into the builder image and is pushed — never installed at job time.
|
||||
|
||||
Windows also gains a native *audio* decoder as a side effect, which is what
|
||||
[windows-native-audio-backend.md](windows-native-audio-backend.md) wants and
|
||||
cannot currently have. If that spec lands first, phase 2 inherits its build work
|
||||
and shrinks to the surface.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Android.** Unchanged in every respect.
|
||||
- **macOS.** Not a shipped target. If it becomes one it joins phase 2's shape.
|
||||
- **Audio backends.** mpv already plays audio on Linux; this adds a video
|
||||
renderer beside it. Windows audio is its own spec.
|
||||
- **HDR, tone mapping, multi-window.** Not exercised by the spike at all.
|
||||
- **Re-deciding what stream to play.** DR-224 owns that. If this spec finds
|
||||
itself choosing a URL, something has gone wrong.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
**Phase 1 — Linux**
|
||||
|
||||
- [ ] Tauri's own webview reparents into the overlay (the untested half of G1),
|
||||
on X11 **and** Wayland.
|
||||
- [ ] Video plays, seeks and switches audio track in mpv, with the Svelte
|
||||
controls composited over it and alpha blending intact.
|
||||
- [ ] The render context is freed on `unrealize` and the update callback
|
||||
unregistered before the free; a test demonstrates the ordering.
|
||||
- [ ] A direct-play negotiation returns `DirectPlay` for an hevc source that
|
||||
today returns `Transcode`, and it plays.
|
||||
- [ ] Direct-play rate over the same 40-item sample rises from 7% toward the
|
||||
Android figure. **Record the number.**
|
||||
- [ ] mpv init failure emits `backend-init-failed` and surfaces an error rather
|
||||
than falling back to a transcode.
|
||||
- [ ] `hwdec-current` is logged and is not copy-back where zero-copy is available.
|
||||
- [ ] A soak covering seek, track switch and fullscreen runs clean for an agreed
|
||||
duration. **The spike's SIGSEGV is why this is a criterion.**
|
||||
|
||||
**Phase 2 — Windows**
|
||||
|
||||
- [ ] libmpv links in the `cargo-xwin` cross-build; the DLL and its licence ship
|
||||
in the NSIS bundle; any new tool lives in the builder image, not in a CI step.
|
||||
- [ ] Video plays composited under a transparent WebView2.
|
||||
- [ ] The device profile, lifetime and hwdec code are **reused, not
|
||||
reimplemented** — a reviewer confirms no `cfg!(target_os = "linux")` guards
|
||||
them.
|
||||
|
||||
**Phase 3 — deletion**
|
||||
|
||||
- [ ] `use_html5_element` is false on every desktop platform.
|
||||
- [ ] `hls.js` is gone from `package.json`; `html5Adapter.ts`, `videoLoaderFor`
|
||||
and the `<video>` element are deleted; Android's opt-out and the
|
||||
background-audio `<audio>` path still work.
|
||||
|
||||
**Throughout**
|
||||
|
||||
- [ ] `bun run check`, `bun run test`, `bun run format:check`, `bun run lint` pass.
|
||||
- [ ] `cargo fmt` clean, `cargo clippy -D warnings` clean, `bun run test:rust` passes.
|
||||
- [ ] `bun run check:boundary` passes, and a reviewer confirms no stream decision
|
||||
was reconstructed in the new backend.
|
||||
- [ ] `bindings.ts` regenerated from Rust.
|
||||
- [ ] `bun run traces:validate` passes; coverage stays ≥ the CI ratchet.
|
||||
- [ ] The spike and this spec are folded into
|
||||
[05-platform-backends.md](../architecture/05-platform-backends.md) and both
|
||||
deleted in the same commit.
|
||||
|
||||
## Testing
|
||||
|
||||
- **Rust, pure:** the device profile per renderer — mpv claims hevc, the webview
|
||||
does not, the multichannel bound survives both. The DR-233 table as a
|
||||
table-driven test.
|
||||
- **Rust, pure:** `PlaybackInfo` fixtures that transcode under the webview
|
||||
profile and direct-play under the mpv profile — the 7%→85% conversion as a unit
|
||||
test, not only as a measurement.
|
||||
- **Rust:** teardown ordering — callback unregistered before context freed, freed
|
||||
before GL context destroyed. Structure it so the ordering is assertable without
|
||||
a live GL context.
|
||||
- **Frontend:** no desktop path selects an HTML5 video adapter. After phase 3,
|
||||
the adapter does not exist and the test goes with it.
|
||||
- **Manual / soak:** the criterion above. The spike's automated fullscreen and
|
||||
resize soaks are reusable and already written.
|
||||
|
||||
## TRACES
|
||||
|
||||
| Piece | Tag |
|
||||
|---|---|
|
||||
| mpv video backend + compositing | `UR-080 \| DR-230, IR-033` |
|
||||
| Render-context lifetime binding | `UR-080 \| DR-231` |
|
||||
| Frame pacing | `UR-080 \| DR-232` |
|
||||
| Renderer-dependent device profile | `UR-080, UR-070 \| DR-233` |
|
||||
| Webview video path removed | `UR-080 \| DR-234` |
|
||||
| Hardware-decode policy | `UR-080 \| DR-235` |
|
||||
| Windows surface + cross-build | `UR-080 \| DR-236` |
|
||||
|
||||
## Notes for the implementer
|
||||
|
||||
- **Read the spike before writing a line.** Its three traps and its
|
||||
hardware-decode table are the most valuable things in this directory, and each
|
||||
cost a debugging cycle to find.
|
||||
- **mpv consumes `StreamSelection`; it does not decide.** The transport is on the
|
||||
queue item (DR-229). If you are parsing a URL, stop.
|
||||
- **Guard nothing on `cfg!(target_os = "linux")` that phase 2 will need.** That is
|
||||
the one avoidable mistake here.
|
||||
- The Android backend is the reference for the *shape* of this — transparent
|
||||
webview over a native surface at index 0. Read `05-platform-backends.md`'s
|
||||
Android section for what shipped and what its defects were (DR-184 surface
|
||||
lifetime, DR-194 letterbox).
|
||||
- Do not call sync/blocking APIs from mpv event callbacks that can re-enter the
|
||||
player or hold a lock. The existing deadlock gotchas apply.
|
||||
- A parallel Claude session may be active in this repo — `git diff` before
|
||||
"repairing" unexpected changes.
|
||||
- This branch is stacked on backend-owned stream selection. Rebase when that
|
||||
merges rather than merging master into it.
|
||||
@@ -2,7 +2,10 @@
|
||||
|
||||
**Status:** **Run 2026-08-21 — compositing works; G5 carries an open crash.**
|
||||
The compositing claim it set out to test is falsified on Linux. See "Result".
|
||||
This file stays open until the implementation spec exists; ABR is unresolved.
|
||||
This file stays open until the implementation spec exists. **ABR is resolved** —
|
||||
the playlist carries one `EXT-X-STREAM-INF`, so finding 3 is false and there is
|
||||
no adaptation for mpv to lose. The remaining blocker is the unexplained SIGSEGV
|
||||
under G5, which is a lifetime problem, not a compositing one.
|
||||
**Requirements:** none allocated. This spike produces a decision record, not
|
||||
product code — same shape as
|
||||
[playback-backend-unification.md](playback-backend-unification.md), which is
|
||||
@@ -258,10 +261,10 @@ anything.
|
||||
Tauri's existing webview into an overlay. Low risk — the same widgets, one
|
||||
extra reparent — but unproven, and it is the only place Tauri-specific
|
||||
behaviour could still bite.
|
||||
- 🔴 **ABR — finding 3's premise is in doubt.** Finding 3 says mpv would regress
|
||||
streaming quality because "the webview path already has real ABR via hls.js".
|
||||
Three pieces of evidence in this repo suggest that is **not true of the URLs we
|
||||
actually build**:
|
||||
- ✅ **ABR — resolved. Finding 3's premise is false.** Finding 3 said mpv would
|
||||
regress streaming quality because "the webview path already has real ABR via
|
||||
hls.js". Three pieces of evidence in this repo suggested that is **not true of
|
||||
the URLs we actually build**:
|
||||
|
||||
1. `get_video_stream_url` (`repository/online.rs`) requests a *single*
|
||||
rendition — one `VideoBitrate`, one `MaxStreamingBitrate`, one `MaxHeight`.
|
||||
@@ -276,21 +279,48 @@ anything.
|
||||
audio-track switch)". Manual selection by stream re-open is what you build
|
||||
when there is no adaptation, and mpv can do the same thing.
|
||||
|
||||
**The decisive test has not been run** and needs a live server plus an API key:
|
||||
count `#EXT-X-STREAM-INF` lines in a real `master.m3u8`. One line means there
|
||||
is no ABR to lose and this blocker disappears. More than one means finding 3
|
||||
stands and the work below applies.
|
||||
**The decisive test has now been run** (2026-08-21, against the development
|
||||
server, Jellyfin 10.11.5):
|
||||
|
||||
If ABR does turn out to be real, it belongs in **Rust**, not in mpv, and there
|
||||
are three designs in increasing cost: pick the variant at open; re-open at a
|
||||
new bitrate on sustained throughput drops (this is the quality-switch path the
|
||||
app already has, so it is nearly free); or run a local proxy serving mpv a
|
||||
synthesized single-variant playlist while swapping renditions underneath. The
|
||||
middle option is almost certainly sufficient.
|
||||
```
|
||||
curl -s ".../Videos/<itemId>/master.m3u8?…&TranscodingProtocol=hls&…" \
|
||||
| grep -c EXT-X-STREAM-INF
|
||||
1
|
||||
```
|
||||
|
||||
Either way the **direct-play path still does not exist** — every video play
|
||||
currently goes through the HLS transcode endpoint. Building it is the real
|
||||
project; the compositing work proven above is the smaller half.
|
||||
**One line.** The playlist carries a single `EXT-X-STREAM-INF` plus an
|
||||
`EXT-X-IMAGE-STREAM-INF` trickplay entry, which is not a rendition. Jellyfin
|
||||
builds the master playlist from the rendition the request asked for; it does
|
||||
not publish a ladder. So **there is no ABR to lose, and this blocker is
|
||||
closed** — hls.js is serving as an HLS demuxer, exactly as (2) above supposed,
|
||||
and mpv gives up nothing by replacing it.
|
||||
|
||||
Recorded as DR-228 (Won't Do) rather than deleted, because it is a
|
||||
measurement: a server that *does* publish a ladder would change the answer, and
|
||||
the re-negotiation path is the hook that work would build on.
|
||||
|
||||
**The direct-play path now exists.** It did not when this spike was written —
|
||||
every video play went through the HLS transcode endpoint. Backend-owned stream
|
||||
selection (DR-224 … DR-229) built it: Rust negotiates direct play / direct
|
||||
stream / transcode and hands every backend one `StreamSelection` carrying the
|
||||
URL, the transport and the chosen rendition. **That is the contract this
|
||||
implementation consumes** — mpv is a consumer of a decision already made, not a
|
||||
place to re-derive it.
|
||||
|
||||
It also sizes the prize precisely. Measured over the same server, 40 items
|
||||
through a real negotiation per profile:
|
||||
|
||||
| Profile | Direct play |
|
||||
|---|---|
|
||||
| Linux / WebKitGTK — `h264` only, 2ch | **7%** |
|
||||
| Android / ExoPlayer — `h264,hevc,vp8,vp9,av1,mpeg4` + `ac3,eac3`, 6ch | **85%** |
|
||||
|
||||
The library sampled is ~80% hevc. Linux sits at 7% **solely because the
|
||||
WebKitGTK profile can only claim h264** — not because of anything about the
|
||||
server or the negotiation. mpv decodes hevc, so widening the Linux device
|
||||
profile once mpv renders the picture is what converts that 7% toward the
|
||||
Android figure. That conversion is the actual product of this work; the
|
||||
compositing proven above is the mechanism that permits it.
|
||||
- 🔴 **One unexplained SIGSEGV.** A ~180s
|
||||
run died in a *decoder* thread (libavcodec -> `av_log` -> libmpv's log handler
|
||||
-> libc). No Tauri, wry, WebKitGTK, GTK or GL frame appears anywhere in the
|
||||
|
||||
Reference in New Issue
Block a user