feat(player): native video on Linux, and one contract for every player (v0.11.0)

mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.

That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.

Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.

  DR-238/246  a seek routed by the stream's container rather than by what the
              engine could do with it - correct only while one player handled
              those streams, silent the moment another did
  DR-239      a property handled but never observed, so the play/pause button
              waited for an event that could not arrive
  DR-240      fullscreen expanding the document while the window stayed put
  DR-241      a seek issued before the engine had a file, failed, and discarded
              - which is why resume began at zero
  DR-247      a Linux-only gate outliving the caller that made it Linux-only,
              breaking the Android build outright
  DR-250      a stop aimed at whichever renderer bookkeeping believed was in
              charge, missing the one actually making sound
  DR-251      a duration of zero believed, leaving the seek bar no scale
  DR-252      a junk float converted to a Duration, panicking the backend the
              instant a length-less stream appeared

So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.

Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.

Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.

Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.

Squashed from worktree-linux-native-video, which keeps the per-defect history.
This commit is contained in:
2026-08-23 10:51:45 +02:00
parent 5fede123e7
commit 11d9d760d8
87 changed files with 15968 additions and 7508 deletions
+52 -18
View File
@@ -2,7 +2,10 @@
**Status:** **Run 2026-08-21 — compositing works; G5 carries an open crash.**
The compositing claim it set out to test is falsified on Linux. See "Result".
This file stays open until the implementation spec exists; ABR is unresolved.
This file stays open until the implementation spec exists. **ABR is resolved**
the playlist carries one `EXT-X-STREAM-INF`, so finding 3 is false and there is
no adaptation for mpv to lose. The remaining blocker is the unexplained SIGSEGV
under G5, which is a lifetime problem, not a compositing one.
**Requirements:** none allocated. This spike produces a decision record, not
product code — same shape as
[playback-backend-unification.md](playback-backend-unification.md), which is
@@ -258,10 +261,10 @@ anything.
Tauri's existing webview into an overlay. Low risk — the same widgets, one
extra reparent — but unproven, and it is the only place Tauri-specific
behaviour could still bite.
- 🔴 **ABR — finding 3's premise is in doubt.** Finding 3 says mpv would regress
streaming quality because "the webview path already has real ABR via hls.js".
Three pieces of evidence in this repo suggest that is **not true of the URLs we
actually build**:
- **ABR — resolved. Finding 3's premise is false.** Finding 3 said mpv would
regress streaming quality because "the webview path already has real ABR via
hls.js". Three pieces of evidence in this repo suggested that is **not true of
the URLs we actually build**:
1. `get_video_stream_url` (`repository/online.rs`) requests a *single*
rendition — one `VideoBitrate`, one `MaxStreamingBitrate`, one `MaxHeight`.
@@ -276,21 +279,52 @@ anything.
audio-track switch)". Manual selection by stream re-open is what you build
when there is no adaptation, and mpv can do the same thing.
**The decisive test has not been run** and needs a live server plus an API key:
count `#EXT-X-STREAM-INF` lines in a real `master.m3u8`. One line means there
is no ABR to lose and this blocker disappears. More than one means finding 3
stands and the work below applies.
**The decisive test has now been run** (2026-08-21, against the development
server, Jellyfin 10.11.5):
If ABR does turn out to be real, it belongs in **Rust**, not in mpv, and there
are three designs in increasing cost: pick the variant at open; re-open at a
new bitrate on sustained throughput drops (this is the quality-switch path the
app already has, so it is nearly free); or run a local proxy serving mpv a
synthesized single-variant playlist while swapping renditions underneath. The
middle option is almost certainly sufficient.
```
curl -s ".../Videos/<itemId>/master.m3u8?…&TranscodingProtocol=hls&…" \
| grep -c EXT-X-STREAM-INF
1
```
Either way the **direct-play path still does not exist** — every video play
currently goes through the HLS transcode endpoint. Building it is the real
project; the compositing work proven above is the smaller half.
**One line.** The playlist carries a single `EXT-X-STREAM-INF` plus an
`EXT-X-IMAGE-STREAM-INF` trickplay entry, which is not a rendition. Jellyfin
builds the master playlist from the rendition the request asked for; it does
not publish a ladder. So **there is no ABR to lose, and this blocker is
closed** — hls.js is serving as an HLS demuxer, exactly as (2) above supposed,
and mpv gives up nothing by replacing it.
Recorded as DR-229 (Won't Do) rather than deleted, because it is a
measurement: a server that *does* publish a ladder would change the answer, and
the re-negotiation path is the hook that work would build on.
**The direct-play path now exists.** It did not when this spike was written —
every video play went through the HLS transcode endpoint. Backend-owned stream
selection (DR-225 … DR-230) built it: Rust negotiates direct play / direct
stream / transcode and hands every backend one `StreamSelection` carrying the
URL, the transport and the chosen rendition. **That is the contract this
implementation consumes** — mpv is a consumer of a decision already made, not a
place to re-derive it.
It also sizes the prize precisely. Measured over the same server, 40 items
through a real negotiation per profile:
| Profile | Direct play |
|---|---|
| Linux / WebKitGTK — `h264` only, 2ch | **7%** |
| Android / ExoPlayer — `h264,hevc,vp8,vp9,av1,mpeg4` + `ac3,eac3`, 6ch | **85%** |
**The 85% is a ceiling, not a shipped result** — it was measured with a
profile containing `ac3,eac3`, which the Android device later used for
verification does not support.
The library sampled is ~80% hevc. Linux sits at 7% **solely because the
WebKitGTK profile can only claim h264** — not because of anything about the
server or the negotiation. mpv decodes hevc, so widening the Linux device
profile once mpv renders the picture is what converts that 7% toward the
Android figure. That conversion is the actual product of this work; the
compositing proven above is the mechanism that permits it.
- 🔴 **One unexplained SIGSEGV.** A ~180s
run died in a *decoder* thread (libavcodec -> `av_log` -> libmpv's log handler
-> libc). No Tauri, wry, WebKitGTK, GTK or GL frame appears anywhere in the