Master allocated DR-224 and UT-211 while this branch was in flight — the third collision on this work. Everything here moves up by one: DR-224..236 become DR-225..237, UT-211..213 become UT-212..214. UR-079, UR-080 and IR-033 were still free and are unchanged. Mechanical, and matched on each row's own text rather than on its number, so a row cannot be shifted twice or the wrong one caught. Master's DR-224 (the background-audio toggle) and UT-211 are untouched.
504 lines
28 KiB
Markdown
504 lines
28 KiB
Markdown
# Spec: Linux native video — bounded compositing spike
|
||
|
||
**Status:** **Run 2026-08-21 — compositing works; G5 carries an open crash.**
|
||
The compositing claim it set out to test is falsified on Linux. See "Result".
|
||
This file stays open until the implementation spec exists. **ABR is resolved** —
|
||
the playlist carries one `EXT-X-STREAM-INF`, so finding 3 is false and there is
|
||
no adaptation for mpv to lose. The remaining blocker is the unexplained SIGSEGV
|
||
under G5, which is a lifetime problem, not a compositing one.
|
||
**Requirements:** none allocated. This spike produces a decision record, not
|
||
product code — same shape as
|
||
[playback-backend-unification.md](playback-backend-unification.md), which is
|
||
Accepted with no requirement ids of its own. Ids are allocated by the
|
||
*implementation* spec that follows a green result.
|
||
**UX spec:** n/a
|
||
**Supersedes / revises:** re-opens finding 2 of
|
||
[playback-backend-unification.md](playback-backend-unification.md) on Linux only.
|
||
Its findings 3, 4, 5 and 6 stand unchallenged and are **not** in scope here.
|
||
|
||
**Destination on completion:**
|
||
[05-platform-backends.md](../architecture/05-platform-backends.md) — a "Native
|
||
Video Compositing (Linux)" section alongside the existing Android one. The
|
||
durable half is the mechanism and the two traps below; the gates and phases are
|
||
disposable.
|
||
|
||
## Summary
|
||
|
||
Test one falsifiable claim: *a native video surface cannot be composited with a
|
||
Tauri webview on Linux.* The claim is load-bearing — it is why Linux video goes
|
||
through an h264 HLS transcode into a WebKitGTK `<video>` element instead of
|
||
decoding directly in the mpv instance we already run. The spike renders one mpv
|
||
frame beneath the webview, on both X11 and Wayland, and stops. It ships no
|
||
product code and flips no defaults.
|
||
|
||
A green result does **not** authorise native video on Linux; it authorises
|
||
writing the spec that would.
|
||
|
||
## Motivation
|
||
|
||
[playback-backend-unification.md](playback-backend-unification.md) finding 2
|
||
concluded that native video cannot be composited with a Tauri webview, on
|
||
evidence from `tauri-plugin-libmpv`'s platform table, wry#284, tauri#6343, and a
|
||
Tauri maintainer's 2024 statement that a GTK widget as a child X11 window is
|
||
"a bit hacky and it is not possible on Wayland at all."
|
||
|
||
Two things have changed since that was written, and one thing was never tested.
|
||
|
||
**1. The general claim has already been falsified on one platform — by us.**
|
||
Android now renders ExoPlayer video on a TextureView at index 0 *behind a
|
||
transparent Tauri WebView*, with the Svelte controls drawn over it, on by
|
||
default. See
|
||
[05-platform-backends.md](../architecture/05-platform-backends.md#native-video-compositing-android).
|
||
That is exactly the composition finding 2 said was impossible, shipped. What
|
||
survives of the finding is a narrower, WebKitGTK-specific claim — which is worth
|
||
testing on its own terms rather than inheriting.
|
||
|
||
**2. A Tauri app now ships Linux native mpv as an active platform.**
|
||
[MaxVideoPlayer](https://github.com/MaxMB15/MaxVideoPlayer) (354 commits) embeds
|
||
libmpv via **EGL + X11 child window / Wayland subsurface**, with Linux and macOS
|
||
active and Windows only planned — the inverse of the plugin matrix finding 2
|
||
sampled. Its existence does not prove our case works, but it does mean the
|
||
Wayland half of the maintainer quote is out of date.
|
||
|
||
**3. The render API was never tested.** Every source in finding 2 describes
|
||
*foreign-window embedding*: `--wid`, child windows, a second toplevel
|
||
position-synced to a `getBoundingClientRect()` div. That is a different mechanism
|
||
from mpv's render API, where **we** own the GL context and mpv draws into an FBO
|
||
we hand it (`mpv_render_context_create` / `mpv_render_context_render`, with an
|
||
upstream [GTK example](https://github.com/mpv-player/mpv-examples/pull/44/files)).
|
||
Tauri v2 exposes `WebviewWindow::gtk_window()` and `default_vbox()`, so the
|
||
target is a widget inside Tauri's own GTK tree — not a foreign window, not a
|
||
second toplevel, and therefore not the thing that was found broken.
|
||
|
||
The prize is direct play: no h264 transcode, hardware decode, libass subtitles,
|
||
and no server CPU burned on every Linux play.
|
||
|
||
## The blocker a green spike does not clear
|
||
|
||
🔴 **Read this before treating a green result as a green light.**
|
||
|
||
Finding 3 of the unification spec stands: **mpv has no adaptive bitrate.** It
|
||
delegates HLS to FFmpeg's demuxer, which picks one variant at open and never
|
||
adapts. The webview path has real ABR via hls.js. Compositing is necessary for
|
||
native video on Linux; it is not sufficient.
|
||
|
||
There is a plausible answer, and this spike exists partly to make it testable:
|
||
**ABR only matters on the transcode path.** A direct-played file has no variant
|
||
ladder to adapt between — the adaptation the server offers *is* the transcode.
|
||
So "mpv when the stream is direct-play, HTML5 + hls.js when the server
|
||
transcodes" would sidestep finding 3 rather than fight it, and it maps onto a
|
||
decision Rust already makes when it builds the stream URL.
|
||
|
||
That is a **hypothesis, not a conclusion.** It is out of scope here. Record it in
|
||
the spike's decision note so the follow-up spec starts from it.
|
||
|
||
## Layer assignment
|
||
|
||
The spike introduces no product logic. The table below is the assignment the
|
||
*follow-up* would inherit, written now so a green result cannot drift into
|
||
frontend decisions during implementation.
|
||
|
||
| Logic / responsibility | Layer | Why it belongs there |
|
||
|------------------------|-------|----------------------|
|
||
| Which backend renders video on this platform (`use_html5_element`, `supports_native_video`) | Rust | Already there — `get_player_status` in `commands/player/mod.rs` computes it from a `cfg!`. The spike would widen that `cfg!`, not relocate the decision. The frontend already consumes it via `createAdapter`. |
|
||
| Whether *this stream* is direct-play or transcoded, and therefore whether mpv or hls.js renders it | Rust | Domain. It depends on Jellyfin's `PlaybackInfo` response, container/codec support, and the bitrate cap — all of which change when Jellyfin's API or our quality ladder changes. The frontend must never re-derive it from a URL shape. |
|
||
| Creating, sizing, and destroying the GL surface; the mpv render context | Rust | Owns the backend and the GTK window handle. There is no presentation decision in it. |
|
||
| Where controls, subtitles, and the mini-player sit above the video, and the letterbox/poster treatment | Frontend | Pure presentation; changes only if the UI is redesigned. Precisely the split the Android path already uses. |
|
||
| Reserving the video rectangle in layout and marking the shell transparent | Frontend | Presentation. `nativeVideo.ts` + the `[data-native-video="active"]` rule in `app.css` already do this for Android and are platform-agnostic. |
|
||
|
||
Borderline row, stated with its tie-breaker: *"is the surface currently
|
||
attached?"* reads like view state, but the Android work found that a surface left
|
||
in the hierarchy outlives its player (DR-184). Attachment is backend lifecycle →
|
||
**Rust**, with the frontend told about it, not asked.
|
||
|
||
## Design
|
||
|
||
A throwaway branch. No merge to `master` except the decision note.
|
||
|
||
### What gets built
|
||
|
||
One `#[cfg(target_os = "linux")]` experiment behind a feature flag, in a scratch
|
||
binary or an ignored test — **not** in `MpvBackend`'s constructor path:
|
||
|
||
1. From `app.get_webview_window(...)`, take `gtk_window()` and `default_vbox()`.
|
||
2. Reparent the webview into a `gtk::Overlay`: `GLArea` as the main child, the
|
||
webview as the overlay child.
|
||
3. Set the webview background to fully transparent (wry does this when
|
||
`"transparent": true`; verify it reaches `webkit_web_view_set_background_color`).
|
||
4. In the `GLArea`'s `render` signal, drive
|
||
`mpv_render_context_render` with `MPV_RENDER_PARAM_OPENGL_FBO` pointing at the
|
||
FBO GTK bound for us.
|
||
5. Play one local file. Draw an opaque HTML element over the video area.
|
||
|
||
`video = no` and `audio-display = no` are set in
|
||
[mpv_backend.rs:135-141](../../src-tauri/src/player/mpv_backend.rs#L135-L141);
|
||
the spike overrides them on its own `Mpv` handle rather than editing that path.
|
||
|
||
### Bindings
|
||
|
||
The current pin is `libmpv = { git = "…/libmpv-rs", branch = "master" }` — the
|
||
dead pin [libmpv2-migration.md](libmpv2-migration.md) exists to replace. The
|
||
render API lives in `libmpv2-sys` (`mpv_render_context_render`); the safe wrapper
|
||
was only ever a PR against the old crate. **Use `libmpv2-sys` raw FFI directly in
|
||
the spike.** Do not block the spike on the migration, and do not let the spike
|
||
half-perform it — if the spike goes green the migration becomes a hard
|
||
prerequisite of the implementation, which is the ordering
|
||
[windows-native-audio-backend.md](windows-native-audio-backend.md) already sits
|
||
in.
|
||
|
||
### IPC
|
||
|
||
None. The spike crosses no boundary. If it goes green, the follow-up changes only
|
||
the *value* of the existing `useHtml5Element` / `supportsNativeVideo` fields — no
|
||
new wire shapes, no `bindings.ts` regeneration.
|
||
|
||
## Gates
|
||
|
||
Each is pass/fail with a named failure. Stop at the first red and write it up —
|
||
a red result is a successful spike.
|
||
|
||
| # | Question | Fails if |
|
||
|---|---|---|
|
||
| G1 | Can a custom GTK widget join Tauri's widget tree and survive the window's lifetime? | `default_vbox()` is absent/unusable, or reparenting the webview breaks input or crashes. |
|
||
| G2 | Does the webview still paint, with a transparent backdrop, over that widget? | The backdrop renders opaque black ([wry#1540](https://github.com/tauri-apps/wry/issues/1540)) or the webview stops repainting ([tauri#12800](https://github.com/tauri-apps/tauri/issues/12800)). **This is the highest-risk gate.** |
|
||
| G3 | Does mpv render a frame into our FBO? | The render context refuses GTK's context, or frames land in the wrong buffer. |
|
||
| G4 | Does HTML drawn over the video area actually appear over it? | Video covers the controls — the exact failure wry#284 and tauri#6343 report. Without this, the whole thing is worthless: our controls, subtitles and mini-player all sit over the video. |
|
||
| G5 | Does it survive resize, fullscreen, and SPA navigation away and back? | Flicker on resize, or a surface that outlives its route. |
|
||
| G6 | Does it hold on **both** X11 and Wayland? | Either session backend fails. Wayland is the one the 2024 maintainer quote says is impossible — test it first, not last. |
|
||
|
||
G6 is not a nice-to-have. A result that only holds on X11 is red for a project
|
||
shipping to current desktops.
|
||
|
||
### Time box
|
||
|
||
If G1–G4 are not all green, stop and write the result up. The value of this spike
|
||
is a dated, method-specific answer — including "still no, and here is the
|
||
mechanism" — not a working player.
|
||
|
||
## Result (2026-08-21)
|
||
|
||
Run on GNOME, kernel 7.1.8, libmpv 2.5.0 (mpv 0.41.0), GTK 3.24.52, WebKitGTK
|
||
2.52.6, wry 0.53.5 — the versions `src-tauri/Cargo.lock` resolves. Spike source:
|
||
a ~250-line standalone crate using wry + gtk + `libmpv2-sys` raw FFI, driving
|
||
mpv's render API with an update callback, frame-gated repaints and
|
||
`report_swap`.
|
||
|
||
| Gate | Result | Observed mechanism |
|
||
|---|---|---|
|
||
| G1 widget in GTK tree | 🟡 **partial** | `GtkOverlay` with `GtkGLArea` as main child and the wry webview as overlay child works, built directly. **Tauri's own `default_vbox()` was not exercised** — see below. |
|
||
| G2 webview paints transparently over it | ✅ green | `with_transparent(true)` alone. No window-level transparency was used or needed. |
|
||
| G3 mpv renders into our FBO | ✅ green | `vo=libmpv` + `mpv_render_context_create` with `MPV_RENDER_PARAM_OPENGL_FBO` into the FBO GTK binds. |
|
||
| G4 HTML over video | ✅ green | Opaque panel and a translucent control bar both drew over moving video. |
|
||
| G5 resize / drag / fullscreen | 🟡 **green on appearance, suspect underneath** | No flicker, gap or misalignment, and smooth once frame pacing was correct (trap 3). But the only crash observed came from the only session where fullscreen was exercised — see "What is still open". |
|
||
| G6 X11 **and** Wayland | ✅ green | Identical on both; `GDK_BACKEND` flipped between runs. |
|
||
|
||
**Finding 2 of [playback-backend-unification.md](playback-backend-unification.md)
|
||
is false on Linux** when tested by the render API rather than by foreign-window
|
||
embedding. Wayland — the half the 2024 maintainer quote called impossible — is
|
||
green.
|
||
|
||
Better than the gate asked for: the translucent bar composited *alpha* against
|
||
the video, not merely opaque-over. Scrims, gradient fades and subtitle backdrops
|
||
therefore work, which is most of how a player UI actually looks. mpv also painted
|
||
the letterbox bars black on its own — the Android equivalent was a shipped defect
|
||
(DR-194).
|
||
|
||
### Three traps, each of which cost a debugging cycle
|
||
|
||
Carry these into the implementation; each produced a failure that looked like a
|
||
platform limitation and was not.
|
||
|
||
1. **`LC_NUMERIC` must be reset *after* `gtk::init()`, not before.** mpv refuses
|
||
to start under a non-C numeric locale. `mpv_backend.rs` already handles this,
|
||
but it has no GTK init in front of it; on this path `gtk::init()` applies the
|
||
user's locale afterwards and `mpv_create` returns null.
|
||
2. **libepoxy exports GL entry points as *data* symbols.** There is no `glFoo`
|
||
function to resolve — there is `epoxy_glFoo`, a variable holding a lazily
|
||
resolving function pointer. `get_proc_address` must return the pointer *stored
|
||
at* that symbol; returning the symbol's own address makes mpv jump into
|
||
non-executable data and take SIGSEGV/SEGV_ACCERR on the first GL call. The
|
||
`epoxy` crate does this correctly but is unusable — its `gl_generator`
|
||
dependency pulls a yanked `xml-rs`.
|
||
3. **Frame pacing is not optional, and its symptom is misleading.** Driving
|
||
`queue_render()` off the widget's frame clock on every tick, without calling
|
||
`mpv_render_context_report_swap` after each render, leaves mpv with nothing to
|
||
time against. Playback looks fine in a window and **judders at fullscreen** —
|
||
which reads as a compositing or GPU limit and is neither. The fix is to
|
||
register `mpv_render_context_set_update_callback`, redraw only when it says a
|
||
frame is ready, and report the swap afterwards. Fullscreen was smooth
|
||
immediately once both were in place.
|
||
|
||
### Hardware decode through the render API
|
||
|
||
Tested by asking mpv what it actually selected (`hwdec-current`), not what it was
|
||
asked for. All three ran 20s clean at a steady 30 fps.
|
||
|
||
| `hwdec` | `hwdec-current` | Note |
|
||
|---|---|---|
|
||
| `vaapi` | `no` | **Did not engage** on this box — silently fell back to software. `vainfo` is not installed, so the libva driver for the Iris Xe iGPU is likely absent. No render-API error; this looks like a missing driver package, not a compositing limit. |
|
||
| `auto` | `nvdec-copy` | Hardware decode **does** work through the render API, on the discrete RTX 3050. Copy-back rather than zero-copy interop. |
|
||
| `no` | `no` | Software. Clean baseline. |
|
||
|
||
The load-bearing result is the middle row: **hardware decode is compatible with
|
||
mpv's render API**, so the direct-play prize is real and not traded away for
|
||
software decoding. Which decoder to prefer is an implementation question — on a
|
||
hybrid Intel+NVIDIA laptop `auto` reached for the discrete GPU in copy-back mode,
|
||
which is the least efficient hardware path. An implementation should evaluate
|
||
zero-copy VA-API on the iGPU (after confirming the driver is installed) before
|
||
accepting `auto`.
|
||
|
||
`hwdec=auto-safe` probes Vulkan video decode, which this GPU does not support.
|
||
It logs two `Failed setup for format vulkan` / `no frame!` pairs at start-up and
|
||
then settles on `nvdec-copy` — the same place `auto` lands. A first reading of
|
||
these logs mistook the start-up pair for a per-frame flood; **it is not**. Every
|
||
run, clean or crashed, contains exactly two. `auto-safe` is not implicated in
|
||
anything.
|
||
|
||
### What is still open
|
||
|
||
- **The Tauri half of G1.** The spike built its own `GtkOverlay`. The app must
|
||
instead reach `WebviewWindow::gtk_window()` / `default_vbox()` and reparent
|
||
Tauri's existing webview into an overlay. Low risk — the same widgets, one
|
||
extra reparent — but unproven, and it is the only place Tauri-specific
|
||
behaviour could still bite.
|
||
- ✅ **ABR — resolved. Finding 3's premise is false.** Finding 3 said mpv would
|
||
regress streaming quality because "the webview path already has real ABR via
|
||
hls.js". Three pieces of evidence in this repo suggested that is **not true of
|
||
the URLs we actually build**:
|
||
|
||
1. `get_video_stream_url` (`repository/online.rs`) requests a *single*
|
||
rendition — one `VideoBitrate`, one `MaxStreamingBitrate`, one `MaxHeight`.
|
||
Jellyfin transcodes to what it is asked for; it does not build a ladder.
|
||
2. The frontend contains **no level-handling code at all** — no `hls.levels`,
|
||
no `LEVEL_SWITCH`, no `currentLevel`. The `abrEwma*` options in
|
||
`VideoPlayer.svelte` are default tuning with nothing to act on. hls.js is
|
||
serving as an HLS *demuxer* (WebKitGTK cannot play HLS natively), not as an
|
||
adaptation engine.
|
||
3. That function's own comment describes a quality switch as **rebuilding the
|
||
URL** — "every path that re-opens a stream (quality switch, transcoded seek,
|
||
audio-track switch)". Manual selection by stream re-open is what you build
|
||
when there is no adaptation, and mpv can do the same thing.
|
||
|
||
**The decisive test has now been run** (2026-08-21, against the development
|
||
server, Jellyfin 10.11.5):
|
||
|
||
```
|
||
curl -s ".../Videos/<itemId>/master.m3u8?…&TranscodingProtocol=hls&…" \
|
||
| grep -c EXT-X-STREAM-INF
|
||
1
|
||
```
|
||
|
||
**One line.** The playlist carries a single `EXT-X-STREAM-INF` plus an
|
||
`EXT-X-IMAGE-STREAM-INF` trickplay entry, which is not a rendition. Jellyfin
|
||
builds the master playlist from the rendition the request asked for; it does
|
||
not publish a ladder. So **there is no ABR to lose, and this blocker is
|
||
closed** — hls.js is serving as an HLS demuxer, exactly as (2) above supposed,
|
||
and mpv gives up nothing by replacing it.
|
||
|
||
Recorded as DR-229 (Won't Do) rather than deleted, because it is a
|
||
measurement: a server that *does* publish a ladder would change the answer, and
|
||
the re-negotiation path is the hook that work would build on.
|
||
|
||
**The direct-play path now exists.** It did not when this spike was written —
|
||
every video play went through the HLS transcode endpoint. Backend-owned stream
|
||
selection (DR-225 … DR-230) built it: Rust negotiates direct play / direct
|
||
stream / transcode and hands every backend one `StreamSelection` carrying the
|
||
URL, the transport and the chosen rendition. **That is the contract this
|
||
implementation consumes** — mpv is a consumer of a decision already made, not a
|
||
place to re-derive it.
|
||
|
||
It also sizes the prize precisely. Measured over the same server, 40 items
|
||
through a real negotiation per profile:
|
||
|
||
| Profile | Direct play |
|
||
|---|---|
|
||
| Linux / WebKitGTK — `h264` only, 2ch | **7%** |
|
||
| Android / ExoPlayer — `h264,hevc,vp8,vp9,av1,mpeg4` + `ac3,eac3`, 6ch | **85%** |
|
||
|
||
**The 85% is a ceiling, not a shipped result** — it was measured with a
|
||
profile containing `ac3,eac3`, which the Android device later used for
|
||
verification does not support.
|
||
|
||
The library sampled is ~80% hevc. Linux sits at 7% **solely because the
|
||
WebKitGTK profile can only claim h264** — not because of anything about the
|
||
server or the negotiation. mpv decodes hevc, so widening the Linux device
|
||
profile once mpv renders the picture is what converts that 7% toward the
|
||
Android figure. That conversion is the actual product of this work; the
|
||
compositing proven above is the mechanism that permits it.
|
||
- 🔴 **One unexplained SIGSEGV.** A ~180s
|
||
run died in a *decoder* thread (libavcodec -> `av_log` -> libmpv's log handler
|
||
-> libc). No Tauri, wry, WebKitGTK, GTK or GL frame appears anywhere in the
|
||
stack, so the fault is on the mpv/ffmpeg side of the process rather than in the
|
||
compositing seam.
|
||
|
||
Three hypotheses were tested and **none reproduced it**:
|
||
|
||
| Hypothesis | Test | Result |
|
||
|---|---|---|
|
||
| `hwdec=auto-safe`'s Vulkan failures | 300s soak on `auto-safe` | Survived. Also based on a misreading — the failures are 2 per run at start-up, not per-frame. Dead. |
|
||
| Fullscreen transitions recreating the GL context under mpv's render context | 240s soak, ~120 automated transitions | Survived, no core dumped. |
|
||
| Continuous resize thrashing the GL framebuffer | 240s soak, ~2000 resizes | Survived, no core dumped. |
|
||
|
||
**The crash is therefore unexplained.** It was observed exactly once, in the
|
||
only session a human interacted with, and did not recur in ~13 minutes of
|
||
targeted stress across the three most plausible causes. It is recorded here
|
||
rather than dismissed precisely because nothing explains it: an intermittent
|
||
fault that nobody can reproduce is worse to inherit than a deterministic one,
|
||
not better.
|
||
|
||
The underlying concern stands regardless of which test eventually reproduces
|
||
it. A SIGSEGV in an unrelated thread is characteristic of memory corruption,
|
||
and this spike never calls `mpv_render_context_free` and never tears down on
|
||
`unrealize` — it has no defence against the GL context being recreated beneath
|
||
the render context. That is DR-184 on Android restated: a surface outliving its
|
||
player. An implementation must bind the two lifetimes together whether or not
|
||
this particular crash is ever explained.
|
||
|
||
**Therefore G5 is recorded green on appearance only**, and this crash is the
|
||
single largest piece of unfinished business in the spike. Do not read the green
|
||
gates above as "safe to build on" until it is explained or a long soak clears
|
||
it.
|
||
- Long-run stability, seeking, track switching, HDR, and multi-window were not
|
||
exercised at all.
|
||
|
||
## Out of scope
|
||
|
||
- Any change to the shipping Linux video path. `experimentalNativeVideo` in
|
||
`adapters/index.ts` is a **suppressor, never a promoter**; the spike must not
|
||
change that.
|
||
- Adaptive bitrate. See "The blocker a green spike does not clear".
|
||
- Windows and macOS — different mechanisms, and **Windows is the easier case, not
|
||
the endangered one**. See below.
|
||
- Android. Already shipped; it is the precedent, not the target.
|
||
- Crossfade, the libmpv2 migration, and the audio-parity work.
|
||
|
||
### Why Windows is unaffected, and cheaper
|
||
|
||
Nothing here can regress Windows. `use_html5_element` is already a per-platform
|
||
`cfg!` in `get_player_status` — Android native, everything else HTML5 — so
|
||
divergent video paths are the existing design rather than something this
|
||
introduces. Windows keeps `<video>` + hls.js whatever this spike returns.
|
||
|
||
The mechanism does not port: `default_vbox()`, `GtkOverlay` and `GtkGLArea` are
|
||
GTK3/WebKitGTK concepts. But the *question* is already answered more favourably
|
||
there. Both mpv plugins list Windows as **fully tested** and Linux as broken,
|
||
because WebView2 honours a transparent background — the "native surface beneath a
|
||
transparent webview" approach that fails on WebKitGTK is the one that works on
|
||
Windows. That asymmetry is why
|
||
[windows-native-audio-backend.md](windows-native-audio-backend.md) can call
|
||
Windows "the cleanest available win".
|
||
|
||
Windows' cost is packaging, not compositing: the build cross-compiles with MSVC +
|
||
`cargo-xwin`, so libmpv arrives as a bundled prebuilt DLL (the ⚠️ in finding 5's
|
||
comparison table). That cost is already committed for *audio*. Once the DLL ships
|
||
to replace `WebviewAudioBackend`, Windows video is largely a follow-on.
|
||
|
||
Sequencing, if native video is ever pursued on both:
|
||
|
||
1. [libmpv2-migration.md](libmpv2-migration.md) — prerequisite for either.
|
||
2. [windows-native-audio-backend.md](windows-native-audio-backend.md) — already
|
||
specced; lands the DLL and a real Windows backend.
|
||
3. Windows native video — cheap once 2 exists, and does not need this spike.
|
||
4. Linux native video — needs this spike, and runs independently of 1–3.
|
||
|
||
### Does this add a backend?
|
||
|
||
No — and the trajectory is convergence, not proliferation.
|
||
|
||
`create_player_backend` in `lib.rs` already selects between four
|
||
`PlayerBackend` impls by `cfg!`: `MpvBackend` (Linux), `ExoPlayerBackend`
|
||
(Android), `WebviewAudioBackend` (Windows and anything else), and `NullBackend`
|
||
as the graceful-init fallback. The HTML5 video path is not among them — it is a
|
||
frontend adapter reporting through `player_report_*`, not a `PlayerBackend`.
|
||
|
||
This spike adds none of these. `MpvBackend` already exists and already runs on
|
||
Linux; it merely sets `video = no` at construction. Giving it video widens an
|
||
existing backend rather than introducing an engine.
|
||
|
||
Following the sequence above, the count goes **down**: replacing
|
||
`WebviewAudioBackend` with mpv on Windows leaves two native engines — mpv
|
||
(Linux + Windows) and ExoPlayer (Android) — with native video riding on both.
|
||
|
||
Two is the floor, for a reason worth stating so nobody re-litigates it: Android
|
||
cannot drop ExoPlayer even if libmpv runs there, because the foreground service,
|
||
`MediaSessionCompat` and lockscreen control are built on it (finding 7 puts the
|
||
cost at that rewrite, not at the bindings). The HTML5 path does not go away
|
||
either — it is the transcode/ABR route and the fallback.
|
||
|
||
The trait surface converges too: `ExoPlayerBackend` already implements the
|
||
video-surface lifecycle for Android native compositing, so teaching `MpvBackend`
|
||
video follows a path already walked rather than opening a second one.
|
||
- Adopting `tauri-plugin-libmpv` or `tauri-plugin-mpv` as dependencies. Both
|
||
report Linux window embedding as not working and are small projects
|
||
(20 and ~70 commits); read them, do not depend on them.
|
||
|
||
## Acceptance criteria
|
||
|
||
The deliverable is a decision, not a feature.
|
||
|
||
- [ ] Each of G1–G6 recorded green/red **with the observed mechanism**, not just
|
||
the verdict.
|
||
- [ ] X11 and Wayland results reported separately, each naming the compositor
|
||
and WebKitGTK version tested.
|
||
- [ ] The direct-play/transcode ABR hypothesis recorded as open, with whatever
|
||
the spike learned about it.
|
||
- [ ] `docs/specs/README.md` updated — this spec listed, and its row moved or
|
||
deleted per the result.
|
||
- [ ] The Linux claim in the `createAdapter` doc comment
|
||
([adapters/index.ts:12-13](../../src/lib/player/adapters/index.ts#L12-L13))
|
||
corrected either way: if red, cite this spike instead of asserting it; if
|
||
green, it is wrong and must be rewritten.
|
||
- [ ] On **red**: finding 2 of
|
||
[playback-backend-unification.md](playback-backend-unification.md) gains a
|
||
dated note naming the render-API method as also tested, and this file is
|
||
deleted. The verdict lives in the design-authority spec, not in a second
|
||
file that contradicts nothing.
|
||
- [ ] On **green**: an implementation spec exists, allocating ids from
|
||
**UR-077 / IR-033 / DR-216** (re-check `requirements.md` — the README's
|
||
"next free DR-215" is stale, DR-215 landed), and it must answer ABR before
|
||
being accepted.
|
||
- [ ] No spike code on `master`. If any lands, the standard gates apply:
|
||
`bun run check`, `bun run test`, `bun run check:boundary`, `cargo fmt`,
|
||
`cargo clippy`, `bun run test:rust`.
|
||
|
||
## Testing
|
||
|
||
No automated tests. A compositing result is a visual, per-session-backend
|
||
observation and cannot be asserted in `cargo test` or vitest — pretending
|
||
otherwise would produce a test that passes on a headless runner and tells us
|
||
nothing.
|
||
|
||
Capture a screenshot per gate. G4 specifically: an opaque HTML element over the
|
||
video area, photographed showing the video *behind* it.
|
||
|
||
If it goes green, the implementation spec inherits the testable surface the
|
||
Android work already established — `nativeVideoLayers.test.ts` asserts the
|
||
`app.css` selector list and the `data-native-video` contract, and both are
|
||
platform-agnostic.
|
||
|
||
## TRACES
|
||
|
||
None. No requirement-implementing code is produced. The implementation spec that
|
||
follows a green result allocates from DR-216 and tags there.
|
||
|
||
## Notes for the implementer
|
||
|
||
- **Read [playback-backend-unification.md](playback-backend-unification.md)
|
||
first, in full.** This spike disputes exactly one of its six findings, on one
|
||
platform, by one method it did not try. Everything else in it is still binding
|
||
— particularly finding 3.
|
||
- Test **Wayland first**. It is the gate most likely to be red and the one that
|
||
makes the rest moot.
|
||
- The frontend plumbing already exists from the Android work: `createAdapter`,
|
||
`NativePlayerAdapter`, `nativeVideo.ts`, `videoSurface.ts`, and the
|
||
`[data-native-video="active"]` rule. A green spike is far cheaper to implement
|
||
than it would have been a year ago — which is itself part of why the question
|
||
is worth re-asking.
|
||
- The Android record in
|
||
[05-platform-backends.md](../architecture/05-platform-backends.md#native-video-compositing-android)
|
||
lists six shipped defects from getting this right on one platform. Expect the
|
||
Linux equivalents (the surface outliving its player, the shell painting over
|
||
it, unpainted letterbox bars) rather than rediscovering them.
|
||
- A parallel Claude session may be active in this repo — `git diff` before
|
||
"repairing" unexpected changes.
|