docs(specs): record the Linux native-video compositing spike
🏗️ Build and Test JellyTau / Run Tests (pull_request) Successful in 15m3s
🏗️ Build and Test JellyTau / Supply Chain (pull_request) Successful in 38s
Traceability Validation / Check Requirement Traces (pull_request) Successful in 20s
🏗️ Build and Test JellyTau / Android Compile Check (pull_request) Successful in 4m11s
🏗️ Build and Test JellyTau / Run Tests (pull_request) Successful in 15m3s
🏗️ Build and Test JellyTau / Supply Chain (pull_request) Successful in 38s
Traceability Validation / Check Requirement Traces (pull_request) Successful in 20s
🏗️ Build and Test JellyTau / Android Compile Check (pull_request) Successful in 4m11s
Adds the spike write-up and re-opens finding 2 of playback-backend-unification.md, which concluded that video cannot unify on a native engine because the webview owns the surface. That finding's general form has since been falsified on Android, where native video composites behind a transparent WebView and ships on by default. The evidence behind it was also entirely about foreign-window embedding -- mpv's render API, drawing into a GL context we own inside Tauri's own GTK tree, was never tested. The spike tests that one claim and comes back green on Linux for both X11 and Wayland, bar the Tauri default_vbox() half of G1. Findings 3-6 are deliberately left standing. Finding 3 in particular -- mpv has no adaptive bitrate -- is an independent disqualifier that a green compositing result does not clear, and the spike says so rather than reading as a green light. The next-free-id line moves to UR-079 / IR-033 / DR-219: this branch allocated UR-077 and UR-078 for the updater and diagnostics work, and DR-215 through DR-218 with them, after that line was last written. Authored in a parallel session working in the same checkout; committed here so it travels with the rest of the branch.
This commit is contained in:
@@ -32,6 +32,7 @@
|
|||||||
- [Spec Template](specs/SPEC-TEMPLATE.md)
|
- [Spec Template](specs/SPEC-TEMPLATE.md)
|
||||||
- [Spec Review Checklist](specs/SPEC-REVIEW-CHECKLIST.md)
|
- [Spec Review Checklist](specs/SPEC-REVIEW-CHECKLIST.md)
|
||||||
- [Playback Backend Unification](specs/playback-backend-unification.md)
|
- [Playback Backend Unification](specs/playback-backend-unification.md)
|
||||||
|
- [Linux Native Video Spike](specs/linux-native-video-spike.md)
|
||||||
- [Player Facade Enforcement](specs/player-facade-enforcement.md)
|
- [Player Facade Enforcement](specs/player-facade-enforcement.md)
|
||||||
- [Windows Native Audio Backend](specs/windows-native-audio-backend.md)
|
- [Windows Native Audio Backend](specs/windows-native-audio-backend.md)
|
||||||
- [libmpv2 Migration](specs/libmpv2-migration.md)
|
- [libmpv2 Migration](specs/libmpv2-migration.md)
|
||||||
|
|||||||
@@ -27,8 +27,8 @@ know how something *works*, read
|
|||||||
| Design authority | No code of its own — it records a decision later specs act on. |
|
| Design authority | No code of its own — it records a decision later specs act on. |
|
||||||
|
|
||||||
**Next free requirement ids** (always re-check
|
**Next free requirement ids** (always re-check
|
||||||
[requirements.md](../requirements.md) before allocating): **UR-077**,
|
[requirements.md](../requirements.md) before allocating): **UR-079**,
|
||||||
**IR-033**, **DR-215**. Three specs below suggested ids that have since been
|
**IR-033**, **DR-219**. Three specs below suggested ids that have since been
|
||||||
taken by other work; each carries a ⚠️ note at the top.
|
taken by other work; each carries a ⚠️ note at the top.
|
||||||
|
|
||||||
## Partially implemented
|
## Partially implemented
|
||||||
@@ -47,6 +47,7 @@ taken by other work; each carries a ⚠️ note at the top.
|
|||||||
| [build-provenance.md](build-provenance.md) | `build.rs` is still bare. ⚠️ suggested id DR-093 is taken. |
|
| [build-provenance.md](build-provenance.md) | `build.rs` is still bare. ⚠️ suggested id DR-093 is taken. |
|
||||||
| [player-facade-enforcement.md](player-facade-enforcement.md) | ~60 `commands.player*` sites still outside the facade; no lint rule. ⚠️ suggested id DR-095 is taken. |
|
| [player-facade-enforcement.md](player-facade-enforcement.md) | ~60 `commands.player*` sites still outside the facade; no lint rule. ⚠️ suggested id DR-095 is taken. |
|
||||||
| [windows-native-audio-backend.md](windows-native-audio-backend.md) | Blocked on the libmpv2 swap. ⚠️ suggested id IR-030 is taken. |
|
| [windows-native-audio-backend.md](windows-native-audio-backend.md) | Blocked on the libmpv2 swap. ⚠️ suggested id IR-030 is taken. |
|
||||||
|
| [linux-native-video-spike.md](linux-native-video-spike.md) | **Spike run 2026-08-21: compositing works on Linux, X11 and Wayland.** G1-G6 green bar the Tauri `default_vbox()` half of G1. Needs an implementation spec that answers adaptive bitrate. |
|
||||||
|
|
||||||
## Design authority
|
## Design authority
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,409 @@
|
|||||||
|
# Spec: Linux native video — bounded compositing spike
|
||||||
|
|
||||||
|
**Status:** **Run 2026-08-21 — G1-G6 green except the Tauri-tree half of G1.**
|
||||||
|
The compositing claim it set out to test is falsified on Linux. See "Result".
|
||||||
|
This file stays open until the implementation spec exists; ABR is unresolved.
|
||||||
|
**Requirements:** none allocated. This spike produces a decision record, not
|
||||||
|
product code — same shape as
|
||||||
|
[playback-backend-unification.md](playback-backend-unification.md), which is
|
||||||
|
Accepted with no requirement ids of its own. Ids are allocated by the
|
||||||
|
*implementation* spec that follows a green result.
|
||||||
|
**UX spec:** n/a
|
||||||
|
**Supersedes / revises:** re-opens finding 2 of
|
||||||
|
[playback-backend-unification.md](playback-backend-unification.md) on Linux only.
|
||||||
|
Its findings 3, 4, 5 and 6 stand unchallenged and are **not** in scope here.
|
||||||
|
|
||||||
|
**Destination on completion:**
|
||||||
|
[05-platform-backends.md](../architecture/05-platform-backends.md) — a "Native
|
||||||
|
Video Compositing (Linux)" section alongside the existing Android one. The
|
||||||
|
durable half is the mechanism and the two traps below; the gates and phases are
|
||||||
|
disposable.
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Test one falsifiable claim: *a native video surface cannot be composited with a
|
||||||
|
Tauri webview on Linux.* The claim is load-bearing — it is why Linux video goes
|
||||||
|
through an h264 HLS transcode into a WebKitGTK `<video>` element instead of
|
||||||
|
decoding directly in the mpv instance we already run. The spike renders one mpv
|
||||||
|
frame beneath the webview, on both X11 and Wayland, and stops. It ships no
|
||||||
|
product code and flips no defaults.
|
||||||
|
|
||||||
|
A green result does **not** authorise native video on Linux; it authorises
|
||||||
|
writing the spec that would.
|
||||||
|
|
||||||
|
## Motivation
|
||||||
|
|
||||||
|
[playback-backend-unification.md](playback-backend-unification.md) finding 2
|
||||||
|
concluded that native video cannot be composited with a Tauri webview, on
|
||||||
|
evidence from `tauri-plugin-libmpv`'s platform table, wry#284, tauri#6343, and a
|
||||||
|
Tauri maintainer's 2024 statement that a GTK widget as a child X11 window is
|
||||||
|
"a bit hacky and it is not possible on Wayland at all."
|
||||||
|
|
||||||
|
Two things have changed since that was written, and one thing was never tested.
|
||||||
|
|
||||||
|
**1. The general claim has already been falsified on one platform — by us.**
|
||||||
|
Android now renders ExoPlayer video on a TextureView at index 0 *behind a
|
||||||
|
transparent Tauri WebView*, with the Svelte controls drawn over it, on by
|
||||||
|
default. See
|
||||||
|
[05-platform-backends.md](../architecture/05-platform-backends.md#native-video-compositing-android).
|
||||||
|
That is exactly the composition finding 2 said was impossible, shipped. What
|
||||||
|
survives of the finding is a narrower, WebKitGTK-specific claim — which is worth
|
||||||
|
testing on its own terms rather than inheriting.
|
||||||
|
|
||||||
|
**2. A Tauri app now ships Linux native mpv as an active platform.**
|
||||||
|
[MaxVideoPlayer](https://github.com/MaxMB15/MaxVideoPlayer) (354 commits) embeds
|
||||||
|
libmpv via **EGL + X11 child window / Wayland subsurface**, with Linux and macOS
|
||||||
|
active and Windows only planned — the inverse of the plugin matrix finding 2
|
||||||
|
sampled. Its existence does not prove our case works, but it does mean the
|
||||||
|
Wayland half of the maintainer quote is out of date.
|
||||||
|
|
||||||
|
**3. The render API was never tested.** Every source in finding 2 describes
|
||||||
|
*foreign-window embedding*: `--wid`, child windows, a second toplevel
|
||||||
|
position-synced to a `getBoundingClientRect()` div. That is a different mechanism
|
||||||
|
from mpv's render API, where **we** own the GL context and mpv draws into an FBO
|
||||||
|
we hand it (`mpv_render_context_create` / `mpv_render_context_render`, with an
|
||||||
|
upstream [GTK example](https://github.com/mpv-player/mpv-examples/pull/44/files)).
|
||||||
|
Tauri v2 exposes `WebviewWindow::gtk_window()` and `default_vbox()`, so the
|
||||||
|
target is a widget inside Tauri's own GTK tree — not a foreign window, not a
|
||||||
|
second toplevel, and therefore not the thing that was found broken.
|
||||||
|
|
||||||
|
The prize is direct play: no h264 transcode, hardware decode, libass subtitles,
|
||||||
|
and no server CPU burned on every Linux play.
|
||||||
|
|
||||||
|
## The blocker a green spike does not clear
|
||||||
|
|
||||||
|
🔴 **Read this before treating a green result as a green light.**
|
||||||
|
|
||||||
|
Finding 3 of the unification spec stands: **mpv has no adaptive bitrate.** It
|
||||||
|
delegates HLS to FFmpeg's demuxer, which picks one variant at open and never
|
||||||
|
adapts. The webview path has real ABR via hls.js. Compositing is necessary for
|
||||||
|
native video on Linux; it is not sufficient.
|
||||||
|
|
||||||
|
There is a plausible answer, and this spike exists partly to make it testable:
|
||||||
|
**ABR only matters on the transcode path.** A direct-played file has no variant
|
||||||
|
ladder to adapt between — the adaptation the server offers *is* the transcode.
|
||||||
|
So "mpv when the stream is direct-play, HTML5 + hls.js when the server
|
||||||
|
transcodes" would sidestep finding 3 rather than fight it, and it maps onto a
|
||||||
|
decision Rust already makes when it builds the stream URL.
|
||||||
|
|
||||||
|
That is a **hypothesis, not a conclusion.** It is out of scope here. Record it in
|
||||||
|
the spike's decision note so the follow-up spec starts from it.
|
||||||
|
|
||||||
|
## Layer assignment
|
||||||
|
|
||||||
|
The spike introduces no product logic. The table below is the assignment the
|
||||||
|
*follow-up* would inherit, written now so a green result cannot drift into
|
||||||
|
frontend decisions during implementation.
|
||||||
|
|
||||||
|
| Logic / responsibility | Layer | Why it belongs there |
|
||||||
|
|------------------------|-------|----------------------|
|
||||||
|
| Which backend renders video on this platform (`use_html5_element`, `supports_native_video`) | Rust | Already there — `get_player_status` in `commands/player/mod.rs` computes it from a `cfg!`. The spike would widen that `cfg!`, not relocate the decision. The frontend already consumes it via `createAdapter`. |
|
||||||
|
| Whether *this stream* is direct-play or transcoded, and therefore whether mpv or hls.js renders it | Rust | Domain. It depends on Jellyfin's `PlaybackInfo` response, container/codec support, and the bitrate cap — all of which change when Jellyfin's API or our quality ladder changes. The frontend must never re-derive it from a URL shape. |
|
||||||
|
| Creating, sizing, and destroying the GL surface; the mpv render context | Rust | Owns the backend and the GTK window handle. There is no presentation decision in it. |
|
||||||
|
| Where controls, subtitles, and the mini-player sit above the video, and the letterbox/poster treatment | Frontend | Pure presentation; changes only if the UI is redesigned. Precisely the split the Android path already uses. |
|
||||||
|
| Reserving the video rectangle in layout and marking the shell transparent | Frontend | Presentation. `nativeVideo.ts` + the `[data-native-video="active"]` rule in `app.css` already do this for Android and are platform-agnostic. |
|
||||||
|
|
||||||
|
Borderline row, stated with its tie-breaker: *"is the surface currently
|
||||||
|
attached?"* reads like view state, but the Android work found that a surface left
|
||||||
|
in the hierarchy outlives its player (DR-184). Attachment is backend lifecycle →
|
||||||
|
**Rust**, with the frontend told about it, not asked.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
A throwaway branch. No merge to `master` except the decision note.
|
||||||
|
|
||||||
|
### What gets built
|
||||||
|
|
||||||
|
One `#[cfg(target_os = "linux")]` experiment behind a feature flag, in a scratch
|
||||||
|
binary or an ignored test — **not** in `MpvBackend`'s constructor path:
|
||||||
|
|
||||||
|
1. From `app.get_webview_window(...)`, take `gtk_window()` and `default_vbox()`.
|
||||||
|
2. Reparent the webview into a `gtk::Overlay`: `GLArea` as the main child, the
|
||||||
|
webview as the overlay child.
|
||||||
|
3. Set the webview background to fully transparent (wry does this when
|
||||||
|
`"transparent": true`; verify it reaches `webkit_web_view_set_background_color`).
|
||||||
|
4. In the `GLArea`'s `render` signal, drive
|
||||||
|
`mpv_render_context_render` with `MPV_RENDER_PARAM_OPENGL_FBO` pointing at the
|
||||||
|
FBO GTK bound for us.
|
||||||
|
5. Play one local file. Draw an opaque HTML element over the video area.
|
||||||
|
|
||||||
|
`video = no` and `audio-display = no` are set in
|
||||||
|
[mpv_backend.rs:135-141](../../src-tauri/src/player/mpv_backend.rs#L135-L141);
|
||||||
|
the spike overrides them on its own `Mpv` handle rather than editing that path.
|
||||||
|
|
||||||
|
### Bindings
|
||||||
|
|
||||||
|
The current pin is `libmpv = { git = "…/libmpv-rs", branch = "master" }` — the
|
||||||
|
dead pin [libmpv2-migration.md](libmpv2-migration.md) exists to replace. The
|
||||||
|
render API lives in `libmpv2-sys` (`mpv_render_context_render`); the safe wrapper
|
||||||
|
was only ever a PR against the old crate. **Use `libmpv2-sys` raw FFI directly in
|
||||||
|
the spike.** Do not block the spike on the migration, and do not let the spike
|
||||||
|
half-perform it — if the spike goes green the migration becomes a hard
|
||||||
|
prerequisite of the implementation, which is the ordering
|
||||||
|
[windows-native-audio-backend.md](windows-native-audio-backend.md) already sits
|
||||||
|
in.
|
||||||
|
|
||||||
|
### IPC
|
||||||
|
|
||||||
|
None. The spike crosses no boundary. If it goes green, the follow-up changes only
|
||||||
|
the *value* of the existing `useHtml5Element` / `supportsNativeVideo` fields — no
|
||||||
|
new wire shapes, no `bindings.ts` regeneration.
|
||||||
|
|
||||||
|
## Gates
|
||||||
|
|
||||||
|
Each is pass/fail with a named failure. Stop at the first red and write it up —
|
||||||
|
a red result is a successful spike.
|
||||||
|
|
||||||
|
| # | Question | Fails if |
|
||||||
|
|---|---|---|
|
||||||
|
| G1 | Can a custom GTK widget join Tauri's widget tree and survive the window's lifetime? | `default_vbox()` is absent/unusable, or reparenting the webview breaks input or crashes. |
|
||||||
|
| G2 | Does the webview still paint, with a transparent backdrop, over that widget? | The backdrop renders opaque black ([wry#1540](https://github.com/tauri-apps/wry/issues/1540)) or the webview stops repainting ([tauri#12800](https://github.com/tauri-apps/tauri/issues/12800)). **This is the highest-risk gate.** |
|
||||||
|
| G3 | Does mpv render a frame into our FBO? | The render context refuses GTK's context, or frames land in the wrong buffer. |
|
||||||
|
| G4 | Does HTML drawn over the video area actually appear over it? | Video covers the controls — the exact failure wry#284 and tauri#6343 report. Without this, the whole thing is worthless: our controls, subtitles and mini-player all sit over the video. |
|
||||||
|
| G5 | Does it survive resize, fullscreen, and SPA navigation away and back? | Flicker on resize, or a surface that outlives its route. |
|
||||||
|
| G6 | Does it hold on **both** X11 and Wayland? | Either session backend fails. Wayland is the one the 2024 maintainer quote says is impossible — test it first, not last. |
|
||||||
|
|
||||||
|
G6 is not a nice-to-have. A result that only holds on X11 is red for a project
|
||||||
|
shipping to current desktops.
|
||||||
|
|
||||||
|
### Time box
|
||||||
|
|
||||||
|
If G1–G4 are not all green, stop and write the result up. The value of this spike
|
||||||
|
is a dated, method-specific answer — including "still no, and here is the
|
||||||
|
mechanism" — not a working player.
|
||||||
|
|
||||||
|
## Result (2026-08-21)
|
||||||
|
|
||||||
|
Run on GNOME, kernel 7.1.8, libmpv 2.5.0 (mpv 0.41.0), GTK 3.24.52, WebKitGTK
|
||||||
|
2.52.6, wry 0.53.5 — the versions `src-tauri/Cargo.lock` resolves. Spike source:
|
||||||
|
a ~250-line standalone crate using wry + gtk + `libmpv2-sys` raw FFI, driving
|
||||||
|
mpv's render API with an update callback, frame-gated repaints and
|
||||||
|
`report_swap`.
|
||||||
|
|
||||||
|
| Gate | Result | Observed mechanism |
|
||||||
|
|---|---|---|
|
||||||
|
| G1 widget in GTK tree | 🟡 **partial** | `GtkOverlay` with `GtkGLArea` as main child and the wry webview as overlay child works, built directly. **Tauri's own `default_vbox()` was not exercised** — see below. |
|
||||||
|
| G2 webview paints transparently over it | ✅ green | `with_transparent(true)` alone. No window-level transparency was used or needed. |
|
||||||
|
| G3 mpv renders into our FBO | ✅ green | `vo=libmpv` + `mpv_render_context_create` with `MPV_RENDER_PARAM_OPENGL_FBO` into the FBO GTK binds. |
|
||||||
|
| G4 HTML over video | ✅ green | Opaque panel and a translucent control bar both drew over moving video. |
|
||||||
|
| G5 resize / drag / fullscreen | ✅ green | No flicker, no gap, no misalignment. Fullscreen juddered until frame pacing was done properly — see trap 3; it is smooth with `report_swap` in place. |
|
||||||
|
| G6 X11 **and** Wayland | ✅ green | Identical on both; `GDK_BACKEND` flipped between runs. |
|
||||||
|
|
||||||
|
**Finding 2 of [playback-backend-unification.md](playback-backend-unification.md)
|
||||||
|
is false on Linux** when tested by the render API rather than by foreign-window
|
||||||
|
embedding. Wayland — the half the 2024 maintainer quote called impossible — is
|
||||||
|
green.
|
||||||
|
|
||||||
|
Better than the gate asked for: the translucent bar composited *alpha* against
|
||||||
|
the video, not merely opaque-over. Scrims, gradient fades and subtitle backdrops
|
||||||
|
therefore work, which is most of how a player UI actually looks. mpv also painted
|
||||||
|
the letterbox bars black on its own — the Android equivalent was a shipped defect
|
||||||
|
(DR-194).
|
||||||
|
|
||||||
|
### Three traps, each of which cost a debugging cycle
|
||||||
|
|
||||||
|
Carry these into the implementation; each produced a failure that looked like a
|
||||||
|
platform limitation and was not.
|
||||||
|
|
||||||
|
1. **`LC_NUMERIC` must be reset *after* `gtk::init()`, not before.** mpv refuses
|
||||||
|
to start under a non-C numeric locale. `mpv_backend.rs` already handles this,
|
||||||
|
but it has no GTK init in front of it; on this path `gtk::init()` applies the
|
||||||
|
user's locale afterwards and `mpv_create` returns null.
|
||||||
|
2. **libepoxy exports GL entry points as *data* symbols.** There is no `glFoo`
|
||||||
|
function to resolve — there is `epoxy_glFoo`, a variable holding a lazily
|
||||||
|
resolving function pointer. `get_proc_address` must return the pointer *stored
|
||||||
|
at* that symbol; returning the symbol's own address makes mpv jump into
|
||||||
|
non-executable data and take SIGSEGV/SEGV_ACCERR on the first GL call. The
|
||||||
|
`epoxy` crate does this correctly but is unusable — its `gl_generator`
|
||||||
|
dependency pulls a yanked `xml-rs`.
|
||||||
|
3. **Frame pacing is not optional, and its symptom is misleading.** Driving
|
||||||
|
`queue_render()` off the widget's frame clock on every tick, without calling
|
||||||
|
`mpv_render_context_report_swap` after each render, leaves mpv with nothing to
|
||||||
|
time against. Playback looks fine in a window and **judders at fullscreen** —
|
||||||
|
which reads as a compositing or GPU limit and is neither. The fix is to
|
||||||
|
register `mpv_render_context_set_update_callback`, redraw only when it says a
|
||||||
|
frame is ready, and report the swap afterwards. Fullscreen was smooth
|
||||||
|
immediately once both were in place.
|
||||||
|
|
||||||
|
### Hardware decode through the render API
|
||||||
|
|
||||||
|
Tested by asking mpv what it actually selected (`hwdec-current`), not what it was
|
||||||
|
asked for. All three ran 20s clean at a steady 30 fps.
|
||||||
|
|
||||||
|
| `hwdec` | `hwdec-current` | Note |
|
||||||
|
|---|---|---|
|
||||||
|
| `vaapi` | `no` | **Did not engage** on this box — silently fell back to software. `vainfo` is not installed, so the libva driver for the Iris Xe iGPU is likely absent. No render-API error; this looks like a missing driver package, not a compositing limit. |
|
||||||
|
| `auto` | `nvdec-copy` | Hardware decode **does** work through the render API, on the discrete RTX 3050. Copy-back rather than zero-copy interop. |
|
||||||
|
| `no` | `no` | Software. Clean baseline. |
|
||||||
|
|
||||||
|
The load-bearing result is the middle row: **hardware decode is compatible with
|
||||||
|
mpv's render API**, so the direct-play prize is real and not traded away for
|
||||||
|
software decoding. Which decoder to prefer is an implementation question — on a
|
||||||
|
hybrid Intel+NVIDIA laptop `auto` reached for the discrete GPU in copy-back mode,
|
||||||
|
which is the least efficient hardware path. An implementation should evaluate
|
||||||
|
zero-copy VA-API on the iGPU (after confirming the driver is installed) before
|
||||||
|
accepting `auto`.
|
||||||
|
|
||||||
|
`hwdec=auto-safe`, the default this spike started with, probes Vulkan video
|
||||||
|
decode, which this GPU does not support. It failed per-frame and logged
|
||||||
|
`no frame!` on every frame. Do not ship `auto-safe` here without checking that.
|
||||||
|
|
||||||
|
### What is still open
|
||||||
|
|
||||||
|
- **The Tauri half of G1.** The spike built its own `GtkOverlay`. The app must
|
||||||
|
instead reach `WebviewWindow::gtk_window()` / `default_vbox()` and reparent
|
||||||
|
Tauri's existing webview into an overlay. Low risk — the same widgets, one
|
||||||
|
extra reparent — but unproven, and it is the only place Tauri-specific
|
||||||
|
behaviour could still bite.
|
||||||
|
- **ABR — unchanged and still the blocker.** Nothing here addresses finding 3.
|
||||||
|
What has changed is that the direct-play/transcode split is now worth designing
|
||||||
|
rather than moot.
|
||||||
|
- **One unexplained SIGSEGV.** A ~180s run crashed in a *decoder* thread
|
||||||
|
(libavcodec -> `av_log` -> libmpv's log handler -> libc), not in the GL or
|
||||||
|
compositing path, while `hwdec=auto-safe` was failing its Vulkan probe on every
|
||||||
|
frame. It did **not** reproduce across five subsequent runs (2x45s, 3x20s) on
|
||||||
|
`no`, `auto` and `vaapi`. Cause unconfirmed; recorded rather than dismissed.
|
||||||
|
Anyone implementing this should run a multi-hour soak before trusting it.
|
||||||
|
- Long-run stability, seeking, track switching, HDR, and multi-window were not
|
||||||
|
exercised at all.
|
||||||
|
|
||||||
|
## Out of scope
|
||||||
|
|
||||||
|
- Any change to the shipping Linux video path. `experimentalNativeVideo` in
|
||||||
|
`adapters/index.ts` is a **suppressor, never a promoter**; the spike must not
|
||||||
|
change that.
|
||||||
|
- Adaptive bitrate. See "The blocker a green spike does not clear".
|
||||||
|
- Windows and macOS — different mechanisms, and **Windows is the easier case, not
|
||||||
|
the endangered one**. See below.
|
||||||
|
- Android. Already shipped; it is the precedent, not the target.
|
||||||
|
- Crossfade, the libmpv2 migration, and the audio-parity work.
|
||||||
|
|
||||||
|
### Why Windows is unaffected, and cheaper
|
||||||
|
|
||||||
|
Nothing here can regress Windows. `use_html5_element` is already a per-platform
|
||||||
|
`cfg!` in `get_player_status` — Android native, everything else HTML5 — so
|
||||||
|
divergent video paths are the existing design rather than something this
|
||||||
|
introduces. Windows keeps `<video>` + hls.js whatever this spike returns.
|
||||||
|
|
||||||
|
The mechanism does not port: `default_vbox()`, `GtkOverlay` and `GtkGLArea` are
|
||||||
|
GTK3/WebKitGTK concepts. But the *question* is already answered more favourably
|
||||||
|
there. Both mpv plugins list Windows as **fully tested** and Linux as broken,
|
||||||
|
because WebView2 honours a transparent background — the "native surface beneath a
|
||||||
|
transparent webview" approach that fails on WebKitGTK is the one that works on
|
||||||
|
Windows. That asymmetry is why
|
||||||
|
[windows-native-audio-backend.md](windows-native-audio-backend.md) can call
|
||||||
|
Windows "the cleanest available win".
|
||||||
|
|
||||||
|
Windows' cost is packaging, not compositing: the build cross-compiles with MSVC +
|
||||||
|
`cargo-xwin`, so libmpv arrives as a bundled prebuilt DLL (the ⚠️ in finding 5's
|
||||||
|
comparison table). That cost is already committed for *audio*. Once the DLL ships
|
||||||
|
to replace `WebviewAudioBackend`, Windows video is largely a follow-on.
|
||||||
|
|
||||||
|
Sequencing, if native video is ever pursued on both:
|
||||||
|
|
||||||
|
1. [libmpv2-migration.md](libmpv2-migration.md) — prerequisite for either.
|
||||||
|
2. [windows-native-audio-backend.md](windows-native-audio-backend.md) — already
|
||||||
|
specced; lands the DLL and a real Windows backend.
|
||||||
|
3. Windows native video — cheap once 2 exists, and does not need this spike.
|
||||||
|
4. Linux native video — needs this spike, and runs independently of 1–3.
|
||||||
|
|
||||||
|
### Does this add a backend?
|
||||||
|
|
||||||
|
No — and the trajectory is convergence, not proliferation.
|
||||||
|
|
||||||
|
`create_player_backend` in `lib.rs` already selects between four
|
||||||
|
`PlayerBackend` impls by `cfg!`: `MpvBackend` (Linux), `ExoPlayerBackend`
|
||||||
|
(Android), `WebviewAudioBackend` (Windows and anything else), and `NullBackend`
|
||||||
|
as the graceful-init fallback. The HTML5 video path is not among them — it is a
|
||||||
|
frontend adapter reporting through `player_report_*`, not a `PlayerBackend`.
|
||||||
|
|
||||||
|
This spike adds none of these. `MpvBackend` already exists and already runs on
|
||||||
|
Linux; it merely sets `video = no` at construction. Giving it video widens an
|
||||||
|
existing backend rather than introducing an engine.
|
||||||
|
|
||||||
|
Following the sequence above, the count goes **down**: replacing
|
||||||
|
`WebviewAudioBackend` with mpv on Windows leaves two native engines — mpv
|
||||||
|
(Linux + Windows) and ExoPlayer (Android) — with native video riding on both.
|
||||||
|
|
||||||
|
Two is the floor, for a reason worth stating so nobody re-litigates it: Android
|
||||||
|
cannot drop ExoPlayer even if libmpv runs there, because the foreground service,
|
||||||
|
`MediaSessionCompat` and lockscreen control are built on it (finding 7 puts the
|
||||||
|
cost at that rewrite, not at the bindings). The HTML5 path does not go away
|
||||||
|
either — it is the transcode/ABR route and the fallback.
|
||||||
|
|
||||||
|
The trait surface converges too: `ExoPlayerBackend` already implements the
|
||||||
|
video-surface lifecycle for Android native compositing, so teaching `MpvBackend`
|
||||||
|
video follows a path already walked rather than opening a second one.
|
||||||
|
- Adopting `tauri-plugin-libmpv` or `tauri-plugin-mpv` as dependencies. Both
|
||||||
|
report Linux window embedding as not working and are small projects
|
||||||
|
(20 and ~70 commits); read them, do not depend on them.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
The deliverable is a decision, not a feature.
|
||||||
|
|
||||||
|
- [ ] Each of G1–G6 recorded green/red **with the observed mechanism**, not just
|
||||||
|
the verdict.
|
||||||
|
- [ ] X11 and Wayland results reported separately, each naming the compositor
|
||||||
|
and WebKitGTK version tested.
|
||||||
|
- [ ] The direct-play/transcode ABR hypothesis recorded as open, with whatever
|
||||||
|
the spike learned about it.
|
||||||
|
- [ ] `docs/specs/README.md` updated — this spec listed, and its row moved or
|
||||||
|
deleted per the result.
|
||||||
|
- [ ] The Linux claim in the `createAdapter` doc comment
|
||||||
|
([adapters/index.ts:12-13](../../src/lib/player/adapters/index.ts#L12-L13))
|
||||||
|
corrected either way: if red, cite this spike instead of asserting it; if
|
||||||
|
green, it is wrong and must be rewritten.
|
||||||
|
- [ ] On **red**: finding 2 of
|
||||||
|
[playback-backend-unification.md](playback-backend-unification.md) gains a
|
||||||
|
dated note naming the render-API method as also tested, and this file is
|
||||||
|
deleted. The verdict lives in the design-authority spec, not in a second
|
||||||
|
file that contradicts nothing.
|
||||||
|
- [ ] On **green**: an implementation spec exists, allocating ids from
|
||||||
|
**UR-077 / IR-033 / DR-216** (re-check `requirements.md` — the README's
|
||||||
|
"next free DR-215" is stale, DR-215 landed), and it must answer ABR before
|
||||||
|
being accepted.
|
||||||
|
- [ ] No spike code on `master`. If any lands, the standard gates apply:
|
||||||
|
`bun run check`, `bun run test`, `bun run check:boundary`, `cargo fmt`,
|
||||||
|
`cargo clippy`, `bun run test:rust`.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
No automated tests. A compositing result is a visual, per-session-backend
|
||||||
|
observation and cannot be asserted in `cargo test` or vitest — pretending
|
||||||
|
otherwise would produce a test that passes on a headless runner and tells us
|
||||||
|
nothing.
|
||||||
|
|
||||||
|
Capture a screenshot per gate. G4 specifically: an opaque HTML element over the
|
||||||
|
video area, photographed showing the video *behind* it.
|
||||||
|
|
||||||
|
If it goes green, the implementation spec inherits the testable surface the
|
||||||
|
Android work already established — `nativeVideoLayers.test.ts` asserts the
|
||||||
|
`app.css` selector list and the `data-native-video` contract, and both are
|
||||||
|
platform-agnostic.
|
||||||
|
|
||||||
|
## TRACES
|
||||||
|
|
||||||
|
None. No requirement-implementing code is produced. The implementation spec that
|
||||||
|
follows a green result allocates from DR-216 and tags there.
|
||||||
|
|
||||||
|
## Notes for the implementer
|
||||||
|
|
||||||
|
- **Read [playback-backend-unification.md](playback-backend-unification.md)
|
||||||
|
first, in full.** This spike disputes exactly one of its six findings, on one
|
||||||
|
platform, by one method it did not try. Everything else in it is still binding
|
||||||
|
— particularly finding 3.
|
||||||
|
- Test **Wayland first**. It is the gate most likely to be red and the one that
|
||||||
|
makes the rest moot.
|
||||||
|
- The frontend plumbing already exists from the Android work: `createAdapter`,
|
||||||
|
`NativePlayerAdapter`, `nativeVideo.ts`, `videoSurface.ts`, and the
|
||||||
|
`[data-native-video="active"]` rule. A green spike is far cheaper to implement
|
||||||
|
than it would have been a year ago — which is itself part of why the question
|
||||||
|
is worth re-asking.
|
||||||
|
- The Android record in
|
||||||
|
[05-platform-backends.md](../architecture/05-platform-backends.md#native-video-compositing-android)
|
||||||
|
lists six shipped defects from getting this right on one platform. Expect the
|
||||||
|
Linux equivalents (the surface outliving its player, the shell painting over
|
||||||
|
it, unpainted letterbox bars) rather than rediscovering them.
|
||||||
|
- A parallel Claude session may be active in this repo — `git diff` before
|
||||||
|
"repairing" unexpected changes.
|
||||||
@@ -90,6 +90,17 @@ The most recent comment on tauri#6343 (2026-05-23) confirms it is still unsolved
|
|||||||
**The principle to carry forward: audio can unify on a native engine; video
|
**The principle to carry forward: audio can unify on a native engine; video
|
||||||
cannot, because video needs a surface and the webview owns the surface.**
|
cannot, because video needs a surface and the webview owns the surface.**
|
||||||
|
|
||||||
|
> **Re-opened on Linux (2026-08-21).** This finding's general form has since been
|
||||||
|
> falsified on Android — native video now composites behind a transparent Tauri
|
||||||
|
> WebView and ships on by default (see
|
||||||
|
> [05-platform-backends.md](../architecture/05-platform-backends.md#native-video-compositing-android)).
|
||||||
|
> The evidence above is also entirely about *foreign-window embedding*; mpv's
|
||||||
|
> render API, drawing into a GL context we own inside Tauri's own GTK tree, was
|
||||||
|
> never tested. [linux-native-video-spike.md](linux-native-video-spike.md) tests
|
||||||
|
> that one claim on Linux. **Findings 3-6 below are untouched by it** - in
|
||||||
|
> particular finding 3, which is an independent disqualifier a green spike would
|
||||||
|
> not clear.
|
||||||
|
|
||||||
### 3. mpv would regress streaming quality
|
### 3. mpv would regress streaming quality
|
||||||
|
|
||||||
mpv has **no adaptive bitrate**. It delegates HLS to FFmpeg's demuxer, which
|
mpv has **no adaptive bitrate**. It delegates HLS to FFmpeg's demuxer, which
|
||||||
|
|||||||
Reference in New Issue
Block a user