Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2e3a864ef0 | ||
|
|
1d56517f07 | ||
|
|
99d96163d8 | ||
|
|
eda6e36d3d | ||
|
|
f11f5eddd5 |
@@ -32,6 +32,8 @@
|
|||||||
- [Spec Template](specs/SPEC-TEMPLATE.md)
|
- [Spec Template](specs/SPEC-TEMPLATE.md)
|
||||||
- [Spec Review Checklist](specs/SPEC-REVIEW-CHECKLIST.md)
|
- [Spec Review Checklist](specs/SPEC-REVIEW-CHECKLIST.md)
|
||||||
- [Playback Backend Unification](specs/playback-backend-unification.md)
|
- [Playback Backend Unification](specs/playback-backend-unification.md)
|
||||||
|
- [Linux Native Video Spike](specs/linux-native-video-spike.md)
|
||||||
|
- [Backend-Owned Stream Selection](specs/backend-owned-stream-selection.md)
|
||||||
- [Player Facade Enforcement](specs/player-facade-enforcement.md)
|
- [Player Facade Enforcement](specs/player-facade-enforcement.md)
|
||||||
- [Windows Native Audio Backend](specs/windows-native-audio-backend.md)
|
- [Windows Native Audio Backend](specs/windows-native-audio-backend.md)
|
||||||
- [libmpv2 Migration](specs/libmpv2-migration.md)
|
- [libmpv2 Migration](specs/libmpv2-migration.md)
|
||||||
|
|||||||
@@ -27,8 +27,8 @@ know how something *works*, read
|
|||||||
| Design authority | No code of its own — it records a decision later specs act on. |
|
| Design authority | No code of its own — it records a decision later specs act on. |
|
||||||
|
|
||||||
**Next free requirement ids** (always re-check
|
**Next free requirement ids** (always re-check
|
||||||
[requirements.md](../requirements.md) before allocating): **UR-077**,
|
[requirements.md](../requirements.md) before allocating): **UR-079**,
|
||||||
**IR-033**, **DR-215**. Three specs below suggested ids that have since been
|
**IR-033**, **DR-219**. Three specs below suggested ids that have since been
|
||||||
taken by other work; each carries a ⚠️ note at the top.
|
taken by other work; each carries a ⚠️ note at the top.
|
||||||
|
|
||||||
## Partially implemented
|
## Partially implemented
|
||||||
@@ -44,9 +44,11 @@ taken by other work; each carries a ⚠️ note at the top.
|
|||||||
|
|
||||||
| Spec | Blocked on / note |
|
| Spec | Blocked on / note |
|
||||||
|---|---|
|
|---|---|
|
||||||
|
| [backend-owned-stream-selection.md](backend-owned-stream-selection.md) | Rust owns direct-play-vs-transcode, transport and quality; players consume one `StreamSelection`. Phase 1 (delete the `.m3u8` sniff) stands alone. Unblocks Linux native video. |
|
||||||
| [build-provenance.md](build-provenance.md) | `build.rs` is still bare. ⚠️ suggested id DR-093 is taken. |
|
| [build-provenance.md](build-provenance.md) | `build.rs` is still bare. ⚠️ suggested id DR-093 is taken. |
|
||||||
| [player-facade-enforcement.md](player-facade-enforcement.md) | ~60 `commands.player*` sites still outside the facade; no lint rule. ⚠️ suggested id DR-095 is taken. |
|
| [player-facade-enforcement.md](player-facade-enforcement.md) | ~60 `commands.player*` sites still outside the facade; no lint rule. ⚠️ suggested id DR-095 is taken. |
|
||||||
| [windows-native-audio-backend.md](windows-native-audio-backend.md) | Blocked on the libmpv2 swap. ⚠️ suggested id IR-030 is taken. |
|
| [windows-native-audio-backend.md](windows-native-audio-backend.md) | Blocked on the libmpv2 swap. ⚠️ suggested id IR-030 is taken. |
|
||||||
|
| [linux-native-video-spike.md](linux-native-video-spike.md) | **Spike run 2026-08-21: compositing works on Linux, X11 and Wayland.** G1-G6 green bar the Tauri `default_vbox()` half of G1. Needs an implementation spec that answers adaptive bitrate. |
|
||||||
|
|
||||||
## Design authority
|
## Design authority
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,242 @@
|
|||||||
|
# Spec: Backend-owned stream selection
|
||||||
|
|
||||||
|
**Status:** Proposed
|
||||||
|
**Requirements:** UR-079 (new) → DR-219 … DR-224 (new); **implements and extends
|
||||||
|
DR-121**, currently allocated to
|
||||||
|
[read-through-media-cache.md](read-through-media-cache.md) and not started.
|
||||||
|
Re-check `requirements.md` before allocating — the ids moved twice while this was
|
||||||
|
being written (`DR` max was 215, then 218).
|
||||||
|
**UX spec:** the quality selector in `VideoPlayer.svelte` already exists; this
|
||||||
|
changes what fills it, not how it looks.
|
||||||
|
**Supersedes / revises:** takes DR-121 out of
|
||||||
|
[read-through-media-cache.md](read-through-media-cache.md), which should keep
|
||||||
|
only its capture/eviction half. Unblocks
|
||||||
|
[linux-native-video-spike.md](linux-native-video-spike.md).
|
||||||
|
|
||||||
|
**Destination on completion:**
|
||||||
|
[01-rust-backend.md](../architecture/01-rust-backend.md) — extends the
|
||||||
|
"Streaming quality ladder" section; and
|
||||||
|
[03-data-flow.md](../architecture/03-data-flow.md) — playback initiation. The
|
||||||
|
durable half is the layer line and the `StreamSelection` contract; phases and
|
||||||
|
acceptance criteria are disposable.
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Make Rust the single owner of *which stream to play* — direct play or transcode,
|
||||||
|
at what ceiling, over what transport — and hand every player backend a
|
||||||
|
self-describing selection instead of a bare URL. mpv, ExoPlayer and the HTML5
|
||||||
|
`<video>`/hls.js path all become consumers of the same decision rather than three
|
||||||
|
places that re-derive it.
|
||||||
|
|
||||||
|
Nothing about how playback *looks* changes. What changes is that the frontend
|
||||||
|
stops inferring transport from a URL string, and that direct play becomes
|
||||||
|
possible at all.
|
||||||
|
|
||||||
|
## Motivation
|
||||||
|
|
||||||
|
Four concrete problems, all the same shape.
|
||||||
|
|
||||||
|
**1. The frontend sniffs transport out of the URL.**
|
||||||
|
[VideoPlayer.svelte:569](../../src/lib/components/player/VideoPlayer.svelte#L569):
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const isHlsStream = currentStreamUrl.includes(".m3u8");
|
||||||
|
```
|
||||||
|
|
||||||
|
and again inline at line 2364. Rust *built* that URL and knows exactly what it
|
||||||
|
is; the frontend re-derives it by substring match. Change the endpoint, add a DASH
|
||||||
|
path, serve a progressive file, and this silently picks wrong. This is the
|
||||||
|
boundary rule in miniature — not item-type taxonomy, but the same error: a
|
||||||
|
domain fact reconstructed in the presentation layer because the wire shape did
|
||||||
|
not carry it.
|
||||||
|
|
||||||
|
**2. There is no direct-play path.** `get_video_stream_url` always builds an HLS
|
||||||
|
transcode URL (`TranscodingProtocol=hls`, `VideoCodec=h264` first). Every video
|
||||||
|
play burns server CPU, even when the file would play untouched. This is the cost
|
||||||
|
the Linux native-video work exists to remove, and it cannot be removed without a
|
||||||
|
decision that does not currently exist anywhere in the codebase.
|
||||||
|
|
||||||
|
**3. Quality is a process-wide global.** `streaming_quality()` /
|
||||||
|
`set_streaming_quality()` in `repository/online.rs` read and write a static.
|
||||||
|
It is not per-session or per-item, so it cannot express "this 4K remux needs a
|
||||||
|
ceiling, that podcast does not", and two concurrent playbacks would share one
|
||||||
|
setting.
|
||||||
|
|
||||||
|
**4. Rust cannot say what qualities *this* media source supports.** The selector
|
||||||
|
is populated from a fixed enum rather than from what the source actually offers.
|
||||||
|
DR-121 already names this; it has not been built.
|
||||||
|
|
||||||
|
### The prior question
|
||||||
|
|
||||||
|
Finding 3 of [playback-backend-unification.md](playback-backend-unification.md)
|
||||||
|
holds that hls.js gives us real adaptive bitrate and mpv would lose it. Evidence
|
||||||
|
in this repo suggests **there is no ABR today**: a single rendition is requested,
|
||||||
|
no level-handling code exists anywhere in the frontend, and a quality switch is
|
||||||
|
implemented by re-opening the stream.
|
||||||
|
|
||||||
|
**Run this before sizing the adaptation work.** It needs a live server:
|
||||||
|
|
||||||
|
```
|
||||||
|
curl -s "https://<server>/Videos/<itemId>/master.m3u8?api_key=<key>&…" \
|
||||||
|
| grep -c EXT-X-STREAM-INF
|
||||||
|
```
|
||||||
|
|
||||||
|
`1` → there is no adaptation to preserve, and the adaptation half of this spec
|
||||||
|
collapses to "pick well at open". `>1` → finding 3 stands and DR-223 applies.
|
||||||
|
**Everything else in this spec is worth doing either way** — the ownership
|
||||||
|
problems above are independent of the answer.
|
||||||
|
|
||||||
|
## Layer assignment
|
||||||
|
|
||||||
|
| Logic / responsibility | Layer | Why it belongs there |
|
||||||
|
|---|---|---|
|
||||||
|
| Direct play vs direct stream vs transcode | Rust | Depends on Jellyfin's `PlaybackInfo`, container/codec support and the device profile. Changes when Jellyfin's API or our profile changes → domain, by the litmus test. |
|
||||||
|
| Transport of the chosen stream (HLS / progressive / local file) | Rust | Rust constructs the URL; it is the only place that *knows* rather than infers. Today the frontend guesses from `.m3u8`. |
|
||||||
|
| Which qualities this media source can offer | Rust | Derived from the source's own streams and the quality→transcode-parameter mapping that `get_video_download_url` already holds. DR-121. |
|
||||||
|
| The quality ceiling in force, per playback session | Rust | Domain state that outlives any one view and must survive a backend swap or a mode transfer. Currently a process-wide static. |
|
||||||
|
| Deciding to re-negotiate mid-playback (if adaptation is needed) | Rust | It performs the HTTP and already derives reachability from real traffic via `ConnectivityMonitor`. Throughput estimation is the same pattern on the same data — a side-channel probe would repeat the mistake that principle exists to prevent. |
|
||||||
|
| Frame-level delivery *within* the selected stream, including a player's own ABR | **Player** | ExoPlayer has genuine adaptive selection; if Rust hands it a multi-variant playlist it should use it. Rust chooses *what to request*, never how a player paces bytes. See "The line". |
|
||||||
|
| Rendering the selector, showing the current quality, ordering the list | Frontend | Pure presentation over a backend-supplied list. |
|
||||||
|
| Poster, letterbox, controls, overlay z-order | Frontend | Unchanged. |
|
||||||
|
|
||||||
|
### The line
|
||||||
|
|
||||||
|
**Rust decides *what stream*. The player decides *how to deliver it*.**
|
||||||
|
|
||||||
|
This matters most for ExoPlayer, which already does real adaptive track selection
|
||||||
|
over HLS. This spec must not reimplement that or fight it — if a multi-variant
|
||||||
|
playlist reaches ExoPlayer, ExoPlayer adapts and Rust stays out of the way. The
|
||||||
|
same restraint applies to any future backend that gains the capability. Rust only
|
||||||
|
steps in where the player has no such ability (mpv) *and* the server actually
|
||||||
|
offers a ladder.
|
||||||
|
|
||||||
|
Borderline row, with its tie-breaker: "which media source of a multi-source item"
|
||||||
|
looks like a user choice, and its *presentation* is. The default and the
|
||||||
|
constraint set are domain → **Rust**, per the borderline-defaults-to-Rust rule.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
### The contract
|
||||||
|
|
||||||
|
One self-describing selection replaces the bare URL. Nested fields are
|
||||||
|
camelCase over the wire (`#[serde(rename_all = "camelCase")]`); the enums are
|
||||||
|
tagged so the frontend matches a tag instead of parsing a string.
|
||||||
|
|
||||||
|
```rust
|
||||||
|
#[derive(Serialize, Type)]
|
||||||
|
#[serde(rename_all = "camelCase")]
|
||||||
|
pub struct StreamSelection {
|
||||||
|
pub url: String,
|
||||||
|
pub transport: Transport,
|
||||||
|
pub playback_kind: PlaybackKind,
|
||||||
|
/// The negotiated rendition; None when direct-playing the source as-is.
|
||||||
|
pub rendition: Option<Rendition>,
|
||||||
|
/// What this media source can offer — fills the selector (DR-121).
|
||||||
|
pub available: Vec<QualityOption>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Serialize, Type)]
|
||||||
|
#[serde(tag = "type", rename_all = "camelCase")]
|
||||||
|
pub enum Transport { Hls, Progressive, LocalFile }
|
||||||
|
|
||||||
|
#[derive(Serialize, Type)]
|
||||||
|
#[serde(tag = "type", rename_all = "camelCase")]
|
||||||
|
pub enum PlaybackKind { DirectPlay, DirectStream, Transcode }
|
||||||
|
```
|
||||||
|
|
||||||
|
`Transport` is the field that deletes the `.m3u8` sniff. The frontend picks
|
||||||
|
hls.js on `Hls` and the element's own loader otherwise — a tag match, not a
|
||||||
|
substring search.
|
||||||
|
|
||||||
|
### Re-negotiation
|
||||||
|
|
||||||
|
Rust emits `stream-selection-changed` (kebab-case, per convention) carrying a new
|
||||||
|
`StreamSelection` plus the position to resume at. The existing
|
||||||
|
`playerSetStreamQuality` response already has exactly the right shape — a tagged
|
||||||
|
`strategy` that tells the caller who reloads, with the backend handling native
|
||||||
|
itself and handing HTML5 a URL for `reloadSource`
|
||||||
|
([index.ts:198](../../src/lib/player/index.ts#L198)). **Extend that; do not
|
||||||
|
invent a second mechanism.** It is the one piece of this that is already right.
|
||||||
|
|
||||||
|
Note the existing wart to preserve or fix deliberately, not accidentally:
|
||||||
|
tauri-specta keeps those response fields snake_case (`new_url`), and the facade
|
||||||
|
comments say so.
|
||||||
|
|
||||||
|
### Phases
|
||||||
|
|
||||||
|
1. **DR-219** `StreamSelection` + `Transport`; delete the `.m3u8` sniff. No
|
||||||
|
behaviour change — pure ownership move, and independently shippable.
|
||||||
|
2. **DR-220** Per-session quality ceiling replacing the `online.rs` static.
|
||||||
|
3. **DR-221** `available` populated from the media source (DR-121's substance).
|
||||||
|
4. **DR-222** Direct-play/direct-stream negotiation via `PlaybackInfo`. This is
|
||||||
|
the phase that unlocks native video and removes the transcode.
|
||||||
|
5. **DR-223** Adaptation, **only if the playlist check says a ladder exists**.
|
||||||
|
Cheapest sufficient design: re-negotiate on sustained throughput drop, reusing
|
||||||
|
the phase-1 re-negotiation path. A local proxy synthesizing a single-variant
|
||||||
|
playlist is a last resort, not a starting point.
|
||||||
|
6. **DR-224** ExoPlayer and mpv consume `StreamSelection` unchanged, proving the
|
||||||
|
contract is player-agnostic rather than HTML5-shaped.
|
||||||
|
|
||||||
|
Phases 1–4 stand on their own merits with no dependency on the ladder question.
|
||||||
|
|
||||||
|
## Out of scope
|
||||||
|
|
||||||
|
- Rendering, compositing, and the Linux native-video work itself. This spec
|
||||||
|
unblocks [linux-native-video-spike.md](linux-native-video-spike.md); it does
|
||||||
|
not contain it.
|
||||||
|
- Replacing hls.js. It stays as the HLS loader for the webview path.
|
||||||
|
- Reimplementing or overriding ExoPlayer's own adaptive selection. See "The line".
|
||||||
|
- The download/capture half of [read-through-media-cache.md](read-through-media-cache.md)
|
||||||
|
(DR-122, DR-124, DR-125), which keeps its own spec.
|
||||||
|
- Audio. The same argument applies, but video is where the transcode cost is.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] The `.m3u8` substring check is gone from `VideoPlayer.svelte` (both sites)
|
||||||
|
and transport comes from the tagged enum.
|
||||||
|
- [ ] `bun run check`, `bun run test`, `bun run format:check`, `bun run lint` pass.
|
||||||
|
- [ ] `cargo fmt` clean, `cargo clippy -D warnings` clean, `bun run test:rust` passes.
|
||||||
|
- [ ] `bun run check:boundary` passes — and the reviewer confirms by reading that
|
||||||
|
no transport/kind decision was reconstructed in `src/`, since the tripwire
|
||||||
|
only catches item-type array literals.
|
||||||
|
- [ ] `bindings.ts` regenerated from Rust, not hand-edited.
|
||||||
|
- [ ] New code carries `// TRACES:` comments; `bun run traces:validate` passes and
|
||||||
|
coverage stays ≥ the CI ratchet.
|
||||||
|
- [ ] The `EXT-X-STREAM-INF` count is recorded in this spec before DR-223 is
|
||||||
|
started or dropped.
|
||||||
|
- [ ] DR-121 is removed from `read-through-media-cache.md` with a pointer here.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
- Rust: `PlaybackInfo` fixtures → expected `PlaybackKind`, one per branch
|
||||||
|
(supported container direct-plays; unsupported codec transcodes; a ceiling
|
||||||
|
below the source bitrate transcodes even when the codec is fine).
|
||||||
|
- Rust: `Transport` round-trips through serde with the tag the frontend matches.
|
||||||
|
- Frontend: adapter selection driven by `transport`, including the case a URL
|
||||||
|
ending `.m3u8` is served as `Progressive` — that test fails on today's code,
|
||||||
|
which is the point.
|
||||||
|
- Extend `tauriIntegration.test.ts` for the new command params (camelCase rule).
|
||||||
|
- No test asserts a URL substring.
|
||||||
|
|
||||||
|
## TRACES
|
||||||
|
|
||||||
|
| Piece | Tag |
|
||||||
|
|---|---|
|
||||||
|
| `StreamSelection` / `Transport` | `UR-079 \| DR-219` |
|
||||||
|
| Per-session ceiling | `UR-074 \| DR-220` |
|
||||||
|
| `available` from media source | `UR-079 \| DR-221, DR-121` |
|
||||||
|
| Direct-play negotiation | `UR-079 \| DR-222` |
|
||||||
|
| Adaptation, if built | `UR-079 \| DR-223` |
|
||||||
|
| ExoPlayer/mpv consumers | `UR-003, UR-004 \| DR-224` |
|
||||||
|
|
||||||
|
## Notes for the implementer
|
||||||
|
|
||||||
|
- **Phase 1 is worth doing on its own**, even if everything after it is dropped.
|
||||||
|
It removes a real leak and costs almost nothing.
|
||||||
|
- Do not frame any phase as "no Rust changes required" — that framing is what
|
||||||
|
produced the leak `scoped-search-boundary.md` records.
|
||||||
|
- `ConnectivityMonitor` is the precedent for DR-223: derive network facts from
|
||||||
|
real traffic, never from a side-channel poller.
|
||||||
|
- A parallel Claude session may be active in this repo — `git diff` before
|
||||||
|
"repairing" unexpected changes. Requirement ids in particular moved twice
|
||||||
|
during the writing of this spec.
|
||||||
@@ -0,0 +1,469 @@
|
|||||||
|
# Spec: Linux native video — bounded compositing spike
|
||||||
|
|
||||||
|
**Status:** **Run 2026-08-21 — compositing works; G5 carries an open crash.**
|
||||||
|
The compositing claim it set out to test is falsified on Linux. See "Result".
|
||||||
|
This file stays open until the implementation spec exists; ABR is unresolved.
|
||||||
|
**Requirements:** none allocated. This spike produces a decision record, not
|
||||||
|
product code — same shape as
|
||||||
|
[playback-backend-unification.md](playback-backend-unification.md), which is
|
||||||
|
Accepted with no requirement ids of its own. Ids are allocated by the
|
||||||
|
*implementation* spec that follows a green result.
|
||||||
|
**UX spec:** n/a
|
||||||
|
**Supersedes / revises:** re-opens finding 2 of
|
||||||
|
[playback-backend-unification.md](playback-backend-unification.md) on Linux only.
|
||||||
|
Its findings 3, 4, 5 and 6 stand unchallenged and are **not** in scope here.
|
||||||
|
|
||||||
|
**Destination on completion:**
|
||||||
|
[05-platform-backends.md](../architecture/05-platform-backends.md) — a "Native
|
||||||
|
Video Compositing (Linux)" section alongside the existing Android one. The
|
||||||
|
durable half is the mechanism and the two traps below; the gates and phases are
|
||||||
|
disposable.
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Test one falsifiable claim: *a native video surface cannot be composited with a
|
||||||
|
Tauri webview on Linux.* The claim is load-bearing — it is why Linux video goes
|
||||||
|
through an h264 HLS transcode into a WebKitGTK `<video>` element instead of
|
||||||
|
decoding directly in the mpv instance we already run. The spike renders one mpv
|
||||||
|
frame beneath the webview, on both X11 and Wayland, and stops. It ships no
|
||||||
|
product code and flips no defaults.
|
||||||
|
|
||||||
|
A green result does **not** authorise native video on Linux; it authorises
|
||||||
|
writing the spec that would.
|
||||||
|
|
||||||
|
## Motivation
|
||||||
|
|
||||||
|
[playback-backend-unification.md](playback-backend-unification.md) finding 2
|
||||||
|
concluded that native video cannot be composited with a Tauri webview, on
|
||||||
|
evidence from `tauri-plugin-libmpv`'s platform table, wry#284, tauri#6343, and a
|
||||||
|
Tauri maintainer's 2024 statement that a GTK widget as a child X11 window is
|
||||||
|
"a bit hacky and it is not possible on Wayland at all."
|
||||||
|
|
||||||
|
Two things have changed since that was written, and one thing was never tested.
|
||||||
|
|
||||||
|
**1. The general claim has already been falsified on one platform — by us.**
|
||||||
|
Android now renders ExoPlayer video on a TextureView at index 0 *behind a
|
||||||
|
transparent Tauri WebView*, with the Svelte controls drawn over it, on by
|
||||||
|
default. See
|
||||||
|
[05-platform-backends.md](../architecture/05-platform-backends.md#native-video-compositing-android).
|
||||||
|
That is exactly the composition finding 2 said was impossible, shipped. What
|
||||||
|
survives of the finding is a narrower, WebKitGTK-specific claim — which is worth
|
||||||
|
testing on its own terms rather than inheriting.
|
||||||
|
|
||||||
|
**2. A Tauri app now ships Linux native mpv as an active platform.**
|
||||||
|
[MaxVideoPlayer](https://github.com/MaxMB15/MaxVideoPlayer) (354 commits) embeds
|
||||||
|
libmpv via **EGL + X11 child window / Wayland subsurface**, with Linux and macOS
|
||||||
|
active and Windows only planned — the inverse of the plugin matrix finding 2
|
||||||
|
sampled. Its existence does not prove our case works, but it does mean the
|
||||||
|
Wayland half of the maintainer quote is out of date.
|
||||||
|
|
||||||
|
**3. The render API was never tested.** Every source in finding 2 describes
|
||||||
|
*foreign-window embedding*: `--wid`, child windows, a second toplevel
|
||||||
|
position-synced to a `getBoundingClientRect()` div. That is a different mechanism
|
||||||
|
from mpv's render API, where **we** own the GL context and mpv draws into an FBO
|
||||||
|
we hand it (`mpv_render_context_create` / `mpv_render_context_render`, with an
|
||||||
|
upstream [GTK example](https://github.com/mpv-player/mpv-examples/pull/44/files)).
|
||||||
|
Tauri v2 exposes `WebviewWindow::gtk_window()` and `default_vbox()`, so the
|
||||||
|
target is a widget inside Tauri's own GTK tree — not a foreign window, not a
|
||||||
|
second toplevel, and therefore not the thing that was found broken.
|
||||||
|
|
||||||
|
The prize is direct play: no h264 transcode, hardware decode, libass subtitles,
|
||||||
|
and no server CPU burned on every Linux play.
|
||||||
|
|
||||||
|
## The blocker a green spike does not clear
|
||||||
|
|
||||||
|
🔴 **Read this before treating a green result as a green light.**
|
||||||
|
|
||||||
|
Finding 3 of the unification spec stands: **mpv has no adaptive bitrate.** It
|
||||||
|
delegates HLS to FFmpeg's demuxer, which picks one variant at open and never
|
||||||
|
adapts. The webview path has real ABR via hls.js. Compositing is necessary for
|
||||||
|
native video on Linux; it is not sufficient.
|
||||||
|
|
||||||
|
There is a plausible answer, and this spike exists partly to make it testable:
|
||||||
|
**ABR only matters on the transcode path.** A direct-played file has no variant
|
||||||
|
ladder to adapt between — the adaptation the server offers *is* the transcode.
|
||||||
|
So "mpv when the stream is direct-play, HTML5 + hls.js when the server
|
||||||
|
transcodes" would sidestep finding 3 rather than fight it, and it maps onto a
|
||||||
|
decision Rust already makes when it builds the stream URL.
|
||||||
|
|
||||||
|
That is a **hypothesis, not a conclusion.** It is out of scope here. Record it in
|
||||||
|
the spike's decision note so the follow-up spec starts from it.
|
||||||
|
|
||||||
|
## Layer assignment
|
||||||
|
|
||||||
|
The spike introduces no product logic. The table below is the assignment the
|
||||||
|
*follow-up* would inherit, written now so a green result cannot drift into
|
||||||
|
frontend decisions during implementation.
|
||||||
|
|
||||||
|
| Logic / responsibility | Layer | Why it belongs there |
|
||||||
|
|------------------------|-------|----------------------|
|
||||||
|
| Which backend renders video on this platform (`use_html5_element`, `supports_native_video`) | Rust | Already there — `get_player_status` in `commands/player/mod.rs` computes it from a `cfg!`. The spike would widen that `cfg!`, not relocate the decision. The frontend already consumes it via `createAdapter`. |
|
||||||
|
| Whether *this stream* is direct-play or transcoded, and therefore whether mpv or hls.js renders it | Rust | Domain. It depends on Jellyfin's `PlaybackInfo` response, container/codec support, and the bitrate cap — all of which change when Jellyfin's API or our quality ladder changes. The frontend must never re-derive it from a URL shape. |
|
||||||
|
| Creating, sizing, and destroying the GL surface; the mpv render context | Rust | Owns the backend and the GTK window handle. There is no presentation decision in it. |
|
||||||
|
| Where controls, subtitles, and the mini-player sit above the video, and the letterbox/poster treatment | Frontend | Pure presentation; changes only if the UI is redesigned. Precisely the split the Android path already uses. |
|
||||||
|
| Reserving the video rectangle in layout and marking the shell transparent | Frontend | Presentation. `nativeVideo.ts` + the `[data-native-video="active"]` rule in `app.css` already do this for Android and are platform-agnostic. |
|
||||||
|
|
||||||
|
Borderline row, stated with its tie-breaker: *"is the surface currently
|
||||||
|
attached?"* reads like view state, but the Android work found that a surface left
|
||||||
|
in the hierarchy outlives its player (DR-184). Attachment is backend lifecycle →
|
||||||
|
**Rust**, with the frontend told about it, not asked.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
A throwaway branch. No merge to `master` except the decision note.
|
||||||
|
|
||||||
|
### What gets built
|
||||||
|
|
||||||
|
One `#[cfg(target_os = "linux")]` experiment behind a feature flag, in a scratch
|
||||||
|
binary or an ignored test — **not** in `MpvBackend`'s constructor path:
|
||||||
|
|
||||||
|
1. From `app.get_webview_window(...)`, take `gtk_window()` and `default_vbox()`.
|
||||||
|
2. Reparent the webview into a `gtk::Overlay`: `GLArea` as the main child, the
|
||||||
|
webview as the overlay child.
|
||||||
|
3. Set the webview background to fully transparent (wry does this when
|
||||||
|
`"transparent": true`; verify it reaches `webkit_web_view_set_background_color`).
|
||||||
|
4. In the `GLArea`'s `render` signal, drive
|
||||||
|
`mpv_render_context_render` with `MPV_RENDER_PARAM_OPENGL_FBO` pointing at the
|
||||||
|
FBO GTK bound for us.
|
||||||
|
5. Play one local file. Draw an opaque HTML element over the video area.
|
||||||
|
|
||||||
|
`video = no` and `audio-display = no` are set in
|
||||||
|
[mpv_backend.rs:135-141](../../src-tauri/src/player/mpv_backend.rs#L135-L141);
|
||||||
|
the spike overrides them on its own `Mpv` handle rather than editing that path.
|
||||||
|
|
||||||
|
### Bindings
|
||||||
|
|
||||||
|
The current pin is `libmpv = { git = "…/libmpv-rs", branch = "master" }` — the
|
||||||
|
dead pin [libmpv2-migration.md](libmpv2-migration.md) exists to replace. The
|
||||||
|
render API lives in `libmpv2-sys` (`mpv_render_context_render`); the safe wrapper
|
||||||
|
was only ever a PR against the old crate. **Use `libmpv2-sys` raw FFI directly in
|
||||||
|
the spike.** Do not block the spike on the migration, and do not let the spike
|
||||||
|
half-perform it — if the spike goes green the migration becomes a hard
|
||||||
|
prerequisite of the implementation, which is the ordering
|
||||||
|
[windows-native-audio-backend.md](windows-native-audio-backend.md) already sits
|
||||||
|
in.
|
||||||
|
|
||||||
|
### IPC
|
||||||
|
|
||||||
|
None. The spike crosses no boundary. If it goes green, the follow-up changes only
|
||||||
|
the *value* of the existing `useHtml5Element` / `supportsNativeVideo` fields — no
|
||||||
|
new wire shapes, no `bindings.ts` regeneration.
|
||||||
|
|
||||||
|
## Gates
|
||||||
|
|
||||||
|
Each is pass/fail with a named failure. Stop at the first red and write it up —
|
||||||
|
a red result is a successful spike.
|
||||||
|
|
||||||
|
| # | Question | Fails if |
|
||||||
|
|---|---|---|
|
||||||
|
| G1 | Can a custom GTK widget join Tauri's widget tree and survive the window's lifetime? | `default_vbox()` is absent/unusable, or reparenting the webview breaks input or crashes. |
|
||||||
|
| G2 | Does the webview still paint, with a transparent backdrop, over that widget? | The backdrop renders opaque black ([wry#1540](https://github.com/tauri-apps/wry/issues/1540)) or the webview stops repainting ([tauri#12800](https://github.com/tauri-apps/tauri/issues/12800)). **This is the highest-risk gate.** |
|
||||||
|
| G3 | Does mpv render a frame into our FBO? | The render context refuses GTK's context, or frames land in the wrong buffer. |
|
||||||
|
| G4 | Does HTML drawn over the video area actually appear over it? | Video covers the controls — the exact failure wry#284 and tauri#6343 report. Without this, the whole thing is worthless: our controls, subtitles and mini-player all sit over the video. |
|
||||||
|
| G5 | Does it survive resize, fullscreen, and SPA navigation away and back? | Flicker on resize, or a surface that outlives its route. |
|
||||||
|
| G6 | Does it hold on **both** X11 and Wayland? | Either session backend fails. Wayland is the one the 2024 maintainer quote says is impossible — test it first, not last. |
|
||||||
|
|
||||||
|
G6 is not a nice-to-have. A result that only holds on X11 is red for a project
|
||||||
|
shipping to current desktops.
|
||||||
|
|
||||||
|
### Time box
|
||||||
|
|
||||||
|
If G1–G4 are not all green, stop and write the result up. The value of this spike
|
||||||
|
is a dated, method-specific answer — including "still no, and here is the
|
||||||
|
mechanism" — not a working player.
|
||||||
|
|
||||||
|
## Result (2026-08-21)
|
||||||
|
|
||||||
|
Run on GNOME, kernel 7.1.8, libmpv 2.5.0 (mpv 0.41.0), GTK 3.24.52, WebKitGTK
|
||||||
|
2.52.6, wry 0.53.5 — the versions `src-tauri/Cargo.lock` resolves. Spike source:
|
||||||
|
a ~250-line standalone crate using wry + gtk + `libmpv2-sys` raw FFI, driving
|
||||||
|
mpv's render API with an update callback, frame-gated repaints and
|
||||||
|
`report_swap`.
|
||||||
|
|
||||||
|
| Gate | Result | Observed mechanism |
|
||||||
|
|---|---|---|
|
||||||
|
| G1 widget in GTK tree | 🟡 **partial** | `GtkOverlay` with `GtkGLArea` as main child and the wry webview as overlay child works, built directly. **Tauri's own `default_vbox()` was not exercised** — see below. |
|
||||||
|
| G2 webview paints transparently over it | ✅ green | `with_transparent(true)` alone. No window-level transparency was used or needed. |
|
||||||
|
| G3 mpv renders into our FBO | ✅ green | `vo=libmpv` + `mpv_render_context_create` with `MPV_RENDER_PARAM_OPENGL_FBO` into the FBO GTK binds. |
|
||||||
|
| G4 HTML over video | ✅ green | Opaque panel and a translucent control bar both drew over moving video. |
|
||||||
|
| G5 resize / drag / fullscreen | 🟡 **green on appearance, suspect underneath** | No flicker, gap or misalignment, and smooth once frame pacing was correct (trap 3). But the only crash observed came from the only session where fullscreen was exercised — see "What is still open". |
|
||||||
|
| G6 X11 **and** Wayland | ✅ green | Identical on both; `GDK_BACKEND` flipped between runs. |
|
||||||
|
|
||||||
|
**Finding 2 of [playback-backend-unification.md](playback-backend-unification.md)
|
||||||
|
is false on Linux** when tested by the render API rather than by foreign-window
|
||||||
|
embedding. Wayland — the half the 2024 maintainer quote called impossible — is
|
||||||
|
green.
|
||||||
|
|
||||||
|
Better than the gate asked for: the translucent bar composited *alpha* against
|
||||||
|
the video, not merely opaque-over. Scrims, gradient fades and subtitle backdrops
|
||||||
|
therefore work, which is most of how a player UI actually looks. mpv also painted
|
||||||
|
the letterbox bars black on its own — the Android equivalent was a shipped defect
|
||||||
|
(DR-194).
|
||||||
|
|
||||||
|
### Three traps, each of which cost a debugging cycle
|
||||||
|
|
||||||
|
Carry these into the implementation; each produced a failure that looked like a
|
||||||
|
platform limitation and was not.
|
||||||
|
|
||||||
|
1. **`LC_NUMERIC` must be reset *after* `gtk::init()`, not before.** mpv refuses
|
||||||
|
to start under a non-C numeric locale. `mpv_backend.rs` already handles this,
|
||||||
|
but it has no GTK init in front of it; on this path `gtk::init()` applies the
|
||||||
|
user's locale afterwards and `mpv_create` returns null.
|
||||||
|
2. **libepoxy exports GL entry points as *data* symbols.** There is no `glFoo`
|
||||||
|
function to resolve — there is `epoxy_glFoo`, a variable holding a lazily
|
||||||
|
resolving function pointer. `get_proc_address` must return the pointer *stored
|
||||||
|
at* that symbol; returning the symbol's own address makes mpv jump into
|
||||||
|
non-executable data and take SIGSEGV/SEGV_ACCERR on the first GL call. The
|
||||||
|
`epoxy` crate does this correctly but is unusable — its `gl_generator`
|
||||||
|
dependency pulls a yanked `xml-rs`.
|
||||||
|
3. **Frame pacing is not optional, and its symptom is misleading.** Driving
|
||||||
|
`queue_render()` off the widget's frame clock on every tick, without calling
|
||||||
|
`mpv_render_context_report_swap` after each render, leaves mpv with nothing to
|
||||||
|
time against. Playback looks fine in a window and **judders at fullscreen** —
|
||||||
|
which reads as a compositing or GPU limit and is neither. The fix is to
|
||||||
|
register `mpv_render_context_set_update_callback`, redraw only when it says a
|
||||||
|
frame is ready, and report the swap afterwards. Fullscreen was smooth
|
||||||
|
immediately once both were in place.
|
||||||
|
|
||||||
|
### Hardware decode through the render API
|
||||||
|
|
||||||
|
Tested by asking mpv what it actually selected (`hwdec-current`), not what it was
|
||||||
|
asked for. All three ran 20s clean at a steady 30 fps.
|
||||||
|
|
||||||
|
| `hwdec` | `hwdec-current` | Note |
|
||||||
|
|---|---|---|
|
||||||
|
| `vaapi` | `no` | **Did not engage** on this box — silently fell back to software. `vainfo` is not installed, so the libva driver for the Iris Xe iGPU is likely absent. No render-API error; this looks like a missing driver package, not a compositing limit. |
|
||||||
|
| `auto` | `nvdec-copy` | Hardware decode **does** work through the render API, on the discrete RTX 3050. Copy-back rather than zero-copy interop. |
|
||||||
|
| `no` | `no` | Software. Clean baseline. |
|
||||||
|
|
||||||
|
The load-bearing result is the middle row: **hardware decode is compatible with
|
||||||
|
mpv's render API**, so the direct-play prize is real and not traded away for
|
||||||
|
software decoding. Which decoder to prefer is an implementation question — on a
|
||||||
|
hybrid Intel+NVIDIA laptop `auto` reached for the discrete GPU in copy-back mode,
|
||||||
|
which is the least efficient hardware path. An implementation should evaluate
|
||||||
|
zero-copy VA-API on the iGPU (after confirming the driver is installed) before
|
||||||
|
accepting `auto`.
|
||||||
|
|
||||||
|
`hwdec=auto-safe` probes Vulkan video decode, which this GPU does not support.
|
||||||
|
It logs two `Failed setup for format vulkan` / `no frame!` pairs at start-up and
|
||||||
|
then settles on `nvdec-copy` — the same place `auto` lands. A first reading of
|
||||||
|
these logs mistook the start-up pair for a per-frame flood; **it is not**. Every
|
||||||
|
run, clean or crashed, contains exactly two. `auto-safe` is not implicated in
|
||||||
|
anything.
|
||||||
|
|
||||||
|
### What is still open
|
||||||
|
|
||||||
|
- **The Tauri half of G1.** The spike built its own `GtkOverlay`. The app must
|
||||||
|
instead reach `WebviewWindow::gtk_window()` / `default_vbox()` and reparent
|
||||||
|
Tauri's existing webview into an overlay. Low risk — the same widgets, one
|
||||||
|
extra reparent — but unproven, and it is the only place Tauri-specific
|
||||||
|
behaviour could still bite.
|
||||||
|
- 🔴 **ABR — finding 3's premise is in doubt.** Finding 3 says mpv would regress
|
||||||
|
streaming quality because "the webview path already has real ABR via hls.js".
|
||||||
|
Three pieces of evidence in this repo suggest that is **not true of the URLs we
|
||||||
|
actually build**:
|
||||||
|
|
||||||
|
1. `get_video_stream_url` (`repository/online.rs`) requests a *single*
|
||||||
|
rendition — one `VideoBitrate`, one `MaxStreamingBitrate`, one `MaxHeight`.
|
||||||
|
Jellyfin transcodes to what it is asked for; it does not build a ladder.
|
||||||
|
2. The frontend contains **no level-handling code at all** — no `hls.levels`,
|
||||||
|
no `LEVEL_SWITCH`, no `currentLevel`. The `abrEwma*` options in
|
||||||
|
`VideoPlayer.svelte` are default tuning with nothing to act on. hls.js is
|
||||||
|
serving as an HLS *demuxer* (WebKitGTK cannot play HLS natively), not as an
|
||||||
|
adaptation engine.
|
||||||
|
3. That function's own comment describes a quality switch as **rebuilding the
|
||||||
|
URL** — "every path that re-opens a stream (quality switch, transcoded seek,
|
||||||
|
audio-track switch)". Manual selection by stream re-open is what you build
|
||||||
|
when there is no adaptation, and mpv can do the same thing.
|
||||||
|
|
||||||
|
**The decisive test has not been run** and needs a live server plus an API key:
|
||||||
|
count `#EXT-X-STREAM-INF` lines in a real `master.m3u8`. One line means there
|
||||||
|
is no ABR to lose and this blocker disappears. More than one means finding 3
|
||||||
|
stands and the work below applies.
|
||||||
|
|
||||||
|
If ABR does turn out to be real, it belongs in **Rust**, not in mpv, and there
|
||||||
|
are three designs in increasing cost: pick the variant at open; re-open at a
|
||||||
|
new bitrate on sustained throughput drops (this is the quality-switch path the
|
||||||
|
app already has, so it is nearly free); or run a local proxy serving mpv a
|
||||||
|
synthesized single-variant playlist while swapping renditions underneath. The
|
||||||
|
middle option is almost certainly sufficient.
|
||||||
|
|
||||||
|
Either way the **direct-play path still does not exist** — every video play
|
||||||
|
currently goes through the HLS transcode endpoint. Building it is the real
|
||||||
|
project; the compositing work proven above is the smaller half.
|
||||||
|
- 🔴 **One unexplained SIGSEGV.** A ~180s
|
||||||
|
run died in a *decoder* thread (libavcodec -> `av_log` -> libmpv's log handler
|
||||||
|
-> libc). No Tauri, wry, WebKitGTK, GTK or GL frame appears anywhere in the
|
||||||
|
stack, so the fault is on the mpv/ffmpeg side of the process rather than in the
|
||||||
|
compositing seam.
|
||||||
|
|
||||||
|
Three hypotheses were tested and **none reproduced it**:
|
||||||
|
|
||||||
|
| Hypothesis | Test | Result |
|
||||||
|
|---|---|---|
|
||||||
|
| `hwdec=auto-safe`'s Vulkan failures | 300s soak on `auto-safe` | Survived. Also based on a misreading — the failures are 2 per run at start-up, not per-frame. Dead. |
|
||||||
|
| Fullscreen transitions recreating the GL context under mpv's render context | 240s soak, ~120 automated transitions | Survived, no core dumped. |
|
||||||
|
| Continuous resize thrashing the GL framebuffer | 240s soak, ~2000 resizes | Survived, no core dumped. |
|
||||||
|
|
||||||
|
**The crash is therefore unexplained.** It was observed exactly once, in the
|
||||||
|
only session a human interacted with, and did not recur in ~13 minutes of
|
||||||
|
targeted stress across the three most plausible causes. It is recorded here
|
||||||
|
rather than dismissed precisely because nothing explains it: an intermittent
|
||||||
|
fault that nobody can reproduce is worse to inherit than a deterministic one,
|
||||||
|
not better.
|
||||||
|
|
||||||
|
The underlying concern stands regardless of which test eventually reproduces
|
||||||
|
it. A SIGSEGV in an unrelated thread is characteristic of memory corruption,
|
||||||
|
and this spike never calls `mpv_render_context_free` and never tears down on
|
||||||
|
`unrealize` — it has no defence against the GL context being recreated beneath
|
||||||
|
the render context. That is DR-184 on Android restated: a surface outliving its
|
||||||
|
player. An implementation must bind the two lifetimes together whether or not
|
||||||
|
this particular crash is ever explained.
|
||||||
|
|
||||||
|
**Therefore G5 is recorded green on appearance only**, and this crash is the
|
||||||
|
single largest piece of unfinished business in the spike. Do not read the green
|
||||||
|
gates above as "safe to build on" until it is explained or a long soak clears
|
||||||
|
it.
|
||||||
|
- Long-run stability, seeking, track switching, HDR, and multi-window were not
|
||||||
|
exercised at all.
|
||||||
|
|
||||||
|
## Out of scope
|
||||||
|
|
||||||
|
- Any change to the shipping Linux video path. `experimentalNativeVideo` in
|
||||||
|
`adapters/index.ts` is a **suppressor, never a promoter**; the spike must not
|
||||||
|
change that.
|
||||||
|
- Adaptive bitrate. See "The blocker a green spike does not clear".
|
||||||
|
- Windows and macOS — different mechanisms, and **Windows is the easier case, not
|
||||||
|
the endangered one**. See below.
|
||||||
|
- Android. Already shipped; it is the precedent, not the target.
|
||||||
|
- Crossfade, the libmpv2 migration, and the audio-parity work.
|
||||||
|
|
||||||
|
### Why Windows is unaffected, and cheaper
|
||||||
|
|
||||||
|
Nothing here can regress Windows. `use_html5_element` is already a per-platform
|
||||||
|
`cfg!` in `get_player_status` — Android native, everything else HTML5 — so
|
||||||
|
divergent video paths are the existing design rather than something this
|
||||||
|
introduces. Windows keeps `<video>` + hls.js whatever this spike returns.
|
||||||
|
|
||||||
|
The mechanism does not port: `default_vbox()`, `GtkOverlay` and `GtkGLArea` are
|
||||||
|
GTK3/WebKitGTK concepts. But the *question* is already answered more favourably
|
||||||
|
there. Both mpv plugins list Windows as **fully tested** and Linux as broken,
|
||||||
|
because WebView2 honours a transparent background — the "native surface beneath a
|
||||||
|
transparent webview" approach that fails on WebKitGTK is the one that works on
|
||||||
|
Windows. That asymmetry is why
|
||||||
|
[windows-native-audio-backend.md](windows-native-audio-backend.md) can call
|
||||||
|
Windows "the cleanest available win".
|
||||||
|
|
||||||
|
Windows' cost is packaging, not compositing: the build cross-compiles with MSVC +
|
||||||
|
`cargo-xwin`, so libmpv arrives as a bundled prebuilt DLL (the ⚠️ in finding 5's
|
||||||
|
comparison table). That cost is already committed for *audio*. Once the DLL ships
|
||||||
|
to replace `WebviewAudioBackend`, Windows video is largely a follow-on.
|
||||||
|
|
||||||
|
Sequencing, if native video is ever pursued on both:
|
||||||
|
|
||||||
|
1. [libmpv2-migration.md](libmpv2-migration.md) — prerequisite for either.
|
||||||
|
2. [windows-native-audio-backend.md](windows-native-audio-backend.md) — already
|
||||||
|
specced; lands the DLL and a real Windows backend.
|
||||||
|
3. Windows native video — cheap once 2 exists, and does not need this spike.
|
||||||
|
4. Linux native video — needs this spike, and runs independently of 1–3.
|
||||||
|
|
||||||
|
### Does this add a backend?
|
||||||
|
|
||||||
|
No — and the trajectory is convergence, not proliferation.
|
||||||
|
|
||||||
|
`create_player_backend` in `lib.rs` already selects between four
|
||||||
|
`PlayerBackend` impls by `cfg!`: `MpvBackend` (Linux), `ExoPlayerBackend`
|
||||||
|
(Android), `WebviewAudioBackend` (Windows and anything else), and `NullBackend`
|
||||||
|
as the graceful-init fallback. The HTML5 video path is not among them — it is a
|
||||||
|
frontend adapter reporting through `player_report_*`, not a `PlayerBackend`.
|
||||||
|
|
||||||
|
This spike adds none of these. `MpvBackend` already exists and already runs on
|
||||||
|
Linux; it merely sets `video = no` at construction. Giving it video widens an
|
||||||
|
existing backend rather than introducing an engine.
|
||||||
|
|
||||||
|
Following the sequence above, the count goes **down**: replacing
|
||||||
|
`WebviewAudioBackend` with mpv on Windows leaves two native engines — mpv
|
||||||
|
(Linux + Windows) and ExoPlayer (Android) — with native video riding on both.
|
||||||
|
|
||||||
|
Two is the floor, for a reason worth stating so nobody re-litigates it: Android
|
||||||
|
cannot drop ExoPlayer even if libmpv runs there, because the foreground service,
|
||||||
|
`MediaSessionCompat` and lockscreen control are built on it (finding 7 puts the
|
||||||
|
cost at that rewrite, not at the bindings). The HTML5 path does not go away
|
||||||
|
either — it is the transcode/ABR route and the fallback.
|
||||||
|
|
||||||
|
The trait surface converges too: `ExoPlayerBackend` already implements the
|
||||||
|
video-surface lifecycle for Android native compositing, so teaching `MpvBackend`
|
||||||
|
video follows a path already walked rather than opening a second one.
|
||||||
|
- Adopting `tauri-plugin-libmpv` or `tauri-plugin-mpv` as dependencies. Both
|
||||||
|
report Linux window embedding as not working and are small projects
|
||||||
|
(20 and ~70 commits); read them, do not depend on them.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
The deliverable is a decision, not a feature.
|
||||||
|
|
||||||
|
- [ ] Each of G1–G6 recorded green/red **with the observed mechanism**, not just
|
||||||
|
the verdict.
|
||||||
|
- [ ] X11 and Wayland results reported separately, each naming the compositor
|
||||||
|
and WebKitGTK version tested.
|
||||||
|
- [ ] The direct-play/transcode ABR hypothesis recorded as open, with whatever
|
||||||
|
the spike learned about it.
|
||||||
|
- [ ] `docs/specs/README.md` updated — this spec listed, and its row moved or
|
||||||
|
deleted per the result.
|
||||||
|
- [ ] The Linux claim in the `createAdapter` doc comment
|
||||||
|
([adapters/index.ts:12-13](../../src/lib/player/adapters/index.ts#L12-L13))
|
||||||
|
corrected either way: if red, cite this spike instead of asserting it; if
|
||||||
|
green, it is wrong and must be rewritten.
|
||||||
|
- [ ] On **red**: finding 2 of
|
||||||
|
[playback-backend-unification.md](playback-backend-unification.md) gains a
|
||||||
|
dated note naming the render-API method as also tested, and this file is
|
||||||
|
deleted. The verdict lives in the design-authority spec, not in a second
|
||||||
|
file that contradicts nothing.
|
||||||
|
- [ ] On **green**: an implementation spec exists, allocating ids from
|
||||||
|
**UR-077 / IR-033 / DR-216** (re-check `requirements.md` — the README's
|
||||||
|
"next free DR-215" is stale, DR-215 landed), and it must answer ABR before
|
||||||
|
being accepted.
|
||||||
|
- [ ] No spike code on `master`. If any lands, the standard gates apply:
|
||||||
|
`bun run check`, `bun run test`, `bun run check:boundary`, `cargo fmt`,
|
||||||
|
`cargo clippy`, `bun run test:rust`.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
No automated tests. A compositing result is a visual, per-session-backend
|
||||||
|
observation and cannot be asserted in `cargo test` or vitest — pretending
|
||||||
|
otherwise would produce a test that passes on a headless runner and tells us
|
||||||
|
nothing.
|
||||||
|
|
||||||
|
Capture a screenshot per gate. G4 specifically: an opaque HTML element over the
|
||||||
|
video area, photographed showing the video *behind* it.
|
||||||
|
|
||||||
|
If it goes green, the implementation spec inherits the testable surface the
|
||||||
|
Android work already established — `nativeVideoLayers.test.ts` asserts the
|
||||||
|
`app.css` selector list and the `data-native-video` contract, and both are
|
||||||
|
platform-agnostic.
|
||||||
|
|
||||||
|
## TRACES
|
||||||
|
|
||||||
|
None. No requirement-implementing code is produced. The implementation spec that
|
||||||
|
follows a green result allocates from DR-216 and tags there.
|
||||||
|
|
||||||
|
## Notes for the implementer
|
||||||
|
|
||||||
|
- **Read [playback-backend-unification.md](playback-backend-unification.md)
|
||||||
|
first, in full.** This spike disputes exactly one of its six findings, on one
|
||||||
|
platform, by one method it did not try. Everything else in it is still binding
|
||||||
|
— particularly finding 3.
|
||||||
|
- Test **Wayland first**. It is the gate most likely to be red and the one that
|
||||||
|
makes the rest moot.
|
||||||
|
- The frontend plumbing already exists from the Android work: `createAdapter`,
|
||||||
|
`NativePlayerAdapter`, `nativeVideo.ts`, `videoSurface.ts`, and the
|
||||||
|
`[data-native-video="active"]` rule. A green spike is far cheaper to implement
|
||||||
|
than it would have been a year ago — which is itself part of why the question
|
||||||
|
is worth re-asking.
|
||||||
|
- The Android record in
|
||||||
|
[05-platform-backends.md](../architecture/05-platform-backends.md#native-video-compositing-android)
|
||||||
|
lists six shipped defects from getting this right on one platform. Expect the
|
||||||
|
Linux equivalents (the surface outliving its player, the shell painting over
|
||||||
|
it, unpainted letterbox bars) rather than rediscovering them.
|
||||||
|
- A parallel Claude session may be active in this repo — `git diff` before
|
||||||
|
"repairing" unexpected changes.
|
||||||
@@ -90,6 +90,17 @@ The most recent comment on tauri#6343 (2026-05-23) confirms it is still unsolved
|
|||||||
**The principle to carry forward: audio can unify on a native engine; video
|
**The principle to carry forward: audio can unify on a native engine; video
|
||||||
cannot, because video needs a surface and the webview owns the surface.**
|
cannot, because video needs a surface and the webview owns the surface.**
|
||||||
|
|
||||||
|
> **Re-opened on Linux (2026-08-21).** This finding's general form has since been
|
||||||
|
> falsified on Android — native video now composites behind a transparent Tauri
|
||||||
|
> WebView and ships on by default (see
|
||||||
|
> [05-platform-backends.md](../architecture/05-platform-backends.md#native-video-compositing-android)).
|
||||||
|
> The evidence above is also entirely about *foreign-window embedding*; mpv's
|
||||||
|
> render API, drawing into a GL context we own inside Tauri's own GTK tree, was
|
||||||
|
> never tested. [linux-native-video-spike.md](linux-native-video-spike.md) tests
|
||||||
|
> that one claim on Linux. **Findings 3-6 below are untouched by it** - in
|
||||||
|
> particular finding 3, which is an independent disqualifier a green spike would
|
||||||
|
> not clear.
|
||||||
|
|
||||||
### 3. mpv would regress streaming quality
|
### 3. mpv would regress streaming quality
|
||||||
|
|
||||||
mpv has **no adaptive bitrate**. It delegates HLS to FFmpeg's demuxer, which
|
mpv has **no adaptive bitrate**. It delegates HLS to FFmpeg's demuxer, which
|
||||||
@@ -100,6 +111,17 @@ The webview path already has real ABR via hls.js. Moving video to mpv would be a
|
|||||||
**downgrade** on every platform — no graceful degradation on weak networks, and
|
**downgrade** on every platform — no graceful degradation on weak networks, and
|
||||||
quality changes requiring teardown and reload.
|
quality changes requiring teardown and reload.
|
||||||
|
|
||||||
|
> **Premise in doubt (2026-08-21).** "The webview path already has real ABR"
|
||||||
|
> was not verified against the URLs this app actually builds.
|
||||||
|
> `get_video_stream_url` requests a *single* rendition (one `VideoBitrate`, one
|
||||||
|
> `MaxHeight`), the frontend has **no** level-handling code (`hls.levels`,
|
||||||
|
> `LEVEL_SWITCH`, `currentLevel` appear nowhere), and this repo implements a
|
||||||
|
> quality switch by *re-opening the stream* — all of which point to a
|
||||||
|
> single-variant playlist, i.e. no ABR to lose. The decisive test is counting
|
||||||
|
> `#EXT-X-STREAM-INF` lines in a real `master.m3u8`; it needs a live server and
|
||||||
|
> has not been run. See
|
||||||
|
> [linux-native-video-spike.md](linux-native-video-spike.md).
|
||||||
|
|
||||||
### 4. Crossfade is architecturally blocked on mpv
|
### 4. Crossfade is architecturally blocked on mpv
|
||||||
|
|
||||||
mpv's audio chain is single-stream. FFmpeg's `acrossfade` is an `N→A` filter
|
mpv's audio chain is single-stream. FFmpeg's `acrossfade` is an `N→A` filter
|
||||||
|
|||||||
Reference in New Issue
Block a user