dtourolle/jellytau
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 40 Wiki Activity

feat(player): native video on Linux, and one contract for every player (v0.11.0)

Browse Source 
mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.

That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.

Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.

  DR-238/246  a seek routed by the stream's container rather than by what the
              engine could do with it - correct only while one player handled
              those streams, silent the moment another did
  DR-239      a property handled but never observed, so the play/pause button
              waited for an event that could not arrive
  DR-240      fullscreen expanding the document while the window stayed put
  DR-241      a seek issued before the engine had a file, failed, and discarded
              - which is why resume began at zero
  DR-247      a Linux-only gate outliving the caller that made it Linux-only,
              breaking the Android build outright
  DR-250      a stop aimed at whichever renderer bookkeeping believed was in
              charge, missing the one actually making sound
  DR-251      a duration of zero believed, leaving the seek bar no scale
  DR-252      a junk float converted to a Duration, panicking the backend the
              instant a length-less stream appeared

So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.

Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.

Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.

Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.

Squashed from worktree-linux-native-video, which keeps the per-defect history.
This commit is contained in:
dtourolle
2026-08-23 10:51:45 +02:00
parent 5fede123e7
commit 11d9d760d8
87 changed files with 15968 additions and 7508 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src-tauri/src/lib.rs
+93 -1
View File
@@ -2,6 +2,10 @@
mod android_context;
mod auth;
mod commands;
/// The MediaPlayer conformance suite, exposed for the `player-conformance`
/// binary. One entry point rather than a public player module tree.
#[cfg(feature = "conformance")]
pub mod conformance_runner;
mod connectivity;
mod credentials;
mod domain;
@@ -96,6 +100,7 @@ use commands::{
lms_unsync_player,
mark_download_completed,
mark_download_failed,
media_local_selection,
media_local_url,
offline_get_items,
offline_is_available,
@@ -224,6 +229,7 @@ use commands::{
repository_get_series_current_episode,
repository_get_series_episodes,
repository_get_similar_items,
repository_get_stream_selection,
repository_get_subtitle_url,
repository_get_video_download_url,
repository_get_video_stream_url,
@@ -732,6 +738,27 @@ fn create_player_backend(
/// Construct the tauri-specta command builder. Shared by `run()` and the
/// bindings-export test so the TypeScript bindings always match the handler.
/// What the engine built for this platform can do.
///
/// Declared per engine, not per category. ExoPlayer speaks HLS and can seek a
/// server-side transcode in place; mpv cannot, because its HLS demuxer will not
/// make the server produce segments from a new offset. Grouping them as "native
/// engines" gets that backwards — being native is not the property that
/// matters, speaking HLS is — and treating a category as a proxy for an ability
/// is exactly the inference DR-246 removed.
///
/// TRACES: UR-081 | DR-246
fn engine_capabilities() -> crate::player::media_player::Capabilities {
#[cfg(target_os = "android")]
{
crate::player::media_player::Capabilities::exoplayer()
}
#[cfg(not(target_os = "android"))]
{
crate::player::media_player::Capabilities::mpv()
}
}
fn specta_builder() -> Builder<tauri::Wry> {
Builder::<tauri::Wry>::new()
// Throw on error so generated `commands.*` return Promise<T> and throw,
@@ -898,6 +925,7 @@ fn specta_builder() -> Builder<tauri::Wry> {
mark_download_completed,
mark_download_failed,
media_local_url,
media_local_selection,
start_download,
enqueue_download,
enqueue_video_downloads,
@@ -984,6 +1012,7 @@ fn specta_builder() -> Builder<tauri::Wry> {
repository_search,
repository_get_playback_info,
repository_get_video_stream_url,
repository_get_stream_selection,
repository_get_audio_stream_url,
repository_get_audio_only_stream_url_for_video,
repository_get_live_tv_channels,
@@ -1206,6 +1235,7 @@ pub fn run() {
// listened for on the frontend via the generated bindings.
builder.mount_events(app);
// In-app update, desktop only.
//
// Registered here rather than in the builder chain above because a
@@ -1345,8 +1375,70 @@ pub fn run() {
playback_reporter.clone(),
position_throttler.clone(),
);
// Attached *after* the backend exists: the mpv handle is registered
// during its construction, and doing this in the order the code
// used to read produced "no mpv handle" every time — the surface was
// built before there was anything to draw from.
// Native video surface: put a GL area under Tauri's webview so mpv
// can draw beneath the controls (UR-080 / DR-231).
//
// 🔴 OFF BY DEFAULT — the naive reparent crashes the app on the
// first click. `tauri-runtime-wry`'s undecorated-resizing handler
// walks a hard-coded two-hop path on every button press in the
// webview:
//
// webview.parent() // "This one should be GtkBox"
// .parent() // ...and this one the GtkWindow
// .downcast::<gtk::Window>().unwrap()
//
// Wrapping the webview in a GtkOverlay makes that chain
// webview → GtkOverlay → GtkBox, the downcast fails, and because the
// panic is non-unwinding it aborts the process. The decoration check
// that would otherwise make this handler inert runs *after* the
// unwrap, so no window configuration avoids it.
//
// This is the "only place Tauri-specific behaviour could still bite"
// that the spike named as the untested half of G1. It bites. The
// surface attaches perfectly and then dies on interaction, so
// "attached successfully" in the log is not the gate — a click is.
//
// Kept behind an env var rather than deleted so the next attempt has
// something to iterate on: JELLYTAU_NATIVE_VIDEO=1 bun run tauri dev
//
// TRACES: UR-080 | DR-231
#[cfg(target_os = "linux")]
if crate::player::native_video::enabled() {
use tauri::Manager;
log::warn!(
"[INIT] JELLYTAU_NATIVE_VIDEO=1 — attaching the experimental \
video surface (mpv drawn behind the webview, no reparenting)"
);
if let Some(window) = app.get_webview_window("main") {
match window.default_vbox() {
Ok(vbox) => {
let handle = crate::player::mpv_backend::registered_handle();
if crate::player::video_surface::attach(&vbox, handle) {
info!("[INIT] Native video surface attached");
} else {
log::warn!("[INIT] Native video surface unavailable");
}
}
Err(e) => {
log::warn!("[INIT] No GTK vbox for the main window: {e}")
}
}
}
}
// Every engine reaches the controller through the one contract.
// `LegacyPlayer` carries the not-yet-ported ones across unchanged,
// so this port swaps a seam rather than four implementations.
// TRACES: UR-081 | DR-245
let player_controller = PlayerController::new(
backend,
Box::new(crate::player::LegacyPlayer::new(
backend,
engine_capabilities(),
)),
playback_reporter.clone(),
position_throttler.clone(),
);