feat(player): native video on Linux, and one contract for every player (v0.11.0)
mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.
That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.
Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.
DR-238/246 a seek routed by the stream's container rather than by what the
engine could do with it - correct only while one player handled
those streams, silent the moment another did
DR-239 a property handled but never observed, so the play/pause button
waited for an event that could not arrive
DR-240 fullscreen expanding the document while the window stayed put
DR-241 a seek issued before the engine had a file, failed, and discarded
- which is why resume began at zero
DR-247 a Linux-only gate outliving the caller that made it Linux-only,
breaking the Android build outright
DR-250 a stop aimed at whichever renderer bookkeeping believed was in
charge, missing the one actually making sound
DR-251 a duration of zero believed, leaving the seek bar no scale
DR-252 a junk float converted to a Duration, panicking the backend the
instant a length-less stream appeared
So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.
Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.
Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.
Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.
Squashed from worktree-linux-native-video, which keeps the per-defect history.
This commit is contained in:
@@ -34,6 +34,19 @@ pub struct MpvBackend {
|
||||
/// through reported 0.0 / unknown exactly when end-of-file handling needed to
|
||||
/// know where playback reached. See [`ObservedTime`].
|
||||
observed: Arc<Mutex<ObservedTime>>,
|
||||
/// A seek that arrived before MPV had a file to seek in.
|
||||
///
|
||||
/// `loadfile` is asynchronous: it returns as soon as the command is queued,
|
||||
/// so `time-pos` is not yet a resolvable property and setting it fails. A
|
||||
/// seek issued in that window used to be dropped on the floor, and the two
|
||||
/// callers that do exactly this are the ones a viewer notices — resume, and
|
||||
/// a transcoded seek, both of which re-open the stream and then ask for a
|
||||
/// position. The stream reloaded and played from zero.
|
||||
///
|
||||
/// Held here and applied by the `FileLoaded` arm.
|
||||
///
|
||||
/// TRACES: UR-040, UR-005 | DR-241
|
||||
pending_seek: Arc<Mutex<Option<f64>>>,
|
||||
}
|
||||
|
||||
struct InternalState {
|
||||
@@ -89,6 +102,32 @@ fn get_stream_url(media: &MediaItem) -> String {
|
||||
}
|
||||
}
|
||||
|
||||
/// The mpv handle of the backend this process created, for the video surface.
|
||||
///
|
||||
/// A `OnceLock` rather than a field reached through `PlayerBackend`, because the
|
||||
/// trait is cross-platform and a raw mpv pointer is not something every backend
|
||||
/// should have to pretend to have. Stored as `usize` because a raw pointer is
|
||||
/// neither `Send` nor `Sync`; the only consumer is the GTK main thread, which is
|
||||
/// also where mpv was created.
|
||||
///
|
||||
/// Written once at construction and never cleared: the backend outlives the
|
||||
/// window, so there is no window in which this could dangle while a surface is
|
||||
/// still using it.
|
||||
///
|
||||
/// TRACES: UR-080 | DR-231
|
||||
static MPV_HANDLE: std::sync::OnceLock<usize> = std::sync::OnceLock::new();
|
||||
|
||||
/// The registered handle, or null if no MPV backend was created (initialisation
|
||||
/// can fail, and the app falls back to a no-op backend rather than dying).
|
||||
///
|
||||
/// TRACES: UR-080 | DR-231
|
||||
pub fn registered_handle() -> *mut libmpv_sys::mpv_handle {
|
||||
MPV_HANDLE
|
||||
.get()
|
||||
.map(|p| *p as *mut libmpv_sys::mpv_handle)
|
||||
.unwrap_or(std::ptr::null_mut())
|
||||
}
|
||||
|
||||
impl MpvBackend {
|
||||
/// Create a new MPV backend
|
||||
pub fn new(
|
||||
@@ -137,9 +176,28 @@ impl MpvBackend {
|
||||
message: format!("Failed to configure MPV audio-display: {:?}", e),
|
||||
})?;
|
||||
|
||||
mpv.set_property("video", "no").map_err(|e| PlayerError {
|
||||
message: format!("Failed to configure MPV video: {:?}", e),
|
||||
})?;
|
||||
// Video is disabled unless this process is drawing it.
|
||||
//
|
||||
// `video: no` is why mpv has never decoded a frame here: Linux video has
|
||||
// always gone through the webview, and decoding it twice would burn a
|
||||
// core for a picture nobody sees. With native video on, mpv needs both
|
||||
// the decoder *and* `vo=libmpv` — the render API only works through that
|
||||
// output, and the default would try to open a window of its own.
|
||||
//
|
||||
// Set at construction because mpv resolves the video output when it
|
||||
// initialises; flipping it later does not re-open one.
|
||||
//
|
||||
// TRACES: UR-080 | DR-231, DR-235
|
||||
if super::native_video::enabled() {
|
||||
mpv.set_property("vo", "libmpv").map_err(|e| PlayerError {
|
||||
message: format!("Failed to select the libmpv video output: {:?}", e),
|
||||
})?;
|
||||
info!("[MpvBackend] native video enabled (vo=libmpv)");
|
||||
} else {
|
||||
mpv.set_property("video", "no").map_err(|e| PlayerError {
|
||||
message: format!("Failed to configure MPV video: {:?}", e),
|
||||
})?;
|
||||
}
|
||||
|
||||
// Set volume to 100% (we'll control via MPV's volume property)
|
||||
mpv.set_property("volume", 100i64)
|
||||
@@ -178,13 +236,21 @@ impl MpvBackend {
|
||||
}));
|
||||
|
||||
let backend = MpvBackend {
|
||||
mpv: Arc::new(mpv),
|
||||
mpv: {
|
||||
let mpv = Arc::new(mpv);
|
||||
// Publish the handle for the video surface (DR-231). Ignores a
|
||||
// second call: only one MPV backend is ever constructed, and a
|
||||
// failed re-init must not replace a live handle.
|
||||
let _ = MPV_HANDLE.set(mpv.ctx.as_ptr() as usize);
|
||||
mpv
|
||||
},
|
||||
state,
|
||||
event_emitter,
|
||||
audio_settings: AudioSettings::default(),
|
||||
playback_reporter,
|
||||
position_throttler,
|
||||
last_seek_time: Arc::new(AtomicU64::new(0)),
|
||||
pending_seek: Arc::new(Mutex::new(None)),
|
||||
observed: Arc::new(Mutex::new(ObservedTime::default())),
|
||||
};
|
||||
|
||||
@@ -202,6 +268,7 @@ impl MpvBackend {
|
||||
let state = self.state.clone();
|
||||
let reporter = self.playback_reporter.clone();
|
||||
let throttler = self.position_throttler.clone();
|
||||
let pending_seek_for_events = self.pending_seek.clone();
|
||||
|
||||
std::thread::spawn(move || {
|
||||
info!("[MpvBackend] Event loop started");
|
||||
@@ -211,6 +278,30 @@ impl MpvBackend {
|
||||
error!("[MpvBackend] Failed to disable deprecated events: {:?}", e);
|
||||
});
|
||||
|
||||
// libmpv delivers PropertyChange only for properties registered
|
||||
// here. Every name matched in the loop below needs a line in this
|
||||
// block or its handler is unreachable — an omission that reads as
|
||||
// working code, because the handler is sitting right there.
|
||||
// UT-218 holds the two lists together.
|
||||
//
|
||||
// `pause` drives the play/pause control: the UI consumes
|
||||
// StateChanged rather than tracking playback itself, per the
|
||||
// one-directional state rule. Unobserved, the event never came and
|
||||
// the button never moved. Invisible until native video shipped,
|
||||
// because the webview <video> element's own DOM events drove that
|
||||
// control on Linux.
|
||||
//
|
||||
// TRACES: UR-005 | DR-239
|
||||
ev_ctx
|
||||
.observe_property("pause", libmpv::Format::Flag, 0)
|
||||
.unwrap_or_else(|e| {
|
||||
error!(
|
||||
"[MpvBackend] Failed to observe 'pause': {:?} — the play/pause \
|
||||
control will not follow the player",
|
||||
e
|
||||
);
|
||||
});
|
||||
|
||||
loop {
|
||||
match ev_ctx.wait_event(1.0) {
|
||||
Some(Ok(event)) => match event {
|
||||
@@ -220,6 +311,43 @@ impl MpvBackend {
|
||||
libmpv::events::Event::FileLoaded => {
|
||||
info!("[MpvBackend] File loaded");
|
||||
|
||||
// Apply a seek that arrived while there was nothing
|
||||
// to seek in. TRACES: UR-040, UR-005 | DR-241
|
||||
{
|
||||
let target = pending_seek_for_events.lock_safe().take();
|
||||
if let Some(position) = target {
|
||||
match mpv.set_property("time-pos", position) {
|
||||
Ok(()) => info!(
|
||||
"[MpvBackend] applied deferred seek to {position}"
|
||||
),
|
||||
Err(e) => warn!(
|
||||
"[MpvBackend] deferred seek to {position} failed: {:?}",
|
||||
e
|
||||
),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Geometry, so "the picture does not fill the screen"
|
||||
// can be attributed rather than guessed at. `width`/
|
||||
// `height` are the decoded frame; `dwidth`/`dheight`
|
||||
// are what mpv will *display* after aspect
|
||||
// correction. A file that carries its letterbox
|
||||
// baked into the picture reports a 16:9 dwidth and
|
||||
// is then pillarboxed on a wider panel — which looks
|
||||
// identical to a rendering bug from outside.
|
||||
{
|
||||
let n = |k: &str| mpv.get_property::<i64>(k).unwrap_or(-1);
|
||||
info!(
|
||||
"[MpvBackend] video geometry: {}x{} decoded, {}x{} display, aspect {:?}",
|
||||
n("width"),
|
||||
n("height"),
|
||||
n("dwidth"),
|
||||
n("dheight"),
|
||||
mpv.get_property::<f64>("video-params/aspect").ok(),
|
||||
);
|
||||
}
|
||||
|
||||
// Get duration
|
||||
if let Ok(duration) = mpv.get_property::<f64>("duration") {
|
||||
if let Some(emitter) = &event_emitter {
|
||||
@@ -522,11 +650,24 @@ impl PlayerBackend for MpvBackend {
|
||||
.as_millis() as u64;
|
||||
self.last_seek_time.store(now, Ordering::Relaxed);
|
||||
|
||||
self.mpv
|
||||
.set_property("time-pos", position)
|
||||
.map_err(|e| PlayerError {
|
||||
message: format!("Failed to seek: {:?}", e),
|
||||
})?;
|
||||
// `time-pos` only resolves while a file is loaded. `loadfile` is
|
||||
// asynchronous, so a seek issued straight after a reload — resume, or a
|
||||
// transcoded seek — lands in a window where this fails, and dropping it
|
||||
// there is what makes the stream play from zero instead of the position
|
||||
// that was asked for. Hold it and let `FileLoaded` apply it.
|
||||
// TRACES: UR-040, UR-005 | DR-241
|
||||
if let Err(e) = self.mpv.set_property("time-pos", position) {
|
||||
debug!(
|
||||
"[MpvBackend] seek to {position} deferred until the file loads ({:?})",
|
||||
e
|
||||
);
|
||||
*self.pending_seek.lock_safe() = Some(position);
|
||||
self.observed.lock_safe().record_position(position);
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// A seek that lands clears any earlier deferred one: the newer intent wins.
|
||||
*self.pending_seek.lock_safe() = None;
|
||||
|
||||
// The poll thread suppresses updates for 150ms after a seek, so without
|
||||
// this a file ending inside that window would report the pre-seek time.
|
||||
|
||||
Reference in New Issue
Block a user