feat(player): native video on Linux, and one contract for every player (v0.11.0)
mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.
That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.
Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.
DR-238/246 a seek routed by the stream's container rather than by what the
engine could do with it - correct only while one player handled
those streams, silent the moment another did
DR-239 a property handled but never observed, so the play/pause button
waited for an event that could not arrive
DR-240 fullscreen expanding the document while the window stayed put
DR-241 a seek issued before the engine had a file, failed, and discarded
- which is why resume began at zero
DR-247 a Linux-only gate outliving the caller that made it Linux-only,
breaking the Android build outright
DR-250 a stop aimed at whichever renderer bookkeeping believed was in
charge, missing the one actually making sound
DR-251 a duration of zero believed, leaving the seek bar no scale
DR-252 a junk float converted to a Duration, panicking the backend the
instant a length-less stream appeared
So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.
Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.
Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.
Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.
Squashed from worktree-linux-native-video, which keeps the per-defect history.
This commit is contained in:
@@ -0,0 +1,328 @@
|
||||
//! The `MediaPlayer` contract: one API, interchangeable engines.
|
||||
//!
|
||||
//! See docs/specs/media-player-controller.md.
|
||||
//!
|
||||
//! This replaces [`PlayerBackend`](super::backend::PlayerBackend), which
|
||||
//! abstracts a *device* — `load`, then `seek` — rather than an *intent*. That
|
||||
//! distinction is not academic; it produced four shipped defects in one day:
|
||||
//!
|
||||
//! * A start position was not expressible, so every caller sequenced
|
||||
//! `load()` + `seek()` itself and each raced the engine's asynchronous load
|
||||
//! independently. Resume worked through one caller and silently failed through
|
||||
//! another (DR-241).
|
||||
//! * Whether a stream could be seeked in place was decided *above* the engines,
|
||||
//! by a truth table in a command handler, for engines it does not own (DR-238).
|
||||
//! * Nothing in the contract obliged an engine to report its own state, so a
|
||||
//! handler for mpv's `pause` property sat unreachable and the play/pause
|
||||
//! control never moved (DR-239).
|
||||
//!
|
||||
//! The contract below is written so each of those is a compile-time or
|
||||
//! conformance-time failure rather than a runtime surprise.
|
||||
//!
|
||||
//! TRACES: UR-081 | DR-242
|
||||
|
||||
// Scaffolding: nothing consumes this contract until `PlayerController` is
|
||||
// ported to it (DR-245). Kept out of `cfg(test)` deliberately — it is production
|
||||
// code being built in shippable steps, not a test fixture. Remove this allow
|
||||
// when the controller talks to `MediaPlayer`.
|
||||
#![allow(dead_code)]
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use super::backend::PlayerError;
|
||||
use super::media::MediaItem;
|
||||
use crate::repository::stream_selection::StreamSelection;
|
||||
use crate::settings::AudioSettings;
|
||||
|
||||
/// Seconds reported by an engine, as a `Duration`, without trusting the number.
|
||||
///
|
||||
/// `Duration::from_secs_f64` **panics** on a negative or non-finite value, and
|
||||
/// no engine promises otherwise. ExoPlayer reports `C.TIME_UNSET` —
|
||||
/// `Long::MIN_VALUE`, about -9.2e15 — for a stream whose length it does not
|
||||
/// know, which is every background-audio handoff: `/Audio/{id}/universal` is a
|
||||
/// chunked, length-less transcode.
|
||||
///
|
||||
/// Held as a float that junk was harmless. Converted to a `Duration` it became
|
||||
/// a panic that killed the backend mid-handoff and left a black screen with no
|
||||
/// controls. Every engine crossing into this contract goes through here.
|
||||
///
|
||||
/// TRACES: UR-005 | DR-252
|
||||
pub fn duration_from_secs(seconds: f64) -> Option<Duration> {
|
||||
(seconds.is_finite() && seconds > 0.0).then(|| Duration::from_secs_f64(seconds))
|
||||
}
|
||||
|
||||
/// What an engine is doing right now.
|
||||
///
|
||||
/// `Opening` is the state the previous design could not express, and is the
|
||||
/// direct cause of DR-241: a seek that arrived while the engine had nothing
|
||||
/// loaded had no phase to be queued against, so it was simply discarded and
|
||||
/// playback began at zero.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum Phase {
|
||||
/// Nothing loaded. `close()` must reach this, and must be silent here.
|
||||
Idle,
|
||||
/// An `open` is in flight. Position is not yet meaningful; a `seek` arriving
|
||||
/// now must be honoured once the engine reaches `Ready`, never dropped.
|
||||
Opening,
|
||||
/// Loaded and able to play, but not advancing.
|
||||
Ready,
|
||||
Playing,
|
||||
Paused,
|
||||
/// Reached the end of the item by itself. Distinct from `Idle`, because
|
||||
/// autoplay cares which one happened.
|
||||
Ended,
|
||||
Failed(String),
|
||||
}
|
||||
|
||||
impl Phase {
|
||||
/// Whether the engine currently holds an item.
|
||||
pub fn has_media(&self) -> bool {
|
||||
!matches!(self, Phase::Idle | Phase::Failed(_))
|
||||
}
|
||||
|
||||
/// Whether playback is advancing.
|
||||
pub fn is_active(&self) -> bool {
|
||||
matches!(self, Phase::Playing)
|
||||
}
|
||||
}
|
||||
|
||||
/// Everything the UI consumes, read as one coherent value.
|
||||
///
|
||||
/// Deliberately a single snapshot rather than a dozen getters: reading position
|
||||
/// and duration through separate calls is how a paused player reported
|
||||
/// `<position> / 0.0` when a file unloaded between them.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PlaybackSnapshot {
|
||||
pub phase: Phase,
|
||||
pub position: Duration,
|
||||
/// `None` while unknown — a live stream, or an item still opening.
|
||||
pub duration: Option<Duration>,
|
||||
/// Whether `seek` can be expected to land. False for live edges.
|
||||
pub seekable: bool,
|
||||
/// 0.0 – 1.0.
|
||||
pub volume: f32,
|
||||
pub muted: bool,
|
||||
pub rate: f64,
|
||||
pub audio_track: Option<i32>,
|
||||
pub subtitle_track: Option<i32>,
|
||||
}
|
||||
|
||||
impl Default for PlaybackSnapshot {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
phase: Phase::Idle,
|
||||
position: Duration::ZERO,
|
||||
duration: None,
|
||||
seekable: false,
|
||||
volume: 1.0,
|
||||
muted: false,
|
||||
rate: 1.0,
|
||||
audio_track: None,
|
||||
subtitle_track: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// What an engine can do, so callers adapt without naming engines.
|
||||
///
|
||||
/// If a caller ever branches on *which* engine it holds, this struct is missing
|
||||
/// something — add it here rather than sniffing. Engine identity leaking into
|
||||
/// callers is the coupling DR-238 came from.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct Capabilities {
|
||||
/// The engine renders pictures, not only sound.
|
||||
pub video: bool,
|
||||
/// Audio settings (EQ, normalisation, gapless) are honoured.
|
||||
pub audio_settings: bool,
|
||||
/// Subtitle tracks can be selected without re-opening.
|
||||
pub subtitle_switching: bool,
|
||||
/// Audio tracks can be selected without re-opening.
|
||||
pub audio_track_switching: bool,
|
||||
/// A *server-side transcode* can be seeked without re-opening the stream.
|
||||
///
|
||||
/// True for hls.js, which seeks within the VOD playlist it is handed and
|
||||
/// lets the server catch up. False for mpv, whose HLS demuxer cannot make
|
||||
/// the server transcode from a new offset.
|
||||
///
|
||||
/// Declared by the engine rather than inferred by the caller. The previous
|
||||
/// design decided this from `is_hls` and `use_html5` in a command handler —
|
||||
/// on behalf of engines it did not own — which is how "who renders" came to
|
||||
/// mean "how do I seek" and why a transcoded seek silently did nothing the
|
||||
/// moment native video changed the renderer (DR-238).
|
||||
///
|
||||
/// Re-negotiating a stream needs the repository, which sits above the
|
||||
/// engine, so the engine states the capability and the caller acts on it.
|
||||
pub seeks_transcoded_in_place: bool,
|
||||
}
|
||||
|
||||
impl Capabilities {
|
||||
/// mpv.
|
||||
///
|
||||
/// Cannot seek a server-side transcode in place: its HLS demuxer will not
|
||||
/// make the server produce segments from a new offset, so the stream has to
|
||||
/// be re-opened.
|
||||
pub fn mpv() -> Self {
|
||||
Self {
|
||||
video: true,
|
||||
audio_settings: true,
|
||||
subtitle_switching: true,
|
||||
audio_track_switching: true,
|
||||
seeks_transcoded_in_place: false,
|
||||
}
|
||||
}
|
||||
|
||||
/// ExoPlayer.
|
||||
///
|
||||
/// **Can** seek a transcode in place. It is a full HLS client, so like
|
||||
/// hls.js it seeks within the VOD playlist it was handed and lets the
|
||||
/// server catch up. Grouping it with mpv as "a native engine" gets this
|
||||
/// exactly backwards — being native is not the property that matters here,
|
||||
/// speaking HLS is, and that is the whole reason this is declared per
|
||||
/// engine rather than inferred from a category.
|
||||
pub fn exoplayer() -> Self {
|
||||
Self {
|
||||
video: true,
|
||||
audio_settings: true,
|
||||
subtitle_switching: true,
|
||||
audio_track_switching: true,
|
||||
seeks_transcoded_in_place: true,
|
||||
}
|
||||
}
|
||||
|
||||
/// An engine that renders through the webview element, where hls.js seeks
|
||||
/// within the playlist it was handed.
|
||||
pub fn webview() -> Self {
|
||||
Self {
|
||||
video: true,
|
||||
audio_settings: false,
|
||||
subtitle_switching: true,
|
||||
audio_track_switching: false,
|
||||
seeks_transcoded_in_place: true,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A request to present an item.
|
||||
///
|
||||
/// `start` is the reason this type exists. Carrying it here — rather than
|
||||
/// leaving callers to `seek` after `open` — is what closes the load/seek race,
|
||||
/// because the engine is the only layer that knows when its pipeline can accept
|
||||
/// a position.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct OpenRequest {
|
||||
pub media: MediaItem,
|
||||
pub selection: StreamSelection,
|
||||
/// Where to begin. `Duration::ZERO` means the start of the item.
|
||||
pub start: Duration,
|
||||
pub audio_track: Option<i32>,
|
||||
pub subtitle_track: Option<i32>,
|
||||
/// Begin playing as soon as the engine is able.
|
||||
pub autoplay: bool,
|
||||
}
|
||||
|
||||
impl OpenRequest {
|
||||
/// Open at the beginning, playing.
|
||||
pub fn new(media: MediaItem, selection: StreamSelection) -> Self {
|
||||
Self {
|
||||
media,
|
||||
selection,
|
||||
start: Duration::ZERO,
|
||||
audio_track: None,
|
||||
subtitle_track: None,
|
||||
autoplay: true,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn starting_at(mut self, start: Duration) -> Self {
|
||||
self.start = start;
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
/// Anything that can present media.
|
||||
///
|
||||
/// Implementations: `MpvPlayer` (Linux/Windows), `ExoPlayerPlayer` (Android),
|
||||
/// `WebviewPlayer` (HTML5 element), and `FakePlayer` for tests. Every one of
|
||||
/// them must pass [`super::conformance`].
|
||||
pub trait MediaPlayer: Send {
|
||||
/// Present `req.selection`, beginning at `req.start`.
|
||||
///
|
||||
/// One operation, deliberately. An engine that cannot start at an offset
|
||||
/// natively absorbs that internally — by deferring until loaded, or by
|
||||
/// re-opening — because it is the only layer that knows when it can.
|
||||
/// Callers must never follow `open` with a `seek` to achieve a start
|
||||
/// position; that is the bug this signature exists to prevent.
|
||||
fn open(&mut self, req: OpenRequest) -> Result<(), PlayerError>;
|
||||
|
||||
fn play(&mut self) -> Result<(), PlayerError>;
|
||||
fn pause(&mut self) -> Result<(), PlayerError>;
|
||||
|
||||
/// Stop and release the current item.
|
||||
///
|
||||
/// Must be **idempotent** and must leave the engine **silent**. "Stopped"
|
||||
/// and "producing no audio" were not the same thing in the previous design,
|
||||
/// and the gap between them is audible.
|
||||
fn close(&mut self) -> Result<(), PlayerError>;
|
||||
|
||||
/// Seek to an absolute position on the item's own timeline.
|
||||
///
|
||||
/// Whether that is an in-place seek or a re-open of the stream is the
|
||||
/// engine's business: hls.js seeks within a VOD playlist, mpv's HLS demuxer
|
||||
/// cannot make a server transcode from a new offset. Callers state the
|
||||
/// destination and nothing else.
|
||||
fn seek(&mut self, to: Duration) -> Result<(), PlayerError>;
|
||||
|
||||
fn set_volume(&mut self, volume: f32) -> Result<(), PlayerError>;
|
||||
fn set_muted(&mut self, muted: bool) -> Result<(), PlayerError>;
|
||||
fn set_rate(&mut self, rate: f64) -> Result<(), PlayerError>;
|
||||
|
||||
fn select_audio_track(&mut self, index: Option<i32>) -> Result<(), PlayerError>;
|
||||
fn select_subtitle_track(&mut self, index: Option<i32>) -> Result<(), PlayerError>;
|
||||
|
||||
/// One coherent read of the engine's state.
|
||||
fn snapshot(&self) -> PlaybackSnapshot;
|
||||
|
||||
fn capabilities(&self) -> Capabilities;
|
||||
|
||||
/// Apply EQ, normalisation and gapless settings.
|
||||
///
|
||||
/// Provided rather than required: engines that cannot honour them say so
|
||||
/// through [`Capabilities::audio_settings`] and inherit this no-op, instead
|
||||
/// of every implementation carrying an `Ok(())` it does not mean.
|
||||
fn set_audio_settings(&mut self, _settings: &AudioSettings) -> Result<(), PlayerError> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn audio_settings(&self) -> AudioSettings {
|
||||
AudioSettings::default()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// The value that killed the backend: `C.TIME_UNSET` as seconds.
|
||||
///
|
||||
/// ExoPlayer reports it for any stream whose length it does not know, and
|
||||
/// `Duration::from_secs_f64` panics on it. A player must not be the place
|
||||
/// anyone discovers a float was strange.
|
||||
///
|
||||
/// TRACES: UR-005 | DR-252 | UT-222
|
||||
#[test]
|
||||
fn test_junk_durations_do_not_panic() {
|
||||
// Long::MIN_VALUE milliseconds, as ExoPlayer hands it over.
|
||||
assert_eq!(duration_from_secs(-9_223_372_036_854_776.0), None);
|
||||
assert_eq!(duration_from_secs(-1.0), None);
|
||||
assert_eq!(duration_from_secs(0.0), None, "zero is not a duration");
|
||||
assert_eq!(duration_from_secs(f64::NAN), None);
|
||||
assert_eq!(duration_from_secs(f64::INFINITY), None);
|
||||
assert_eq!(duration_from_secs(f64::NEG_INFINITY), None);
|
||||
|
||||
// A real one still survives.
|
||||
assert_eq!(
|
||||
duration_from_secs(6997.024),
|
||||
Some(Duration::from_secs_f64(6997.024))
|
||||
);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user