mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.
That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.
Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.
DR-238/246 a seek routed by the stream's container rather than by what the
engine could do with it - correct only while one player handled
those streams, silent the moment another did
DR-239 a property handled but never observed, so the play/pause button
waited for an event that could not arrive
DR-240 fullscreen expanding the document while the window stayed put
DR-241 a seek issued before the engine had a file, failed, and discarded
- which is why resume began at zero
DR-247 a Linux-only gate outliving the caller that made it Linux-only,
breaking the Android build outright
DR-250 a stop aimed at whichever renderer bookkeeping believed was in
charge, missing the one actually making sound
DR-251 a duration of zero believed, leaving the seek bar no scale
DR-252 a junk float converted to a Duration, panicking the backend the
instant a length-less stream appeared
So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.
Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.
Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.
Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.
Squashed from worktree-linux-native-video, which keeps the per-defect history.
456 lines
19 KiB
Rust
456 lines
19 KiB
Rust
//! What stream to play, decided in Rust and handed to a player whole.
|
|
//!
|
|
//! Every player backend — mpv, ExoPlayer, the webview `<video>`/hls.js path —
|
|
//! used to receive a bare URL and re-derive the rest: the frontend decided
|
|
//! "is this HLS?" by looking for `.m3u8` in the string, and nothing anywhere
|
|
//! carried *why* a stream was transcoded or what else the source could have
|
|
//! offered. This module is the replacement contract: one self-describing
|
|
//! [`StreamSelection`] that says what the stream is, how to fetch it, and what
|
|
//! the alternatives were.
|
|
//!
|
|
//! The division of labour it encodes — **Rust decides *what stream*, the player
|
|
//! decides *how to deliver it*** — is the point. A multi-variant playlist handed
|
|
//! to ExoPlayer is still ExoPlayer's to adapt over; Rust never paces bytes.
|
|
//!
|
|
//! TRACES: UR-079 | DR-225
|
|
|
|
use serde::{Deserialize, Serialize};
|
|
|
|
use crate::settings::StreamingQuality;
|
|
|
|
/// How the bytes of a chosen stream are fetched.
|
|
///
|
|
/// This field exists to delete a substring search. The frontend previously
|
|
/// decided which loader to attach by testing `url.contains(".m3u8")`, which is a
|
|
/// domain fact reconstructed in the presentation layer — the same class of leak
|
|
/// as the item-type taxonomy that `check:boundary` guards, and one that breaks
|
|
/// silently the moment a server serves a playlist from a path that does not end
|
|
/// in `.m3u8`, or serves a progressive file from one that does.
|
|
///
|
|
/// Tagged (`{"type":"hls"}`) rather than a bare string so the frontend matches a
|
|
/// discriminant instead of comparing text.
|
|
///
|
|
/// TRACES: UR-079 | DR-225
|
|
#[derive(specta::Type, Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
|
#[serde(tag = "type", rename_all = "camelCase")]
|
|
pub enum Transport {
|
|
/// An HLS playlist. The webview attaches hls.js (or Safari's native loader);
|
|
/// ExoPlayer uses its HLS media source.
|
|
Hls,
|
|
/// A single progressive HTTP resource, seekable by byte range.
|
|
Progressive,
|
|
/// A file already on disk — a completed download, or the loopback media
|
|
/// server standing in front of one.
|
|
LocalFile,
|
|
}
|
|
|
|
/// What the server is doing to the source to produce this stream.
|
|
///
|
|
/// Distinct from [`Transport`] because the two are genuinely independent: a
|
|
/// direct-streamed remux and a transcode can both arrive over HLS, and a direct
|
|
/// play can arrive progressively or as a local file. Keeping them apart is what
|
|
/// lets the UI say "this is not costing the server anything" without inferring
|
|
/// it from a URL shape.
|
|
///
|
|
/// TRACES: UR-079 | DR-228
|
|
#[derive(specta::Type, Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
|
#[serde(tag = "type", rename_all = "camelCase")]
|
|
pub enum PlaybackKind {
|
|
/// The source file is served untouched. No server CPU, no quality loss.
|
|
DirectPlay,
|
|
/// The container is repackaged but the codecs are copied — cheap, and
|
|
/// visually identical to the source.
|
|
DirectStream,
|
|
/// The server is re-encoding. The only case where a bitrate ceiling can
|
|
/// actually be honoured, and the only one that costs the server real work.
|
|
Transcode,
|
|
}
|
|
|
|
impl PlaybackKind {
|
|
/// Whether the server is spending encoder time on this stream.
|
|
///
|
|
/// The queue carries a `needs_transcoding` flag that predates this enum and
|
|
/// that several seek/reload paths still branch on; this keeps the two from
|
|
/// drifting by making one derive from the other.
|
|
///
|
|
/// TRACES: UR-079 | DR-228
|
|
pub fn needs_transcoding(&self) -> bool {
|
|
matches!(self, PlaybackKind::Transcode)
|
|
}
|
|
}
|
|
|
|
/// The rendition actually negotiated — what the viewer is receiving right now.
|
|
///
|
|
/// `None` on a [`StreamSelection`] when the source is being direct-played as-is:
|
|
/// there is no *chosen* rendition in that case, only the file itself, and
|
|
/// reporting the ceiling that happened to be set would misdescribe it.
|
|
///
|
|
/// TRACES: UR-079 | DR-225, DR-226
|
|
#[derive(specta::Type, Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
|
#[serde(rename_all = "camelCase")]
|
|
pub struct Rendition {
|
|
/// The rung of the ladder this stream was built against.
|
|
pub quality: StreamingQuality,
|
|
/// Total bits per second the stream may use, when a ceiling applies.
|
|
pub max_bitrate: Option<u64>,
|
|
/// Resolution ceiling, when one applies. `None` preserves the source's.
|
|
pub max_height: Option<u32>,
|
|
/// Video codec the server was asked to produce.
|
|
pub video_codec: Option<String>,
|
|
/// Audio codec the server was asked to produce.
|
|
pub audio_codec: Option<String>,
|
|
}
|
|
|
|
/// One rung of the quality picker, as it applies to *this* media source.
|
|
///
|
|
/// The picker used to be filled from the fixed [`StreamingQuality::ALL`] ladder,
|
|
/// which meant offering "20 Mbps" for a 1.1 Mbps podcast — eight rungs, six of
|
|
/// them indistinguishable from Original. `exceeds_source` is what lets the
|
|
/// frontend render that honestly without knowing anything about bitrates.
|
|
///
|
|
/// TRACES: UR-070, UR-079 | DR-227, DR-121
|
|
#[derive(specta::Type, Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
|
#[serde(rename_all = "camelCase")]
|
|
pub struct QualityOption {
|
|
pub quality: StreamingQuality,
|
|
/// Human label ("8 Mbps"). Lives in Rust beside the number it describes.
|
|
pub label: String,
|
|
/// Secondary line ("1080p").
|
|
pub detail: String,
|
|
/// True when this rung's ceiling is at or above what the source itself
|
|
/// carries, so selecting it yields the same stream as `Original`.
|
|
///
|
|
/// The frontend renders these differently (or hides them); it does not
|
|
/// decide which they are.
|
|
pub exceeds_source: bool,
|
|
/// The source's own bitrate, when the server reported one. Presentation
|
|
/// only — the picker shows "Original (6.7 Mbps)" rather than a bare word.
|
|
pub source_bitrate: Option<u64>,
|
|
}
|
|
|
|
/// Everything a player backend needs to open a stream, and everything the UI
|
|
/// needs to describe it.
|
|
///
|
|
/// Replaces the bare `String` URL that `get_video_stream_url` used to return.
|
|
///
|
|
/// TRACES: UR-079 | DR-225, DR-227, DR-228
|
|
#[derive(specta::Type, Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
|
#[serde(rename_all = "camelCase")]
|
|
pub struct StreamSelection {
|
|
/// The URL (or loopback URL) to open.
|
|
pub url: String,
|
|
/// How to fetch it. Replaces the `.m3u8` substring check.
|
|
pub transport: Transport,
|
|
/// What the server is doing to the source to produce it.
|
|
pub playback_kind: PlaybackKind,
|
|
/// The negotiated rendition; `None` when direct-playing the source as-is.
|
|
pub rendition: Option<Rendition>,
|
|
/// What this media source can offer, for the quality picker (DR-227).
|
|
pub available: Vec<QualityOption>,
|
|
/// The media source this selection is for, so a later re-open (quality
|
|
/// change, audio-track switch, transcoded seek) targets the same one.
|
|
pub media_source_id: Option<String>,
|
|
/// The transcode identity the server keyed this job by, when there is one.
|
|
pub play_session_id: Option<String>,
|
|
/// Whether the server is spending encoder time on this stream.
|
|
///
|
|
/// Derived from [`playback_kind`](Self::playback_kind) rather than left for
|
|
/// the frontend to compute: "which kinds count as transcoding" is a domain
|
|
/// rule, and a direct *stream* is a remux that must not be counted. The
|
|
/// queue's long-standing `needs_transcoding` flag and the seek strategy both
|
|
/// read this, so there is one answer rather than three.
|
|
///
|
|
/// TRACES: UR-079 | DR-225, DR-228
|
|
pub needs_transcoding: bool,
|
|
}
|
|
|
|
impl StreamSelection {
|
|
/// A selection for a file already on disk.
|
|
///
|
|
/// A downloaded file is a direct play by definition — the bytes are the
|
|
/// source's — and offering a quality ladder over it would be a lie, since
|
|
/// nothing about a local file can be re-negotiated.
|
|
///
|
|
/// TRACES: UR-071, UR-079 | DR-225
|
|
pub fn local_file(url: impl Into<String>) -> Self {
|
|
Self {
|
|
url: url.into(),
|
|
transport: Transport::LocalFile,
|
|
playback_kind: PlaybackKind::DirectPlay,
|
|
rendition: None,
|
|
available: Vec::new(),
|
|
media_source_id: None,
|
|
play_session_id: None,
|
|
needs_transcoding: false,
|
|
}
|
|
}
|
|
|
|
/// A selection for an item already sitting in the queue.
|
|
///
|
|
/// The queue predates `StreamSelection`: its items carry a URL, an optional
|
|
/// transport and the older `needs_transcoding` flag. This rebuilds a
|
|
/// selection from those without re-negotiating with the server, so the
|
|
/// controller can hand an engine an `OpenRequest` for an item it already
|
|
/// holds.
|
|
///
|
|
/// The transport falls back rather than being sniffed from the URL — the
|
|
/// substring check is exactly what DR-230 removed. `needs_transcoding` is an
|
|
/// exact stand-in because every transcode this app requests is HLS (DR-140).
|
|
///
|
|
/// TRACES: UR-079, UR-081 | DR-225, DR-245
|
|
pub fn for_queued_item(
|
|
url: impl Into<String>,
|
|
transport: Option<Transport>,
|
|
needs_transcoding: bool,
|
|
) -> Self {
|
|
let transport = transport.unwrap_or(if needs_transcoding {
|
|
Transport::Hls
|
|
} else {
|
|
Transport::Progressive
|
|
});
|
|
Self {
|
|
url: url.into(),
|
|
transport,
|
|
playback_kind: if needs_transcoding {
|
|
PlaybackKind::Transcode
|
|
} else {
|
|
PlaybackKind::DirectPlay
|
|
},
|
|
rendition: None,
|
|
available: Vec::new(),
|
|
media_source_id: None,
|
|
play_session_id: None,
|
|
needs_transcoding,
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Build the quality ladder as it applies to a source of a known bitrate.
|
|
///
|
|
/// Every rung is returned — the picker stays a fixed, predictable list rather
|
|
/// than one that changes length per item — but each is marked with whether it
|
|
/// would actually constrain *this* source. A rung whose ceiling is at or above
|
|
/// the source bitrate produces the same bytes as `Original`, so presenting it as
|
|
/// a distinct choice is noise.
|
|
///
|
|
/// `source_bitrate` is `None` when the server did not report one (it is absent
|
|
/// for some containers — the sampled library has `avi` files with no bitrate at
|
|
/// all). In that case nothing can be judged redundant and every rung is offered,
|
|
/// which is the safe direction: the viewer keeps every choice they had before.
|
|
///
|
|
/// TRACES: UR-070, UR-079 | DR-227, DR-121 | UT-212
|
|
pub fn quality_options_for_source(source_bitrate: Option<u64>) -> Vec<QualityOption> {
|
|
StreamingQuality::ALL
|
|
.iter()
|
|
.map(|quality| QualityOption {
|
|
quality: *quality,
|
|
label: quality.label().to_string(),
|
|
detail: quality.detail().to_string(),
|
|
exceeds_source: match (quality.max_bitrate(), source_bitrate) {
|
|
// `Original` is the source; it never "exceeds" it.
|
|
(None, _) => false,
|
|
// Nothing known about the source — judge nothing redundant.
|
|
(Some(_), None) => false,
|
|
(Some(cap), Some(source)) => cap >= source,
|
|
},
|
|
source_bitrate,
|
|
})
|
|
.collect()
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
/// The tag the frontend matches on has to be exactly what it expects, and
|
|
/// it is a *string in TypeScript* — nothing but a test keeps the two in step.
|
|
///
|
|
/// TRACES: UR-079 | DR-225 | UT-212
|
|
#[test]
|
|
fn test_transport_serialises_with_the_tag_the_frontend_matches() {
|
|
let cases = [
|
|
(Transport::Hls, r#"{"type":"hls"}"#),
|
|
(Transport::Progressive, r#"{"type":"progressive"}"#),
|
|
(Transport::LocalFile, r#"{"type":"localFile"}"#),
|
|
];
|
|
for (transport, expected) in cases {
|
|
let json = serde_json::to_string(&transport).expect("serialises");
|
|
assert_eq!(json, expected, "wire shape of {transport:?}");
|
|
let back: Transport = serde_json::from_str(&json).expect("round-trips");
|
|
assert_eq!(back, transport);
|
|
}
|
|
}
|
|
|
|
/// TRACES: UR-079 | DR-228 | UT-212
|
|
#[test]
|
|
fn test_playback_kind_serialises_with_the_tag_the_frontend_matches() {
|
|
let cases = [
|
|
(PlaybackKind::DirectPlay, r#"{"type":"directPlay"}"#),
|
|
(PlaybackKind::DirectStream, r#"{"type":"directStream"}"#),
|
|
(PlaybackKind::Transcode, r#"{"type":"transcode"}"#),
|
|
];
|
|
for (kind, expected) in cases {
|
|
let json = serde_json::to_string(&kind).expect("serialises");
|
|
assert_eq!(json, expected, "wire shape of {kind:?}");
|
|
let back: PlaybackKind = serde_json::from_str(&json).expect("round-trips");
|
|
assert_eq!(back, kind);
|
|
}
|
|
}
|
|
|
|
/// Only a transcode costs the server encoder time. A direct *stream* is a
|
|
/// remux — cheap, and not what `needs_transcoding` has ever meant.
|
|
///
|
|
/// TRACES: UR-079 | DR-228 | UT-212
|
|
#[test]
|
|
fn test_only_transcode_counts_as_transcoding() {
|
|
assert!(PlaybackKind::Transcode.needs_transcoding());
|
|
assert!(!PlaybackKind::DirectStream.needs_transcoding());
|
|
assert!(!PlaybackKind::DirectPlay.needs_transcoding());
|
|
}
|
|
|
|
/// A local file is a direct play over a local transport, with no ladder:
|
|
/// nothing about a file on disk can be re-negotiated.
|
|
///
|
|
/// TRACES: UR-071, UR-079 | DR-225 | UT-212
|
|
#[test]
|
|
fn test_local_file_selection_offers_no_ladder() {
|
|
let selection = StreamSelection::local_file("http://127.0.0.1:9000/media/x.mkv");
|
|
assert_eq!(selection.transport, Transport::LocalFile);
|
|
assert_eq!(selection.playback_kind, PlaybackKind::DirectPlay);
|
|
assert!(selection.rendition.is_none());
|
|
assert!(selection.available.is_empty());
|
|
assert!(!selection.needs_transcoding);
|
|
}
|
|
|
|
/// The camelCase rule applies to nested struct fields too, and
|
|
/// `playbackKind` is the one the frontend branches on.
|
|
///
|
|
/// TRACES: UR-079 | DR-225 | UT-212
|
|
#[test]
|
|
fn test_stream_selection_fields_are_camel_case_on_the_wire() {
|
|
let selection = StreamSelection {
|
|
url: "https://example/master.m3u8".to_string(),
|
|
transport: Transport::Hls,
|
|
playback_kind: PlaybackKind::Transcode,
|
|
rendition: Some(Rendition {
|
|
quality: StreamingQuality::Mbps8,
|
|
max_bitrate: Some(8_000_000),
|
|
max_height: Some(1080),
|
|
video_codec: Some("h264".to_string()),
|
|
audio_codec: Some("aac".to_string()),
|
|
}),
|
|
available: Vec::new(),
|
|
media_source_id: Some("src-1".to_string()),
|
|
play_session_id: Some("sess-1".to_string()),
|
|
needs_transcoding: true,
|
|
};
|
|
let json = serde_json::to_string(&selection).expect("serialises");
|
|
assert!(
|
|
json.contains(r#""playbackKind":{"type":"transcode"}"#),
|
|
"{json}"
|
|
);
|
|
assert!(json.contains(r#""transport":{"type":"hls"}"#), "{json}");
|
|
assert!(json.contains(r#""mediaSourceId":"src-1""#), "{json}");
|
|
assert!(json.contains(r#""playSessionId":"sess-1""#), "{json}");
|
|
assert!(json.contains(r#""maxBitrate":8000000"#), "{json}");
|
|
assert!(json.contains(r#""maxHeight":1080"#), "{json}");
|
|
assert!(json.contains(r#""needsTranscoding":true"#), "{json}");
|
|
}
|
|
|
|
/// The measured library has 1.1 Mbps sources in it. Offering those a choice
|
|
/// of 20, 10, 8, 4 and 2 Mbps is offering five ways to spell "Original".
|
|
///
|
|
/// TRACES: UR-070, UR-079 | DR-227, DR-121 | UT-212
|
|
#[test]
|
|
fn test_rungs_above_the_source_bitrate_are_marked_redundant() {
|
|
let options = quality_options_for_source(Some(1_122_137));
|
|
let redundant: Vec<_> = options
|
|
.iter()
|
|
.filter(|o| o.exceeds_source)
|
|
.map(|o| o.quality)
|
|
.collect();
|
|
assert_eq!(
|
|
redundant,
|
|
vec![
|
|
StreamingQuality::Mbps20,
|
|
StreamingQuality::Mbps10,
|
|
StreamingQuality::Mbps8,
|
|
StreamingQuality::Mbps4,
|
|
StreamingQuality::Mbps2,
|
|
],
|
|
"every rung at or above a 1.12 Mbps source is the source"
|
|
);
|
|
|
|
// The rungs that genuinely constrain it are not marked.
|
|
let constraining: Vec<_> = options
|
|
.iter()
|
|
.filter(|o| !o.exceeds_source)
|
|
.map(|o| o.quality)
|
|
.collect();
|
|
assert_eq!(
|
|
constraining,
|
|
vec![
|
|
StreamingQuality::Original,
|
|
StreamingQuality::Mbps1,
|
|
StreamingQuality::Kbps720,
|
|
]
|
|
);
|
|
}
|
|
|
|
/// `Original` is the source, so it is never "above" it — not even for a
|
|
/// source whose bitrate is unknown or zero.
|
|
///
|
|
/// TRACES: UR-070, UR-079 | DR-227 | UT-212
|
|
#[test]
|
|
fn test_original_is_never_marked_as_exceeding_the_source() {
|
|
for bitrate in [None, Some(0), Some(1), Some(50_000_000)] {
|
|
let options = quality_options_for_source(bitrate);
|
|
let original = options
|
|
.iter()
|
|
.find(|o| o.quality == StreamingQuality::Original)
|
|
.expect("Original is always offered");
|
|
assert!(!original.exceeds_source, "bitrate {bitrate:?}");
|
|
}
|
|
}
|
|
|
|
/// An `avi` with no reported bitrate must not lose the picker. Judging
|
|
/// nothing redundant is the safe direction — the viewer keeps every choice.
|
|
///
|
|
/// TRACES: UR-070, UR-079 | DR-227 | UT-212
|
|
#[test]
|
|
fn test_an_unknown_source_bitrate_keeps_every_rung_offered() {
|
|
let options = quality_options_for_source(None);
|
|
assert_eq!(options.len(), StreamingQuality::ALL.len());
|
|
assert!(
|
|
options.iter().all(|o| !o.exceeds_source),
|
|
"nothing can be judged redundant without a source bitrate"
|
|
);
|
|
assert!(options.iter().all(|o| o.source_bitrate.is_none()));
|
|
}
|
|
|
|
/// A 4K remux constrains at every rung — the ladder is fully meaningful.
|
|
///
|
|
/// TRACES: UR-070, UR-079 | DR-227 | UT-212
|
|
#[test]
|
|
fn test_a_source_above_the_ladder_marks_nothing_redundant() {
|
|
let options = quality_options_for_source(Some(40_000_000));
|
|
assert!(options.iter().all(|o| !o.exceeds_source));
|
|
}
|
|
|
|
/// The picker's text comes from Rust, beside the numbers it describes, so a
|
|
/// relabelled rung cannot drift out of step with what it does.
|
|
///
|
|
/// TRACES: UR-070, UR-079 | DR-227 | UT-212
|
|
#[test]
|
|
fn test_options_carry_the_ladder_labels() {
|
|
let options = quality_options_for_source(Some(6_652_961));
|
|
assert_eq!(options.len(), StreamingQuality::ALL.len());
|
|
for (option, quality) in options.iter().zip(StreamingQuality::ALL) {
|
|
assert_eq!(option.quality, quality);
|
|
assert_eq!(option.label, quality.label());
|
|
assert_eq!(option.detail, quality.detail());
|
|
assert_eq!(option.source_bitrate, Some(6_652_961));
|
|
}
|
|
}
|
|
}
|