Files
jellytau/src-tauri/src/player/backend.rs
T
dtourolle 83dc8c7028 feat(playback): let Rust decide what stream to play, and say so
Playing a video meant asking the server to re-encode it, always. That
decision was made nowhere and written down nowhere, so whoever needed it
re-derived it downstream — the player worked out whether it had been handed
a playlist by looking for ".m3u8" in the URL, in two places. A viewer paid
for a transcode of a file their device could have played untouched, and the
app could not tell them which it was.

One negotiation now produces one self-describing StreamSelection — direct
play, remux or transcode; over a playlist, a plain HTTP file, or a local one
— and every renderer consumes that same answer.

Measured against the development server (Jellyfin 10.11.5), 400 items
sampled for codec mix and 40 put through a real PlaybackInfo negotiation
per profile:

  Linux / WebKitGTK (h264 only, 2ch)          3/40 —  7% direct play
  Android / ExoPlayer (hevc, ac3/eac3, 6ch)  34/40 — 85% direct play

The library is ~80% hevc, which is why the two diverge so hard. The payoff
is overwhelmingly Android, where 85% of plays were starting a transcode
nobody needed. Linux stays near 7% until libmpv decodes the picture — the
h264-only profile is a WebKitGTK constraint, not a JellyTau choice.

DR-219  StreamSelection: url + tagged Transport (hls/progressive/localFile)
        + PlaybackKind (directPlay/directStream/transcode) + the negotiated
        rendition + this source's ladder + a needs_transcoding flag derived
        in Rust so the rule is answered once. Both enums are serde-tagged
        so the frontend matches a discriminant, not a substring. The paths
        that never negotiate get the same shape from Rust rather than
        assembling one — media_local_selection for a downloaded file,
        LiveStreamInfo.transport for a live channel — so there is no second
        place where a transport is decided.

DR-220  The ceiling becomes two levels: a durable device default (Settings,
        persisted) and a per-playback override the in-player picker sets.
        The picker had called itself a "this film, this connection" control
        since it was written but wrote the process-wide default, so dropping
        one awkward film to 2 Mbps silently capped every video played
        afterwards for the rest of the process, with Settings still showing
        the old value. The override is cleared whenever playback moves to a
        new item, which stops it surviving into an autoplayed next episode.
        effective_streaming_quality() is the single resolution point.

DR-221  The quality picker is filled from what this media source can offer.
        Rust marks a rung exceeds_source when its ceiling is at or above the
        source's own bitrate — such a rung is another way to spell Original
        — and the frontend does not draw those. Original is never marked; a
        source whose bitrate the server does not report marks nothing, which
        keeps every rung offered.

DR-222  Direct play and direct stream are negotiated, with two client-side
        overrides on top because the server's answer is right about the file
        and wrong about what this app will do with it: undecodable audio
        (Jellyfin 10.11.5 honours a DirectPlayProfile's container and video
        codec but ignores its audio codec, so it offers direct play for an
        E-AC-3 track the webview renders in silence) and a viewer-pinned
        audio track the file does not default to. A direct stream is a remux
        and is deliberately not counted as transcoding.

DR-223  Dropped on measurement, not deferred. A master playlist from this
        server carries exactly one EXT-X-STREAM-INF: Jellyfin builds it from
        the single rendition the request asked for rather than publishing a
        ladder. So there is no adaptation for hls.js to be preserving and
        none mpv would lose — the claim that there was, in
        playback-backend-unification.md, does not hold. Recorded rather than
        deleted because it is a measurement: a server that does publish a
        ladder would change the answer.

DR-224  Every backend consumes the same selection. The queue item carries
        the transport, so player_seek_video picks its seek strategy from the
        backend's decision instead of the last stream_url.contains(".m3u8")
        in the codebase. Items queued by a path that never negotiated carry
        None and fall back to needs_transcoding, which is exact rather than
        a guess because every transcode this app requests is HLS (DR-140).

The frontend loader decision moves to streamTransport.ts so it can be
tested: the two cases that pin it are the ones that failed against the old
implementation — a progressive stream whose URL contains ".m3u8" must not
get an HLS loader, and an HLS stream whose URL contains none must.

Also verified the URL the direct-play branch builds actually serves playable
bytes: 206, video/mp4, valid ISO-BMFF, and a mid-file range works, so
seeking a direct play works.

The spec is folded into docs/architecture/{01,02,03} and deleted, per the
rule that docs/specs holds only work that has not shipped. DR-121 leaves
read-through-media-cache.md with a pointer; that spec keeps its capture half.

Not verified: real playback on a device. Direct play changes what actually
gets played, and neither fixtures nor curl prove the WebKitGTK and ExoPlayer
paths render it.
2026-08-21 22:44:13 +02:00

540 lines
17 KiB
Rust

use super::media::MediaItem;
use super::state::PlayerState;
use crate::settings::AudioSettings;
/// Error type for player operations
#[derive(Debug, Clone)]
pub struct PlayerError {
pub message: String,
}
impl std::fmt::Display for PlayerError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.message)
}
}
impl std::error::Error for PlayerError {}
impl PlayerError {
pub fn not_implemented() -> Self {
Self {
message: "Not implemented".to_string(),
}
}
/// Create a playback failure error
///
/// Only available on Android where ExoPlayer uses it for JNI errors
#[cfg(target_os = "android")]
pub fn playback_failed<S: Into<String>>(message: S) -> Self {
Self {
message: message.into(),
}
}
}
/// Player backend trait - implemented by platform-specific players
///
/// TRACES: UR-003, UR-004 | IR-003, IR-004 | DR-004
pub trait PlayerBackend: Send + Sync {
/// Load a media item for playback
/// TRACES: UR-005
fn load(&mut self, media: &MediaItem) -> Result<(), PlayerError>;
/// Start or resume playback
/// TRACES: UR-005
fn play(&mut self) -> Result<(), PlayerError>;
/// Pause playback
/// TRACES: UR-005
fn pause(&mut self) -> Result<(), PlayerError>;
/// Stop playback and unload media
/// TRACES: UR-005
fn stop(&mut self) -> Result<(), PlayerError>;
/// Seek to a position in seconds
/// TRACES: UR-005
fn seek(&mut self, position: f64) -> Result<(), PlayerError>;
/// Set volume (0.0 - 1.0)
/// TRACES: UR-016
fn set_volume(&mut self, volume: f32) -> Result<(), PlayerError>;
/// Get current playback position in seconds
fn position(&self) -> f64;
/// Get total duration in seconds
fn duration(&self) -> Option<f64>;
/// Get current player state
fn state(&self) -> PlayerState;
/// Get current volume
fn volume(&self) -> f32;
/// Apply audio settings (crossfade, gapless, normalization)
///
/// @req-partial: UR-031 (Linux only) - Crossfade between audio tracks
/// @req-partial: UR-032 (Linux only) - Gapless playback for seamless album listening
/// @req-partial: UR-033 (Linux only) - Volume normalization to prevent volume jumps
/// @req: DR-034 - Crossfade engine with configurable duration (0-12s)
/// @req: DR-035 - Gapless playback between sequential tracks
/// @req: DR-036 - Volume normalization with preset levels (Loud/Normal/Quiet)
fn set_audio_settings(&mut self, _settings: &AudioSettings) -> Result<(), PlayerError> {
// Default implementation does nothing - override in platform-specific backends
Ok(())
}
/// Get current audio settings
///
/// @req: DR-034 - Crossfade engine
/// @req: DR-035 - Gapless playback
/// @req: DR-036 - Volume normalization
fn audio_settings(&self) -> AudioSettings {
AudioSettings::default()
}
/// Set the active audio track by stream index
///
/// Overridden by the Android (ExoPlayer) backend. `MpvBackend` deliberately
/// does **not** override it — MPV is the audio-only backend here, so it keeps
/// this `not_implemented()` default and the Linux video path switches track by
/// re-opening the stream instead (`player_switch_audio_track`).
///
/// TRACES: UR-021 | IR-019, DR-024
fn set_audio_track(&mut self, _stream_index: i32) -> Result<(), PlayerError> {
// Default implementation does nothing - override in platform-specific backends
Err(PlayerError::not_implemented())
}
/// Set the active subtitle track by stream index (None to disable subtitles)
///
/// Overridden by the Android (ExoPlayer) backend. `MpvBackend` deliberately
/// does **not** override it, so it keeps this `not_implemented()` default;
/// the Linux video path renders subtitles as `<track>` children of the
/// WebKitGTK HTML5 `<video>` element and never calls this.
///
/// TRACES: UR-020 | IR-018, DR-023
fn set_subtitle_track(&mut self, _stream_index: Option<i32>) -> Result<(), PlayerError> {
// Default implementation does nothing - override in platform-specific backends
Err(PlayerError::not_implemented())
}
}
/// Null player backend (for testing or when no real player is available)
///
/// @req: DR-004 - PlayerBackend trait (mock implementation for testing)
pub struct NullBackend {
state: PlayerState,
volume: f32,
position: f64,
duration: Option<f64>,
audio_settings: AudioSettings,
}
impl Default for NullBackend {
fn default() -> Self {
Self::new()
}
}
impl NullBackend {
pub fn new() -> Self {
Self {
state: PlayerState::Idle,
volume: 1.0,
position: 0.0,
duration: None,
audio_settings: AudioSettings::default(),
}
}
}
impl PlayerBackend for NullBackend {
fn load(&mut self, media: &MediaItem) -> Result<(), PlayerError> {
self.state = PlayerState::Loading {
media: media.clone(),
};
// Simulate immediate load
self.duration = media.duration;
self.position = 0.0;
self.state = PlayerState::Paused {
media: media.clone(),
position: 0.0,
duration: media.duration.unwrap_or(0.0),
};
Ok(())
}
fn play(&mut self) -> Result<(), PlayerError> {
if let PlayerState::Paused {
media,
position,
duration,
} = &self.state
{
self.state = PlayerState::Playing {
media: media.clone(),
position: *position,
duration: *duration,
};
}
Ok(())
}
fn pause(&mut self) -> Result<(), PlayerError> {
if let PlayerState::Playing {
media,
position,
duration,
} = &self.state
{
self.state = PlayerState::Paused {
media: media.clone(),
position: *position,
duration: *duration,
};
}
Ok(())
}
fn stop(&mut self) -> Result<(), PlayerError> {
self.state = PlayerState::Idle;
self.position = 0.0;
self.duration = None;
Ok(())
}
fn seek(&mut self, position: f64) -> Result<(), PlayerError> {
self.position = position;
match &mut self.state {
PlayerState::Playing { position: pos, .. } => *pos = position,
PlayerState::Paused { position: pos, .. } => *pos = position,
_ => {}
}
Ok(())
}
fn set_volume(&mut self, volume: f32) -> Result<(), PlayerError> {
self.volume = volume.clamp(0.0, 1.0);
Ok(())
}
fn position(&self) -> f64 {
self.position
}
fn duration(&self) -> Option<f64> {
self.duration
}
fn state(&self) -> PlayerState {
self.state.clone()
}
fn volume(&self) -> f32 {
self.volume
}
fn set_audio_settings(&mut self, settings: &AudioSettings) -> Result<(), PlayerError> {
self.audio_settings = settings.clone().with_crossfade_clamped();
Ok(())
}
fn audio_settings(&self) -> AudioSettings {
self.audio_settings.clone()
}
}
// TRACES: UR-003, UR-004 | DR-004 | UT-026, UT-027, UT-028, UT-029, UT-030, UT-031, UT-032, UT-033
#[cfg(test)]
mod tests {
use super::*;
/// Test NullBackend volume default value
/// TRACES: UR-016 | DR-004 | UT-026
#[test]
fn test_null_backend_volume_default() {
let backend = NullBackend::new();
assert_eq!(backend.volume(), 1.0);
}
/// Test NullBackend set volume
///
/// @req-test: UT-027 - NullBackend set volume
/// @req-test: UR-016 - Change system settings while playing (volume)
#[test]
fn test_null_backend_set_volume() {
let mut backend = NullBackend::new();
backend.set_volume(0.5).unwrap();
assert_eq!(backend.volume(), 0.5);
}
/// Test NullBackend volume clamping (high)
///
/// @req-test: UT-028 - NullBackend volume clamping (high/low)
/// @req-test: UR-016 - Change system settings while playing (volume)
#[test]
fn test_null_backend_volume_clamping_high() {
let mut backend = NullBackend::new();
backend.set_volume(1.5).unwrap();
assert_eq!(backend.volume(), 1.0);
}
/// Test NullBackend volume clamping (low)
///
/// @req-test: UT-028 - NullBackend volume clamping (high/low)
/// @req-test: UR-016 - Change system settings while playing (volume)
#[test]
fn test_null_backend_volume_clamping_low() {
let mut backend = NullBackend::new();
backend.set_volume(-0.5).unwrap();
assert_eq!(backend.volume(), 0.0);
}
/// Test NullBackend volume boundary values
///
/// @req-test: UT-029 - NullBackend volume boundary values
/// @req-test: UR-016 - Change system settings while playing (volume)
#[test]
fn test_null_backend_volume_boundary() {
let mut backend = NullBackend::new();
backend.set_volume(0.0).unwrap();
assert_eq!(backend.volume(), 0.0);
backend.set_volume(1.0).unwrap();
assert_eq!(backend.volume(), 1.0);
}
/// Test NullBackend audio settings default values
///
/// @req-test: DR-034 - Crossfade engine
/// @req-test: DR-035 - Gapless playback
/// @req-test: DR-036 - Volume normalization
#[test]
fn test_null_backend_audio_settings_default() {
let backend = NullBackend::new();
let settings = backend.audio_settings();
assert_eq!(settings.crossfade_duration, 0.0);
assert!(settings.gapless_playback);
assert!(!settings.normalize_volume);
}
/// Test NullBackend set audio settings
///
/// @req-test: DR-034 - Crossfade engine with configurable duration
/// @req-test: DR-035 - Gapless playback between sequential tracks
/// @req-test: DR-036 - Volume normalization with preset levels
#[test]
fn test_null_backend_set_audio_settings() {
use crate::settings::VolumeLevel;
let mut backend = NullBackend::new();
let settings = AudioSettings {
crossfade_duration: 5.0,
gapless_playback: false,
normalize_volume: true,
volume_level: VolumeLevel::Loud,
..Default::default()
};
backend.set_audio_settings(&settings).unwrap();
let result = backend.audio_settings();
assert_eq!(result.crossfade_duration, 5.0);
assert!(!result.gapless_playback);
assert!(result.normalize_volume);
assert_eq!(result.volume_level, VolumeLevel::Loud);
}
/// Test NullBackend audio settings crossfade clamping to 12s max
///
/// @req-test: DR-034 - Crossfade engine with configurable duration (0-12s)
#[test]
fn test_null_backend_audio_settings_crossfade_clamping() {
let mut backend = NullBackend::new();
let settings = AudioSettings {
crossfade_duration: 20.0,
..Default::default()
};
backend.set_audio_settings(&settings).unwrap();
assert_eq!(backend.audio_settings().crossfade_duration, 12.0);
}
/// Test NullBackend seek updates position
///
/// @req-test: UR-005 - Control media playback (scrub operation)
/// @req-test: DR-004 - PlayerBackend trait
#[test]
fn test_null_backend_seek_updates_position() {
use crate::player::media::{MediaItem, MediaSource, MediaType};
let mut backend = NullBackend::new();
// Create a test media item
let media = MediaItem {
// Audio and direct-URL items never negotiate a transport.
transport: None,
id: "test_media".to_string(),
title: "Test Track".to_string(),
name: Some("Test Track".to_string()),
artist: Some("Test Artist".to_string()),
album: Some("Test Album".to_string()),
album_name: Some("Test Album".to_string()),
album_id: None,
artist_items: None,
artists: Some(vec!["Test Artist".to_string()]),
primary_image_tag: None,
image_id: None,
item_type: Some("Audio".to_string()),
playlist_id: None,
duration: Some(180.0),
artwork_url: None,
media_type: MediaType::Audio,
source: MediaSource::DirectUrl {
url: "http://example.com/test.mp3".to_string(),
},
video_codec: None,
needs_transcoding: false,
video_width: None,
video_height: None,
subtitles: vec![],
series_id: None,
server_id: None,
};
// Load and play the media
backend.load(&media).unwrap();
backend.play().unwrap();
// Verify initial position
assert_eq!(backend.position(), 0.0);
// Seek to 30 seconds
backend.seek(30.0).unwrap();
assert_eq!(backend.position(), 30.0);
// Seek to 60 seconds
backend.seek(60.0).unwrap();
assert_eq!(backend.position(), 60.0);
// Seek backward
backend.seek(15.0).unwrap();
assert_eq!(backend.position(), 15.0);
}
/// Test NullBackend seek while paused
///
/// @req-test: UR-005 - Control media playback (scrub while paused)
/// @req-test: DR-001 - Player state machine (seeking from paused state)
#[test]
fn test_null_backend_seek_while_paused() {
use crate::player::media::{MediaItem, MediaSource, MediaType};
let mut backend = NullBackend::new();
let media = MediaItem {
// Audio and direct-URL items never negotiate a transport.
transport: None,
id: "test_media".to_string(),
title: "Test Track".to_string(),
name: Some("Test Track".to_string()),
artist: Some("Test Artist".to_string()),
album: Some("Test Album".to_string()),
album_name: Some("Test Album".to_string()),
album_id: None,
artist_items: None,
artists: Some(vec!["Test Artist".to_string()]),
primary_image_tag: None,
image_id: None,
item_type: Some("Audio".to_string()),
playlist_id: None,
duration: Some(180.0),
artwork_url: None,
media_type: MediaType::Audio,
source: MediaSource::DirectUrl {
url: "http://example.com/test.mp3".to_string(),
},
video_codec: None,
needs_transcoding: false,
video_width: None,
video_height: None,
subtitles: vec![],
series_id: None,
server_id: None,
};
// Load media (starts paused)
backend.load(&media).unwrap();
// Verify state is paused
assert!(matches!(backend.state(), PlayerState::Paused { .. }));
// Seek while paused
backend.seek(45.0).unwrap();
assert_eq!(backend.position(), 45.0);
// Verify still paused
assert!(matches!(backend.state(), PlayerState::Paused { .. }));
}
/// Test NullBackend position updates reflected in state
///
/// @req-test: DR-001 - Player state machine (position tracking)
/// @req-test: UR-005 - Control media playback (position accuracy)
#[test]
fn test_null_backend_position_updates_in_state() {
use crate::player::media::{MediaItem, MediaSource, MediaType};
let mut backend = NullBackend::new();
let media = MediaItem {
// Audio and direct-URL items never negotiate a transport.
transport: None,
id: "test_media".to_string(),
title: "Test Track".to_string(),
name: Some("Test Track".to_string()),
artist: Some("Test Artist".to_string()),
album: Some("Test Album".to_string()),
album_name: Some("Test Album".to_string()),
album_id: None,
artist_items: None,
artists: Some(vec!["Test Artist".to_string()]),
primary_image_tag: None,
image_id: None,
item_type: Some("Audio".to_string()),
playlist_id: None,
duration: Some(180.0),
artwork_url: None,
media_type: MediaType::Audio,
source: MediaSource::DirectUrl {
url: "http://example.com/test.mp3".to_string(),
},
video_codec: None,
needs_transcoding: false,
video_width: None,
video_height: None,
subtitles: vec![],
series_id: None,
server_id: None,
};
backend.load(&media).unwrap();
backend.play().unwrap();
// Seek to 30 seconds
backend.seek(30.0).unwrap();
// Verify the state reflects the new position
if let PlayerState::Playing { position, .. } = backend.state() {
assert_eq!(position, 30.0);
} else {
panic!("Expected Playing state");
}
}
}