dtourolle/jellytau
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 39 Wiki Activity

fix(player): change the audio track, and load subtitles at all

Browse Source 
Two faults, both present since v0.0.1, both found and confirmed on a device.

Audio track (DR-258). Jellyfin builds a transcode around one AudioStreamIndex,
so the alternate tracks are not in the stream that arrives — but the native
path only ever called setAudioTrack(n), which indexes ExoPlayer's audio track
*groups*. On Android that is the common case, since any source whose default
audio codec the device cannot decode is transcoded: logcat showed ExoPlayer
holding `Audio tracks: 1` while the menu listed every track in the file, so
each selection warned `Invalid audio track index` and was dropped, leaving the
default track playing with nothing in the UI saying so.

determine_audio_track_switch_strategy now decides by whether the stream in
front of the engine carries the track at all — a direct play still selects in
place, a transcode is re-negotiated at the chosen index and resumed. Where it
resumes is the player's answer rather than the UI's: the native path has no
<video> element to read, so it sends no position, and defaulting that to zero
re-opened the film at the beginning (caught on device before it shipped).

Subtitles (DR-259). The URL was missing its `Stream.` route segment, so every
sideloaded subtitle 404ed; since media3 1.5 a sideloaded text track only
becomes a track group once its file is parsed, so 42 failed fetches left
ExoPlayer with no text tracks and selection warned `available: 0`. Verified
against a live server: the built URL answers 404, the corrected one 200. The
tests that should have caught this asserted the shape of a mock helper that
restated the format string instead of the URL the app requests — so the new
test drives the repository itself, and failed red on the old URL.
This commit is contained in:
dtourolle
2026-08-23 19:15:05 +02:00
parent 231ffae626
commit 64de22bd51
8 changed files with 397 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src-tauri/src/commands/player/mod.rs
+141 -49
View File
@@ -25,8 +25,9 @@ use super::DatabaseWrapper;
use crate::download::cache::{CacheConfig, SmartCache};
use crate::jellyfin::{JellyfinClient, JellyfinConfig};
use crate::player::{
determine_video_seek_strategy, MediaItem, MediaSessionManager, MediaSource, MediaType,
PlayerController, PlayerState, PlayerStatusEvent, QueueContext, RepeatMode, VideoSeekStrategy,
determine_audio_track_switch_strategy, determine_video_seek_strategy, AudioTrackSwitchStrategy,
MediaItem, MediaSessionManager, MediaSource, MediaType, PlayerController, PlayerState,
PlayerStatusEvent, QueueContext, RepeatMode, VideoSeekStrategy,
};
use crate::repository::{
types::{GetItemsOptions, ImageOptions, ImageType},
@@ -1595,18 +1596,37 @@ pub async fn player_seek_video(
}
}
/// Switch audio track - handles both HTML5 (stream reload) and native (direct switch)
/// Switch audio track.
/// Note: Frontend should handle saving series preferences after this command succeeds
///
/// The split is the requirement: an HTML5 `<video>` element cannot be told to
/// change audio track, so the stream is re-opened at the chosen
/// `AudioStreamIndex` and the frontend seeks the reloaded element back to
/// `position`; a native backend (ExoPlayer) switches in place by track-group
/// index. libmpv implements neither — it is the audio-only backend here and
/// leaves `PlayerBackend::set_audio_track` at its `not_implemented()` default,
/// which is why IR-019 is met by these two paths rather than by MPV.
/// What decides the route is **whether the stream in front of the engine
/// carries the requested track at all** — see
/// [`determine_audio_track_switch_strategy`]:
///
/// TRACES: UR-021 | IR-019, DR-024
/// - An HTML5 `<video>` element has no track-selection API, so the stream is
/// always re-opened at the chosen `AudioStreamIndex` and the frontend seeks
/// the reloaded element back to `position`.
/// - A native backend playing a **direct play** holds the source file with
/// every track in it, so ExoPlayer selects in place by track-group index.
/// - A native backend playing a **transcode** does not. Jellyfin builds a
/// transcode around one `AudioStreamIndex`, so the alternate tracks are not
/// in the stream; the switch has to re-open it, which this command does
/// itself and resumes at `current_position`.
///
/// That last case is a bug fix, and it was the common case on Android: any
/// source whose default audio codec the device cannot decode is transcoded, so
/// ExoPlayer saw `Audio tracks: 1` while the menu listed every track in the
/// file. The old code called `setAudioTrack(n)` regardless, which indexes
/// ExoPlayer's audio track *groups*, found nothing at `n`, warned `Invalid
/// audio track index` and dropped the request — the default track just kept
/// playing, with nothing in the UI saying so.
///
/// libmpv implements neither selection nor reload here — it is the audio-only
/// backend and leaves `PlayerBackend::set_audio_track` at its
/// `not_implemented()` default, which is why IR-019 is met by these paths
/// rather than by MPV.
///
/// TRACES: UR-021, UR-005 | IR-019, DR-024, DR-258
#[tauri::command]
#[specta::specta]
// Two of the eight arguments are Tauri `State<'_, _>` injections, not caller
@@ -1626,56 +1646,128 @@ pub async fn player_switch_audio_track(
info!("[player_switch_audio_track] Switching to audio track - stream_index: {}, array_index: {}, use_html5: {}",
stream_index, array_index, use_html5);
if use_html5 {
// HTML5 backend needs stream reload
let repository = repository_manager
.0
.get(&repository_handle)
.ok_or("Repository not found - user may need to log in")?;
// Read what the engine is playing before deciding anything — including
// where it is, which has to be captured before the stop below wipes it.
// Locks are dropped at the end of this block so none is held across an
// await.
let (jellyfin_item_id, needs_transcoding, engine_position) = {
let controller = player.0.lock().await;
let engine_position = controller.absolute_position();
let queue_arc = controller.queue();
let queue = queue_arc.lock().map_err(|e| e.to_string())?;
// Get current item to find Jellyfin ID
let jellyfin_item_id = {
let controller = player.0.lock().await;
let queue_arc = controller.queue();
let queue = queue_arc.lock().map_err(|e| e.to_string())?;
let current_item = queue.current().ok_or("No item currently playing")?;
let current_item = queue.current().ok_or("No item currently playing")?;
(
current_item
.jellyfin_id()
.ok_or("Current item has no Jellyfin ID")?
.to_string()
};
.to_string(),
current_item.needs_transcoding,
engine_position,
)
};
// Select a stream carrying the chosen audio track. It starts at zero —
// an HLS playlist cannot carry a position (DR-181) — and `position`
// below tells the frontend where to seek the reloaded element back to.
//
// Pinning a track is itself a reason the source cannot be direct-played:
// the file has one default track and the viewer asked for another, so
// the negotiation returns a transcode. That decision lives in
// `decide_playback_kind`, not here.
let selection = repository
.get_stream_selection(
&jellyfin_item_id,
media_source_id.as_deref(),
Some(stream_index),
)
.await
.map_err(|e| format!("Failed to select a stream: {:?}", e))?;
let strategy = determine_audio_track_switch_strategy(needs_transcoding, use_html5);
Ok(AudioTrackSwitchResponse::ReloadStream {
selection,
position: current_position.unwrap_or(0.0),
})
} else {
// Native backend (Android ExoPlayer) - use array index
info!(
"[player_switch_audio_track] needs_transcoding={}, use_html5={}, strategy={:?}",
needs_transcoding, use_html5, strategy
);
if strategy == AudioTrackSwitchStrategy::BackendSelectInPlace {
// A direct play: the engine holds the source file, every track included.
let controller = player.0.lock().await;
controller
.set_audio_track(array_index)
.map_err(|e| e.to_string())?;
Ok(AudioTrackSwitchResponse::Native { success: true })
return Ok(AudioTrackSwitchResponse::Native { success: true });
}
// Both reload strategies need a stream built around the chosen track.
let repository = repository_manager
.0
.get(&repository_handle)
.ok_or("Repository not found - user may need to log in")?;
// Select a stream carrying the chosen audio track. It starts at zero —
// an HLS playlist cannot carry a position (DR-181) — so the position is
// restored by seeking afterwards, here or in the frontend.
//
// Pinning a track is itself a reason the source cannot be direct-played:
// the file has one default track and the viewer asked for another, so
// the negotiation returns a transcode. That decision lives in
// `decide_playback_kind`, not here.
let selection = repository
.get_stream_selection(
&jellyfin_item_id,
media_source_id.as_deref(),
Some(stream_index),
)
.await
.map_err(|e| format!("Failed to select a stream: {:?}", e))?;
// The caller's position if it has one, the engine's otherwise. The native
// path has no `<video>` element to read, so it sends none — and defaulting
// that to zero re-opened the stream at the start of the film.
let position = crate::player::track_switch::resume_position(current_position, engine_position);
match strategy {
AudioTrackSwitchStrategy::Html5ReloadStream => Ok(AudioTrackSwitchResponse::ReloadStream {
selection,
position,
}),
AudioTrackSwitchStrategy::BackendReloadStream => {
// The native backend re-opens its own stream, the same sequence the
// transcoded seek and quality change use: stop, repoint the queue
// entry at the new URL, load, then seek back to where the viewer
// was. Nothing is left for the frontend to do.
let new_url = selection.url.clone();
{
let controller = player.0.lock().await;
controller.stop().map_err(|e| e.to_string())?;
}
{
let controller = player.0.lock().await;
let queue_arc = controller.queue();
let mut queue = queue_arc.lock().map_err(|e| e.to_string())?;
if !queue.update_current_stream_url(new_url) {
return Err("Failed to update stream URL in queue".to_string());
}
}
{
let controller = player.0.lock().await;
let queue_arc = controller.queue();
let queue = queue_arc.lock().map_err(|e| e.to_string())?;
let updated_item = queue
.current()
.ok_or("No current item after URL update")?
.clone();
drop(queue);
controller
.load_and_play(&updated_item)
.map_err(|e| e.to_string())?;
controller.seek(position).map_err(|e| e.to_string())?;
}
info!(
"[player_switch_audio_track] Re-opened the stream on audio stream {} and resumed at {}",
stream_index, position
);
Ok(AudioTrackSwitchResponse::Native { success: true })
}
// Handled above, before the stream was negotiated.
AudioTrackSwitchStrategy::BackendSelectInPlace => {
Ok(AudioTrackSwitchResponse::Native { success: true })
}
}
}
src-tauri/src/player/mod.rs
+2
View File
@@ -23,6 +23,7 @@ pub mod session;
pub mod sleep_timer;
pub mod state;
pub mod stream_end;
pub mod track_switch;
#[cfg(test)]
mod mpv_backend_test;
@@ -71,6 +72,7 @@ pub use seek::{determine_video_seek_strategy, VideoSeekStrategy};
pub use session::{MediaSessionManager, MediaSessionType};
pub use sleep_timer::{SleepTimerMode, SleepTimerState};
pub use state::{EndReason, PlayerState};
pub use track_switch::{determine_audio_track_switch_strategy, AudioTrackSwitchStrategy};
// Re-export platform-specific backends
#[cfg(target_os = "android")]
src-tauri/src/player/track_switch.rs
+155
View File
@@ -0,0 +1,155 @@
//! Audio-track switch strategy decision logic.
//!
//! Pure logic, extracted from the command layer so it can be unit-tested in the
//! player core — the sibling of [`super::seek`]. `player_switch_audio_track`
//! turns the resulting [`AudioTrackSwitchStrategy`] into a concrete action.
//!
//! The rule this module exists to state: **an engine can only select a track
//! the stream in front of it actually carries.** A Jellyfin transcode is built
//! around one `AudioStreamIndex`, so the alternate tracks are not in the stream
//! at all — the switch has to re-open it. Only a direct play/stream hands the
//! engine the source file with every track present.
/// How a request to change audio track has to be carried out.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AudioTrackSwitchStrategy {
/// Re-open the stream pinned to the chosen track; the frontend reloads its
/// `<video>` element. An HTML5 element cannot select an audio track at all,
/// so this holds whether or not the current stream is a transcode.
Html5ReloadStream,
/// Re-open the stream pinned to the chosen track; the backend reloads
/// itself and restores the position.
BackendReloadStream,
/// The engine already holds every track — select in place, no reload.
BackendSelectInPlace,
}
/// Decide how to honour an audio-track change.
///
/// # Arguments
/// * `needs_transcoding` - Whether the stream now playing is a server-side
/// transcode, which carries exactly the one audio track it was built around.
/// * `use_html5` - Whether the frontend `<video>` element is rendering.
///
/// TRACES: UR-021 | IR-019, DR-024, DR-258 | UT-232
pub fn determine_audio_track_switch_strategy(
needs_transcoding: bool,
use_html5: bool,
) -> AudioTrackSwitchStrategy {
if use_html5 {
return AudioTrackSwitchStrategy::Html5ReloadStream;
}
if needs_transcoding {
AudioTrackSwitchStrategy::BackendReloadStream
} else {
AudioTrackSwitchStrategy::BackendSelectInPlace
}
}
/// Where to resume after re-opening the stream for a track change.
///
/// `requested` is what the caller supplied; `engine_position` is where the
/// engine itself says it is. The caller wins when it has something real to say,
/// and the engine answers otherwise — which is the whole point: **position is
/// the player's to know**, not the UI's to remember.
///
/// The native path proved why. It has no `<video>` element, so the frontend
/// sent `null`, the command defaulted to `0.0`, and switching audio track
/// re-opened the stream at the beginning of the film — the track changed and
/// the viewer lost their place. A non-finite or negative value is treated the
/// same as absent rather than passed through to a backend that would reject it.
///
/// TRACES: UR-021, UR-005 | IR-019, DR-024, DR-258 | UT-233
pub fn resume_position(requested: Option<f64>, engine_position: f64) -> f64 {
let usable = requested.filter(|p| p.is_finite() && *p > 0.0);
let fallback = if engine_position.is_finite() && engine_position > 0.0 {
engine_position
} else {
0.0
};
usable.unwrap_or(fallback)
}
#[cfg(test)]
mod tests {
use super::*;
/// The reported bug, seen on a device: switching audio track changed the
/// track but "restarts from zero". The native path has no `<video>`
/// element, so the frontend passed `null` and the re-opened stream began at
/// the start of the film — logcat: `Re-opened the stream on audio stream 2
/// and resumed at 0` while playback was 22 minutes in.
#[test]
fn a_caller_with_no_position_resumes_where_the_engine_is() {
assert_eq!(resume_position(None, 1337.5), 1337.5);
}
/// The HTML5 path does have an element and its clock is the honest answer
/// there, so what the caller supplies wins.
#[test]
fn a_caller_that_knows_its_position_is_believed() {
assert_eq!(resume_position(Some(42.0), 1337.5), 42.0);
}
/// A position that is not a position — NaN from an element with no
/// metadata, or a negative from a clock read mid-teardown — is treated as
/// absent. Passing it through re-opens at a place no backend accepts.
#[test]
fn a_nonsense_position_falls_back_to_the_engine() {
assert_eq!(resume_position(Some(f64::NAN), 90.0), 90.0);
assert_eq!(resume_position(Some(-5.0), 90.0), 90.0);
assert_eq!(resume_position(None, f64::NAN), 0.0);
}
/// Switching track in the first moments of playback resumes at the start,
/// which is where the viewer actually is.
#[test]
fn the_very_beginning_stays_the_very_beginning() {
assert_eq!(resume_position(None, 0.0), 0.0);
}
/// The reported bug: on Android the audio-track menu did nothing and the
/// default track kept playing.
///
/// Jellyfin had negotiated a transcode (`TranscodeReasons=AudioCodecNot
/// Supported`) whose URL pins `AudioStreamIndex=1`, so ExoPlayer was handed
/// a stream with exactly one audio track — logcat: `Audio tracks: 1`. The
/// native path nonetheless only ever called `setAudioTrack(n)`, which
/// indexes ExoPlayer's audio track *groups* and so found nothing to select:
/// `Invalid audio track index: 1 (available: 1)`, warned and dropped. The
/// track the viewer asked for is not in the stream; it has to be re-opened.
#[test]
fn a_transcode_is_re_opened_because_it_carries_only_one_track() {
assert_eq!(
determine_audio_track_switch_strategy(true, false),
AudioTrackSwitchStrategy::BackendReloadStream
);
}
/// A direct play hands the engine the source file, every track included, so
/// ExoPlayer selects in place — no reload, no re-buffer, no lost position.
#[test]
fn a_direct_play_switches_in_place() {
assert_eq!(
determine_audio_track_switch_strategy(false, false),
AudioTrackSwitchStrategy::BackendSelectInPlace
);
}
/// An HTML5 `<video>` element has no track-selection API, so it reloads
/// either way. This is the path that already worked, and it must keep
/// working: the fix is about the native side only.
#[test]
fn html5_always_reloads_because_the_element_cannot_select() {
assert_eq!(
determine_audio_track_switch_strategy(true, true),
AudioTrackSwitchStrategy::Html5ReloadStream
);
assert_eq!(
determine_audio_track_switch_strategy(false, true),
AudioTrackSwitchStrategy::Html5ReloadStream
);
}
}
src-tauri/src/repository/online.rs
+37 -1
View File
@@ -2469,8 +2469,15 @@ impl MediaRepository for OnlineRepository {
stream_index: i32,
format: &str,
) -> String {
// `Stream.{format}` is the route, not a filename we get to choose:
// Jellyfin exposes the subtitle as
// `/Videos/{item}/{source}/Subtitles/{index}/Stream.{format}`, and
// stopping at the format alone matches no route and 404s. Every
// sideloaded subtitle failed to load on Android because of it, leaving
// ExoPlayer with no text tracks to select.
// TRACES: UR-020 | JA-008, DR-259 | UT-234
format!(
"{}/Videos/{}/{}/Subtitles/{}/{}",
"{}/Videos/{}/{}/Subtitles/{}/Stream.{}",
self.server_url, item_id, media_source_id, stream_index, format
)
}
@@ -3019,6 +3026,35 @@ mod tests {
)
}
/// The reported bug: on Android every subtitle track was inert — the menu
/// listed 42 languages and picking one changed nothing.
///
/// The cause is here rather than in the player. ExoPlayer sideloads each
/// subtitle as its own media source, and since media3 1.5 a sideloaded text
/// track only becomes a *track group* once its file has been fetched and
/// parsed. Every fetch 404ed, so `Tracks` carried no text group at all and
/// `setSubtitleTrack(1)` warned `available: 0` and dropped the request.
///
/// Jellyfin's route is `/Videos/{item}/{source}/Subtitles/{index}/Stream.{fmt}`
/// (verified against a live server: this shape answers 200, the one built
/// here answered 404). The `Stream.` segment is not decoration — without it
/// the path matches no route.
///
/// The old mock-based URL tests could not catch this: they asserted the
/// shape of a *test helper* that duplicated the format string, not of the
/// URL the app actually requests.
///
/// TRACES: UR-020 | JA-008, DR-259 | UT-234
#[test]
fn subtitle_url_uses_jellyfins_stream_route() {
let repo = create_test_repository();
assert_eq!(
repo.get_subtitle_url("item123", "source456", 2, "vtt"),
"https://test.server.com/Videos/item123/source456/Subtitles/2/Stream.vtt"
);
}
/// Build a repository wired to a real ConnectivityReporter so we can assert
/// how `report_outcome` classifies each `RepoError` into reachability.
/// (No app handle → event emission is a harmless no-op.)