fix(playback): stop pinning the video stream as the audio track (DR-140)
Jellyfin's MediaStream.Index is global across every stream in a media source, so index 0 is the video stream on virtually all files. We sent AudioStreamIndex=0 as "the first audio track" on the HLS transcode URL, the background audio-only handoff URL, the direct-play fallback URL and the PlaybackInfo negotiation body — asking the server to use the video stream as audio. Servers that honour it produce a picture with no sound; only those that silently correct the index hid the bug, which is why it surfaced as "some videos have no audio". Omit the parameter unless a track was actually chosen, so the server resolves the source's DefaultAudioStreamIndex. An explicit selection from player_switch_audio_track still passes through unchanged. Dropped outright from the static=true direct-play URL, which serves the original file untouched.
This commit is contained in:
@@ -299,6 +299,7 @@ Internal architecture, components, and application logic.
|
||||
| DR-128 | Audio-only playback of *downloaded* media reads the local file rather than fetching an audio-only stream. No transcode is involved or wanted: the Linux backend already runs MPV with `video: no`, so handing it the downloaded video file decodes the audio track and ignores the video, and ExoPlayer disables its video renderer equivalently. Transcoding to a separate audio artifact would cost CPU and battery, need an encoder the project does not ship, and produce a second file to keep in step — for no gain over simply not decoding the video | Playback | UR-071 | Done |
|
||||
| DR-129 | A stream that stops delivering is recovered, not treated as terminal. Two failure shapes, because the streams differ. (a) *Phantom end* — the background audio-only handoff uses a progressive mp3 transcode over plain HTTP, chunked and therefore length-less, so a dropped connection reaches the player as end-of-input and ExoPlayer reports `STATE_ENDED` indistinguishably from the real end. The item's runtime is the only thing that can tell them apart: an end reported more than a tolerance short of it (comparing the *absolute* position — handoff base plus the player's relative position) is a truncation. Left unhandled, playback parked in `STATE_ENDED` and the next play intent from the lockscreen, notification or a Bluetooth reconnect seeks an ended player to position 0 — the user-visible "the episode randomly restarted". (b) *Recoverable error* — music (`/Audio/{id}/stream?Static=true`) and video (`/Videos/{id}/master.m3u8`) declare their length, so the player detects the truncation itself and raises an error; the frontend's handler stopped playback outright, turning a hiccup into silence. Both resume the current item **in place** (never via `play_item`, which would replace the queue with a single item and lose the album), the error path after a per-attempt backoff. Seekable streams are re-prepared at the URL they already have and seeked; the length-less transcode, which cannot be seeked, has `StartTimeTicks` rewritten into its existing URL so the user's audio-track selection survives and recovery needs no network round-trip. Only `Remote` sources qualify — a local file cannot fail from the network. A shared budget of consecutive attempts at the same position, refilled whenever playback progresses, stops an unreachable server from looping | Playback | UR-040, UR-004 | Done |
|
||||
| DR-130 | A backend's position and duration must survive the end of the file they describe. MPV exposes `time-pos`/`duration` as properties of the *loaded* file, so at EOF it unloads and both stop resolving — the accessors reported `0.0`/unknown at exactly the moment end-of-file handling asks where playback reached, and any position-versus-runtime check would have read every natural end as a truncation. The poll thread records the last reading and the accessors fall back to it. Linux resilience is layered on the same principle that the stream, not the player, is what failed: MPV is configured with ffmpeg reconnection (`stream-lavf-o`, `network-timeout`) so ordinary blips never surface, and `EndFile(ERROR)` — previously a bare log, which left playback halted while the UI still showed "playing" — is emitted as a *recoverable* error. Because MpvBackend is constructed before `PlayerController` exists, it cannot decide in-process like the Android JNI callback: the frontend echoes the error into `player_recover_stream`, which keeps the decision in Rust (the same shape as `PlaybackEnded` → `player_on_playback_ended`). Android reports errors it has already declined as *unrecoverable*, so the echo never asks twice | Playback | UR-004, UR-040 | Done |
|
||||
| DR-140 | An audio track is pinned only when the user picked one. Jellyfin's `MediaStream.Index` is global across every stream in a media source, so index 0 is the *video* stream on virtually all files — yet `AudioStreamIndex=0` was sent as "the first audio track" on the HLS transcode URL, the background audio-only handoff URL, the direct-play fallback URL, and the `PlaybackInfo` negotiation body. A server that honours the request literally then transcodes the video stream into the audio slot and the result plays as a picture with no sound; only servers that silently correct the index hid the bug, which is why it presented as "some videos have no audio". The parameter is now omitted whenever no track has been chosen, so the server resolves the source's `DefaultAudioStreamIndex`; an explicit selection from `player_switch_audio_track` is still carried through unchanged. On the `static=true` direct-play URL it is dropped outright — the original file is served untouched, so the parameter could only mislead | Playback | UR-004, UR-040 | Done |
|
||||
| DR-093 | Traceability coverage gate derives its requirement denominators from `requirements.md` at run time rather than hardcoded literals: `countDefinedRequirements` counts an ID only where it leads a markdown table row (ignoring the "Traces To" column and prose) and deduplicates IDs listed both in the definition tables and in the §3 traceability matrix; `computeCoverage` reports the *intersection* of traced and defined IDs so an ID traced in code but absent from `requirements.md` is surfaced as `orphaned` instead of inflating the ratio past 100%. UT/IT test identifiers are excluded as a separate taxonomy. CI and `bun run traces:coverage` share this computation and fail on both a sub-threshold and an impossible >100% result | Tooling | - | Done |
|
||||
|
||||
---
|
||||
@@ -506,6 +507,7 @@ Internal architecture, components, and application logic.
|
||||
| UT-108 | LRU eviction reclaims only `'auto'` downloads and never a user's own, even when the user's is the oldest | DR-126 | Done |
|
||||
| UT-117 | A background audio-only stream cut short resumes where it died instead of ending the episode; a real end still advances; the absolute position is compared against the runtime; retries at a stuck position give up. A recoverable error resumes music and video too, with growing backoff, leaving the rest of the queue intact and the seekable stream's URL untouched; local and DirectUrl sources are excluded | DR-129 | Done |
|
||||
| UT-121 | An EOF reads as the last observed timestamp, not zero: live readings win while the file is loaded, a not-yet-established duration is not recorded as a real zero, a seek updates the position before the next poll, and loading a new file clears the previous one's | DR-130 | Done |
|
||||
| UT-130 | Video and background-audio stream URLs omit `AudioStreamIndex` when no track was chosen, and carry the exact index when one was | DR-140 | Done |
|
||||
|
||||
### Integration Tests
|
||||
|
||||
|
||||
@@ -382,6 +382,8 @@ impl OnlineRepository {
|
||||
/// forces the server to transcode the whole file before playback can begin —
|
||||
/// which manifests as playback never starting. `StartTimeTicks` makes the
|
||||
/// server begin the transcode at the requested position.
|
||||
///
|
||||
/// TRACES: UR-004 | DR-140 | UT-130
|
||||
pub async fn get_video_stream_url(
|
||||
&self,
|
||||
item_id: &str,
|
||||
@@ -392,9 +394,6 @@ impl OnlineRepository {
|
||||
// Convert seconds to ticks (10,000,000 ticks per second)
|
||||
let start_time_ticks = start_time_seconds.map(|seconds| (seconds * 10_000_000.0) as i64);
|
||||
|
||||
// Use provided audio stream index, or default to 0
|
||||
let audio_index = audio_stream_index.unwrap_or(0).to_string();
|
||||
|
||||
// Build an HLS transcode URL. VideoCodec lists h264 first so the server
|
||||
// transcodes HEVC/10-bit/unsupported sources to h264 the WebView can decode.
|
||||
let mut params = vec![
|
||||
@@ -402,7 +401,6 @@ impl OnlineRepository {
|
||||
("DeviceId", "jellytau-tauri".to_string()),
|
||||
("VideoCodec", "h264".to_string()),
|
||||
("AudioCodec", "aac".to_string()),
|
||||
("AudioStreamIndex", audio_index),
|
||||
("MaxStreamingBitrate", "20000000".to_string()),
|
||||
("VideoBitrate", "18000000".to_string()),
|
||||
("AudioBitrate", "384000".to_string()),
|
||||
@@ -412,6 +410,16 @@ impl OnlineRepository {
|
||||
("TranscodingProtocol", "hls".to_string()),
|
||||
];
|
||||
|
||||
// Only pin an audio track when the user actually picked one. Jellyfin's
|
||||
// `MediaStream.Index` is global across *all* streams in a media source, so
|
||||
// index 0 is the video stream on virtually every file — defaulting to 0
|
||||
// asks the server to transcode the video stream as the audio track, which
|
||||
// yields a picture with no sound. Omitting the param lets the server use
|
||||
// the source's `DefaultAudioStreamIndex`.
|
||||
if let Some(index) = audio_stream_index {
|
||||
params.push(("AudioStreamIndex", index.to_string()));
|
||||
}
|
||||
|
||||
if let Some(source_id) = media_source_id {
|
||||
params.push(("MediaSourceId", source_id.to_string()));
|
||||
}
|
||||
@@ -438,7 +446,7 @@ impl OnlineRepository {
|
||||
/// Get an **audio-only** stream URL for a *video* item, for the
|
||||
/// background-audio handoff (UR-040).
|
||||
///
|
||||
/// TRACES: UR-040 | JA-032 | UT-059
|
||||
/// TRACES: UR-040 | JA-032, DR-140 | UT-059, UT-130
|
||||
///
|
||||
/// This deliberately targets `/Audio/{id}/universal`, NOT the video stream:
|
||||
/// the server extracts/transcodes only the item's audio track and streams
|
||||
@@ -463,13 +471,10 @@ impl OnlineRepository {
|
||||
start_time_seconds: Option<f64>,
|
||||
audio_stream_index: Option<i32>,
|
||||
) -> Result<String, RepoError> {
|
||||
let audio_index = audio_stream_index.unwrap_or(0).to_string();
|
||||
|
||||
let mut params = vec![
|
||||
("UserId", self.user_id.clone()),
|
||||
("api_key", self.access_token.clone()),
|
||||
("DeviceId", "jellytau-tauri".to_string()),
|
||||
("AudioStreamIndex", audio_index),
|
||||
// Progressive mp3 over HTTP — ExoPlayer-friendly; no HLS/ts.
|
||||
("Container", "mp3".to_string()),
|
||||
("AudioCodec", "mp3".to_string()),
|
||||
@@ -478,6 +483,12 @@ impl OnlineRepository {
|
||||
("MaxStreamingBitrate", "384000".to_string()),
|
||||
];
|
||||
|
||||
// Carry the track over only if one was actually selected — index 0 is the
|
||||
// video stream, not "the first audio track" (see `get_video_stream_url`).
|
||||
if let Some(index) = audio_stream_index {
|
||||
params.push(("AudioStreamIndex", index.to_string()));
|
||||
}
|
||||
|
||||
if let Some(source_id) = media_source_id {
|
||||
params.push(("MediaSourceId", source_id.to_string()));
|
||||
}
|
||||
@@ -1239,7 +1250,11 @@ impl MediaRepository for OnlineRepository {
|
||||
#[serde(rename_all = "PascalCase")]
|
||||
struct PlaybackInfoRequest {
|
||||
user_id: String,
|
||||
audio_stream_index: i32,
|
||||
/// Omitted so the server resolves the source's default audio stream.
|
||||
/// Never send 0 here: the index is global across all streams, so 0 is
|
||||
/// the video stream and the negotiated source comes back soundless.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
audio_stream_index: Option<i32>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
subtitle_stream_index: Option<i32>,
|
||||
start_time_ticks: i64,
|
||||
@@ -1399,7 +1414,7 @@ impl MediaRepository for OnlineRepository {
|
||||
// POST to PlaybackInfo with device profile containing detected codecs
|
||||
let request_body = PlaybackInfoRequest {
|
||||
user_id: self.user_id.clone(),
|
||||
audio_stream_index: 0, // Request first audio stream
|
||||
audio_stream_index: None, // Let the server pick the source default
|
||||
subtitle_stream_index: None,
|
||||
start_time_ticks: 0,
|
||||
is_playback: true,
|
||||
@@ -1430,9 +1445,11 @@ impl MediaRepository for OnlineRepository {
|
||||
let stream_url = if let Some(transcoding_url) = &source.transcoding_url {
|
||||
format!("{}{}", self.server_url, transcoding_url)
|
||||
} else {
|
||||
// Fall back to direct stream URL
|
||||
// Fall back to direct stream URL. No audioStreamIndex: static=true
|
||||
// serves the original file untouched, and pinning index 0 (the video
|
||||
// stream) only misleads servers that do honour it.
|
||||
format!(
|
||||
"{}/Videos/{}/stream?static=true&container=mp4&mediaSourceId={}&deviceId=jellytau&api_key={}&audioStreamIndex=0&userId={}",
|
||||
"{}/Videos/{}/stream?static=true&container=mp4&mediaSourceId={}&deviceId=jellytau&api_key={}&userId={}",
|
||||
self.server_url,
|
||||
item_id,
|
||||
source.id,
|
||||
@@ -2267,8 +2284,14 @@ mod tests {
|
||||
assert!(url.starts_with("https://test.server.com/Videos/vid-1/master.m3u8?"));
|
||||
assert!(!url.contains("StartTimeTicks"));
|
||||
assert!(!url.contains("MediaSourceId"));
|
||||
// Defaults to first audio stream
|
||||
assert!(url.contains("AudioStreamIndex=0"));
|
||||
// With no track chosen, the param must be OMITTED so the server picks the
|
||||
// source's DefaultAudioStreamIndex. `MediaStream.Index` is global across
|
||||
// all streams of a source, so index 0 is the *video* stream on virtually
|
||||
// every file — sending it asks for a "audio track" that has no audio.
|
||||
assert!(
|
||||
!url.contains("AudioStreamIndex"),
|
||||
"must not pin an audio index when none was chosen: {url}"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -2325,8 +2348,12 @@ mod tests {
|
||||
assert!(url.starts_with("https://test.server.com/Audio/vid-1/universal?"));
|
||||
assert!(!url.contains("StartTimeTicks"));
|
||||
assert!(!url.contains("MediaSourceId"));
|
||||
// Defaults to first audio stream.
|
||||
assert!(url.contains("AudioStreamIndex=0"));
|
||||
// Same as the video path: omit rather than pin index 0 (the video stream),
|
||||
// and let the server fall back to the source's default audio stream.
|
||||
assert!(
|
||||
!url.contains("AudioStreamIndex"),
|
||||
"must not pin an audio index when none was chosen: {url}"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
||||
Reference in New Issue
Block a user