fix(playback): bound the device profile by the audio route's channels (DR-141)

MediaCodecList answers "can this device decode 5.1", which is not the
question that decides whether the user hears anything: a phone decodes an
AC-3 5.1 track happily and still has two channels to play it out of. The
DeviceProfile carried no MaxAudioChannels, so Jellyfin was free to
direct-play the multichannel track to a two-channel sink — silence or
dialogue folded into surround channels that go nowhere, depending on the
device.

Report media3 AudioCapabilities.maxChannelCount for the current route over
JNI alongside the codec lists, and bound the direct-play and transcoding
profiles (and the HLS URL's TranscodingMaxAudioChannels, previously
hardcoded to 2) by it. No codec is ever removed, so a device with genuine
surround output keeps direct-playing it. A missing or zero reading means
"route not yet established", not "no audio", and falls back to stereo.
This commit is contained in:
2026-08-09 15:01:57 +02:00
parent db520c6551
commit a53042fe80
7 changed files with 199 additions and 15 deletions
+2
View File
@@ -300,6 +300,7 @@ Internal architecture, components, and application logic.
| DR-129 | A stream that stops delivering is recovered, not treated as terminal. Two failure shapes, because the streams differ. (a) *Phantom end* — the background audio-only handoff uses a progressive mp3 transcode over plain HTTP, chunked and therefore length-less, so a dropped connection reaches the player as end-of-input and ExoPlayer reports `STATE_ENDED` indistinguishably from the real end. The item's runtime is the only thing that can tell them apart: an end reported more than a tolerance short of it (comparing the *absolute* position — handoff base plus the player's relative position) is a truncation. Left unhandled, playback parked in `STATE_ENDED` and the next play intent from the lockscreen, notification or a Bluetooth reconnect seeks an ended player to position 0 — the user-visible "the episode randomly restarted". (b) *Recoverable error* — music (`/Audio/{id}/stream?Static=true`) and video (`/Videos/{id}/master.m3u8`) declare their length, so the player detects the truncation itself and raises an error; the frontend's handler stopped playback outright, turning a hiccup into silence. Both resume the current item **in place** (never via `play_item`, which would replace the queue with a single item and lose the album), the error path after a per-attempt backoff. Seekable streams are re-prepared at the URL they already have and seeked; the length-less transcode, which cannot be seeked, has `StartTimeTicks` rewritten into its existing URL so the user's audio-track selection survives and recovery needs no network round-trip. Only `Remote` sources qualify — a local file cannot fail from the network. A shared budget of consecutive attempts at the same position, refilled whenever playback progresses, stops an unreachable server from looping | Playback | UR-040, UR-004 | Done |
| DR-130 | A backend's position and duration must survive the end of the file they describe. MPV exposes `time-pos`/`duration` as properties of the *loaded* file, so at EOF it unloads and both stop resolving — the accessors reported `0.0`/unknown at exactly the moment end-of-file handling asks where playback reached, and any position-versus-runtime check would have read every natural end as a truncation. The poll thread records the last reading and the accessors fall back to it. Linux resilience is layered on the same principle that the stream, not the player, is what failed: MPV is configured with ffmpeg reconnection (`stream-lavf-o`, `network-timeout`) so ordinary blips never surface, and `EndFile(ERROR)` — previously a bare log, which left playback halted while the UI still showed "playing" — is emitted as a *recoverable* error. Because MpvBackend is constructed before `PlayerController` exists, it cannot decide in-process like the Android JNI callback: the frontend echoes the error into `player_recover_stream`, which keeps the decision in Rust (the same shape as `PlaybackEnded``player_on_playback_ended`). Android reports errors it has already declined as *unrecoverable*, so the echo never asks twice | Playback | UR-004, UR-040 | Done |
| DR-140 | An audio track is pinned only when the user picked one. Jellyfin's `MediaStream.Index` is global across every stream in a media source, so index 0 is the *video* stream on virtually all files — yet `AudioStreamIndex=0` was sent as "the first audio track" on the HLS transcode URL, the background audio-only handoff URL, the direct-play fallback URL, and the `PlaybackInfo` negotiation body. A server that honours the request literally then transcodes the video stream into the audio slot and the result plays as a picture with no sound; only servers that silently correct the index hid the bug, which is why it presented as "some videos have no audio". The parameter is now omitted whenever no track has been chosen, so the server resolves the source's `DefaultAudioStreamIndex`; an explicit selection from `player_switch_audio_track` is still carried through unchanged. On the `static=true` direct-play URL it is dropped outright — the original file is served untouched, so the parameter could only mislead | Playback | UR-004, UR-040 | Done |
| DR-141 | One search input per screen, and the URL is the search's single source of truth. The header bar rendered only under `/library/**` and merely *navigated* to `/search` (DR-063), so a desktop search handed the user to a screen whose input was a different element — the header box cleared itself and vanished, and the page's own box took over mid-word. That page then re-derived its input from `?q=` against `library.searchQuery` on every store write, so the next keystroke re-ran the effect and snapped the text back to the query the header had sent (and a scope chip back to the URL's scope); entering from the bottom-nav Search tab skipped it only because the effect early-returned on an empty query. The bar now renders on `/search` too (`showHeaderSearch`) and is the sole md+ input — the page's own input is `md:hidden` — and on that route it republishes the query into the URL with `replaceState`, so a whole session of typing costs one history entry. The page *consumes* that URL once per distinct value (`seedFromSearchUrl` against a non-reactive `applied` marker) instead of continuously reconciling it, and the scope chips publish through the same URL so the bar and the chips cannot disagree. Landing on `/search` with a seeded query focuses the bar and puts the caret at the end, because the box the user was typing in belonged to the unmounted route | UI | UR-049, UR-054 | Done |
| DR-093 | Traceability coverage gate derives its requirement denominators from `requirements.md` at run time rather than hardcoded literals: `countDefinedRequirements` counts an ID only where it leads a markdown table row (ignoring the "Traces To" column and prose) and deduplicates IDs listed both in the definition tables and in the §3 traceability matrix; `computeCoverage` reports the *intersection* of traced and defined IDs so an ID traced in code but absent from `requirements.md` is surfaced as `orphaned` instead of inflating the ratio past 100%. UT/IT test identifiers are excluded as a separate taxonomy. CI and `bun run traces:coverage` share this computation and fail on both a sub-threshold and an impossible >100% result | Tooling | - | Done |
---
@@ -508,6 +509,7 @@ Internal architecture, components, and application logic.
| UT-117 | A background audio-only stream cut short resumes where it died instead of ending the episode; a real end still advances; the absolute position is compared against the runtime; retries at a stuck position give up. A recoverable error resumes music and video too, with growing backoff, leaving the rest of the queue intact and the seekable stream's URL untouched; local and DirectUrl sources are excluded | DR-129 | Done |
| UT-121 | An EOF reads as the last observed timestamp, not zero: live readings win while the file is loaded, a not-yet-established duration is not recorded as a real zero, a seek updates the position before the next poll, and loading a new file clears the previous one's | DR-130 | Done |
| UT-130 | Video and background-audio stream URLs omit `AudioStreamIndex` when no track was chosen, and carry the exact index when one was | DR-140 | Done |
| UT-131 | The Episode Focus View hero offers a download control | DR-142 | Done |
### Integration Tests
@@ -1,7 +1,11 @@
package com.dtourolle.jellytau.player
import android.content.Context
import android.media.MediaCodecList
import android.util.Log
import androidx.media3.common.AudioAttributes
import androidx.media3.common.util.UnstableApi
import androidx.media3.exoplayer.audio.AudioCapabilities
/**
* Detects hardware codec capabilities using MediaCodecList.
@@ -75,6 +79,36 @@ object CodecDetector {
)
}
/**
* Report how many channels the *current audio output route* can voice.
*
* This is a different question from "can this device decode 5.1", which the
* codec list already answers: a phone decodes a 5.1 AC-3 track happily and
* still has only two channels to play it out of. Left unreported, Jellyfin
* is free to direct-play the multichannel track, and what the user hears is
* device dependent a failed AudioSink configuration (silence) or dialogue
* folded into the surround channels and lost.
*
* Returns 0 when there is no answer; Rust reads that as "unknown" and falls
* back to stereo rather than claiming a capability we have not observed.
*/
@UnstableApi
fun detectMaxAudioChannels(context: Context): Int {
return try {
val capabilities = AudioCapabilities.getCapabilities(
context,
AudioAttributes.DEFAULT,
/* routedDevice= */ null
)
val channels = capabilities.maxChannelCount
Log.i(TAG, "Audio route max channel count: $channels")
channels.coerceAtLeast(0)
} catch (e: Exception) {
Log.e(TAG, "Error querying audio capabilities", e)
0
}
}
/**
* Map Android MIME types to Jellyfin codec names.
*
@@ -130,7 +130,7 @@ class JellyTauPlayer(private val appContext: Context) {
}
// Detect and report hardware codec capabilities to Rust
detectAndReportCodecs()
detectAndReportCodecs(context.applicationContext)
}
}
}
@@ -141,23 +141,31 @@ class JellyTauPlayer(private val appContext: Context) {
* Called during player initialization.
*/
@JvmStatic
fun detectAndReportCodecs() {
fun detectAndReportCodecs(context: Context) {
val capabilities = CodecDetector.detectHardwareCodecs()
// Convert lists to comma-separated strings for JNI transfer
val videoCodecsStr = capabilities.videoCodecs.joinToString(",")
val audioCodecsStr = capabilities.audioCodecs.joinToString(",")
android.util.Log.i("JellyTauPlayer", "Reporting codecs to Rust: video=$videoCodecsStr, audio=$audioCodecsStr")
// What the route can *voice*, which the codec list does not answer.
// 0 means "no answer"; Rust falls back to stereo.
val maxAudioChannels = CodecDetector.detectMaxAudioChannels(context)
android.util.Log.i("JellyTauPlayer", "Reporting codecs to Rust: video=$videoCodecsStr, audio=$audioCodecsStr, maxAudioChannels=$maxAudioChannels")
// Call native method to store in Rust
nativeOnCodecsDetected(videoCodecsStr, audioCodecsStr)
nativeOnCodecsDetected(videoCodecsStr, audioCodecsStr, maxAudioChannels)
}
/**
* Native method to report detected codecs to Rust.
*/
private external fun nativeOnCodecsDetected(videoCodecs: String, audioCodecs: String)
private external fun nativeOnCodecsDetected(
videoCodecs: String,
audioCodecs: String,
maxAudioChannels: Int
)
/**
* Check if the player is initialized.
+33 -7
View File
@@ -58,11 +58,20 @@ static POSITION_THROTTLER: OnceLock<Arc<EventThrottler>> = OnceLock::new();
struct DetectedCodecs {
video_codecs: Vec<String>,
audio_codecs: Vec<String>,
/// Channels the *current audio output route* accepts, as reported by
/// media3's `AudioCapabilities`. Distinct from the codec lists: a device
/// decodes 5.1 happily and still has only two channels to play it out of.
/// `None` when the platform had no answer.
max_audio_channels: Option<u32>,
}
impl DetectedCodecs {
/// Create from comma-separated codec strings (from JNI)
fn from_jni_strings(video_codecs: &str, audio_codecs: &str) -> Self {
fn from_jni_strings(
video_codecs: &str,
audio_codecs: &str,
max_audio_channels: Option<u32>,
) -> Self {
Self {
video_codecs: video_codecs
.split(',')
@@ -74,6 +83,7 @@ impl DetectedCodecs {
.filter(|s| !s.is_empty())
.map(|s| s.to_string())
.collect(),
max_audio_channels,
}
}
@@ -88,11 +98,19 @@ impl DetectedCodecs {
}
}
/// Public function to get detected codecs (for use in repository layer)
pub fn get_detected_codecs() -> Option<(String, String)> {
DETECTED_CODECS
.get()
.map(|codecs| (codecs.video_codecs_string(), codecs.audio_codecs_string()))
/// Public function to get detected codecs (for use in repository layer).
///
/// Returns `(video, audio, max_audio_channels)` — the third element is how many
/// channels the current audio output can actually voice, which bounds what the
/// server may direct-play.
pub fn get_detected_codecs() -> Option<(String, String, Option<u32>)> {
DETECTED_CODECS.get().map(|codecs| {
(
codecs.video_codecs_string(),
codecs.audio_codecs_string(),
codecs.max_audio_channels,
)
})
}
/// Trait for handling media commands from Android MediaSession.
@@ -1125,6 +1143,7 @@ pub extern "system" fn Java_com_dtourolle_jellytau_player_JellyTauPlayer_00024Co
_class: JClass,
video_codecs: JString,
audio_codecs: JString,
max_audio_channels: jint,
) {
let video_str: String = env
.get_string(&video_codecs)
@@ -1136,7 +1155,10 @@ pub extern "system" fn Java_com_dtourolle_jellytau_player_JellyTauPlayer_00024Co
.map(|s| s.into())
.unwrap_or_default();
let codecs = DetectedCodecs::from_jni_strings(&video_str, &audio_str);
// Kotlin sends 0 when AudioCapabilities had no answer for the current route.
let channels = u32::try_from(max_audio_channels).ok().filter(|c| *c > 0);
let codecs = DetectedCodecs::from_jni_strings(&video_str, &audio_str, channels);
log::info!(
"[CodecDetection] Detected {} video codecs: {}",
@@ -1148,6 +1170,10 @@ pub extern "system" fn Java_com_dtourolle_jellytau_player_JellyTauPlayer_00024Co
codecs.audio_codecs.len(),
codecs.audio_codecs_string()
);
log::info!(
"[CodecDetection] Audio route max channels: {:?}",
codecs.max_audio_channels
);
// Store in global state
if DETECTED_CODECS.set(codecs).is_err() {
@@ -0,0 +1,95 @@
//! Device-profile policy: turning what a device *reports* about its audio
//! output into the constraints we send Jellyfin.
//!
//! The platform layer reports raw facts (what `MediaCodecList` enumerates, how
//! many channels the current audio route accepts); deciding what those facts
//! mean for a `DeviceProfile` is domain logic and lives here, on the Rust side
//! of the boundary, where it is testable without a device.
/// Channel count assumed when the platform cannot tell us — every audio route
/// can voice stereo, so it is the only safe floor.
const FALLBACK_AUDIO_CHANNELS: u32 = 2;
/// Upper bound we are willing to claim. Jellyfin profiles top out at 7.1, and a
/// nonsense reading from a driver should not become a nonsense profile.
const MAX_SUPPORTED_AUDIO_CHANNELS: u32 = 8;
/// Decide the `MaxAudioChannels` to advertise, given what the current audio
/// route reported.
///
/// Without this constraint Jellyfin is free to direct-play a 5.1 or 7.1 track to
/// a sink that only has two channels. What the user hears then is device
/// dependent and rarely correct — a failed `AudioSink` configuration (silence),
/// or centre-channel dialogue folded away to near-inaudibility. Naming the real
/// channel count makes the server downmix instead, which is always audible.
///
/// A missing or zero reading means "route not established yet", not "no audio":
/// fall back to stereo rather than claiming a capability we have not seen.
///
/// TRACES: UR-004 | DR-141 | UT-131
pub fn clamp_max_audio_channels(reported: Option<u32>) -> u32 {
match reported {
Some(channels) if channels >= 1 => channels.min(MAX_SUPPORTED_AUDIO_CHANNELS),
_ => FALLBACK_AUDIO_CHANNELS,
}
}
/// The channel cap for this device, reading the platform's report where one
/// exists.
///
/// TRACES: UR-004 | DR-141 | UT-131
pub fn max_audio_channels() -> u32 {
#[cfg(target_os = "android")]
let reported = crate::player::get_detected_codecs().and_then(|(_, _, channels)| channels);
// Desktop plays video through the WebKitGTK HTML5 <video> element, which we
// do not interrogate for a channel count; stereo is the safe assumption.
#[cfg(not(target_os = "android"))]
let reported: Option<u32> = None;
clamp_max_audio_channels(reported)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn an_unknown_route_falls_back_to_stereo() {
// Codec detection has not run yet, or the platform has no answer. Never
// claim surround we have not seen — every sink can do stereo.
assert_eq!(clamp_max_audio_channels(None), 2);
}
#[test]
fn a_zero_reading_is_not_a_capability() {
// A route that has not been established reports 0; taking that literally
// would advertise a device with no audio at all.
assert_eq!(clamp_max_audio_channels(Some(0)), 2);
}
#[test]
fn a_stereo_sink_is_reported_as_stereo() {
// The phone speaker / Bluetooth headset case: the server must downmix
// 5.1 rather than direct-play it.
assert_eq!(clamp_max_audio_channels(Some(2)), 2);
}
#[test]
fn a_surround_route_keeps_its_channels() {
// HDMI to an AVR: 5.1 and 7.1 direct play stay available.
assert_eq!(clamp_max_audio_channels(Some(6)), 6);
assert_eq!(clamp_max_audio_channels(Some(8)), 8);
}
#[test]
fn an_absurd_reading_is_capped_rather_than_forwarded() {
// Some drivers report the AudioTrack maximum rather than the route's.
assert_eq!(clamp_max_audio_channels(Some(32)), 8);
}
#[test]
fn a_mono_route_is_taken_at_its_word() {
assert_eq!(clamp_max_audio_channels(Some(1)), 1);
}
}
+1
View File
@@ -1,3 +1,4 @@
pub mod device_profile;
pub mod hybrid;
pub mod offline;
pub mod online;
+21 -3
View File
@@ -404,7 +404,10 @@ impl OnlineRepository {
("MaxStreamingBitrate", "20000000".to_string()),
("VideoBitrate", "18000000".to_string()),
("AudioBitrate", "384000".to_string()),
("TranscodingMaxAudioChannels", "2".to_string()),
(
"TranscodingMaxAudioChannels",
super::device_profile::max_audio_channels().to_string(),
),
("SegmentContainer", "ts".to_string()),
("TranscodingContainer", "ts".to_string()),
("TranscodingProtocol", "hls".to_string()),
@@ -1271,6 +1274,10 @@ impl MediaRepository for OnlineRepository {
name: String,
max_streaming_bitrate: i64,
max_static_bitrate: i64,
/// Channels the device's audio route can actually voice. Without it
/// the server may direct-play a 5.1 track to a two-channel sink,
/// which is silence or inaudible dialogue depending on the device.
max_audio_channels: String,
direct_play_profiles: Vec<DirectPlayProfile>,
transcoding_profiles: Vec<TranscodingProfile>,
subtitle_profiles: Vec<SubtitleProfile>,
@@ -1298,6 +1305,7 @@ impl MediaRepository for OnlineRepository {
#[serde(skip_serializing_if = "Option::is_none")]
video_codec: Option<String>,
audio_codec: String,
max_audio_channels: String,
}
#[derive(Debug, Serialize)]
@@ -1338,8 +1346,9 @@ impl MediaRepository for OnlineRepository {
// Get detected codecs from Android MediaCodecList or use platform defaults
#[cfg(target_os = "android")]
let (video_codecs, audio_codecs) =
crate::player::get_detected_codecs().unwrap_or_else(|| {
let (video_codecs, audio_codecs) = crate::player::get_detected_codecs()
.map(|(video, audio, _channels)| (video, audio))
.unwrap_or_else(|| {
warn!("[DeviceProfile] Codec detection not complete, using conservative defaults");
("h264,hevc".to_string(), "aac,mp3".to_string())
});
@@ -1362,11 +1371,18 @@ impl MediaRepository for OnlineRepository {
info!("[DeviceProfile] Using video codecs: {}", video_codecs);
info!("[DeviceProfile] Using audio codecs: {}", audio_codecs);
// Bound every profile by what the audio route can actually voice, so a
// multichannel track is downmixed by the server rather than direct-played
// into a sink that has nowhere to put the extra channels.
let max_audio_channels = super::device_profile::max_audio_channels().to_string();
info!("[DeviceProfile] Max audio channels: {}", max_audio_channels);
// Create device profile with detected hardware capabilities
let device_profile = DeviceProfile {
name: "JellyTau Native Player".to_string(),
max_streaming_bitrate: 999_999_999,
max_static_bitrate: 999_999_999,
max_audio_channels: max_audio_channels.clone(),
direct_play_profiles: vec![
DirectPlayProfile {
profile_type: "Video".to_string(),
@@ -1389,6 +1405,7 @@ impl MediaRepository for OnlineRepository {
container: "ts".to_string(),
video_codec: Some("h264,hevc".to_string()),
audio_codec: "aac,mp3".to_string(),
max_audio_channels: max_audio_channels.clone(),
},
TranscodingProfile {
profile_type: "Audio".to_string(),
@@ -1397,6 +1414,7 @@ impl MediaRepository for OnlineRepository {
container: "mp3".to_string(),
video_codec: None,
audio_codec: "mp3".to_string(),
max_audio_channels: max_audio_channels.clone(),
},
],
subtitle_profiles: vec![