fix(player): stop the server burning subtitles into the picture

A transcoded episode stalled every few seconds and seeking took five to
nine seconds to produce a frame. Neither was a seek bug: both seeks in the
capture landed correctly. The stream itself could not keep up.

The episode was HEVC video, E-AC-3 audio, and a PGSSUB subtitle track.
Only the audio needed transcoding — the device profile supports HEVC and
the server would have remuxed the video untouched. But the PlaybackInfo
request omitted SubtitleStreamIndex, and omitting it does not mean "no
subtitles": the server then honours the source's default/forced flag and
picks a track itself. It picked the PGS one. PGS is a bitmap, and the
profile advertised only srt/vtt as External, so it could not go out as a
sidecar — leaving SubtitleMethod=Encode, burn-in.

Burn-in is a video cost, not a subtitle cost. Compositing rules out
remuxing, so the whole HEVC stream was re-encoded to h264 frame by frame.
The server could not sustain that in real time: the buffer never grew past
one segment and playback ran waiting -> HLS error -> canplay -> three
seconds of picture, indefinitely, while each seek restarted the encoder
from scratch. TranscodeReasons named it — SubtitleCodecNotSupported — but
nothing in the log connected that to the stall, so the diagnostic now says
which track it is declining and why.

Ask for SubtitleStreamIndex=-1 explicitly, and advertise every text format
we can render (srt/subrip/ass/ssa/vtt) as External so a subtitle can only
ever arrive as a sidecar. Nothing is lost: the app already fetches subtitle
tracks itself and draws them over the video (UR-020), so the server's
composited copy was always redundant. Image-based tracks are consequently
not offered, which is honest rather than a regression — the renderer cannot
composite a bitmap, and the previous behaviour paid for them by making the
stream unwatchable.

The policy lives beside the other device-profile rules in Rust, where it is
testable without a device.

TRACES: UR-020, UR-004 | DR-176 | UT-168
This commit is contained in:
2026-08-16 09:23:39 +02:00
parent 1a9805f0f3
commit 041969f446
3 changed files with 134 additions and 13 deletions
@@ -61,6 +61,67 @@ const WEBVIEW_AUDIO_CODECS: &[&str] = &["aac", "mp3", "opus", "vorbis", "flac"];
/// it rather than give up.
const FALLBACK_AUDIO_CODEC: &str = "aac";
/// Jellyfin's sentinel for "negotiate no subtitle stream at all".
///
/// Omitting `SubtitleStreamIndex` does **not** mean this: the server then applies
/// the source's default/forced flags and picks a track itself. See
/// [`playback_subtitle_stream_index`] for why that is never what we want.
pub const NO_SUBTITLE_STREAM: i32 = -1;
/// Subtitle formats we can render ourselves, delivered as an external sidecar
/// track rather than painted into the video.
///
/// Every entry here is *text*. Image-based subtitles (PGS, DVD, DVB) are
/// deliberately absent: they are bitmaps, so the only way a server can show them
/// on a client that cannot composite them is to burn them into the picture.
const EXTERNAL_SUBTITLE_FORMATS: &[&str] = &["srt", "subrip", "ass", "ssa", "vtt"];
/// The `SubtitleProfile` entries to advertise, as `(format, method)`.
///
/// All `External`: the app fetches subtitle tracks itself and renders them over
/// the video (UR-020), so it never needs the server to composite them.
///
/// TRACES: UR-020 | DR-176 | UT-168
pub fn subtitle_profiles() -> Vec<(&'static str, &'static str)> {
EXTERNAL_SUBTITLE_FORMATS
.iter()
.map(|format| (*format, "External"))
.collect()
}
/// Whether asking the server to serve this subtitle codec forces it to burn the
/// subtitle into the picture.
///
/// Burn-in is not a subtitle cost — it is a *video* cost. It rules out remuxing
/// the video stream, so a source we would otherwise have passed through untouched
/// gets fully re-encoded frame by frame.
///
/// TRACES: UR-020 | DR-176 | UT-168
pub fn subtitle_forces_burn_in(codec: &str) -> bool {
!EXTERNAL_SUBTITLE_FORMATS
.iter()
.any(|format| format.eq_ignore_ascii_case(codec.trim()))
}
/// The `SubtitleStreamIndex` to negotiate with: always "none".
///
/// The reported bug: a source with an E-AC-3 track and a **PGSSUB** default
/// subtitle track. Sending no index let the server honour that default, and since
/// PGS cannot go out as a sidecar it chose `SubtitleMethod=Encode` — burn-in.
/// That turned an audio-only transcode (the HEVC video was directly supported)
/// into a full HEVC→h264 re-encode, which the server could not sustain in real
/// time: the buffer never grew beyond one segment and playback stalled every few
/// seconds, taking seeking down with it.
///
/// Asking for no subtitle stream costs nothing, because the app never wanted the
/// server's composited version — it fetches the text tracks separately and
/// renders them itself (UR-020).
///
/// TRACES: UR-020, UR-004 | DR-176 | UT-168
pub fn playback_subtitle_stream_index() -> i32 {
NO_SUBTITLE_STREAM
}
/// Narrow a detected audio-codec list to what the renderer that will actually
/// play the **video** can decode.
///
@@ -162,6 +223,43 @@ pub fn served_audio_codec<'a>(streams: &[(Option<&'a str>, bool)]) -> Option<&'a
mod tests {
use super::*;
/// The reported bug, at the level it was decided: a source whose default
/// subtitle track is PGSSUB must not drag the video into a re-encode.
///
/// TRACES: UR-020 | DR-176 | UT-168
#[test]
fn no_subtitle_stream_is_negotiated_so_the_server_never_burns_one_in() {
assert_eq!(playback_subtitle_stream_index(), NO_SUBTITLE_STREAM);
// Not `None`/omitted: that is what let the server pick the PGS track.
assert_eq!(playback_subtitle_stream_index(), -1);
}
/// TRACES: UR-020 | DR-176 | UT-168
#[test]
fn text_subtitles_are_advertised_as_external_sidecars() {
let profiles = subtitle_profiles();
for format in ["srt", "subrip", "ass", "ssa", "vtt"] {
let entry = profiles.iter().find(|(f, _)| *f == format);
assert!(
entry.is_some(),
"{format} must be advertised or the server burns it into the picture"
);
assert_eq!(entry.unwrap().1, "External");
}
}
/// TRACES: UR-020 | DR-176 | UT-168
#[test]
fn text_subtitles_never_force_burn_in_but_image_ones_do() {
// Text: deliverable as a sidecar, so the video can still be remuxed.
assert!(!subtitle_forces_burn_in("subrip"));
assert!(!subtitle_forces_burn_in("ASS"));
assert!(!subtitle_forces_burn_in("ssa"));
// Image formats are bitmaps — the server can only composite them.
assert!(subtitle_forces_burn_in("PGSSUB"));
assert!(subtitle_forces_burn_in("dvdsub"));
}
#[test]
fn an_undecodable_default_track_forces_a_transcode() {
// The reported bug: one E-AC-3 track, which the webview cannot decode.
+32 -11
View File
@@ -1530,23 +1530,27 @@ impl MediaRepository for OnlineRepository {
max_audio_channels: max_audio_channels.clone(),
},
],
subtitle_profiles: vec![
SubtitleProfile {
format: "srt".to_string(),
method: "External".to_string(),
},
SubtitleProfile {
format: "vtt".to_string(),
method: "External".to_string(),
},
],
subtitle_profiles: super::device_profile::subtitle_profiles()
.into_iter()
.map(|(format, method)| SubtitleProfile {
format: format.to_string(),
method: method.to_string(),
})
.collect(),
};
// POST to PlaybackInfo with device profile containing detected codecs
let request_body = PlaybackInfoRequest {
user_id: self.user_id.clone(),
audio_stream_index: None, // Let the server pick the source default
subtitle_stream_index: None,
// Never let the server choose a subtitle track for us. Omitting this
// makes it honour the source's default/forced flag, and an image-based
// default (PGS) it cannot send as a sidecar becomes SubtitleMethod=Encode
// — burn-in, which forces a full video re-encode of a stream that would
// otherwise be remuxed. The app renders subtitles itself (UR-020).
//
// TRACES: UR-020, UR-004 | DR-176 | UT-168
subtitle_stream_index: Some(super::device_profile::playback_subtitle_stream_index()),
start_time_ticks: 0,
is_playback: true,
auto_open_live_stream: true,
@@ -1573,6 +1577,23 @@ impl MediaRepository for OnlineRepository {
);
}
// Name the tracks we are declining to have the server composite. Burn-in
// rules out remuxing the video, so a single image-based track can turn a
// free passthrough into a full re-encode; when that used to happen there
// was nothing in the log connecting the stall to the subtitle.
for stream in &source.media_streams {
if stream.stream_type == "Subtitle" {
if let Some(codec) = stream.codec.as_deref() {
if super::device_profile::subtitle_forces_burn_in(codec) {
info!(
" Subtitle index={} ({}) is image-based — not requested; the app renders text tracks itself rather than have the server burn it in (which would force a video re-encode)",
stream.index, codec
);
}
}
}
}
// Jellyfin 10.11.5 honours a DirectPlayProfile's container and video codec
// but ignores its audio codec, so it offers an E-AC-3 track for direct
// play even though DR-148 advertises only AAC — and the webview renders