fix(player): send subtitle tracks to ExoPlayer on Android (UR-020)
Selecting a subtitle on Android did nothing. The Kotlin side has been
complete for a long time — JellyTauPlayer.load() parses a subtitles JSON
array into MediaItem.SubtitleConfigurations and setSubtitleTrack() drives a
TrackSelectionOverride — but nothing ever reached it.
VideoPlayer built the list and then threw it away: it resolved every
subtitle stream's URL into a subtitleTracks array and the
commands.playerPlayItem({...}) call two lines below passed only streamUrl,
title, id, videoCodec and needsTranscoding. PlayItemRequest had no subtitle
field to put them in, so create_media_item hardcoded subtitles: vec![],
android/mod.rs serialized "[]" across JNI, and every MediaItem reached
ExoPlayer with zero SubtitleConfigurations. A later set_subtitle_track then
found no text track groups and logged "Invalid subtitle track index".
PlayItemRequest now carries the tracks (defaulted, so the background-audio
handoff and next-episode callers are unchanged) and create_media_item
threads them onto the MediaItem.
Serialization: SubtitleTrack is reused verbatim rather than given an
IPC-specific twin, and deliberately keeps snake_case. The same struct feeds
two consumers that both spell mime_type — the JNI JSON that
JellyTauPlayer.load() reads with optString("mime_type"), and the generated
binding the frontend types against. camelCasing it would not fail the build
or the IPC; Kotlin would silently fall back to its default MIME type for
every track. UT-146 asserts the exact serialized keys so a future
rename_all cannot pass unnoticed.
The index mapping was NOT already correct. setSubtitleTrack(n) indexes
ExoPlayer's filtered text track groups, i.e. the position of the sideloaded
configuration — but the menu passed its own {#each} row number, which counts
every subtitle *stream*, including ones whose URL failed to resolve and were
therefore never sideloaded. One failed URL and every track below it selected
the wrong subtitle. The position is now looked up in the exact array that
was sent (nativeSubtitleArrayIndex), and a stream that was never sent maps
to "off" rather than to a guessed position.
The resolution loop also reuses resolveSubtitleTracks() from the Linux fix
instead of duplicating it, which fans the URL requests out in parallel
rather than awaiting them one per stream before playback can start. The
awaits are safe where they sit: the native-mode pitfall is about Svelte
lifecycle calls after an await, and nothing is registered here — the
background-audio subscriptions above still run synchronously.
No Kotlin change was needed.
Tests (UT-145, UT-146, UT-147) were written first and failed: PlayItemRequest
had no subtitles field to compile against, nativeSubtitleTracks and
nativeSubtitleArrayIndex did not exist, and the playerPlayItem call carried
no subtitles key.
TRACES: UR-020 | IR-016, JA-008 | UT-145, UT-146, UT-147
This commit is contained in:
@@ -202,6 +202,27 @@ pub struct PlayItemRequest {
|
||||
/// look up the next episode when a background-audio track ends.
|
||||
#[serde(default)]
|
||||
pub series_id: Option<String>,
|
||||
/// Subtitle tracks to sideload, with URLs the frontend has already resolved.
|
||||
///
|
||||
/// Only the native backends use these: on Android they become the
|
||||
/// `MediaItem.SubtitleConfiguration`s ExoPlayer renders. The HTML5 path
|
||||
/// builds its own `<track>` children instead and ignores this list.
|
||||
///
|
||||
/// **Order is the contract.** `player_set_subtitle_track(n)` reaches
|
||||
/// `JellyTauPlayer.setSubtitleTrack(n)`, which indexes into ExoPlayer's
|
||||
/// *text track groups* — i.e. the position of the sideloaded configuration,
|
||||
/// not the Jellyfin stream index (which is kept on each entry for the UI's
|
||||
/// benefit). So `n` must be a position in this very array, and the array
|
||||
/// must not be reordered or filtered between building it and sending it.
|
||||
/// `nativeSubtitleArrayIndex()` on the frontend computes `n` from the same
|
||||
/// list that is sent here, for exactly this reason.
|
||||
///
|
||||
/// Defaulted so the background-audio handoff and the autoplay/next-episode
|
||||
/// callers, which have no subtitles to offer, need not send the field.
|
||||
///
|
||||
/// TRACES: UR-020 | IR-016, JA-008 | UT-145
|
||||
#[serde(default)]
|
||||
pub subtitles: Vec<crate::player::SubtitleTrack>,
|
||||
}
|
||||
|
||||
/// Queue context for remote transfer - what type of queue is this?
|
||||
@@ -373,7 +394,10 @@ pub(super) async fn create_media_item(
|
||||
needs_transcoding: req.needs_transcoding,
|
||||
video_width: None, // Not available from video-only request
|
||||
video_height: None, // Not available from video-only request
|
||||
subtitles: vec![],
|
||||
// Sideloaded subtitles, in the order the frontend sent them — that order
|
||||
// is what `player_set_subtitle_track(n)` indexes into on Android.
|
||||
// TRACES: UR-020 | IR-016 | UT-145
|
||||
subtitles: req.subtitles,
|
||||
series_id: None, // Not available from video-only request
|
||||
server_id: None, // Not available from video-only request
|
||||
})
|
||||
@@ -2459,6 +2483,141 @@ pub async fn player_disable_jellyfin(player: State<'_, PlayerStateWrapper>) -> R
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
/// The subtitle list the frontend resolved must survive the IPC hop and end
|
||||
/// up on the `MediaItem` the native backend loads.
|
||||
///
|
||||
/// The bug: `VideoPlayer.svelte` built a fully-resolved subtitle array and
|
||||
/// then dropped it on the floor — `PlayItemRequest` had no field to put it
|
||||
/// in — so `create_media_item` always produced `subtitles: vec![]`,
|
||||
/// `android/mod.rs` serialized `[]` across JNI, and ExoPlayer was handed a
|
||||
/// `MediaItem` with zero `SubtitleConfiguration`s. Every later
|
||||
/// `setSubtitleTrack(n)` then found no text track groups and logged
|
||||
/// "Invalid subtitle track index".
|
||||
///
|
||||
/// The payload below is exactly what the frontend sends: camelCase for the
|
||||
/// top-level command params (Tauri v2 converts them), and the subtitle
|
||||
/// entries in the casing of `SubtitleTrack` itself — note `mime_type`.
|
||||
///
|
||||
/// TRACES: UR-020 | IR-016 | UT-145
|
||||
#[tokio::test]
|
||||
async fn test_play_item_request_carries_subtitles_into_media_item() {
|
||||
use super::{create_media_item, PlayItemRequest};
|
||||
|
||||
let payload = serde_json::json!({
|
||||
"id": "ep-1",
|
||||
"title": "Pilot",
|
||||
"streamUrl": "https://jelly.example/Videos/ep-1/master.m3u8",
|
||||
"videoCodec": "h264",
|
||||
"needsTranscoding": false,
|
||||
"subtitles": [
|
||||
{
|
||||
"index": 2,
|
||||
"url": "https://jelly.example/Videos/ep-1/2/Subtitles/subtitles.vtt",
|
||||
"language": "eng",
|
||||
"label": "English (SRT)",
|
||||
"mime_type": "text/vtt"
|
||||
},
|
||||
{
|
||||
"index": 3,
|
||||
"url": "https://jelly.example/Videos/ep-1/3/Subtitles/subtitles.vtt",
|
||||
"language": null,
|
||||
"label": null,
|
||||
"mime_type": "text/vtt"
|
||||
}
|
||||
]
|
||||
});
|
||||
|
||||
let req: PlayItemRequest =
|
||||
serde_json::from_value(payload).expect("frontend payload must deserialize");
|
||||
assert_eq!(
|
||||
req.subtitles.len(),
|
||||
2,
|
||||
"PlayItemRequest must carry the subtitle tracks, not silently ignore them"
|
||||
);
|
||||
|
||||
let media = create_media_item(req, None).await.unwrap();
|
||||
assert_eq!(
|
||||
media.subtitles.len(),
|
||||
2,
|
||||
"create_media_item must thread the tracks onto the MediaItem the backend loads"
|
||||
);
|
||||
assert_eq!(media.subtitles[0].index, 2);
|
||||
assert_eq!(media.subtitles[0].language.as_deref(), Some("eng"));
|
||||
assert_eq!(media.subtitles[0].label.as_deref(), Some("English (SRT)"));
|
||||
assert_eq!(media.subtitles[0].mime_type, "text/vtt");
|
||||
// Order is the contract: `player_set_subtitle_track(n)` is a position in
|
||||
// this list (see the note on `PlayItemRequest::subtitles`).
|
||||
assert_eq!(media.subtitles[1].index, 3);
|
||||
assert!(media.subtitles[1].language.is_none());
|
||||
}
|
||||
|
||||
/// A request without subtitles must still deserialize — the field is
|
||||
/// defaulted so the background-audio handoff and the autoplay/next-episode
|
||||
/// callers keep compiling and sending what they always sent.
|
||||
///
|
||||
/// TRACES: UR-020 | IR-016 | UT-145
|
||||
#[tokio::test]
|
||||
async fn test_play_item_request_without_subtitles_defaults_to_empty() {
|
||||
use super::{create_media_item, PlayItemRequest};
|
||||
|
||||
let req: PlayItemRequest = serde_json::from_value(serde_json::json!({
|
||||
"id": "movie-1",
|
||||
"title": "Movie",
|
||||
"streamUrl": "https://jelly.example/Videos/movie-1/stream.mp4",
|
||||
"videoCodec": "h264",
|
||||
"needsTranscoding": false
|
||||
}))
|
||||
.expect("a subtitle-less payload must still deserialize");
|
||||
|
||||
assert!(req.subtitles.is_empty());
|
||||
assert!(create_media_item(req, None)
|
||||
.await
|
||||
.unwrap()
|
||||
.subtitles
|
||||
.is_empty());
|
||||
}
|
||||
|
||||
/// The JSON handed to Kotlin over JNI must use the keys
|
||||
/// `JellyTauPlayer.load()` actually reads.
|
||||
///
|
||||
/// `MediaItem` is `rename_all = "camelCase"`, and the instinct (and the
|
||||
/// house IPC rule) is to camelCase nested structs too — but
|
||||
/// `JellyTauPlayer.kt` reads `subtitle.optString("mime_type", …)`. Renaming
|
||||
/// the field would not fail to compile or fail the IPC; it would silently
|
||||
/// fall back to the default MIME type for every track, so this is asserted
|
||||
/// on the exact bytes `android/mod.rs` sends.
|
||||
///
|
||||
/// TRACES: UR-020 | IR-016, JA-008 | UT-146
|
||||
#[test]
|
||||
fn test_subtitle_json_for_jni_uses_the_keys_kotlin_reads() {
|
||||
use crate::player::media::SubtitleTrack;
|
||||
|
||||
let subtitles = vec![SubtitleTrack {
|
||||
index: 2,
|
||||
url: "https://jelly.example/subs.vtt".to_string(),
|
||||
language: Some("eng".to_string()),
|
||||
label: Some("English".to_string()),
|
||||
mime_type: "text/vtt".to_string(),
|
||||
}];
|
||||
|
||||
// Exactly what player/android/mod.rs passes to loadWithMetadata.
|
||||
let json = serde_json::to_string(&subtitles).unwrap();
|
||||
let parsed: Vec<serde_json::Value> = serde_json::from_str(&json).unwrap();
|
||||
let obj = parsed[0].as_object().unwrap();
|
||||
|
||||
for key in ["url", "language", "label", "mime_type"] {
|
||||
assert!(
|
||||
obj.contains_key(key),
|
||||
"JellyTauPlayer.load() reads `{key}`; serialized keys were {:?}",
|
||||
obj.keys().collect::<Vec<_>>()
|
||||
);
|
||||
}
|
||||
assert!(
|
||||
!obj.contains_key("mimeType"),
|
||||
"camelCasing mime_type silently drops every track's MIME type on Android"
|
||||
);
|
||||
}
|
||||
|
||||
/// The audio-only handoff must play a downloaded file when there is one,
|
||||
/// rather than fetching an audio-only stream for media already on disk.
|
||||
///
|
||||
|
||||
Reference in New Issue
Block a user