fix(player): send subtitle tracks to ExoPlayer on Android (UR-020)
Selecting a subtitle on Android did nothing. The Kotlin side has been
complete for a long time — JellyTauPlayer.load() parses a subtitles JSON
array into MediaItem.SubtitleConfigurations and setSubtitleTrack() drives a
TrackSelectionOverride — but nothing ever reached it.
VideoPlayer built the list and then threw it away: it resolved every
subtitle stream's URL into a subtitleTracks array and the
commands.playerPlayItem({...}) call two lines below passed only streamUrl,
title, id, videoCodec and needsTranscoding. PlayItemRequest had no subtitle
field to put them in, so create_media_item hardcoded subtitles: vec![],
android/mod.rs serialized "[]" across JNI, and every MediaItem reached
ExoPlayer with zero SubtitleConfigurations. A later set_subtitle_track then
found no text track groups and logged "Invalid subtitle track index".
PlayItemRequest now carries the tracks (defaulted, so the background-audio
handoff and next-episode callers are unchanged) and create_media_item
threads them onto the MediaItem.
Serialization: SubtitleTrack is reused verbatim rather than given an
IPC-specific twin, and deliberately keeps snake_case. The same struct feeds
two consumers that both spell mime_type — the JNI JSON that
JellyTauPlayer.load() reads with optString("mime_type"), and the generated
binding the frontend types against. camelCasing it would not fail the build
or the IPC; Kotlin would silently fall back to its default MIME type for
every track. UT-146 asserts the exact serialized keys so a future
rename_all cannot pass unnoticed.
The index mapping was NOT already correct. setSubtitleTrack(n) indexes
ExoPlayer's filtered text track groups, i.e. the position of the sideloaded
configuration — but the menu passed its own {#each} row number, which counts
every subtitle *stream*, including ones whose URL failed to resolve and were
therefore never sideloaded. One failed URL and every track below it selected
the wrong subtitle. The position is now looked up in the exact array that
was sent (nativeSubtitleArrayIndex), and a stream that was never sent maps
to "off" rather than to a guessed position.
The resolution loop also reuses resolveSubtitleTracks() from the Linux fix
instead of duplicating it, which fans the URL requests out in parallel
rather than awaiting them one per stream before playback can start. The
awaits are safe where they sit: the native-mode pitfall is about Svelte
lifecycle calls after an await, and nothing is registered here — the
background-audio subscriptions above still run synchronously.
No Kotlin change was needed.
Tests (UT-145, UT-146, UT-147) were written first and failed: PlayItemRequest
had no subtitles field to compile against, nativeSubtitleTracks and
nativeSubtitleArrayIndex did not exist, and the playerPlayItem call carried
no subtitles key.
TRACES: UR-020 | IR-016, JA-008 | UT-145, UT-146, UT-147
This commit is contained in:
@@ -18,6 +18,8 @@
|
||||
resolveSubtitleTracks,
|
||||
reconcileSelectedSubtitle,
|
||||
videoCrossOriginMode,
|
||||
nativeSubtitleTracks,
|
||||
nativeSubtitleArrayIndex,
|
||||
type RenderableSubtitleTrack,
|
||||
} from "./subtitleTracks";
|
||||
import { sleepTimerActive, sleepTimerExpiredSignal } from "$lib/stores/sleepTimer";
|
||||
@@ -291,6 +293,15 @@
|
||||
// TRACES: UR-020 | DR-023 | UT-143, UT-144
|
||||
let renderedSubtitleTracks = $state<RenderableSubtitleTrack[]>([]);
|
||||
|
||||
// The subtitle list actually handed to the native backend at load time
|
||||
// (Android/ExoPlayer). Kept because `player_set_subtitle_track` takes a
|
||||
// *position in this list*, not a Jellyfin stream index — see
|
||||
// nativeSubtitleArrayIndex. It is written once, from onMount, before the
|
||||
// play request; it is not derived, because the request is what fixed the
|
||||
// backend's idea of the track order.
|
||||
// TRACES: UR-020 | IR-016 | UT-147
|
||||
let sentSubtitleTracks = $state<RenderableSubtitleTrack[]>([]);
|
||||
|
||||
// Cross-origin <track> fetches use the media element's CORS setting; see
|
||||
// videoCrossOriginMode for why this is opt-in and same-origin-only.
|
||||
const videoCrossOrigin = $derived(
|
||||
@@ -596,28 +607,23 @@
|
||||
console.log("[VideoPlayer] Initializing player for:", media.name);
|
||||
console.log("[VideoPlayer] Stream URL:", currentStreamUrl);
|
||||
|
||||
// Build subtitle tracks for native player
|
||||
const subtitleTracks = [];
|
||||
if (media.mediaStreams && mediaSourceId) {
|
||||
const subtitles = media.mediaStreams.filter(s => s.kind === "subtitle");
|
||||
for (const sub of subtitles) {
|
||||
try {
|
||||
const url = await getSubtitleUrl(sub.index);
|
||||
if (url) {
|
||||
subtitleTracks.push({
|
||||
index: sub.index,
|
||||
url: url,
|
||||
language: sub.language || null,
|
||||
label: sub.displayTitle || sub.language || `Track ${sub.index}`,
|
||||
mime_type: "text/vtt" // Jellyfin converts to WebVTT
|
||||
});
|
||||
}
|
||||
} catch (err) {
|
||||
console.warn(`[VideoPlayer] Failed to build subtitle URL for track ${sub.index}:`, err);
|
||||
}
|
||||
}
|
||||
console.log(`[VideoPlayer] Built ${subtitleTracks.length} subtitle tracks for native player`);
|
||||
}
|
||||
// Resolve subtitle URLs for the native (ExoPlayer) path. These must be
|
||||
// in hand *before* the play request: ExoPlayer sideloads subtitles as
|
||||
// MediaItem.SubtitleConfigurations, which have to exist before
|
||||
// prepare() — there is no way to add one to a loaded item afterwards.
|
||||
//
|
||||
// Awaiting here is safe despite the native-mode pitfall: that rule is
|
||||
// about Svelte *lifecycle* calls (onMount/onDestroy) after an await,
|
||||
// which throw lifecycle_outside_component and used to be misread as an
|
||||
// init failure. Nothing is registered here, and the background-audio
|
||||
// subscriptions above already ran synchronously. resolveSubtitleTracks
|
||||
// fans the requests out in parallel, so this costs one round trip, not
|
||||
// one per subtitle stream as the old serial loop did.
|
||||
// TRACES: UR-020 | IR-016, JA-008 | UT-147
|
||||
sentSubtitleTracks = mediaSourceId
|
||||
? await resolveSubtitleTracks(media.mediaStreams, (index) => getSubtitleUrl(index))
|
||||
: [];
|
||||
console.log(`[VideoPlayer] Sending ${sentSubtitleTracks.length} subtitle tracks to the backend`);
|
||||
|
||||
// Call Rust backend to start playback
|
||||
// Rust will choose ExoPlayer (Android), libmpv (Linux), or tell us to use HTML5
|
||||
@@ -628,6 +634,10 @@
|
||||
id: media.id,
|
||||
videoCodec: needsTranscoding ? "hevc" : "h264",
|
||||
needsTranscoding: needsTranscoding,
|
||||
// Order matters: player_set_subtitle_track(n) is a position in this
|
||||
// array. Previously this array was built and then dropped, so
|
||||
// ExoPlayer got a MediaItem with no subtitles at all.
|
||||
subtitles: nativeSubtitleTracks(sentSubtitleTracks),
|
||||
});
|
||||
|
||||
// Rust tells us which backend it's using
|
||||
@@ -1747,8 +1757,22 @@
|
||||
});
|
||||
}
|
||||
|
||||
async function selectSubtitle(streamIndex: number | null, arrayIndex?: number) {
|
||||
console.log("[VideoPlayer] Selecting subtitle - streamIndex:", streamIndex, "arrayIndex:", arrayIndex);
|
||||
/**
|
||||
* Apply the menu's choice. `streamIndex` is always the Jellyfin media-stream
|
||||
* index (or `null` for "Off") — the UI speaks stream indices throughout.
|
||||
*
|
||||
* The native backend does not: `player_set_subtitle_track(n)` reaches
|
||||
* `JellyTauPlayer.setSubtitleTrack(n)`, which indexes ExoPlayer's text track
|
||||
* groups, i.e. the position of the sideloaded subtitle configuration. That
|
||||
* position is derived from `sentSubtitleTracks` — the exact array sent with
|
||||
* the play request — and not from the menu's row number, which counts every
|
||||
* subtitle *stream* including ones whose URL never resolved and so were never
|
||||
* sideloaded.
|
||||
*
|
||||
* TRACES: UR-020 | DR-023, IR-016 | UT-147
|
||||
*/
|
||||
async function selectSubtitle(streamIndex: number | null) {
|
||||
console.log("[VideoPlayer] Selecting subtitle - streamIndex:", streamIndex);
|
||||
selectedSubtitleIndex = streamIndex;
|
||||
showSubtitleMenu = false;
|
||||
|
||||
@@ -1758,11 +1782,9 @@
|
||||
} else {
|
||||
// For native backend (Android), send command to change subtitle track
|
||||
try {
|
||||
// Use array index for ExoPlayer (0-based position in subtitle tracks array)
|
||||
// If streamIndex is null (disable), pass null; otherwise use arrayIndex
|
||||
const indexToUse = streamIndex === null ? null : (arrayIndex !== undefined ? arrayIndex : streamIndex);
|
||||
const indexToUse = nativeSubtitleArrayIndex(sentSubtitleTracks, streamIndex);
|
||||
await commands.playerSetSubtitleTrack(indexToUse);
|
||||
console.log("[VideoPlayer] Native backend subtitle track changed - arrayIndex:", arrayIndex, "used:", indexToUse);
|
||||
console.log("[VideoPlayer] Native backend subtitle track changed - streamIndex:", streamIndex, "position:", indexToUse);
|
||||
} catch (error) {
|
||||
console.error("[VideoPlayer] Failed to set subtitle track:", error);
|
||||
}
|
||||
@@ -2135,9 +2157,9 @@
|
||||
{/if}
|
||||
</button>
|
||||
<!-- Subtitle tracks -->
|
||||
{#each subtitleTracks() as track, i}
|
||||
{#each subtitleTracks() as track}
|
||||
<button
|
||||
onclick={() => selectSubtitle(track.index, i)}
|
||||
onclick={() => selectSubtitle(track.index)}
|
||||
class="w-full text-left px-3 py-2 text-white hover:bg-white/10 rounded transition-colors flex items-center justify-between {selectedSubtitleIndex === track.index ? 'bg-white/20' : ''}"
|
||||
>
|
||||
<div class="flex flex-col">
|
||||
|
||||
Reference in New Issue
Block a user