fix(player): send subtitle tracks to ExoPlayer on Android (UR-020)

Selecting a subtitle on Android did nothing. The Kotlin side has been
complete for a long time — JellyTauPlayer.load() parses a subtitles JSON
array into MediaItem.SubtitleConfigurations and setSubtitleTrack() drives a
TrackSelectionOverride — but nothing ever reached it.

VideoPlayer built the list and then threw it away: it resolved every
subtitle stream's URL into a subtitleTracks array and the
commands.playerPlayItem({...}) call two lines below passed only streamUrl,
title, id, videoCodec and needsTranscoding. PlayItemRequest had no subtitle
field to put them in, so create_media_item hardcoded subtitles: vec![],
android/mod.rs serialized "[]" across JNI, and every MediaItem reached
ExoPlayer with zero SubtitleConfigurations. A later set_subtitle_track then
found no text track groups and logged "Invalid subtitle track index".

PlayItemRequest now carries the tracks (defaulted, so the background-audio
handoff and next-episode callers are unchanged) and create_media_item
threads them onto the MediaItem.

Serialization: SubtitleTrack is reused verbatim rather than given an
IPC-specific twin, and deliberately keeps snake_case. The same struct feeds
two consumers that both spell mime_type — the JNI JSON that
JellyTauPlayer.load() reads with optString("mime_type"), and the generated
binding the frontend types against. camelCasing it would not fail the build
or the IPC; Kotlin would silently fall back to its default MIME type for
every track. UT-146 asserts the exact serialized keys so a future
rename_all cannot pass unnoticed.

The index mapping was NOT already correct. setSubtitleTrack(n) indexes
ExoPlayer's filtered text track groups, i.e. the position of the sideloaded
configuration — but the menu passed its own {#each} row number, which counts
every subtitle *stream*, including ones whose URL failed to resolve and were
therefore never sideloaded. One failed URL and every track below it selected
the wrong subtitle. The position is now looked up in the exact array that
was sent (nativeSubtitleArrayIndex), and a stream that was never sent maps
to "off" rather than to a guessed position.

The resolution loop also reuses resolveSubtitleTracks() from the Linux fix
instead of duplicating it, which fans the URL requests out in parallel
rather than awaiting them one per stream before playback can start. The
awaits are safe where they sit: the native-mode pitfall is about Svelte
lifecycle calls after an await, and nothing is registered here — the
background-audio subscriptions above still run synchronously.

No Kotlin change was needed.

Tests (UT-145, UT-146, UT-147) were written first and failed: PlayItemRequest
had no subtitles field to compile against, nativeSubtitleTracks and
nativeSubtitleArrayIndex did not exist, and the playerPlayItem call carried
no subtitles key.

TRACES: UR-020 | IR-016, JA-008 | UT-145, UT-146, UT-147

src/lib/api/bindings.ts

@@ -2231,7 +2231,29 @@ itemType?: string | null;
  * Series ID for TV episodes. Needed alongside `item_type` so the backend can
  * look up the next episode when a background-audio track ends.
  */
-seriesId?: string | null }
+seriesId?: string | null; 
+/**
+ * Subtitle tracks to sideload, with URLs the frontend has already resolved.
+ * 
+ * Only the native backends use these: on Android they become the
+ * `MediaItem.SubtitleConfiguration`s ExoPlayer renders. The HTML5 path
+ * builds its own `<track>` children instead and ignores this list.
+ * 
+ * **Order is the contract.** `player_set_subtitle_track(n)` reaches
+ * `JellyTauPlayer.setSubtitleTrack(n)`, which indexes into ExoPlayer's
+ * *text track groups* — i.e. the position of the sideloaded configuration,
+ * not the Jellyfin stream index (which is kept on each entry for the UI's
+ * benefit). So `n` must be a position in this very array, and the array
+ * must not be reordered or filtered between building it and sending it.
+ * `nativeSubtitleArrayIndex()` on the frontend computes `n` from the same
+ * list that is sent here, for exactly this reason.
+ * 
+ * Defaulted so the background-audio handoff and the autoplay/next-episode
+ * callers, which have no subtitles to offer, need not send the field.
+ * 
+ * TRACES: UR-020 | IR-016, JA-008 | UT-145
+ */
+subtitles?: SubtitleTrack[] }
 /**
  * Queue context for remote transfer - what type of queue is this?
  */
@@ -2769,6 +2791,23 @@ export type StreamKind = "audio" | "video" | "subtitle" |
 "other"
 /**
  * Represents a subtitle track
+ * 
+ * 🔴 **Do not add `#[serde(rename_all = "camelCase")]` here.** This is the one
+ * struct in the player that deliberately keeps snake_case on the wire, because
+ * the *same* serialization feeds two consumers that both spell `mime_type`:
+ * 
+ * * the JNI boundary — `player/android/mod.rs` serializes `MediaItem::subtitles`
+ * with `serde_json` and hands the string to `JellyTauPlayer.loadWithMetadata`,
+ * whose parser reads `url`, `language`, `label` and `optString("mime_type")`;
+ * * the IPC boundary — `PlayItemRequest::subtitles` deserializes this same type
+ * from the frontend, and the generated binding (`SubtitleTrack` in
+ * `bindings.ts`) therefore also declares `mime_type`.
+ * 
+ * Renaming would not break the build and would not fail the IPC: Kotlin's
+ * `optString` would just fall back to its default MIME type for every track, so
+ * the failure would be silent. UT-146 asserts the serialized keys.
+ * 
+ * TRACES: UR-020 | IR-016, JA-008 | UT-146
  */
 export type SubtitleTrack = { 
 /**
@@ -2788,7 +2827,8 @@ language: string | null;
  */
 label: string | null; 
 /**
- * MIME type (e.g., "text/vtt", "application/x-subrip")
+ * MIME type (e.g., "text/vtt", "application/x-subrip").
+ * Snake_case on purpose — see the note on the struct.
  */
 mime_type: string }
 /**

src/lib/components/player/VideoPlayer.svelte

@@ -18,6 +18,8 @@
     resolveSubtitleTracks,
     reconcileSelectedSubtitle,
     videoCrossOriginMode,
+    nativeSubtitleTracks,
+    nativeSubtitleArrayIndex,
     type RenderableSubtitleTrack,
   } from "./subtitleTracks";
   import { sleepTimerActive, sleepTimerExpiredSignal } from "$lib/stores/sleepTimer";
@@ -291,6 +293,15 @@
   // TRACES: UR-020 | DR-023 | UT-143, UT-144
   let renderedSubtitleTracks = $state<RenderableSubtitleTrack[]>([]);
 
+  // The subtitle list actually handed to the native backend at load time
+  // (Android/ExoPlayer). Kept because `player_set_subtitle_track` takes a
+  // *position in this list*, not a Jellyfin stream index — see
+  // nativeSubtitleArrayIndex. It is written once, from onMount, before the
+  // play request; it is not derived, because the request is what fixed the
+  // backend's idea of the track order.
+  // TRACES: UR-020 | IR-016 | UT-147
+  let sentSubtitleTracks = $state<RenderableSubtitleTrack[]>([]);
+
   // Cross-origin <track> fetches use the media element's CORS setting; see
   // videoCrossOriginMode for why this is opt-in and same-origin-only.
   const videoCrossOrigin = $derived(
@@ -596,28 +607,23 @@
         console.log("[VideoPlayer] Initializing player for:", media.name);
         console.log("[VideoPlayer] Stream URL:", currentStreamUrl);
 
-        // Build subtitle tracks for native player
-        const subtitleTracks = [];
-        if (media.mediaStreams && mediaSourceId) {
-          const subtitles = media.mediaStreams.filter(s => s.kind === "subtitle");
-          for (const sub of subtitles) {
-            try {
-              const url = await getSubtitleUrl(sub.index);
-              if (url) {
-                subtitleTracks.push({
-                  index: sub.index,
-                  url: url,
-                  language: sub.language || null,
-                  label: sub.displayTitle || sub.language || `Track ${sub.index}`,
-                  mime_type: "text/vtt" // Jellyfin converts to WebVTT
-                });
-              }
-            } catch (err) {
-              console.warn(`[VideoPlayer] Failed to build subtitle URL for track ${sub.index}:`, err);
-            }
-          }
-          console.log(`[VideoPlayer] Built ${subtitleTracks.length} subtitle tracks for native player`);
-        }
+        // Resolve subtitle URLs for the native (ExoPlayer) path. These must be
+        // in hand *before* the play request: ExoPlayer sideloads subtitles as
+        // MediaItem.SubtitleConfigurations, which have to exist before
+        // prepare() — there is no way to add one to a loaded item afterwards.
+        //
+        // Awaiting here is safe despite the native-mode pitfall: that rule is
+        // about Svelte *lifecycle* calls (onMount/onDestroy) after an await,
+        // which throw lifecycle_outside_component and used to be misread as an
+        // init failure. Nothing is registered here, and the background-audio
+        // subscriptions above already ran synchronously. resolveSubtitleTracks
+        // fans the requests out in parallel, so this costs one round trip, not
+        // one per subtitle stream as the old serial loop did.
+        // TRACES: UR-020 | IR-016, JA-008 | UT-147
+        sentSubtitleTracks = mediaSourceId
+          ? await resolveSubtitleTracks(media.mediaStreams, (index) => getSubtitleUrl(index))
+          : [];
+        console.log(`[VideoPlayer] Sending ${sentSubtitleTracks.length} subtitle tracks to the backend`);
 
         // Call Rust backend to start playback
         // Rust will choose ExoPlayer (Android), libmpv (Linux), or tell us to use HTML5
@@ -628,6 +634,10 @@
           id: media.id,
           videoCodec: needsTranscoding ? "hevc" : "h264",
           needsTranscoding: needsTranscoding,
+          // Order matters: player_set_subtitle_track(n) is a position in this
+          // array. Previously this array was built and then dropped, so
+          // ExoPlayer got a MediaItem with no subtitles at all.
+          subtitles: nativeSubtitleTracks(sentSubtitleTracks),
         });
 
         // Rust tells us which backend it's using
@@ -1747,8 +1757,22 @@
     });
   }
 
-  async function selectSubtitle(streamIndex: number | null, arrayIndex?: number) {
-    console.log("[VideoPlayer] Selecting subtitle - streamIndex:", streamIndex, "arrayIndex:", arrayIndex);
+  /**
+   * Apply the menu's choice. `streamIndex` is always the Jellyfin media-stream
+   * index (or `null` for "Off") — the UI speaks stream indices throughout.
+   *
+   * The native backend does not: `player_set_subtitle_track(n)` reaches
+   * `JellyTauPlayer.setSubtitleTrack(n)`, which indexes ExoPlayer's text track
+   * groups, i.e. the position of the sideloaded subtitle configuration. That
+   * position is derived from `sentSubtitleTracks` — the exact array sent with
+   * the play request — and not from the menu's row number, which counts every
+   * subtitle *stream* including ones whose URL never resolved and so were never
+   * sideloaded.
+   *
+   * TRACES: UR-020 | DR-023, IR-016 | UT-147
+   */
+  async function selectSubtitle(streamIndex: number | null) {
+    console.log("[VideoPlayer] Selecting subtitle - streamIndex:", streamIndex);
     selectedSubtitleIndex = streamIndex;
     showSubtitleMenu = false;
 
@@ -1758,11 +1782,9 @@
     } else {
       // For native backend (Android), send command to change subtitle track
       try {
-        // Use array index for ExoPlayer (0-based position in subtitle tracks array)
-        // If streamIndex is null (disable), pass null; otherwise use arrayIndex
-        const indexToUse = streamIndex === null ? null : (arrayIndex !== undefined ? arrayIndex : streamIndex);
+        const indexToUse = nativeSubtitleArrayIndex(sentSubtitleTracks, streamIndex);
         await commands.playerSetSubtitleTrack(indexToUse);
-        console.log("[VideoPlayer] Native backend subtitle track changed - arrayIndex:", arrayIndex, "used:", indexToUse);
+        console.log("[VideoPlayer] Native backend subtitle track changed - streamIndex:", streamIndex, "position:", indexToUse);
       } catch (error) {
         console.error("[VideoPlayer] Failed to set subtitle track:", error);
       }
@@ -2135,9 +2157,9 @@
                     {/if}
                   </button>
                   <!-- Subtitle tracks -->
-                  {#each subtitleTracks() as track, i}
+                  {#each subtitleTracks() as track}
                     <button
-                      onclick={() => selectSubtitle(track.index, i)}
+                      onclick={() => selectSubtitle(track.index)}
                       class="w-full text-left px-3 py-2 text-white hover:bg-white/10 rounded transition-colors flex items-center justify-between {selectedSubtitleIndex === track.index ? 'bg-white/20' : ''}"
                     >
                       <div class="flex flex-col">

src/lib/components/player/subtitleTracks.test.ts

@@ -7,6 +7,8 @@ import {
   resolveSubtitleTracks,
   reconcileSelectedSubtitle,
   videoCrossOriginMode,
+  nativeSubtitleTracks,
+  nativeSubtitleArrayIndex,
   type SubtitleStreamLike,
 } from "./subtitleTracks";
 
@@ -151,6 +153,99 @@ describe("videoCrossOriginMode", () => {
   });
 });
 
+/**
+ * Subtitles on the Android / ExoPlayer native path.
+ *
+ * TRACES: UR-020 | IR-016, JA-008 | UT-147
+ *
+ * The bug this guards: VideoPlayer built a fully-resolved subtitle array in
+ * onMount and then never sent it — `commands.playerPlayItem({...})` passed only
+ * streamUrl/title/id/videoCodec/needsTranscoding — so every MediaItem reached
+ * ExoPlayer with zero SubtitleConfigurations and `setSubtitleTrack(n)` logged
+ * "Invalid subtitle track index".
+ *
+ * And the second half: `setSubtitleTrack(n)` indexes ExoPlayer's *text track
+ * groups*, i.e. the position of the sideloaded configuration — not the Jellyfin
+ * stream index. The menu used to pass its own row position, which is a position
+ * in the *unresolved* stream list; the moment one subtitle URL failed to
+ * resolve, the two lists diverged and every track below the gap selected the
+ * wrong subtitle.
+ */
+describe("nativeSubtitleTracks", () => {
+  it("maps to the wire shape Rust deserializes and Kotlin parses", async () => {
+    const resolved = await resolveSubtitleTracks(SUBS, async (i) => url(i));
+    const payload = nativeSubtitleTracks(resolved);
+
+    expect(payload).toHaveLength(2);
+    // Kotlin reads url/language/label/mime_type; Rust's SubtitleTrack keeps
+    // snake_case for exactly that reason, and so does the generated binding.
+    for (const track of payload) {
+      expect(Object.keys(track).sort()).toEqual(
+        ["index", "label", "language", "mime_type", "url"].sort(),
+      );
+      expect(track).not.toHaveProperty("mimeType");
+      expect(track.mime_type).toBe("text/vtt");
+    }
+    // Jellyfin serves every subtitle stream as WebVTT here, and the stream index
+    // rides along so the UI can keep talking in stream indices.
+    expect(payload.map((t) => t.index)).toEqual([2, 3]);
+    expect(payload[0].url).toContain("subtitles.vtt");
+    expect(payload[0].language).toBe("eng");
+    expect(payload[0].label).toBe("English (SRT)");
+  });
+
+  it("preserves stream order, because that order is the selection index", async () => {
+    const resolved = await resolveSubtitleTracks(STREAMS, async (i) => url(i));
+    expect(nativeSubtitleTracks(resolved).map((t) => t.index)).toEqual(
+      resolved.map((t) => t.streamIndex),
+    );
+  });
+
+  it("has nothing to send when no subtitle URL resolved", async () => {
+    expect(nativeSubtitleTracks(await resolveSubtitleTracks(SUBS, async () => ""))).toEqual([]);
+    expect(nativeSubtitleTracks([])).toEqual([]);
+  });
+
+  it("carries a null language/label through rather than inventing one", () => {
+    const payload = nativeSubtitleTracks([
+      { streamIndex: 5, url: "u.vtt", srclang: "und", label: "Track 5", isDefault: false },
+    ]);
+    expect(payload[0].language).toBeNull();
+    expect(payload[0].label).toBe("Track 5");
+  });
+});
+
+describe("nativeSubtitleArrayIndex", () => {
+  it("returns the position in the list that was actually sent, not the stream index", async () => {
+    const resolved = await resolveSubtitleTracks(SUBS, async (i) => url(i));
+    expect(nativeSubtitleArrayIndex(resolved, 2)).toBe(0);
+    expect(nativeSubtitleArrayIndex(resolved, 3)).toBe(1);
+  });
+
+  it("stays aligned when a subtitle URL failed to resolve (the mis-selection bug)", async () => {
+    // Stream 2 has no URL, so it is not among the sideloaded configurations.
+    // The menu's own row for stream 3 is position 1, but ExoPlayer only has one
+    // text track group — position 0. Sending 1 would select nothing.
+    const resolved = await resolveSubtitleTracks(SUBS, async (i) => {
+      if (i === 2) throw new Error("no repository");
+      return url(i);
+    });
+    expect(resolved).toHaveLength(1);
+    expect(nativeSubtitleArrayIndex(resolved, 3)).toBe(0);
+  });
+
+  it("maps 'Off' to null so the backend disables text instead of selecting track 0", async () => {
+    const resolved = await resolveSubtitleTracks(SUBS, async (i) => url(i));
+    expect(nativeSubtitleArrayIndex(resolved, null)).toBeNull();
+  });
+
+  it("maps a track that was never sent to null rather than to a wrong position", async () => {
+    const resolved = await resolveSubtitleTracks(SUBS, async (i) => url(i));
+    expect(nativeSubtitleArrayIndex(resolved, 99)).toBeNull();
+    expect(nativeSubtitleArrayIndex([], 3)).toBeNull();
+  });
+});
+
 describe("VideoPlayer markup (the regression that made the menu inert)", () => {
   const source = readFileSync(
     resolve(__dirname, "VideoPlayer.svelte"),
@@ -178,3 +273,31 @@ describe("VideoPlayer markup (the regression that made the menu inert)", () => {
     expect(trackElement).not.toMatch(/\bdefault=/);
   });
 });
+
+/**
+ * The half of the Android fix that lives in the component: the resolved list has
+ * to actually be handed to `playerPlayItem`, and the index sent to the backend
+ * has to be computed from that same list.
+ *
+ * TRACES: UR-020 | IR-016 | UT-147
+ */
+describe("VideoPlayer -> playerPlayItem (the tracks that were built and thrown away)", () => {
+  const source = readFileSync(resolve(__dirname, "VideoPlayer.svelte"), "utf-8");
+
+  /** The playerPlayItem({...}) argument object. */
+  const playItemCall = (() => {
+    const start = source.indexOf("commands.playerPlayItem(");
+    expect(start).toBeGreaterThan(-1);
+    return source.slice(start, source.indexOf("});", start) + 3);
+  })();
+
+  it("sends the subtitle tracks it resolved", () => {
+    expect(playItemCall).toMatch(/\bsubtitles:/);
+  });
+
+  it("selects by position in the sent list, not by the menu's row number", () => {
+    expect(source).toContain("nativeSubtitleArrayIndex");
+    // The old code forwarded the `{#each}` index straight to the backend.
+    expect(source).not.toMatch(/playerSetSubtitleTrack\(\s*arrayIndex\s*\)/);
+  });
+});

src/lib/components/player/subtitleTracks.ts

@@ -12,7 +12,14 @@
 // the render path, and only tracks that actually resolved are handed to the
 // markup.
 //
-// TRACES: UR-020 | DR-023 | UT-143, UT-144
+// The Android / ExoPlayer native path shares this module (see
+// nativeSubtitleTracks / nativeSubtitleArrayIndex at the bottom): it needs the
+// exact same "resolve the URLs first, keep only what resolved" list, just handed
+// to Rust instead of to `<track>` elements.
+//
+// TRACES: UR-020 | DR-023, IR-016 | UT-143, UT-144, UT-147
+
+import type { SubtitleTrack } from "$lib/api/bindings";
 
 /**
  * The subset of `MediaStream` (from the generated bindings) this module needs.
@@ -149,3 +156,72 @@ export function videoCrossOriginMode(
   if (subtitleStreamCount <= 0) return undefined;
   return originOf(streamUrl) ? "anonymous" : undefined;
 }
+
+// ===== Native (Android / ExoPlayer) path ====================================
+//
+// The HTML5 element gets `<track>` children; the native backend instead gets the
+// list *up front*, as part of the play request, because ExoPlayer sideloads
+// subtitles as `MediaItem.SubtitleConfiguration`s that must exist before
+// `prepare()`. There is no "add a subtitle later" — a track absent from the
+// MediaItem simply does not exist as far as the player is concerned.
+
+/**
+ * Map resolved tracks onto the wire shape `PlayItemRequest.subtitles` carries.
+ *
+ * The element type is the *generated* `SubtitleTrack` binding on purpose, so
+ * `bun run check` fails if the Rust struct's field names ever move. In
+ * particular `mime_type` is snake_case and must stay that way: the very same
+ * bytes are re-serialized across JNI in `player/android/mod.rs`, and
+ * `JellyTauPlayer.load()` reads `optString("mime_type")`. Renaming it to
+ * `mimeType` would not error anywhere — Kotlin would just silently fall back to
+ * its default MIME type for every track.
+ *
+ * Jellyfin is asked for every subtitle stream as WebVTT (see
+ * `getSubtitleUrl(..., "vtt")`), so the MIME type is fixed rather than derived
+ * from the source subtitle codec.
+ *
+ * TRACES: UR-020 | IR-016, JA-008 | UT-147
+ */
+export function nativeSubtitleTracks(
+  tracks: readonly RenderableSubtitleTrack[],
+): SubtitleTrack[] {
+  return tracks.map((track) => ({
+    index: track.streamIndex,
+    url: track.url,
+    // `srclang` carries "und" for a stream with no language, which is the right
+    // value for a `<track>` but is not a language the native side should claim.
+    language: track.srclang === "und" ? null : track.srclang,
+    label: track.label,
+    mime_type: "text/vtt",
+  }));
+}
+
+/**
+ * The argument for `player_set_subtitle_track` on the native backend.
+ *
+ * 🔴 This is **not** the Jellyfin stream index.
+ * `JellyTauPlayer.setSubtitleTrack(n)` filters ExoPlayer's track groups down to
+ * `C.TRACK_TYPE_TEXT` and indexes that list with `n`, so `n` is the *position of
+ * the sideloaded subtitle configuration* — which is the position in the array
+ * that `nativeSubtitleTracks()` produced and `playerPlayItem` sent.
+ *
+ * The menu's own row number is not that position: the menu lists every subtitle
+ * *stream*, while only the streams whose URL resolved are sent. One failed URL
+ * and everything below it selects the wrong subtitle. So the index is looked up
+ * in the sent list instead of being passed down from the `{#each}`.
+ *
+ * `null` (the menu's "Off") stays `null`, which the backend turns into -1 and
+ * Kotlin turns into "disable text tracks". A stream that was never sent also
+ * maps to `null`: disabling subtitles is a truthful outcome, whereas guessing a
+ * position would show the user a different language than the one they clicked.
+ *
+ * TRACES: UR-020 | IR-016 | UT-147
+ */
+export function nativeSubtitleArrayIndex(
+  tracks: readonly RenderableSubtitleTrack[],
+  streamIndex: number | null,
+): number | null {
+  if (streamIndex === null) return null;
+  const position = tracks.findIndex((t) => t.streamIndex === streamIndex);
+  return position === -1 ? null : position;
+}