dtourolle/jellytau
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases 37 Wiki Activity
Files
bbdcdca47e8adf160bc1eec8218141272d17274c
jellytau/src-tauri/src/player/media.rs
T
dtourolle 6a712c46cb fix(player): send subtitle tracks to ExoPlayer on Android (UR-020) 
Selecting a subtitle on Android did nothing. The Kotlin side has been
complete for a long time — JellyTauPlayer.load() parses a subtitles JSON
array into MediaItem.SubtitleConfigurations and setSubtitleTrack() drives a
TrackSelectionOverride — but nothing ever reached it.

VideoPlayer built the list and then threw it away: it resolved every
subtitle stream's URL into a subtitleTracks array and the
commands.playerPlayItem({...}) call two lines below passed only streamUrl,
title, id, videoCodec and needsTranscoding. PlayItemRequest had no subtitle
field to put them in, so create_media_item hardcoded subtitles: vec![],
android/mod.rs serialized "[]" across JNI, and every MediaItem reached
ExoPlayer with zero SubtitleConfigurations. A later set_subtitle_track then
found no text track groups and logged "Invalid subtitle track index".

PlayItemRequest now carries the tracks (defaulted, so the background-audio
handoff and next-episode callers are unchanged) and create_media_item
threads them onto the MediaItem.

Serialization: SubtitleTrack is reused verbatim rather than given an
IPC-specific twin, and deliberately keeps snake_case. The same struct feeds
two consumers that both spell mime_type — the JNI JSON that
JellyTauPlayer.load() reads with optString("mime_type"), and the generated
binding the frontend types against. camelCasing it would not fail the build
or the IPC; Kotlin would silently fall back to its default MIME type for
every track. UT-146 asserts the exact serialized keys so a future
rename_all cannot pass unnoticed.

The index mapping was NOT already correct. setSubtitleTrack(n) indexes
ExoPlayer's filtered text track groups, i.e. the position of the sideloaded
configuration — but the menu passed its own {#each} row number, which counts
every subtitle *stream*, including ones whose URL failed to resolve and were
therefore never sideloaded. One failed URL and every track below it selected
the wrong subtitle. The position is now looked up in the exact array that
was sent (nativeSubtitleArrayIndex), and a stream that was never sent maps
to "off" rather than to a guessed position.

The resolution loop also reuses resolveSubtitleTracks() from the Linux fix
instead of duplicating it, which fans the URL requests out in parallel
rather than awaiting them one per stream before playback can start. The
awaits are safe where they sit: the native-mode pitfall is about Svelte
lifecycle calls after an await, and nothing is registered here — the
background-audio subscriptions above still run synchronously.

No Kotlin change was needed.

Tests (UT-145, UT-146, UT-147) were written first and failed: PlayItemRequest
had no subtitles field to compile against, nativeSubtitleTracks and
nativeSubtitleArrayIndex did not exist, and the playerPlayItem call carried
no subtitles key.

TRACES: UR-020 | IR-016, JA-008 | UT-145, UT-146, UT-147
2026-08-11 20:03:19 +02:00

581 lines
18 KiB
Rust
Raw Blame History

use serde::{Deserialize, Serialize};
use std::path::PathBuf;
/// Context for the current queue - where did the queue items come from?
/// This is used for remote playback transfer to send album/playlist context.
#[derive(specta::Type, Debug, Clone, Serialize, Deserialize, PartialEq, Default)]
#[serde(tag = "type", rename_all = "lowercase")]
pub enum QueueContext {
/// Playing from a specific album
Album {
album_id: String,
album_name: String,
},
/// Playing from a specific playlist
Playlist {
playlist_id: String,
playlist_name: String,
},
/// Custom queue (search results, manual queue, etc.)
/// Will create a temporary playlist on remote transfer
#[default]
Custom,
}
/// Represents a subtitle track
///
/// 🔴 **Do not add `#[serde(rename_all = "camelCase")]` here.** This is the one
/// struct in the player that deliberately keeps snake_case on the wire, because
/// the *same* serialization feeds two consumers that both spell `mime_type`:
///
/// * the JNI boundary — `player/android/mod.rs` serializes `MediaItem::subtitles`
/// with `serde_json` and hands the string to `JellyTauPlayer.loadWithMetadata`,
/// whose parser reads `url`, `language`, `label` and `optString("mime_type")`;
/// * the IPC boundary — `PlayItemRequest::subtitles` deserializes this same type
/// from the frontend, and the generated binding (`SubtitleTrack` in
/// `bindings.ts`) therefore also declares `mime_type`.
///
/// Renaming would not break the build and would not fail the IPC: Kotlin's
/// `optString` would just fall back to its default MIME type for every track, so
/// the failure would be silent. UT-146 asserts the serialized keys.
///
/// TRACES: UR-020 | IR-016, JA-008 | UT-146
#[derive(specta::Type, Debug, Clone, Serialize, Deserialize, PartialEq)]
pub struct SubtitleTrack {
/// Stream index in the media source
pub index: i32,
/// Subtitle URL
pub url: String,
/// Language code (e.g., "eng", "spa")
pub language: Option<String>,
/// Display title
pub label: Option<String>,
/// MIME type (e.g., "text/vtt", "application/x-subrip").
/// Snake_case on purpose — see the note on the struct.
pub mime_type: String,
}
/// Represents a media item that can be played
///
/// TRACES: UR-003, UR-004 | DR-002
#[derive(specta::Type, Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "camelCase")]
#[specta(rename = "PlayerMediaItem")]
pub struct MediaItem {
/// Unique identifier
pub id: String,
/// Display title
pub title: String,
/// Name (alias for title - for frontend compatibility)
#[serde(default, skip_serializing_if = "Option::is_none")]
pub name: Option<String>,
/// Artist name(s) for audio
pub artist: Option<String>,
/// Album name for audio
pub album: Option<String>,
/// Album name (alias - for frontend compatibility)
#[serde(default, skip_serializing_if = "Option::is_none")]
pub album_name: Option<String>,
/// Album ID (Jellyfin ID) for remote transfer context
#[serde(default)]
pub album_id: Option<String>,
/// Artist items with IDs for clickable links
#[serde(default, skip_serializing_if = "Option::is_none")]
pub artist_items: Option<Vec<crate::repository::types::ArtistItem>>,
/// Artists as array of strings (fallback when artist_items not available)
#[serde(default, skip_serializing_if = "Option::is_none")]
pub artists: Option<Vec<String>>,
/// Primary image tag for artwork.
///
/// Legacy Jellyfin name; being replaced by `image_id` (same value). Dual-carried
/// while the frontend migrates (docs/specs/frontend-domain-model.md).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub primary_image_tag: Option<String>,
/// Neutral image identifier the frontend resolves to a URL — replaces
/// `primary_image_tag`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub image_id: Option<String>,
/// Item type (Audio, Movie, Episode, etc.)
#[serde(rename = "type", default, skip_serializing_if = "Option::is_none")]
pub item_type: Option<String>,
/// Playlist ID (Jellyfin ID) for remote transfer context
#[serde(default)]
pub playlist_id: Option<String>,
/// Duration in seconds
pub duration: Option<f64>,
/// URL or path to artwork image
pub artwork_url: Option<String>,
/// Type of media
pub media_type: MediaType,
/// Source of the media
pub source: MediaSource,
/// Video codec (e.g., "h264", "hevc") for video media
#[serde(default)]
pub video_codec: Option<String>,
/// Whether the video requires server-side transcoding
#[serde(default)]
pub needs_transcoding: bool,
/// Video width in pixels
#[serde(default)]
pub video_width: Option<u32>,
/// Video height in pixels
#[serde(default)]
pub video_height: Option<u32>,
/// Available subtitle tracks
#[serde(default)]
pub subtitles: Vec<SubtitleTrack>,
/// Series ID (for TV show episodes) - used for series audio preferences
#[serde(default)]
pub series_id: Option<String>,
/// Server ID - used for series audio preferences
#[serde(default)]
pub server_id: Option<String>,
}
#[derive(specta::Type, Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum MediaType {
Audio,
Video,
}
/// TRACES: UR-002, UR-003, UR-004, UR-011 | DR-003
#[derive(specta::Type, Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(tag = "type", rename_all = "lowercase")]
#[specta(rename = "PlayerMediaSource")]
pub enum MediaSource {
/// Streaming from Jellyfin server
Remote {
stream_url: String,
jellyfin_item_id: String,
},
/// Downloaded/cached locally
Local {
file_path: PathBuf,
/// Original Jellyfin ID for sync-back
jellyfin_item_id: Option<String>,
},
/// Direct URL (e.g., channel plugins)
DirectUrl { url: String },
}
impl MediaItem {
/// Get the Jellyfin item ID if available
pub fn jellyfin_id(&self) -> Option<&str> {
match &self.source {
MediaSource::Remote {
jellyfin_item_id, ..
} => Some(jellyfin_item_id),
MediaSource::Local {
jellyfin_item_id, ..
} => jellyfin_item_id.as_deref(),
MediaSource::DirectUrl { .. } => None,
}
}
/// Get the playback URL or file path
///
/// Only available on Android where ExoPlayer needs direct URL access
#[cfg(target_os = "android")]
pub fn playback_url(&self) -> String {
match &self.source {
MediaSource::Remote { stream_url, .. } => stream_url.clone(),
MediaSource::Local { file_path, .. } => file_path.to_string_lossy().to_string(),
MediaSource::DirectUrl { url } => url.clone(),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::PathBuf;
#[test]
fn test_queue_context_album() {
let context = QueueContext::Album {
album_id: "album-123".to_string(),
album_name: "Test Album".to_string(),
};
assert!(matches!(context, QueueContext::Album { .. }));
}
#[test]
fn test_queue_context_playlist() {
let context = QueueContext::Playlist {
playlist_id: "playlist-456".to_string(),
playlist_name: "Test Playlist".to_string(),
};
assert!(matches!(context, QueueContext::Playlist { .. }));
}
#[test]
fn test_queue_context_custom() {
let context = QueueContext::Custom;
assert!(matches!(context, QueueContext::Custom));
}
#[test]
fn test_queue_context_serialization() {
let context = QueueContext::Album {
album_id: "alb-001".to_string(),
album_name: "Album 001".to_string(),
};
let json = serde_json::to_string(&context);
assert!(json.is_ok());
let serialized = json.unwrap();
assert!(serialized.contains("album"));
}
#[test]
fn test_queue_context_default() {
let context = QueueContext::default();
assert!(matches!(context, QueueContext::Custom));
}
#[test]
fn test_queue_context_clone() {
let context = QueueContext::Album {
album_id: "clone-alb".to_string(),
album_name: "Clone Album".to_string(),
};
let cloned = context.clone();
assert_eq!(context, cloned);
}
#[test]
fn test_subtitle_track_creation() {
let track = SubtitleTrack {
index: 0,
url: "https://example.com/subs.vtt".to_string(),
language: Some("eng".to_string()),
label: Some("English".to_string()),
mime_type: "text/vtt".to_string(),
};
assert_eq!(track.index, 0);
assert_eq!(track.language, Some("eng".to_string()));
}
#[test]
fn test_subtitle_track_without_language() {
let track = SubtitleTrack {
index: 1,
url: "https://example.com/subs.srt".to_string(),
language: None,
label: Some("Subtitles".to_string()),
mime_type: "application/x-subrip".to_string(),
};
assert!(track.language.is_none());
assert!(track.label.is_some());
}
#[test]
fn test_subtitle_track_serialization() {
let track = SubtitleTrack {
index: 0,
url: "url.vtt".to_string(),
language: Some("eng".to_string()),
label: None,
mime_type: "text/vtt".to_string(),
};
let json = serde_json::to_string(&track);
assert!(json.is_ok());
let serialized = json.unwrap();
assert!(serialized.contains("0"));
assert!(serialized.contains("eng"));
}
#[test]
fn test_media_type_audio() {
let media_type = MediaType::Audio;
let json = serde_json::to_string(&media_type).unwrap();
assert!(json.contains("audio"));
}
#[test]
fn test_media_type_video() {
let media_type = MediaType::Video;
let json = serde_json::to_string(&media_type).unwrap();
assert!(json.contains("video"));
}
#[test]
fn test_media_source_remote() {
let source = MediaSource::Remote {
stream_url: "https://server.com/video.mp4".to_string(),
jellyfin_item_id: "item-123".to_string(),
};
assert!(matches!(source, MediaSource::Remote { .. }));
}
#[test]
fn test_media_source_local() {
let source = MediaSource::Local {
file_path: PathBuf::from("/path/to/video.mp4"),
jellyfin_item_id: Some("item-456".to_string()),
};
assert!(matches!(source, MediaSource::Local { .. }));
}
#[test]
fn test_media_source_local_without_jellyfin_id() {
let source = MediaSource::Local {
file_path: PathBuf::from("/downloads/audio.mp3"),
jellyfin_item_id: None,
};
assert!(matches!(source, MediaSource::Local { .. }));
}
#[test]
fn test_media_source_direct_url() {
let source = MediaSource::DirectUrl {
url: "https://external.com/stream".to_string(),
};
assert!(matches!(source, MediaSource::DirectUrl { .. }));
}
#[test]
fn test_media_source_serialization() {
let source = MediaSource::DirectUrl {
url: "https://example.com/stream".to_string(),
};
let json = serde_json::to_string(&source);
assert!(json.is_ok());
let serialized = json.unwrap();
assert!(serialized.contains("directurl"));
}
#[test]
fn test_media_item_creation_minimal() {
let item = MediaItem {
id: "item-1".to_string(),
title: "Test Item".to_string(),
name: None,
artist: None,
album: None,
album_name: None,
album_id: None,
artist_items: None,
artists: None,
primary_image_tag: None,
image_id: None,
item_type: None,
playlist_id: None,
duration: None,
artwork_url: None,
media_type: MediaType::Video,
source: MediaSource::DirectUrl {
url: "https://example.com/video".to_string(),
},
video_codec: None,
needs_transcoding: false,
video_width: None,
video_height: None,
subtitles: vec![],
series_id: None,
server_id: None,
};
assert_eq!(item.id, "item-1");
assert_eq!(item.title, "Test Item");
assert!(!item.needs_transcoding);
}
#[test]
fn test_media_item_jellyfin_id() {
let item = MediaItem {
id: "item-2".to_string(),
title: "Test".to_string(),
name: None,
artist: None,
album: None,
album_name: None,
album_id: None,
artist_items: None,
artists: None,
primary_image_tag: None,
image_id: None,
item_type: None,
playlist_id: None,
duration: None,
artwork_url: None,
media_type: MediaType::Audio,
source: MediaSource::Remote {
stream_url: "https://server/stream".to_string(),
jellyfin_item_id: "jf-id-123".to_string(),
},
video_codec: None,
needs_transcoding: false,
video_width: None,
video_height: None,
subtitles: vec![],
series_id: None,
server_id: None,
};
assert_eq!(item.jellyfin_id(), Some("jf-id-123"));
}
#[test]
fn test_media_item_jellyfin_id_local() {
let item = MediaItem {
id: "item-3".to_string(),
title: "Local".to_string(),
name: None,
artist: None,
album: None,
album_name: None,
album_id: None,
artist_items: None,
artists: None,
primary_image_tag: None,
image_id: None,
item_type: None,
playlist_id: None,
duration: None,
artwork_url: None,
media_type: MediaType::Video,
source: MediaSource::Local {
file_path: PathBuf::from("/local/video.mp4"),
jellyfin_item_id: Some("jf-local".to_string()),
},
video_codec: None,
needs_transcoding: false,
video_width: None,
video_height: None,
subtitles: vec![],
series_id: None,
server_id: None,
};
assert_eq!(item.jellyfin_id(), Some("jf-local"));
}
#[test]
fn test_media_item_jellyfin_id_direct_url() {
let item = MediaItem {
id: "item-4".to_string(),
title: "Direct".to_string(),
name: None,
artist: None,
album: None,
album_name: None,
album_id: None,
artist_items: None,
artists: None,
primary_image_tag: None,
image_id: None,
item_type: None,
playlist_id: None,
duration: None,
artwork_url: None,
media_type: MediaType::Video,
source: MediaSource::DirectUrl {
url: "https://external.com/media".to_string(),
},
video_codec: None,
needs_transcoding: false,
video_width: None,
video_height: None,
subtitles: vec![],
series_id: None,
server_id: None,
};
assert_eq!(item.jellyfin_id(), None);
}
#[test]
fn test_media_item_with_subtitles() {
let sub = SubtitleTrack {
index: 0,
url: "subs.vtt".to_string(),
language: Some("eng".to_string()),
label: Some("English".to_string()),
mime_type: "text/vtt".to_string(),
};
let item = MediaItem {
id: "item-subs".to_string(),
title: "With Subs".to_string(),
name: None,
artist: None,
album: None,
album_name: None,
album_id: None,
artist_items: None,
artists: None,
primary_image_tag: None,
image_id: None,
item_type: None,
playlist_id: None,
duration: None,
artwork_url: None,
media_type: MediaType::Video,
source: MediaSource::DirectUrl {
url: "video.mp4".to_string(),
},
video_codec: None,
needs_transcoding: false,
video_width: None,
video_height: None,
subtitles: vec![sub],
series_id: None,
server_id: None,
};
assert_eq!(item.subtitles.len(), 1);
assert_eq!(item.subtitles[0].language, Some("eng".to_string()));
}
#[test]
fn test_media_item_serialization() {
let item = MediaItem {
id: "serial-item".to_string(),
title: "Serial Test".to_string(),
name: Some("Name".to_string()),
artist: None,
album: None,
album_name: None,
album_id: None,
artist_items: None,
artists: None,
primary_image_tag: None,
image_id: None,
item_type: Some("Movie".to_string()),
playlist_id: None,
duration: Some(120.0),
artwork_url: None,
media_type: MediaType::Video,
source: MediaSource::DirectUrl {
url: "https://example.com/movie.mp4".to_string(),
},
video_codec: Some("h264".to_string()),
needs_transcoding: false,
video_width: Some(1920),
video_height: Some(1080),
subtitles: vec![],
series_id: None,
server_id: None,
};
let json = serde_json::to_string(&item);
assert!(json.is_ok());
let serialized = json.unwrap();
assert!(serialized.contains("serial-item"));
assert!(serialized.contains("Serial Test"));
}
}
Reference in New Issue View Git Blame Copy Permalink