486 lines
15 KiB
Markdown
486 lines
15 KiB
Markdown
# Platform-Specific Player Backends
|
|
|
|
## Player Events System
|
|
|
|
**Location**: `src-tauri/src/player/events.rs`
|
|
|
|
The player uses a push-based event system to notify the frontend of state changes:
|
|
|
|
```rust
|
|
pub enum PlayerStatusEvent {
|
|
/// Playback position updated (emitted periodically during playback)
|
|
PositionUpdate { position: f64, duration: f64 },
|
|
|
|
/// Player state changed
|
|
StateChanged { state: String, media_id: Option<String> },
|
|
|
|
/// Media has finished loading and is ready to play
|
|
MediaLoaded { duration: f64 },
|
|
|
|
/// Playback has ended naturally
|
|
PlaybackEnded,
|
|
|
|
/// Buffering state changed
|
|
Buffering { percent: u8 },
|
|
|
|
/// An error occurred during playback
|
|
Error { message: String, recoverable: bool },
|
|
|
|
/// Volume changed
|
|
VolumeChanged { volume: f32, muted: bool },
|
|
|
|
/// Sleep timer state changed
|
|
SleepTimerChanged {
|
|
mode: SleepTimerMode,
|
|
remaining_seconds: u32,
|
|
},
|
|
|
|
/// Show next episode popup with countdown
|
|
ShowNextEpisodePopup {
|
|
current_episode: MediaItem,
|
|
next_episode: MediaItem,
|
|
countdown_seconds: u32,
|
|
auto_advance: bool,
|
|
},
|
|
|
|
/// Countdown tick (emitted every second during autoplay countdown)
|
|
CountdownTick { remaining_seconds: u32 },
|
|
|
|
/// Queue changed (items added, removed, reordered, or playback mode changed)
|
|
QueueChanged {
|
|
items: Vec<MediaItem>,
|
|
current_index: Option<usize>,
|
|
shuffle: bool,
|
|
repeat: RepeatMode,
|
|
has_next: bool,
|
|
has_previous: bool,
|
|
},
|
|
|
|
/// Media session changed (activity context changed: Audio/Movie/TvShow/Idle)
|
|
SessionChanged { session: MediaSessionType },
|
|
}
|
|
```
|
|
|
|
Events are emitted via Tauri's event system:
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
subgraph Backend["Player Backend"]
|
|
MPV["MPV/ExoPlayer"]
|
|
end
|
|
|
|
subgraph EventSystem["Event System"]
|
|
Emitter["TauriEventEmitter<br/>emit()"]
|
|
Bus["Tauri Event Bus<br/>'player-event'"]
|
|
end
|
|
|
|
subgraph Frontend["Frontend"]
|
|
Listener["playerEvents.ts<br/>Frontend Listener"]
|
|
Store["Player Store Update<br/>(position, state, etc)"]
|
|
end
|
|
|
|
MPV --> Emitter --> Bus --> Listener --> Store
|
|
```
|
|
|
|
**Frontend Listener** (`src/lib/services/playerEvents.ts`):
|
|
- Listens for `player-event` Tauri events
|
|
- Updates player/queue stores based on event type
|
|
- Auto-advances to next track on `PlaybackEnded`
|
|
- On `StateChanged` events, calls `invoke("player_get_queue")` to update `appState.hasNext`/`hasPrevious` -- this enables MiniPlayer skip button state
|
|
|
|
**Important**: The command is `player_get_queue` (returns `QueueStatus` with `hasNext`/`hasPrevious`). There is no `player_get_queue_status` command.
|
|
|
|
## MpvBackend (Linux)
|
|
|
|
**Location**: `src-tauri/src/player/mpv/`
|
|
|
|
The MPV backend uses libmpv for audio playback on Linux. Since MPV handles are not `Send`, all operations occur on a dedicated thread.
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
subgraph MainThread["Main Thread"]
|
|
MpvBackend["MpvBackend<br/>- command_tx<br/>- shared_state<br/>- shutdown"]
|
|
Commands["Commands:<br/>Load, Play, Pause<br/>Stop, Seek, SetVolume"]
|
|
end
|
|
|
|
subgraph EventLoopThread["MPV Event Loop Thread"]
|
|
EventLoop["event_loop.rs<br/>- MPV Handle<br/>- command_rx<br/>- Event Emitter"]
|
|
TauriEmitter["TauriEventEmitter"]
|
|
end
|
|
|
|
MpvBackend -->|"MpvCommand"| EventLoop
|
|
MpvBackend <-->|"Arc<Mutex<>>"| EventLoop
|
|
EventLoop -->|"Events"| TauriEmitter
|
|
TauriEmitter --> FrontendStore["Frontend Store"]
|
|
```
|
|
|
|
**Key Components:**
|
|
|
|
```rust
|
|
// Command enum sent to event loop thread
|
|
pub enum MpvCommand {
|
|
Load { url: String, media: MediaItem },
|
|
Play,
|
|
Pause,
|
|
Stop,
|
|
Seek(f64),
|
|
SetVolume(f32),
|
|
Quit,
|
|
}
|
|
|
|
// Shared state between main thread and event loop
|
|
pub struct MpvSharedState {
|
|
pub state: PlayerState,
|
|
pub position: f64,
|
|
pub duration: Option<f64>,
|
|
pub volume: f32,
|
|
pub is_loaded: bool,
|
|
pub current_media: Option<MediaItem>,
|
|
}
|
|
```
|
|
|
|
**Event Loop** (`event_loop.rs`):
|
|
- Initializes MPV with audio-only config (`vo=null`, `video=false`)
|
|
- Observes properties: `time-pos`, `duration`, `pause`, `volume`
|
|
- Emits position updates every 250ms during playback
|
|
- Processes commands from channel (non-blocking)
|
|
- Handles MPV events: `FileLoaded`, `EndFile`, `PropertyChange`
|
|
|
|
## ExoPlayerBackend (Android)
|
|
|
|
**Location**: `src-tauri/src/player/android/` and Kotlin sources
|
|
|
|
The ExoPlayer backend uses Android's Media3/ExoPlayer library via JNI.
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
subgraph RustNative["Rust (Native)"]
|
|
ExoBackend["ExoPlayerBackend<br/>- player_ref<br/>- shared_state"]
|
|
NativeFuncs["JNI Callbacks<br/>nativeOnPosition...<br/>nativeOnState...<br/>nativeOnMediaLoaded<br/>nativeOnPlaybackEnd"]
|
|
TauriEmitter2["TauriEventEmitter"]
|
|
end
|
|
|
|
subgraph KotlinJVM["Kotlin (JVM)"]
|
|
JellyTauPlayer["JellyTauPlayer<br/>- ExoPlayer<br/>- Player.Listener"]
|
|
end
|
|
|
|
ExoBackend -->|"JNI Calls"| JellyTauPlayer
|
|
JellyTauPlayer -->|"Callbacks"| NativeFuncs
|
|
NativeFuncs --> TauriEmitter2
|
|
TauriEmitter2 --> FrontendStore2["Frontend Store"]
|
|
```
|
|
|
|
**Kotlin Player** (`JellyTauPlayer.kt`):
|
|
```kotlin
|
|
class JellyTauPlayer(context: Context) {
|
|
private val exoPlayer: ExoPlayer
|
|
private var positionUpdateJob: Job?
|
|
|
|
// Methods callable from Rust via JNI
|
|
fun load(url: String, mediaId: String)
|
|
fun play()
|
|
fun pause()
|
|
fun stop()
|
|
fun seek(positionSeconds: Double)
|
|
fun setVolume(volume: Float)
|
|
|
|
// Native callbacks to Rust
|
|
private external fun nativeOnPositionUpdate(position: Double, duration: Double)
|
|
private external fun nativeOnStateChanged(state: String, mediaId: String?)
|
|
private external fun nativeOnMediaLoaded(duration: Double)
|
|
private external fun nativeOnPlaybackEnded()
|
|
}
|
|
```
|
|
|
|
**JNI Callbacks** (Rust):
|
|
```rust
|
|
#[no_mangle]
|
|
pub extern "system" fn Java_com_dtourolle_jellytau_player_JellyTauPlayer_nativeOnPositionUpdate(
|
|
_env: JNIEnv, _class: JClass, position: jdouble, duration: jdouble
|
|
) {
|
|
// Update shared state
|
|
// Emit PlayerStatusEvent::PositionUpdate
|
|
}
|
|
```
|
|
|
|
## Android MediaSession & Remote Volume Control
|
|
|
|
**Location**: `JellyTauPlaybackService.kt`
|
|
|
|
JellyTau uses a dual MediaSession architecture for Android to support both Media3 playback controls and remote volume control:
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
subgraph Service["JellyTauPlaybackService"]
|
|
MediaSession["Media3 MediaSession<br/>- Lockscreen controls<br/>- Media notifications<br/>- Play/Pause/Next/Previous"]
|
|
|
|
MediaSessionCompat["MediaSessionCompat<br/>- Remote volume control<br/>- Hardware button interception"]
|
|
|
|
VolumeProvider["VolumeProviderCompat<br/>- onSetVolumeTo()<br/>- onAdjustVolume()"]
|
|
|
|
MediaSessionCompat --> VolumeProvider
|
|
end
|
|
|
|
subgraph Hardware["System"]
|
|
VolumeButtons["Hardware Volume Buttons"]
|
|
Lockscreen["Lockscreen Controls"]
|
|
Notification["Media Notification"]
|
|
end
|
|
|
|
subgraph Rust["Rust Backend"]
|
|
JNI["JNI Callbacks<br/>nativeOnRemoteVolumeChange()"]
|
|
PlaybackMode["PlaybackModeManager<br/>send_remote_volume_command()"]
|
|
JellyfinAPI["Jellyfin API<br/>session_set_volume()"]
|
|
end
|
|
|
|
VolumeButtons --> VolumeProvider
|
|
Lockscreen --> MediaSession
|
|
Notification --> MediaSession
|
|
|
|
VolumeProvider --> JNI
|
|
JNI --> PlaybackMode
|
|
PlaybackMode --> JellyfinAPI
|
|
```
|
|
|
|
**Architecture Rationale:**
|
|
|
|
JellyTau maintains both MediaSession types because they serve different purposes:
|
|
|
|
1. **Media3 MediaSession**: Handles lockscreen/notification playback controls (play/pause/next/previous)
|
|
2. **MediaSessionCompat**: Intercepts hardware volume button presses for remote playback control
|
|
|
|
When in remote playback mode (controlling a Jellyfin session on another device):
|
|
- Volume buttons are routed through `VolumeProviderCompat`
|
|
- Volume changes are sent to the remote session via Jellyfin API
|
|
- System volume UI shows the remote session's volume level
|
|
|
|
**Remote Volume Flow:**
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant User
|
|
participant VolumeButton as Hardware Volume Button
|
|
participant VolumeProvider as VolumeProviderCompat
|
|
participant JNI as nativeOnRemoteVolumeChange
|
|
participant PlaybackMode as PlaybackModeManager
|
|
participant Jellyfin as Jellyfin Server
|
|
participant RemoteSession as Remote Session (TV/Browser)
|
|
|
|
User->>VolumeButton: Press Volume Up
|
|
VolumeButton->>VolumeProvider: onAdjustVolume(ADJUST_RAISE)
|
|
VolumeProvider->>VolumeProvider: remoteVolumeLevel += 2
|
|
VolumeProvider->>VolumeProvider: currentVolume = remoteVolumeLevel
|
|
VolumeProvider->>JNI: nativeOnRemoteVolumeChange("VolumeUp", level)
|
|
JNI->>PlaybackMode: send_remote_volume_command("VolumeUp", level)
|
|
PlaybackMode->>Jellyfin: POST /Sessions/{id}/Command/VolumeUp
|
|
Jellyfin->>RemoteSession: Set volume to new level
|
|
RemoteSession-->>User: Volume changes on TV/Browser
|
|
```
|
|
|
|
**Key Implementation Details:**
|
|
|
|
**Enabling Remote Volume** (`enableRemoteVolume()`):
|
|
```kotlin
|
|
fun enableRemoteVolume(initialVolume: Int) {
|
|
volumeProvider = object : VolumeProviderCompat(
|
|
VolumeProviderCompat.VOLUME_CONTROL_ABSOLUTE,
|
|
100, // Max volume
|
|
initialVolume
|
|
) {
|
|
override fun onSetVolumeTo(volume: Int) {
|
|
remoteVolumeLevel = volume.coerceIn(0, 100)
|
|
nativeOnRemoteVolumeChange("SetVolume", remoteVolumeLevel)
|
|
}
|
|
|
|
override fun onAdjustVolume(direction: Int) {
|
|
when (direction) {
|
|
AudioManager.ADJUST_RAISE -> {
|
|
remoteVolumeLevel = (remoteVolumeLevel + 2).coerceAtMost(100)
|
|
nativeOnRemoteVolumeChange("VolumeUp", remoteVolumeLevel)
|
|
currentVolume = remoteVolumeLevel
|
|
}
|
|
AudioManager.ADJUST_LOWER -> {
|
|
remoteVolumeLevel = (remoteVolumeLevel - 2).coerceAtLeast(0)
|
|
nativeOnRemoteVolumeChange("VolumeDown", remoteVolumeLevel)
|
|
currentVolume = remoteVolumeLevel
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
mediaSessionCompat.setPlaybackToRemote(volumeProvider)
|
|
}
|
|
```
|
|
|
|
**Disabling Remote Volume** (`disableRemoteVolume()`):
|
|
```kotlin
|
|
fun disableRemoteVolume() {
|
|
mediaSessionCompat.setPlaybackToLocal(AudioManager.STREAM_MUSIC)
|
|
volumeProvider = null
|
|
}
|
|
```
|
|
|
|
**Rust Integration** (`src-tauri/src/player/android/mod.rs`):
|
|
```rust
|
|
/// Enable remote volume control on Android
|
|
pub fn enable_remote_volume(initial_volume: i32) -> Result<(), String> {
|
|
start_playback_service()?;
|
|
let service_instance = get_playback_service_instance()?;
|
|
env.call_method(&service_instance, "enableRemoteVolume", "(I)V",
|
|
&[JValue::Int(initial_volume)])?;
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
**Dependencies** (`src-tauri/android/build.gradle.kts`):
|
|
```kotlin
|
|
dependencies {
|
|
implementation("androidx.media3:media3-session:1.5.1") // Media3 MediaSession
|
|
implementation("androidx.media:media:1.7.0") // MediaSessionCompat
|
|
}
|
|
```
|
|
|
|
**Integration with Playback Mode:**
|
|
|
|
Remote volume is automatically enabled/disabled during playback mode transfers:
|
|
|
|
```rust
|
|
// In PlaybackModeManager::transfer_to_remote()
|
|
#[cfg(target_os = "android")]
|
|
{
|
|
if let Err(e) = crate::player::enable_remote_volume(50) {
|
|
log::warn!("Failed to enable remote volume: {}", e);
|
|
}
|
|
}
|
|
|
|
// In PlaybackModeManager::transfer_to_local()
|
|
#[cfg(target_os = "android")]
|
|
{
|
|
if let Err(e) = crate::player::disable_remote_volume() {
|
|
log::warn!("Failed to disable remote volume: {}", e);
|
|
}
|
|
}
|
|
```
|
|
|
|
## Android Album Art Caching
|
|
|
|
**Location**: `src-tauri/android/src/main/java/com/dtourolle/jellytau/player/AlbumArtCache.kt`
|
|
|
|
Album art caching provides efficient bitmap storage for lock screen notifications with automatic LRU eviction and memory management.
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
subgraph JellyTauPlayer["JellyTauPlayer.kt"]
|
|
LoadMedia["loadWithMetadata()<br/>- Store artworkUrl<br/>- Launch async download"]
|
|
AsyncDownload["Coroutine<br/>- Non-blocking<br/>- Dispatchers.IO"]
|
|
end
|
|
|
|
subgraph Cache["AlbumArtCache.kt"]
|
|
MemoryCache["LruCache<String, Bitmap><br/>- 1/8 of heap<br/>- ~12-16MB typical<br/>- 50-100 albums capacity"]
|
|
Download["Download & Scale<br/>- 512x512 max<br/>- Exponential backoff"]
|
|
ErrorHandle["Error Handling<br/>- Graceful fallback<br/>- Auto-retry"]
|
|
end
|
|
|
|
subgraph Service["JellyTauPlaybackService.kt"]
|
|
UpdateMeta["updateMediaMetadata()<br/>- Accept Bitmap parameter<br/>- Add METADATA_KEY_ALBUM_ART"]
|
|
Notification["Notification<br/>- setLargeIcon()<br/>- Lock screen display"]
|
|
end
|
|
|
|
LoadMedia --> AsyncDownload
|
|
AsyncDownload --> MemoryCache
|
|
MemoryCache --> Download
|
|
Download --> ErrorHandle
|
|
AsyncDownload --> UpdateMeta
|
|
UpdateMeta --> Notification
|
|
```
|
|
|
|
**AlbumArtCache Singleton:**
|
|
|
|
```kotlin
|
|
class AlbumArtCache(context: Context) {
|
|
private val memoryCache = object : LruCache<String, Bitmap>(cacheSize) {
|
|
override fun sizeOf(key: String, bitmap: Bitmap): Int {
|
|
return bitmap.byteCount / 1024 // Size in KB
|
|
}
|
|
}
|
|
|
|
suspend fun getArtwork(url: String): Bitmap? {
|
|
memoryCache.get(url)?.let { return it }
|
|
return downloadAndCache(url)
|
|
}
|
|
|
|
private suspend fun downloadAndCache(url: String): Bitmap? =
|
|
withContext(Dispatchers.IO) {
|
|
// HTTP download with 5s timeout
|
|
// Scale to 512x512 max
|
|
// Auto-evict LRU if needed
|
|
}
|
|
}
|
|
```
|
|
|
|
**Integration Flow:**
|
|
|
|
1. **Track Load** (`loadWithMetadata()`):
|
|
- Store artwork URL in `currentArtworkUrl`
|
|
- Reset bitmap to null
|
|
- Start playback immediately (non-blocking)
|
|
|
|
2. **Async Download** (Background Coroutine):
|
|
- Check cache: instant hit if available
|
|
- Network miss: download, scale, cache
|
|
- Auto-retry on network failure with exponential backoff
|
|
- Graceful fallback if artwork unavailable
|
|
|
|
3. **Notification Update**:
|
|
- Pass bitmap to `updatePlaybackServiceNotification()`
|
|
- Add to `MediaMetadataCompat` with `METADATA_KEY_ALBUM_ART`
|
|
- Display as large icon in notification
|
|
- Show on lock screen
|
|
|
|
**Memory Management:**
|
|
|
|
| Metric | Value |
|
|
|--------|-------|
|
|
| Cache Size | 1/8 of heap (12-16MB typical) |
|
|
| Max Resolution | 512x512 pixels |
|
|
| Capacity | ~50-100 album arts |
|
|
| Eviction Policy | LRU (Least Recently Used) |
|
|
| Lifetime | In-memory only (app session) |
|
|
| Network Timeout | 5 seconds per download |
|
|
|
|
**Performance Characteristics:**
|
|
|
|
- **Cache Hit**: ~1ms (in-memory retrieval)
|
|
- **Cache Miss**: ~200-500ms (download + scale)
|
|
- **Playback Impact**: Zero (async downloads)
|
|
- **Memory Overhead**: Max 16MB (auto-eviction)
|
|
- **Error Recovery**: Automatic with exponential backoff
|
|
|
|
## Backend Initialization
|
|
|
|
**Location**: `src-tauri/src/lib.rs`
|
|
|
|
Backend selection is platform-specific:
|
|
|
|
```rust
|
|
fn create_player_backend(app_handle: tauri::AppHandle) -> Box<dyn PlayerBackend> {
|
|
let event_emitter = Arc::new(TauriEventEmitter::new(app_handle));
|
|
|
|
#[cfg(target_os = "linux")]
|
|
{
|
|
match MpvBackend::new(event_emitter.clone()) {
|
|
Ok(backend) => return Box::new(backend),
|
|
Err(e) => eprintln!("MPV init failed: {}", e),
|
|
}
|
|
}
|
|
|
|
#[cfg(target_os = "android")]
|
|
{
|
|
// ExoPlayer requires Activity context, initialized separately
|
|
}
|
|
|
|
// Fallback
|
|
Box::new(NullBackend::new())
|
|
}
|
|
```
|