Split software arch desc for easier manintenance. Many fixes related to next video playing and remote playback
This commit is contained in:
@@ -0,0 +1,485 @@
|
||||
# Platform-Specific Player Backends
|
||||
|
||||
## Player Events System
|
||||
|
||||
**Location**: `src-tauri/src/player/events.rs`
|
||||
|
||||
The player uses a push-based event system to notify the frontend of state changes:
|
||||
|
||||
```rust
|
||||
pub enum PlayerStatusEvent {
|
||||
/// Playback position updated (emitted periodically during playback)
|
||||
PositionUpdate { position: f64, duration: f64 },
|
||||
|
||||
/// Player state changed
|
||||
StateChanged { state: String, media_id: Option<String> },
|
||||
|
||||
/// Media has finished loading and is ready to play
|
||||
MediaLoaded { duration: f64 },
|
||||
|
||||
/// Playback has ended naturally
|
||||
PlaybackEnded,
|
||||
|
||||
/// Buffering state changed
|
||||
Buffering { percent: u8 },
|
||||
|
||||
/// An error occurred during playback
|
||||
Error { message: String, recoverable: bool },
|
||||
|
||||
/// Volume changed
|
||||
VolumeChanged { volume: f32, muted: bool },
|
||||
|
||||
/// Sleep timer state changed
|
||||
SleepTimerChanged {
|
||||
mode: SleepTimerMode,
|
||||
remaining_seconds: u32,
|
||||
},
|
||||
|
||||
/// Show next episode popup with countdown
|
||||
ShowNextEpisodePopup {
|
||||
current_episode: MediaItem,
|
||||
next_episode: MediaItem,
|
||||
countdown_seconds: u32,
|
||||
auto_advance: bool,
|
||||
},
|
||||
|
||||
/// Countdown tick (emitted every second during autoplay countdown)
|
||||
CountdownTick { remaining_seconds: u32 },
|
||||
|
||||
/// Queue changed (items added, removed, reordered, or playback mode changed)
|
||||
QueueChanged {
|
||||
items: Vec<MediaItem>,
|
||||
current_index: Option<usize>,
|
||||
shuffle: bool,
|
||||
repeat: RepeatMode,
|
||||
has_next: bool,
|
||||
has_previous: bool,
|
||||
},
|
||||
|
||||
/// Media session changed (activity context changed: Audio/Movie/TvShow/Idle)
|
||||
SessionChanged { session: MediaSessionType },
|
||||
}
|
||||
```
|
||||
|
||||
Events are emitted via Tauri's event system:
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Backend["Player Backend"]
|
||||
MPV["MPV/ExoPlayer"]
|
||||
end
|
||||
|
||||
subgraph EventSystem["Event System"]
|
||||
Emitter["TauriEventEmitter<br/>emit()"]
|
||||
Bus["Tauri Event Bus<br/>'player-event'"]
|
||||
end
|
||||
|
||||
subgraph Frontend["Frontend"]
|
||||
Listener["playerEvents.ts<br/>Frontend Listener"]
|
||||
Store["Player Store Update<br/>(position, state, etc)"]
|
||||
end
|
||||
|
||||
MPV --> Emitter --> Bus --> Listener --> Store
|
||||
```
|
||||
|
||||
**Frontend Listener** (`src/lib/services/playerEvents.ts`):
|
||||
- Listens for `player-event` Tauri events
|
||||
- Updates player/queue stores based on event type
|
||||
- Auto-advances to next track on `PlaybackEnded`
|
||||
- On `StateChanged` events, calls `invoke("player_get_queue")` to update `appState.hasNext`/`hasPrevious` -- this enables MiniPlayer skip button state
|
||||
|
||||
**Important**: The command is `player_get_queue` (returns `QueueStatus` with `hasNext`/`hasPrevious`). There is no `player_get_queue_status` command.
|
||||
|
||||
## MpvBackend (Linux)
|
||||
|
||||
**Location**: `src-tauri/src/player/mpv/`
|
||||
|
||||
The MPV backend uses libmpv for audio playback on Linux. Since MPV handles are not `Send`, all operations occur on a dedicated thread.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph MainThread["Main Thread"]
|
||||
MpvBackend["MpvBackend<br/>- command_tx<br/>- shared_state<br/>- shutdown"]
|
||||
Commands["Commands:<br/>Load, Play, Pause<br/>Stop, Seek, SetVolume"]
|
||||
end
|
||||
|
||||
subgraph EventLoopThread["MPV Event Loop Thread"]
|
||||
EventLoop["event_loop.rs<br/>- MPV Handle<br/>- command_rx<br/>- Event Emitter"]
|
||||
TauriEmitter["TauriEventEmitter"]
|
||||
end
|
||||
|
||||
MpvBackend -->|"MpvCommand"| EventLoop
|
||||
MpvBackend <-->|"Arc<Mutex<>>"| EventLoop
|
||||
EventLoop -->|"Events"| TauriEmitter
|
||||
TauriEmitter --> FrontendStore["Frontend Store"]
|
||||
```
|
||||
|
||||
**Key Components:**
|
||||
|
||||
```rust
|
||||
// Command enum sent to event loop thread
|
||||
pub enum MpvCommand {
|
||||
Load { url: String, media: MediaItem },
|
||||
Play,
|
||||
Pause,
|
||||
Stop,
|
||||
Seek(f64),
|
||||
SetVolume(f32),
|
||||
Quit,
|
||||
}
|
||||
|
||||
// Shared state between main thread and event loop
|
||||
pub struct MpvSharedState {
|
||||
pub state: PlayerState,
|
||||
pub position: f64,
|
||||
pub duration: Option<f64>,
|
||||
pub volume: f32,
|
||||
pub is_loaded: bool,
|
||||
pub current_media: Option<MediaItem>,
|
||||
}
|
||||
```
|
||||
|
||||
**Event Loop** (`event_loop.rs`):
|
||||
- Initializes MPV with audio-only config (`vo=null`, `video=false`)
|
||||
- Observes properties: `time-pos`, `duration`, `pause`, `volume`
|
||||
- Emits position updates every 250ms during playback
|
||||
- Processes commands from channel (non-blocking)
|
||||
- Handles MPV events: `FileLoaded`, `EndFile`, `PropertyChange`
|
||||
|
||||
## ExoPlayerBackend (Android)
|
||||
|
||||
**Location**: `src-tauri/src/player/android/` and Kotlin sources
|
||||
|
||||
The ExoPlayer backend uses Android's Media3/ExoPlayer library via JNI.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph RustNative["Rust (Native)"]
|
||||
ExoBackend["ExoPlayerBackend<br/>- player_ref<br/>- shared_state"]
|
||||
NativeFuncs["JNI Callbacks<br/>nativeOnPosition...<br/>nativeOnState...<br/>nativeOnMediaLoaded<br/>nativeOnPlaybackEnd"]
|
||||
TauriEmitter2["TauriEventEmitter"]
|
||||
end
|
||||
|
||||
subgraph KotlinJVM["Kotlin (JVM)"]
|
||||
JellyTauPlayer["JellyTauPlayer<br/>- ExoPlayer<br/>- Player.Listener"]
|
||||
end
|
||||
|
||||
ExoBackend -->|"JNI Calls"| JellyTauPlayer
|
||||
JellyTauPlayer -->|"Callbacks"| NativeFuncs
|
||||
NativeFuncs --> TauriEmitter2
|
||||
TauriEmitter2 --> FrontendStore2["Frontend Store"]
|
||||
```
|
||||
|
||||
**Kotlin Player** (`JellyTauPlayer.kt`):
|
||||
```kotlin
|
||||
class JellyTauPlayer(context: Context) {
|
||||
private val exoPlayer: ExoPlayer
|
||||
private var positionUpdateJob: Job?
|
||||
|
||||
// Methods callable from Rust via JNI
|
||||
fun load(url: String, mediaId: String)
|
||||
fun play()
|
||||
fun pause()
|
||||
fun stop()
|
||||
fun seek(positionSeconds: Double)
|
||||
fun setVolume(volume: Float)
|
||||
|
||||
// Native callbacks to Rust
|
||||
private external fun nativeOnPositionUpdate(position: Double, duration: Double)
|
||||
private external fun nativeOnStateChanged(state: String, mediaId: String?)
|
||||
private external fun nativeOnMediaLoaded(duration: Double)
|
||||
private external fun nativeOnPlaybackEnded()
|
||||
}
|
||||
```
|
||||
|
||||
**JNI Callbacks** (Rust):
|
||||
```rust
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_dtourolle_jellytau_player_JellyTauPlayer_nativeOnPositionUpdate(
|
||||
_env: JNIEnv, _class: JClass, position: jdouble, duration: jdouble
|
||||
) {
|
||||
// Update shared state
|
||||
// Emit PlayerStatusEvent::PositionUpdate
|
||||
}
|
||||
```
|
||||
|
||||
## Android MediaSession & Remote Volume Control
|
||||
|
||||
**Location**: `JellyTauPlaybackService.kt`
|
||||
|
||||
JellyTau uses a dual MediaSession architecture for Android to support both Media3 playback controls and remote volume control:
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Service["JellyTauPlaybackService"]
|
||||
MediaSession["Media3 MediaSession<br/>- Lockscreen controls<br/>- Media notifications<br/>- Play/Pause/Next/Previous"]
|
||||
|
||||
MediaSessionCompat["MediaSessionCompat<br/>- Remote volume control<br/>- Hardware button interception"]
|
||||
|
||||
VolumeProvider["VolumeProviderCompat<br/>- onSetVolumeTo()<br/>- onAdjustVolume()"]
|
||||
|
||||
MediaSessionCompat --> VolumeProvider
|
||||
end
|
||||
|
||||
subgraph Hardware["System"]
|
||||
VolumeButtons["Hardware Volume Buttons"]
|
||||
Lockscreen["Lockscreen Controls"]
|
||||
Notification["Media Notification"]
|
||||
end
|
||||
|
||||
subgraph Rust["Rust Backend"]
|
||||
JNI["JNI Callbacks<br/>nativeOnRemoteVolumeChange()"]
|
||||
PlaybackMode["PlaybackModeManager<br/>send_remote_volume_command()"]
|
||||
JellyfinAPI["Jellyfin API<br/>session_set_volume()"]
|
||||
end
|
||||
|
||||
VolumeButtons --> VolumeProvider
|
||||
Lockscreen --> MediaSession
|
||||
Notification --> MediaSession
|
||||
|
||||
VolumeProvider --> JNI
|
||||
JNI --> PlaybackMode
|
||||
PlaybackMode --> JellyfinAPI
|
||||
```
|
||||
|
||||
**Architecture Rationale:**
|
||||
|
||||
JellyTau maintains both MediaSession types because they serve different purposes:
|
||||
|
||||
1. **Media3 MediaSession**: Handles lockscreen/notification playback controls (play/pause/next/previous)
|
||||
2. **MediaSessionCompat**: Intercepts hardware volume button presses for remote playback control
|
||||
|
||||
When in remote playback mode (controlling a Jellyfin session on another device):
|
||||
- Volume buttons are routed through `VolumeProviderCompat`
|
||||
- Volume changes are sent to the remote session via Jellyfin API
|
||||
- System volume UI shows the remote session's volume level
|
||||
|
||||
**Remote Volume Flow:**
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User
|
||||
participant VolumeButton as Hardware Volume Button
|
||||
participant VolumeProvider as VolumeProviderCompat
|
||||
participant JNI as nativeOnRemoteVolumeChange
|
||||
participant PlaybackMode as PlaybackModeManager
|
||||
participant Jellyfin as Jellyfin Server
|
||||
participant RemoteSession as Remote Session (TV/Browser)
|
||||
|
||||
User->>VolumeButton: Press Volume Up
|
||||
VolumeButton->>VolumeProvider: onAdjustVolume(ADJUST_RAISE)
|
||||
VolumeProvider->>VolumeProvider: remoteVolumeLevel += 2
|
||||
VolumeProvider->>VolumeProvider: currentVolume = remoteVolumeLevel
|
||||
VolumeProvider->>JNI: nativeOnRemoteVolumeChange("VolumeUp", level)
|
||||
JNI->>PlaybackMode: send_remote_volume_command("VolumeUp", level)
|
||||
PlaybackMode->>Jellyfin: POST /Sessions/{id}/Command/VolumeUp
|
||||
Jellyfin->>RemoteSession: Set volume to new level
|
||||
RemoteSession-->>User: Volume changes on TV/Browser
|
||||
```
|
||||
|
||||
**Key Implementation Details:**
|
||||
|
||||
**Enabling Remote Volume** (`enableRemoteVolume()`):
|
||||
```kotlin
|
||||
fun enableRemoteVolume(initialVolume: Int) {
|
||||
volumeProvider = object : VolumeProviderCompat(
|
||||
VolumeProviderCompat.VOLUME_CONTROL_ABSOLUTE,
|
||||
100, // Max volume
|
||||
initialVolume
|
||||
) {
|
||||
override fun onSetVolumeTo(volume: Int) {
|
||||
remoteVolumeLevel = volume.coerceIn(0, 100)
|
||||
nativeOnRemoteVolumeChange("SetVolume", remoteVolumeLevel)
|
||||
}
|
||||
|
||||
override fun onAdjustVolume(direction: Int) {
|
||||
when (direction) {
|
||||
AudioManager.ADJUST_RAISE -> {
|
||||
remoteVolumeLevel = (remoteVolumeLevel + 2).coerceAtMost(100)
|
||||
nativeOnRemoteVolumeChange("VolumeUp", remoteVolumeLevel)
|
||||
currentVolume = remoteVolumeLevel
|
||||
}
|
||||
AudioManager.ADJUST_LOWER -> {
|
||||
remoteVolumeLevel = (remoteVolumeLevel - 2).coerceAtLeast(0)
|
||||
nativeOnRemoteVolumeChange("VolumeDown", remoteVolumeLevel)
|
||||
currentVolume = remoteVolumeLevel
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
mediaSessionCompat.setPlaybackToRemote(volumeProvider)
|
||||
}
|
||||
```
|
||||
|
||||
**Disabling Remote Volume** (`disableRemoteVolume()`):
|
||||
```kotlin
|
||||
fun disableRemoteVolume() {
|
||||
mediaSessionCompat.setPlaybackToLocal(AudioManager.STREAM_MUSIC)
|
||||
volumeProvider = null
|
||||
}
|
||||
```
|
||||
|
||||
**Rust Integration** (`src-tauri/src/player/android/mod.rs`):
|
||||
```rust
|
||||
/// Enable remote volume control on Android
|
||||
pub fn enable_remote_volume(initial_volume: i32) -> Result<(), String> {
|
||||
start_playback_service()?;
|
||||
let service_instance = get_playback_service_instance()?;
|
||||
env.call_method(&service_instance, "enableRemoteVolume", "(I)V",
|
||||
&[JValue::Int(initial_volume)])?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
**Dependencies** (`src-tauri/android/build.gradle.kts`):
|
||||
```kotlin
|
||||
dependencies {
|
||||
implementation("androidx.media3:media3-session:1.5.1") // Media3 MediaSession
|
||||
implementation("androidx.media:media:1.7.0") // MediaSessionCompat
|
||||
}
|
||||
```
|
||||
|
||||
**Integration with Playback Mode:**
|
||||
|
||||
Remote volume is automatically enabled/disabled during playback mode transfers:
|
||||
|
||||
```rust
|
||||
// In PlaybackModeManager::transfer_to_remote()
|
||||
#[cfg(target_os = "android")]
|
||||
{
|
||||
if let Err(e) = crate::player::enable_remote_volume(50) {
|
||||
log::warn!("Failed to enable remote volume: {}", e);
|
||||
}
|
||||
}
|
||||
|
||||
// In PlaybackModeManager::transfer_to_local()
|
||||
#[cfg(target_os = "android")]
|
||||
{
|
||||
if let Err(e) = crate::player::disable_remote_volume() {
|
||||
log::warn!("Failed to disable remote volume: {}", e);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Android Album Art Caching
|
||||
|
||||
**Location**: `src-tauri/android/src/main/java/com/dtourolle/jellytau/player/AlbumArtCache.kt`
|
||||
|
||||
Album art caching provides efficient bitmap storage for lock screen notifications with automatic LRU eviction and memory management.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph JellyTauPlayer["JellyTauPlayer.kt"]
|
||||
LoadMedia["loadWithMetadata()<br/>- Store artworkUrl<br/>- Launch async download"]
|
||||
AsyncDownload["Coroutine<br/>- Non-blocking<br/>- Dispatchers.IO"]
|
||||
end
|
||||
|
||||
subgraph Cache["AlbumArtCache.kt"]
|
||||
MemoryCache["LruCache<String, Bitmap><br/>- 1/8 of heap<br/>- ~12-16MB typical<br/>- 50-100 albums capacity"]
|
||||
Download["Download & Scale<br/>- 512x512 max<br/>- Exponential backoff"]
|
||||
ErrorHandle["Error Handling<br/>- Graceful fallback<br/>- Auto-retry"]
|
||||
end
|
||||
|
||||
subgraph Service["JellyTauPlaybackService.kt"]
|
||||
UpdateMeta["updateMediaMetadata()<br/>- Accept Bitmap parameter<br/>- Add METADATA_KEY_ALBUM_ART"]
|
||||
Notification["Notification<br/>- setLargeIcon()<br/>- Lock screen display"]
|
||||
end
|
||||
|
||||
LoadMedia --> AsyncDownload
|
||||
AsyncDownload --> MemoryCache
|
||||
MemoryCache --> Download
|
||||
Download --> ErrorHandle
|
||||
AsyncDownload --> UpdateMeta
|
||||
UpdateMeta --> Notification
|
||||
```
|
||||
|
||||
**AlbumArtCache Singleton:**
|
||||
|
||||
```kotlin
|
||||
class AlbumArtCache(context: Context) {
|
||||
private val memoryCache = object : LruCache<String, Bitmap>(cacheSize) {
|
||||
override fun sizeOf(key: String, bitmap: Bitmap): Int {
|
||||
return bitmap.byteCount / 1024 // Size in KB
|
||||
}
|
||||
}
|
||||
|
||||
suspend fun getArtwork(url: String): Bitmap? {
|
||||
memoryCache.get(url)?.let { return it }
|
||||
return downloadAndCache(url)
|
||||
}
|
||||
|
||||
private suspend fun downloadAndCache(url: String): Bitmap? =
|
||||
withContext(Dispatchers.IO) {
|
||||
// HTTP download with 5s timeout
|
||||
// Scale to 512x512 max
|
||||
// Auto-evict LRU if needed
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Integration Flow:**
|
||||
|
||||
1. **Track Load** (`loadWithMetadata()`):
|
||||
- Store artwork URL in `currentArtworkUrl`
|
||||
- Reset bitmap to null
|
||||
- Start playback immediately (non-blocking)
|
||||
|
||||
2. **Async Download** (Background Coroutine):
|
||||
- Check cache: instant hit if available
|
||||
- Network miss: download, scale, cache
|
||||
- Auto-retry on network failure with exponential backoff
|
||||
- Graceful fallback if artwork unavailable
|
||||
|
||||
3. **Notification Update**:
|
||||
- Pass bitmap to `updatePlaybackServiceNotification()`
|
||||
- Add to `MediaMetadataCompat` with `METADATA_KEY_ALBUM_ART`
|
||||
- Display as large icon in notification
|
||||
- Show on lock screen
|
||||
|
||||
**Memory Management:**
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Cache Size | 1/8 of heap (12-16MB typical) |
|
||||
| Max Resolution | 512x512 pixels |
|
||||
| Capacity | ~50-100 album arts |
|
||||
| Eviction Policy | LRU (Least Recently Used) |
|
||||
| Lifetime | In-memory only (app session) |
|
||||
| Network Timeout | 5 seconds per download |
|
||||
|
||||
**Performance Characteristics:**
|
||||
|
||||
- **Cache Hit**: ~1ms (in-memory retrieval)
|
||||
- **Cache Miss**: ~200-500ms (download + scale)
|
||||
- **Playback Impact**: Zero (async downloads)
|
||||
- **Memory Overhead**: Max 16MB (auto-eviction)
|
||||
- **Error Recovery**: Automatic with exponential backoff
|
||||
|
||||
## Backend Initialization
|
||||
|
||||
**Location**: `src-tauri/src/lib.rs`
|
||||
|
||||
Backend selection is platform-specific:
|
||||
|
||||
```rust
|
||||
fn create_player_backend(app_handle: tauri::AppHandle) -> Box<dyn PlayerBackend> {
|
||||
let event_emitter = Arc::new(TauriEventEmitter::new(app_handle));
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
{
|
||||
match MpvBackend::new(event_emitter.clone()) {
|
||||
Ok(backend) => return Box::new(backend),
|
||||
Err(e) => eprintln!("MPV init failed: {}", e),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(target_os = "android")]
|
||||
{
|
||||
// ExoPlayer requires Activity context, initialized separately
|
||||
}
|
||||
|
||||
// Fallback
|
||||
Box::new(NullBackend::new())
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user