jellytau/docs/architecture/05-platform-backends.md

# Platform-Specific Player Backends

## Player Events System

**Location**: `src-tauri/src/player/events.rs`

The player uses a push-based event system to notify the frontend of state changes:

```rust
pub enum PlayerStatusEvent {
    /// Playback position updated (emitted periodically during playback)
    PositionUpdate { position: f64, duration: f64 },

    /// Player state changed
    StateChanged { state: String, media_id: Option<String> },

    /// Media has finished loading and is ready to play
    MediaLoaded { duration: f64 },

    /// Playback has ended naturally
    PlaybackEnded,

    /// Buffering state changed
    Buffering { percent: u8 },

    /// An error occurred during playback
    Error { message: String, recoverable: bool },

    /// Volume changed
    VolumeChanged { volume: f32, muted: bool },

    /// Sleep timer state changed
    SleepTimerChanged {
        mode: SleepTimerMode,
        remaining_seconds: u32,
    },

    /// Show next episode popup with countdown
    ShowNextEpisodePopup {
        current_episode: MediaItem,
        next_episode: MediaItem,
        countdown_seconds: u32,
        auto_advance: bool,
    },

    /// Countdown tick (emitted every second during autoplay countdown)
    CountdownTick { remaining_seconds: u32 },

    /// Queue changed (items added, removed, reordered, or playback mode changed)
    QueueChanged {
        items: Vec<MediaItem>,
        current_index: Option<usize>,
        shuffle: bool,
        repeat: RepeatMode,
        has_next: bool,
        has_previous: bool,
    },

    /// Media session changed (activity context changed: Audio/Movie/TvShow/Idle)
    SessionChanged { session: MediaSessionType },
}
```

Events are emitted via Tauri's event system:

```mermaid
flowchart LR
    subgraph Backend["Player Backend"]
        MPV["MPV/ExoPlayer"]
    end

    subgraph EventSystem["Event System"]
        Emitter["TauriEventEmitter<br/>emit()"]
        Bus["Tauri Event Bus<br/>'player-event'"]
    end

    subgraph Frontend["Frontend"]
        Listener["playerEvents.ts<br/>Frontend Listener"]
        Store["Player Store Update<br/>(position, state, etc)"]
    end

    MPV --> Emitter --> Bus --> Listener --> Store
```

**Frontend Listener** (`src/lib/services/playerEvents.ts`):
- Listens for `player-event` Tauri events
- Updates player/queue stores based on event type
- Auto-advances to next track on `PlaybackEnded`
- On `StateChanged` events, calls `invoke("player_get_queue")` to update `appState.hasNext`/`hasPrevious` -- this enables MiniPlayer skip button state

**Important**: The command is `player_get_queue` (returns `QueueStatus` with `hasNext`/`hasPrevious`). There is no `player_get_queue_status` command.

## HTML5 Video Adapter (webview-rendered video)

**Location**: `src/lib/player/html5Adapter.ts`, `src/lib/player/index.ts`, report commands in
`src-tauri/src/commands/player/timers.rs`

Video on desktop (Linux WebKitGTK) — and, per current interim behavior, Android — is rendered by an
HTML5 `<video>`/HLS element **inside the webview**. libmpv is initialized audio-only (`vo=null`,
`video=false`), so the native backend cannot render or observe this element. The `<video>` is therefore
the real player, living outside Rust's reach.

To keep the `PlayerController` the single source of truth (matching the audio path), the HTML5 element
is treated as **a dumb output device that reports back into Rust**, rather than an independent state
authority:

```mermaid
flowchart LR
    subgraph Webview["Webview"]
        Video["HTML5 <video> / HLS.js"]
        Adapter["html5Adapter.ts<br/>(reports DOM events)"]
    end
    subgraph Backend["Rust"]
        Cmds["player_report_state<br/>player_report_position<br/>player_report_media_loaded"]
        Controller["PlayerController"]
        Emitter["TauriEventEmitter"]
    end
    subgraph Frontend["Frontend"]
        Events["playerEvents.ts"]
        Store["player store"]
    end

    Video -->|DOM events| Adapter --> Cmds --> Controller --> Emitter --> Events --> Store
```

**Key points:**
- The adapter re-emits the *same* `PlayerStatusEvent`s (`StateChanged`, `PositionUpdate`, `MediaLoaded`)
  the native backends emit, so `playerEvents.ts` needs **no** HTML5-specific branch — HTML5 is just
  another event source feeding the existing pipeline.
- Position reports are throttled (~250ms) to match the MPV cadence and avoid flooding IPC from the
  60fps RAF loop.
- **Boundary rule**: UI components never touch the report commands or `videoElement` state directly.
  Playback *control* goes through the unified facade `src/lib/player/index.ts` (`playerController`);
  HTML5 *state reporting* goes through `html5Adapter.ts`. This restores the documented invariant
  ("frontend only displays state and invokes commands") for the video path.

## MpvBackend (Linux)

**Location**: `src-tauri/src/player/mpv/`

The MPV backend uses libmpv for audio playback on Linux. Since MPV handles are not `Send`, all operations occur on a dedicated thread.

```mermaid
flowchart TB
    subgraph MainThread["Main Thread"]
        MpvBackend["MpvBackend<br/>- command_tx<br/>- shared_state<br/>- shutdown"]
        Commands["Commands:<br/>Load, Play, Pause<br/>Stop, Seek, SetVolume"]
    end

    subgraph EventLoopThread["MPV Event Loop Thread"]
        EventLoop["event_loop.rs<br/>- MPV Handle<br/>- command_rx<br/>- Event Emitter"]
        TauriEmitter["TauriEventEmitter"]
    end

    MpvBackend -->|"MpvCommand"| EventLoop
    MpvBackend <-->|"Arc<Mutex<>>"| EventLoop
    EventLoop -->|"Events"| TauriEmitter
    TauriEmitter --> FrontendStore["Frontend Store"]
```

**Key Components:**

```rust
// Command enum sent to event loop thread
pub enum MpvCommand {
    Load { url: String, media: MediaItem },
    Play,
    Pause,
    Stop,
    Seek(f64),
    SetVolume(f32),
    Quit,
}

// Shared state between main thread and event loop
pub struct MpvSharedState {
    pub state: PlayerState,
    pub position: f64,
    pub duration: Option<f64>,
    pub volume: f32,
    pub is_loaded: bool,
    pub current_media: Option<MediaItem>,
}
```

**Event Loop** (`event_loop.rs`):
- Initializes MPV with audio-only config (`vo=null`, `video=false`)
- Observes properties: `time-pos`, `duration`, `pause`, `volume`
- Emits position updates every 250ms during playback
- Processes commands from channel (non-blocking)
- Handles MPV events: `FileLoaded`, `EndFile`, `PropertyChange`

## ExoPlayerBackend (Android)

**Location**: `src-tauri/src/player/android/` and Kotlin sources

The ExoPlayer backend uses Android's Media3/ExoPlayer library via JNI.

```mermaid
flowchart TB
    subgraph RustNative["Rust (Native)"]
        ExoBackend["ExoPlayerBackend<br/>- player_ref<br/>- shared_state"]
        NativeFuncs["JNI Callbacks<br/>nativeOnPosition...<br/>nativeOnState...<br/>nativeOnMediaLoaded<br/>nativeOnPlaybackEnd"]
        TauriEmitter2["TauriEventEmitter"]
    end

    subgraph KotlinJVM["Kotlin (JVM)"]
        JellyTauPlayer["JellyTauPlayer<br/>- ExoPlayer<br/>- Player.Listener"]
    end

    ExoBackend -->|"JNI Calls"| JellyTauPlayer
    JellyTauPlayer -->|"Callbacks"| NativeFuncs
    NativeFuncs --> TauriEmitter2
    TauriEmitter2 --> FrontendStore2["Frontend Store"]
```

**Kotlin Player** (`JellyTauPlayer.kt`):
```kotlin
class JellyTauPlayer(context: Context) {
    private val exoPlayer: ExoPlayer
    private var positionUpdateJob: Job?

    // Methods callable from Rust via JNI
    fun load(url: String, mediaId: String)
    fun play()
    fun pause()
    fun stop()
    fun seek(positionSeconds: Double)
    fun setVolume(volume: Float)

    // Native callbacks to Rust
    private external fun nativeOnPositionUpdate(position: Double, duration: Double)
    private external fun nativeOnStateChanged(state: String, mediaId: String?)
    private external fun nativeOnMediaLoaded(duration: Double)
    private external fun nativeOnPlaybackEnded()
}
```

**JNI Callbacks** (Rust):
```rust
#[no_mangle]
pub extern "system" fn Java_com_dtourolle_jellytau_player_JellyTauPlayer_nativeOnPositionUpdate(
    _env: JNIEnv, _class: JClass, position: jdouble, duration: jdouble
) {
    // Update shared state
    // Emit PlayerStatusEvent::PositionUpdate
}
```

## Android MediaSession & Remote Volume Control

**Location**: `JellyTauPlaybackService.kt`

JellyTau uses a dual MediaSession architecture for Android to support both Media3 playback controls and remote volume control:

```mermaid
flowchart TB
    subgraph Service["JellyTauPlaybackService"]
        MediaSession["Media3 MediaSession<br/>- Lockscreen controls<br/>- Media notifications<br/>- Play/Pause/Next/Previous"]

        MediaSessionCompat["MediaSessionCompat<br/>- Remote volume control<br/>- Hardware button interception"]

        VolumeProvider["VolumeProviderCompat<br/>- onSetVolumeTo()<br/>- onAdjustVolume()"]

        MediaSessionCompat --> VolumeProvider
    end

    subgraph Hardware["System"]
        VolumeButtons["Hardware Volume Buttons"]
        Lockscreen["Lockscreen Controls"]
        Notification["Media Notification"]
    end

    subgraph Rust["Rust Backend"]
        JNI["JNI Callbacks<br/>nativeOnRemoteVolumeChange()"]
        PlaybackMode["PlaybackModeManager<br/>send_remote_volume_command()"]
        JellyfinAPI["Jellyfin API<br/>session_set_volume()"]
    end

    VolumeButtons --> VolumeProvider
    Lockscreen --> MediaSession
    Notification --> MediaSession

    VolumeProvider --> JNI
    JNI --> PlaybackMode
    PlaybackMode --> JellyfinAPI
```

**Architecture Rationale:**

JellyTau maintains both MediaSession types because they serve different purposes:

1. **Media3 MediaSession**: Handles lockscreen/notification playback controls (play/pause/next/previous)
2. **MediaSessionCompat**: Intercepts hardware volume button presses for remote playback control

When in remote playback mode (controlling a Jellyfin session on another device):
- Volume buttons are routed through `VolumeProviderCompat`
- Volume changes are sent to the remote session via Jellyfin API
- System volume UI shows the remote session's volume level

**Remote Volume Flow:**

```mermaid
sequenceDiagram
    participant User
    participant VolumeButton as Hardware Volume Button
    participant VolumeProvider as VolumeProviderCompat
    participant JNI as nativeOnRemoteVolumeChange
    participant PlaybackMode as PlaybackModeManager
    participant Jellyfin as Jellyfin Server
    participant RemoteSession as Remote Session (TV/Browser)

    User->>VolumeButton: Press Volume Up
    VolumeButton->>VolumeProvider: onAdjustVolume(ADJUST_RAISE)
    VolumeProvider->>VolumeProvider: remoteVolumeLevel += 2
    VolumeProvider->>VolumeProvider: currentVolume = remoteVolumeLevel
    VolumeProvider->>JNI: nativeOnRemoteVolumeChange("VolumeUp", level)
    JNI->>PlaybackMode: send_remote_volume_command("VolumeUp", level)
    PlaybackMode->>Jellyfin: POST /Sessions/{id}/Command/VolumeUp
    Jellyfin->>RemoteSession: Set volume to new level
    RemoteSession-->>User: Volume changes on TV/Browser
```

**Key Implementation Details:**

**Enabling Remote Volume** (`enableRemoteVolume()`):
```kotlin
fun enableRemoteVolume(initialVolume: Int) {
    volumeProvider = object : VolumeProviderCompat(
        VolumeProviderCompat.VOLUME_CONTROL_ABSOLUTE,
        100,  // Max volume
        initialVolume
    ) {
        override fun onSetVolumeTo(volume: Int) {
            remoteVolumeLevel = volume.coerceIn(0, 100)
            nativeOnRemoteVolumeChange("SetVolume", remoteVolumeLevel)
        }

        override fun onAdjustVolume(direction: Int) {
            when (direction) {
                AudioManager.ADJUST_RAISE -> {
                    remoteVolumeLevel = (remoteVolumeLevel + 2).coerceAtMost(100)
                    nativeOnRemoteVolumeChange("VolumeUp", remoteVolumeLevel)
                    currentVolume = remoteVolumeLevel
                }
                AudioManager.ADJUST_LOWER -> {
                    remoteVolumeLevel = (remoteVolumeLevel - 2).coerceAtLeast(0)
                    nativeOnRemoteVolumeChange("VolumeDown", remoteVolumeLevel)
                    currentVolume = remoteVolumeLevel
                }
            }
        }
    }

    mediaSessionCompat.setPlaybackToRemote(volumeProvider)
}
```

**Disabling Remote Volume** (`disableRemoteVolume()`):
```kotlin
fun disableRemoteVolume() {
    mediaSessionCompat.setPlaybackToLocal(AudioManager.STREAM_MUSIC)
    volumeProvider = null
}
```

**Rust Integration** (`src-tauri/src/player/android/mod.rs`):
```rust
/// Enable remote volume control on Android
pub fn enable_remote_volume(initial_volume: i32) -> Result<(), String> {
    start_playback_service()?;
    let service_instance = get_playback_service_instance()?;
    env.call_method(&service_instance, "enableRemoteVolume", "(I)V",
        &[JValue::Int(initial_volume)])?;
    Ok(())
}
```

**Dependencies** (`src-tauri/android/build.gradle.kts`):
```kotlin
dependencies {
    implementation("androidx.media3:media3-session:1.5.1")  // Media3 MediaSession
    implementation("androidx.media:media:1.7.0")            // MediaSessionCompat
}
```

**Integration with Playback Mode:**

Remote volume is automatically enabled/disabled during playback mode transfers:

```rust
// In PlaybackModeManager::transfer_to_remote()
#[cfg(target_os = "android")]
{
    if let Err(e) = crate::player::enable_remote_volume(50) {
        log::warn!("Failed to enable remote volume: {}", e);
    }
}

// In PlaybackModeManager::transfer_to_local()
#[cfg(target_os = "android")]
{
    if let Err(e) = crate::player::disable_remote_volume() {
        log::warn!("Failed to disable remote volume: {}", e);
    }
}
```

## Android Album Art Caching

**Location**: `src-tauri/android/src/main/java/com/dtourolle/jellytau/player/AlbumArtCache.kt`

Album art caching provides efficient bitmap storage for lock screen notifications with automatic LRU eviction and memory management.

```mermaid
flowchart TB
    subgraph JellyTauPlayer["JellyTauPlayer.kt"]
        LoadMedia["loadWithMetadata()<br/>- Store artworkUrl<br/>- Launch async download"]
        AsyncDownload["Coroutine<br/>- Non-blocking<br/>- Dispatchers.IO"]
    end

    subgraph Cache["AlbumArtCache.kt"]
        MemoryCache["LruCache<String, Bitmap><br/>- 1/8 of heap<br/>- ~12-16MB typical<br/>- 50-100 albums capacity"]
        Download["Download & Scale<br/>- 512x512 max<br/>- Exponential backoff"]
        ErrorHandle["Error Handling<br/>- Graceful fallback<br/>- Auto-retry"]
    end

    subgraph Service["JellyTauPlaybackService.kt"]
        UpdateMeta["updateMediaMetadata()<br/>- Accept Bitmap parameter<br/>- Add METADATA_KEY_ALBUM_ART"]
        Notification["Notification<br/>- setLargeIcon()<br/>- Lock screen display"]
    end

    LoadMedia --> AsyncDownload
    AsyncDownload --> MemoryCache
    MemoryCache --> Download
    Download --> ErrorHandle
    AsyncDownload --> UpdateMeta
    UpdateMeta --> Notification
```

**AlbumArtCache Singleton:**

```kotlin
class AlbumArtCache(context: Context) {
    private val memoryCache = object : LruCache<String, Bitmap>(cacheSize) {
        override fun sizeOf(key: String, bitmap: Bitmap): Int {
            return bitmap.byteCount / 1024  // Size in KB
        }
    }

    suspend fun getArtwork(url: String): Bitmap? {
        memoryCache.get(url)?.let { return it }
        return downloadAndCache(url)
    }

    private suspend fun downloadAndCache(url: String): Bitmap? =
        withContext(Dispatchers.IO) {
            // HTTP download with 5s timeout
            // Scale to 512x512 max
            // Auto-evict LRU if needed
        }
}
```

**Integration Flow:**

1. **Track Load** (`loadWithMetadata()`):
   - Store artwork URL in `currentArtworkUrl`
   - Reset bitmap to null
   - Start playback immediately (non-blocking)

2. **Async Download** (Background Coroutine):
   - Check cache: instant hit if available
   - Network miss: download, scale, cache
   - Auto-retry on network failure with exponential backoff
   - Graceful fallback if artwork unavailable

3. **Notification Update**:
   - Pass bitmap to `updatePlaybackServiceNotification()`
   - Add to `MediaMetadataCompat` with `METADATA_KEY_ALBUM_ART`
   - Display as large icon in notification
   - Show on lock screen

**Memory Management:**

| Metric | Value |
|--------|-------|
| Cache Size | 1/8 of heap (12-16MB typical) |
| Max Resolution | 512x512 pixels |
| Capacity | ~50-100 album arts |
| Eviction Policy | LRU (Least Recently Used) |
| Lifetime | In-memory only (app session) |
| Network Timeout | 5 seconds per download |

**Performance Characteristics:**

- **Cache Hit**: ~1ms (in-memory retrieval)
- **Cache Miss**: ~200-500ms (download + scale)
- **Playback Impact**: Zero (async downloads)
- **Memory Overhead**: Max 16MB (auto-eviction)
- **Error Recovery**: Automatic with exponential backoff

## Backend Initialization

**Location**: `src-tauri/src/lib.rs`

Backend selection is platform-specific:

```rust
fn create_player_backend(app_handle: tauri::AppHandle) -> Box<dyn PlayerBackend> {
    let event_emitter = Arc::new(TauriEventEmitter::new(app_handle));

    #[cfg(target_os = "linux")]
    {
        match MpvBackend::new(event_emitter.clone()) {
            Ok(backend) => return Box::new(backend),
            Err(e) => eprintln!("MPV init failed: {}", e),
        }
    }

    #[cfg(target_os = "android")]
    {
        // ExoPlayer requires Activity context, initialized separately
    }

    // Fallback
    Box::new(NullBackend::new())
}
```