mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.
That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.
Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.
DR-238/246 a seek routed by the stream's container rather than by what the
engine could do with it - correct only while one player handled
those streams, silent the moment another did
DR-239 a property handled but never observed, so the play/pause button
waited for an event that could not arrive
DR-240 fullscreen expanding the document while the window stayed put
DR-241 a seek issued before the engine had a file, failed, and discarded
- which is why resume began at zero
DR-247 a Linux-only gate outliving the caller that made it Linux-only,
breaking the Android build outright
DR-250 a stop aimed at whichever renderer bookkeeping believed was in
charge, missing the one actually making sound
DR-251 a duration of zero believed, leaving the seek bar no scale
DR-252 a junk float converted to a Duration, panicking the backend the
instant a length-less stream appeared
So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.
Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.
Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.
Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.
Squashed from worktree-linux-native-video, which keeps the per-defect history.
268 lines
10 KiB
Markdown
268 lines
10 KiB
Markdown
# Data Flow
|
|
|
|
## Repository Query Flow (Cache-First)
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant UI as Svelte Component
|
|
participant Client as RepositoryClient (TS)
|
|
participant Rust as Tauri Command
|
|
participant Hybrid as HybridRepository
|
|
participant Cache as OfflineRepository (SQLite)
|
|
participant Server as OnlineRepository (HTTP)
|
|
participant Conn as ConnectivityMonitor
|
|
|
|
UI->>Client: getItems(parentId)
|
|
Client->>Rust: invoke("repository_get_items", {handle, parentId})
|
|
Rust->>Hybrid: get_items()
|
|
|
|
par Parallel Racing
|
|
Hybrid->>Cache: get_items() with 100ms timeout
|
|
Hybrid->>Server: get_items() (no timeout)
|
|
end
|
|
|
|
Note over Server,Conn: Every server request reports its outcome
|
|
alt Server succeeds (or answers with 4xx/5xx)
|
|
Server->>Conn: mark_reachable() (server is up)
|
|
else Network failure / timeout
|
|
Server->>Conn: mark_unreachable() (debounced)
|
|
end
|
|
|
|
alt Cache returns with content
|
|
Cache-->>Hybrid: Result with items
|
|
Hybrid-->>Rust: Return cache result
|
|
else Cache timeout or empty
|
|
Server-->>Hybrid: Fresh result
|
|
Hybrid-->>Rust: Return server result
|
|
end
|
|
|
|
Rust-->>Client: SearchResult
|
|
Client-->>UI: items[]
|
|
Note over UI: Reactive update
|
|
```
|
|
|
|
**Key Points:**
|
|
- Cache queries have 100ms timeout for responsiveness
|
|
- Server queries always run for fresh data
|
|
- Cache wins if it has meaningful content
|
|
- Automatic fallback to server if cache is empty/stale
|
|
- Background cache updates (planned)
|
|
- **Connectivity side-effect**: each server request feeds the `ConnectivityMonitor`, which is the source of truth for the offline/online banner (see [07-connectivity.md](07-connectivity.md)). A server-answered error (401/404/5xx) still counts as *reachable* — only network failures, sustained past a debounce window, flip the app to offline.
|
|
|
|
## Search Flow (Locally Indexed)
|
|
|
|
**TRACES**: UR-065 | DR-108 … DR-111, IR-030
|
|
|
|
Search does not depend on a per-keystroke round trip to Jellyfin. The instant leg
|
|
reads the **local SQLite catalog**, which is already synced and already
|
|
FTS5-indexed, so results appear as fast as SQLite can answer — online or offline.
|
|
The server query stays, demoted to a background reconciliation that merges in
|
|
late results.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant UI as Search UI
|
|
participant Rust as repository_search
|
|
participant Cache as Local catalog (FTS5)
|
|
participant Server as Jellyfin
|
|
participant Indexer as spawn_catalog_indexer
|
|
|
|
UI->>Rust: search(query, scope)
|
|
Rust->>Cache: FTS5 query, scope expanded by SearchScope::item_types()
|
|
Cache-->>UI: instant results
|
|
Rust->>Server: reconciliation query (background)
|
|
Server-->>UI: search-event with late/merged results
|
|
Note over Indexer,Cache: Independent of any query:<br/>scheduled crawl keeps the index fresh,<br/>prunes items deleted on the server
|
|
```
|
|
|
|
**Key points:**
|
|
|
|
- The **scope is opaque on the wire**. The frontend sends a `SearchScope`
|
|
variant; Rust expands it to item types
|
|
([01-rust-backend.md](01-rust-backend.md#search-scope-and-the-taxonomy-boundary)).
|
|
- **Index freshness is a Rust policy**, not a frontend startup call — a scheduled
|
|
background pass, not "whatever was synced when the app last launched"
|
|
(DR-109). See
|
|
[Background workers](01-rust-backend.md#background-workers).
|
|
- **Index hygiene matters as much as freshness**: the catalog save path uses
|
|
`INSERT OR REPLACE` and the crawl prunes rows for content deleted on the
|
|
server, or search keeps returning items that no longer exist (DR-110).
|
|
- The index covers **exactly the types the result groups render** (DR-111) —
|
|
including Artists, which the crawl must reach or the Artists group is silently
|
|
always empty.
|
|
|
|
**Deliberately not done, with reasons:**
|
|
|
|
- **Incremental indexing** (Jellyfin's `MinDateLastSaved`). A *full* crawl is
|
|
what makes the deletion sweep sound — it yields the authoritative id set per
|
|
library, and an incremental pass cannot detect deletions. Worth revisiting if
|
|
full crawls prove slow on large libraries; measure first.
|
|
- **Removing the server leg.** The reconciliation query stays.
|
|
|
|
> ⚠️ Two dead search implementations still exist: `storage_search_items`
|
|
> (`commands/storage/mod.rs`) and `offline_search` (`commands/offline.rs`). Both
|
|
> are registered in `lib.rs` and exported to `bindings.ts`; neither is called
|
|
> from the frontend. Deleting them is correct and unclaimed.
|
|
|
|
## Playback Initiation Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant User
|
|
participant AudioPlayer
|
|
participant Tauri as Tauri IPC
|
|
participant Command as player_play_item()
|
|
participant Controller as PlayerController
|
|
participant Backend as PlayerBackend
|
|
participant Store as Frontend Store
|
|
|
|
User->>AudioPlayer: clicks play
|
|
AudioPlayer->>Tauri: invoke("player_play_item", {item})
|
|
Tauri->>Command: player_play_item()
|
|
Command->>Command: Convert PlayItemRequest -> MediaItem
|
|
Command->>Controller: play_item(item)
|
|
Controller->>Backend: load(item)
|
|
Note over Backend: State -> Loading
|
|
Controller->>Backend: play()
|
|
Note over Backend: State -> Playing
|
|
Controller-->>Command: Ok(())
|
|
Command-->>Tauri: PlayerStatus {state, position, duration, volume}
|
|
Tauri-->>AudioPlayer: status
|
|
AudioPlayer->>Store: player.setPlaying(media, position, duration)
|
|
Note over Store: UI updates reactively
|
|
```
|
|
|
|
## Video Stream Selection Flow
|
|
|
|
**TRACES: UR-070, UR-079 | DR-225, DR-227, DR-228**
|
|
|
|
Before a video plays, Rust decides *what stream* — direct play, remux or
|
|
transcode, over which transport — and hands the player one self-describing
|
|
`StreamSelection`. The page no longer inspects the URL to work any of this out.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Page as player/[id]/+page.svelte
|
|
participant Repo as HybridRepository
|
|
participant Online as OnlineRepository
|
|
participant Server as Jellyfin
|
|
participant VP as VideoPlayer.svelte
|
|
|
|
Page->>Repo: playerLocalMediaPath(id)
|
|
alt a completed download exists
|
|
Page->>Repo: mediaLocalSelection(path)
|
|
Note over Page: LocalFile / DirectPlay, no ladder —<br/>nothing about a file on disk re-negotiates
|
|
else stream from the server
|
|
Page->>Repo: getStreamSelection(id, mediaSourceId)
|
|
Repo->>Online: get_stream_selection()
|
|
Online->>Online: effective_streaming_quality()
|
|
Note over Online: per-playback override, else device default
|
|
Online->>Server: POST /Items/{id}/PlaybackInfo<br/>(device profile + ceiling)
|
|
Server-->>Online: MediaSource {supportsDirectPlay,<br/>supportsDirectStream, transcodingUrl, bitrate}
|
|
Online->>Online: decide_playback_kind()
|
|
alt Transcode
|
|
Online->>Online: adopt/stop prior play session,<br/>build HLS URL
|
|
Note over Online: Transport::Hls
|
|
else DirectPlay / DirectStream
|
|
Online->>Online: /Videos/{id}/stream?static=true
|
|
Note over Online: Transport::Progressive,<br/>rendition = None (it IS the source)
|
|
end
|
|
Online->>Online: quality_options_for_source(bitrate)
|
|
Online-->>Page: StreamSelection
|
|
end
|
|
Page->>VP: selection
|
|
VP->>VP: videoLoaderFor(selection, caps)
|
|
Note over VP: hls.js / native HLS / direct —<br/>from the tag, never from the URL
|
|
```
|
|
|
|
The selection travels with the stream from then on. A reload — a quality change,
|
|
an audio-track switch, a transcoded seek — returns a *new* selection through the
|
|
same tagged `strategy` response, so transport and URL can never disagree; and the
|
|
queue item carries the transport so `player_seek_video` picks its seek strategy
|
|
from the backend's decision rather than from the URL string.
|
|
|
|
## Playback Mode Transfer Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant UI as Cast Button
|
|
participant Store as playbackMode store
|
|
participant Rust as Tauri Command
|
|
participant Manager as PlaybackModeManager
|
|
participant Player as PlayerController
|
|
participant Jellyfin as Jellyfin API
|
|
|
|
UI->>Store: transferToRemote(sessionId)
|
|
Store->>Rust: invoke("playback_mode_transfer_to_remote", {sessionId})
|
|
Rust->>Manager: transfer_to_remote()
|
|
|
|
Manager->>Player: Get current queue
|
|
Player-->>Manager: Vec<MediaItem>
|
|
Manager->>Manager: Extract Jellyfin IDs
|
|
|
|
Manager->>Jellyfin: POST /Sessions/{id}/Playing<br/>{itemIds, startIndex}
|
|
Jellyfin-->>Manager: 200 OK
|
|
|
|
Manager->>Jellyfin: POST /Sessions/{id}/Playing/Seek<br/>{positionTicks}
|
|
Jellyfin-->>Manager: 200 OK
|
|
|
|
Manager->>Player: stop()
|
|
Manager->>Manager: mode = Remote {sessionId}
|
|
|
|
Manager-->>Rust: Ok(())
|
|
Rust-->>Store: PlaybackMode
|
|
Store->>UI: Update cast icon
|
|
```
|
|
|
|
## Queue Navigation Flow
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
User["User clicks Next"] --> Invoke["invoke('player_next')"]
|
|
Invoke --> ControllerNext["controller.next()"]
|
|
ControllerNext --> QueueNext["queue.next()<br/>- Check repeat mode<br/>- Check shuffle<br/>- Update history"]
|
|
|
|
QueueNext --> None["None<br/>(at end)"]
|
|
QueueNext --> Some["Some(next)"]
|
|
QueueNext --> Same["Same<br/>(repeat one)"]
|
|
|
|
Some --> PlayItem["play_item(next)<br/>Returns new status"]
|
|
```
|
|
|
|
## Volume Control Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant User
|
|
participant Slider as Volume Slider
|
|
participant Handler as handleVolumeChange()
|
|
participant Tauri as Tauri IPC
|
|
participant Command as player_set_volume
|
|
participant Controller as PlayerController
|
|
participant Backend as MpvBackend/NullBackend
|
|
participant Events as playerEvents.ts
|
|
participant Store as Player Store
|
|
participant UI
|
|
|
|
User->>Slider: adjusts (0-100)
|
|
Slider->>Handler: oninput event
|
|
Handler->>Handler: Convert 0-100 -> 0.0-1.0
|
|
Handler->>Tauri: invoke("player_set_volume", {volume})
|
|
Tauri->>Command: player_set_volume
|
|
Command->>Controller: set_volume(volume)
|
|
Controller->>Backend: set_volume(volume)
|
|
Backend->>Backend: Clamp to 0.0-1.0
|
|
Note over Backend: MpvBackend: Send to MPV loop
|
|
Backend-->>Tauri: emit "player-event"
|
|
Tauri-->>Events: VolumeChanged event
|
|
Events->>Store: player.setVolume(volume)
|
|
Store-->>UI: Reactive update
|
|
Note over UI: Both AudioPlayer and<br/>MiniPlayer stay in sync
|
|
```
|
|
|
|
**Key Implementation Details:**
|
|
- Volume is stored in the backend (NullBackend/MpvBackend)
|
|
- `PlayerController.volume()` delegates to backend
|
|
- `get_player_status()` returns `controller.volume()` (not hardcoded)
|
|
- Frontend uses normalized 0.0-1.0 scale, UI shows 0-100
|