Playing a video meant asking the server to re-encode it, always. That
decision was made nowhere and written down nowhere, so whoever needed it
re-derived it downstream — the player worked out whether it had been handed
a playlist by looking for ".m3u8" in the URL, in two places. A viewer paid
for a transcode of a file their device could have played untouched, and the
app could not tell them which it was.
One negotiation now produces one self-describing StreamSelection — direct
play, remux or transcode; over a playlist, a plain HTTP file, or a local one
— and every renderer consumes that same answer.
Measured against the development server (Jellyfin 10.11.5), 400 items
sampled for codec mix and 40 put through a real PlaybackInfo negotiation
per profile:
Linux / WebKitGTK (h264 only, 2ch) 3/40 — 7% direct play
Android / ExoPlayer (hevc, ac3/eac3, 6ch) 34/40 — 85% direct play
The library is ~80% hevc, which is why the two diverge so hard. The payoff
is overwhelmingly Android, where 85% of plays were starting a transcode
nobody needed. Linux stays near 7% until libmpv decodes the picture — the
h264-only profile is a WebKitGTK constraint, not a JellyTau choice.
DR-219 StreamSelection: url + tagged Transport (hls/progressive/localFile)
+ PlaybackKind (directPlay/directStream/transcode) + the negotiated
rendition + this source's ladder + a needs_transcoding flag derived
in Rust so the rule is answered once. Both enums are serde-tagged
so the frontend matches a discriminant, not a substring. The paths
that never negotiate get the same shape from Rust rather than
assembling one — media_local_selection for a downloaded file,
LiveStreamInfo.transport for a live channel — so there is no second
place where a transport is decided.
DR-220 The ceiling becomes two levels: a durable device default (Settings,
persisted) and a per-playback override the in-player picker sets.
The picker had called itself a "this film, this connection" control
since it was written but wrote the process-wide default, so dropping
one awkward film to 2 Mbps silently capped every video played
afterwards for the rest of the process, with Settings still showing
the old value. The override is cleared whenever playback moves to a
new item, which stops it surviving into an autoplayed next episode.
effective_streaming_quality() is the single resolution point.
DR-221 The quality picker is filled from what this media source can offer.
Rust marks a rung exceeds_source when its ceiling is at or above the
source's own bitrate — such a rung is another way to spell Original
— and the frontend does not draw those. Original is never marked; a
source whose bitrate the server does not report marks nothing, which
keeps every rung offered.
DR-222 Direct play and direct stream are negotiated, with two client-side
overrides on top because the server's answer is right about the file
and wrong about what this app will do with it: undecodable audio
(Jellyfin 10.11.5 honours a DirectPlayProfile's container and video
codec but ignores its audio codec, so it offers direct play for an
E-AC-3 track the webview renders in silence) and a viewer-pinned
audio track the file does not default to. A direct stream is a remux
and is deliberately not counted as transcoding.
DR-223 Dropped on measurement, not deferred. A master playlist from this
server carries exactly one EXT-X-STREAM-INF: Jellyfin builds it from
the single rendition the request asked for rather than publishing a
ladder. So there is no adaptation for hls.js to be preserving and
none mpv would lose — the claim that there was, in
playback-backend-unification.md, does not hold. Recorded rather than
deleted because it is a measurement: a server that does publish a
ladder would change the answer.
DR-224 Every backend consumes the same selection. The queue item carries
the transport, so player_seek_video picks its seek strategy from the
backend's decision instead of the last stream_url.contains(".m3u8")
in the codebase. Items queued by a path that never negotiated carry
None and fall back to needs_transcoding, which is exact rather than
a guess because every transcode this app requests is HLS (DR-140).
The frontend loader decision moves to streamTransport.ts so it can be
tested: the two cases that pin it are the ones that failed against the old
implementation — a progressive stream whose URL contains ".m3u8" must not
get an HLS loader, and an HLS stream whose URL contains none must.
Also verified the URL the direct-play branch builds actually serves playable
bytes: 206, video/mp4, valid ISO-BMFF, and a mid-file range works, so
seeking a direct play works.
The spec is folded into docs/architecture/{01,02,03} and deleted, per the
rule that docs/specs holds only work that has not shipped. DR-121 leaves
read-through-media-cache.md with a pointer; that spec keeps its capture half.
Not verified: real playback on a device. Direct play changes what actually
gets played, and neither fixtures nor curl prove the WebKitGTK and ExoPlayer
paths render it.
268 lines
10 KiB
Markdown
268 lines
10 KiB
Markdown
# Data Flow
|
|
|
|
## Repository Query Flow (Cache-First)
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant UI as Svelte Component
|
|
participant Client as RepositoryClient (TS)
|
|
participant Rust as Tauri Command
|
|
participant Hybrid as HybridRepository
|
|
participant Cache as OfflineRepository (SQLite)
|
|
participant Server as OnlineRepository (HTTP)
|
|
participant Conn as ConnectivityMonitor
|
|
|
|
UI->>Client: getItems(parentId)
|
|
Client->>Rust: invoke("repository_get_items", {handle, parentId})
|
|
Rust->>Hybrid: get_items()
|
|
|
|
par Parallel Racing
|
|
Hybrid->>Cache: get_items() with 100ms timeout
|
|
Hybrid->>Server: get_items() (no timeout)
|
|
end
|
|
|
|
Note over Server,Conn: Every server request reports its outcome
|
|
alt Server succeeds (or answers with 4xx/5xx)
|
|
Server->>Conn: mark_reachable() (server is up)
|
|
else Network failure / timeout
|
|
Server->>Conn: mark_unreachable() (debounced)
|
|
end
|
|
|
|
alt Cache returns with content
|
|
Cache-->>Hybrid: Result with items
|
|
Hybrid-->>Rust: Return cache result
|
|
else Cache timeout or empty
|
|
Server-->>Hybrid: Fresh result
|
|
Hybrid-->>Rust: Return server result
|
|
end
|
|
|
|
Rust-->>Client: SearchResult
|
|
Client-->>UI: items[]
|
|
Note over UI: Reactive update
|
|
```
|
|
|
|
**Key Points:**
|
|
- Cache queries have 100ms timeout for responsiveness
|
|
- Server queries always run for fresh data
|
|
- Cache wins if it has meaningful content
|
|
- Automatic fallback to server if cache is empty/stale
|
|
- Background cache updates (planned)
|
|
- **Connectivity side-effect**: each server request feeds the `ConnectivityMonitor`, which is the source of truth for the offline/online banner (see [07-connectivity.md](07-connectivity.md)). A server-answered error (401/404/5xx) still counts as *reachable* — only network failures, sustained past a debounce window, flip the app to offline.
|
|
|
|
## Search Flow (Locally Indexed)
|
|
|
|
**TRACES**: UR-065 | DR-108 … DR-111, IR-030
|
|
|
|
Search does not depend on a per-keystroke round trip to Jellyfin. The instant leg
|
|
reads the **local SQLite catalog**, which is already synced and already
|
|
FTS5-indexed, so results appear as fast as SQLite can answer — online or offline.
|
|
The server query stays, demoted to a background reconciliation that merges in
|
|
late results.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant UI as Search UI
|
|
participant Rust as repository_search
|
|
participant Cache as Local catalog (FTS5)
|
|
participant Server as Jellyfin
|
|
participant Indexer as spawn_catalog_indexer
|
|
|
|
UI->>Rust: search(query, scope)
|
|
Rust->>Cache: FTS5 query, scope expanded by SearchScope::item_types()
|
|
Cache-->>UI: instant results
|
|
Rust->>Server: reconciliation query (background)
|
|
Server-->>UI: search-event with late/merged results
|
|
Note over Indexer,Cache: Independent of any query:<br/>scheduled crawl keeps the index fresh,<br/>prunes items deleted on the server
|
|
```
|
|
|
|
**Key points:**
|
|
|
|
- The **scope is opaque on the wire**. The frontend sends a `SearchScope`
|
|
variant; Rust expands it to item types
|
|
([01-rust-backend.md](01-rust-backend.md#search-scope-and-the-taxonomy-boundary)).
|
|
- **Index freshness is a Rust policy**, not a frontend startup call — a scheduled
|
|
background pass, not "whatever was synced when the app last launched"
|
|
(DR-109). See
|
|
[Background workers](01-rust-backend.md#background-workers).
|
|
- **Index hygiene matters as much as freshness**: the catalog save path uses
|
|
`INSERT OR REPLACE` and the crawl prunes rows for content deleted on the
|
|
server, or search keeps returning items that no longer exist (DR-110).
|
|
- The index covers **exactly the types the result groups render** (DR-111) —
|
|
including Artists, which the crawl must reach or the Artists group is silently
|
|
always empty.
|
|
|
|
**Deliberately not done, with reasons:**
|
|
|
|
- **Incremental indexing** (Jellyfin's `MinDateLastSaved`). A *full* crawl is
|
|
what makes the deletion sweep sound — it yields the authoritative id set per
|
|
library, and an incremental pass cannot detect deletions. Worth revisiting if
|
|
full crawls prove slow on large libraries; measure first.
|
|
- **Removing the server leg.** The reconciliation query stays.
|
|
|
|
> ⚠️ Two dead search implementations still exist: `storage_search_items`
|
|
> (`commands/storage/mod.rs`) and `offline_search` (`commands/offline.rs`). Both
|
|
> are registered in `lib.rs` and exported to `bindings.ts`; neither is called
|
|
> from the frontend. Deleting them is correct and unclaimed.
|
|
|
|
## Playback Initiation Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant User
|
|
participant AudioPlayer
|
|
participant Tauri as Tauri IPC
|
|
participant Command as player_play_item()
|
|
participant Controller as PlayerController
|
|
participant Backend as PlayerBackend
|
|
participant Store as Frontend Store
|
|
|
|
User->>AudioPlayer: clicks play
|
|
AudioPlayer->>Tauri: invoke("player_play_item", {item})
|
|
Tauri->>Command: player_play_item()
|
|
Command->>Command: Convert PlayItemRequest -> MediaItem
|
|
Command->>Controller: play_item(item)
|
|
Controller->>Backend: load(item)
|
|
Note over Backend: State -> Loading
|
|
Controller->>Backend: play()
|
|
Note over Backend: State -> Playing
|
|
Controller-->>Command: Ok(())
|
|
Command-->>Tauri: PlayerStatus {state, position, duration, volume}
|
|
Tauri-->>AudioPlayer: status
|
|
AudioPlayer->>Store: player.setPlaying(media, position, duration)
|
|
Note over Store: UI updates reactively
|
|
```
|
|
|
|
## Video Stream Selection Flow
|
|
|
|
**TRACES: UR-070, UR-079 | DR-224, DR-226, DR-227**
|
|
|
|
Before a video plays, Rust decides *what stream* — direct play, remux or
|
|
transcode, over which transport — and hands the player one self-describing
|
|
`StreamSelection`. The page no longer inspects the URL to work any of this out.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Page as player/[id]/+page.svelte
|
|
participant Repo as HybridRepository
|
|
participant Online as OnlineRepository
|
|
participant Server as Jellyfin
|
|
participant VP as VideoPlayer.svelte
|
|
|
|
Page->>Repo: playerLocalMediaPath(id)
|
|
alt a completed download exists
|
|
Page->>Repo: mediaLocalSelection(path)
|
|
Note over Page: LocalFile / DirectPlay, no ladder —<br/>nothing about a file on disk re-negotiates
|
|
else stream from the server
|
|
Page->>Repo: getStreamSelection(id, mediaSourceId)
|
|
Repo->>Online: get_stream_selection()
|
|
Online->>Online: effective_streaming_quality()
|
|
Note over Online: per-playback override, else device default
|
|
Online->>Server: POST /Items/{id}/PlaybackInfo<br/>(device profile + ceiling)
|
|
Server-->>Online: MediaSource {supportsDirectPlay,<br/>supportsDirectStream, transcodingUrl, bitrate}
|
|
Online->>Online: decide_playback_kind()
|
|
alt Transcode
|
|
Online->>Online: adopt/stop prior play session,<br/>build HLS URL
|
|
Note over Online: Transport::Hls
|
|
else DirectPlay / DirectStream
|
|
Online->>Online: /Videos/{id}/stream?static=true
|
|
Note over Online: Transport::Progressive,<br/>rendition = None (it IS the source)
|
|
end
|
|
Online->>Online: quality_options_for_source(bitrate)
|
|
Online-->>Page: StreamSelection
|
|
end
|
|
Page->>VP: selection
|
|
VP->>VP: videoLoaderFor(selection, caps)
|
|
Note over VP: hls.js / native HLS / direct —<br/>from the tag, never from the URL
|
|
```
|
|
|
|
The selection travels with the stream from then on. A reload — a quality change,
|
|
an audio-track switch, a transcoded seek — returns a *new* selection through the
|
|
same tagged `strategy` response, so transport and URL can never disagree; and the
|
|
queue item carries the transport so `player_seek_video` picks its seek strategy
|
|
from the backend's decision rather than from the URL string.
|
|
|
|
## Playback Mode Transfer Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant UI as Cast Button
|
|
participant Store as playbackMode store
|
|
participant Rust as Tauri Command
|
|
participant Manager as PlaybackModeManager
|
|
participant Player as PlayerController
|
|
participant Jellyfin as Jellyfin API
|
|
|
|
UI->>Store: transferToRemote(sessionId)
|
|
Store->>Rust: invoke("playback_mode_transfer_to_remote", {sessionId})
|
|
Rust->>Manager: transfer_to_remote()
|
|
|
|
Manager->>Player: Get current queue
|
|
Player-->>Manager: Vec<MediaItem>
|
|
Manager->>Manager: Extract Jellyfin IDs
|
|
|
|
Manager->>Jellyfin: POST /Sessions/{id}/Playing<br/>{itemIds, startIndex}
|
|
Jellyfin-->>Manager: 200 OK
|
|
|
|
Manager->>Jellyfin: POST /Sessions/{id}/Playing/Seek<br/>{positionTicks}
|
|
Jellyfin-->>Manager: 200 OK
|
|
|
|
Manager->>Player: stop()
|
|
Manager->>Manager: mode = Remote {sessionId}
|
|
|
|
Manager-->>Rust: Ok(())
|
|
Rust-->>Store: PlaybackMode
|
|
Store->>UI: Update cast icon
|
|
```
|
|
|
|
## Queue Navigation Flow
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
User["User clicks Next"] --> Invoke["invoke('player_next')"]
|
|
Invoke --> ControllerNext["controller.next()"]
|
|
ControllerNext --> QueueNext["queue.next()<br/>- Check repeat mode<br/>- Check shuffle<br/>- Update history"]
|
|
|
|
QueueNext --> None["None<br/>(at end)"]
|
|
QueueNext --> Some["Some(next)"]
|
|
QueueNext --> Same["Same<br/>(repeat one)"]
|
|
|
|
Some --> PlayItem["play_item(next)<br/>Returns new status"]
|
|
```
|
|
|
|
## Volume Control Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant User
|
|
participant Slider as Volume Slider
|
|
participant Handler as handleVolumeChange()
|
|
participant Tauri as Tauri IPC
|
|
participant Command as player_set_volume
|
|
participant Controller as PlayerController
|
|
participant Backend as MpvBackend/NullBackend
|
|
participant Events as playerEvents.ts
|
|
participant Store as Player Store
|
|
participant UI
|
|
|
|
User->>Slider: adjusts (0-100)
|
|
Slider->>Handler: oninput event
|
|
Handler->>Handler: Convert 0-100 -> 0.0-1.0
|
|
Handler->>Tauri: invoke("player_set_volume", {volume})
|
|
Tauri->>Command: player_set_volume
|
|
Command->>Controller: set_volume(volume)
|
|
Controller->>Backend: set_volume(volume)
|
|
Backend->>Backend: Clamp to 0.0-1.0
|
|
Note over Backend: MpvBackend: Send to MPV loop
|
|
Backend-->>Tauri: emit "player-event"
|
|
Tauri-->>Events: VolumeChanged event
|
|
Events->>Store: player.setVolume(volume)
|
|
Store-->>UI: Reactive update
|
|
Note over UI: Both AudioPlayer and<br/>MiniPlayer stay in sync
|
|
```
|
|
|
|
**Key Implementation Details:**
|
|
- Volume is stored in the backend (NullBackend/MpvBackend)
|
|
- `PlayerController.volume()` delegates to backend
|
|
- `get_player_status()` returns `controller.volume()` (not hardcoded)
|
|
- Frontend uses normalized 0.0-1.0 scale, UI shows 0-100
|