Files
jellytau/docs/architecture/03-data-flow.md
dtourolle 83dc8c7028 feat(playback): let Rust decide what stream to play, and say so
Playing a video meant asking the server to re-encode it, always. That
decision was made nowhere and written down nowhere, so whoever needed it
re-derived it downstream — the player worked out whether it had been handed
a playlist by looking for ".m3u8" in the URL, in two places. A viewer paid
for a transcode of a file their device could have played untouched, and the
app could not tell them which it was.

One negotiation now produces one self-describing StreamSelection — direct
play, remux or transcode; over a playlist, a plain HTTP file, or a local one
— and every renderer consumes that same answer.

Measured against the development server (Jellyfin 10.11.5), 400 items
sampled for codec mix and 40 put through a real PlaybackInfo negotiation
per profile:

  Linux / WebKitGTK (h264 only, 2ch)          3/40 —  7% direct play
  Android / ExoPlayer (hevc, ac3/eac3, 6ch)  34/40 — 85% direct play

The library is ~80% hevc, which is why the two diverge so hard. The payoff
is overwhelmingly Android, where 85% of plays were starting a transcode
nobody needed. Linux stays near 7% until libmpv decodes the picture — the
h264-only profile is a WebKitGTK constraint, not a JellyTau choice.

DR-219  StreamSelection: url + tagged Transport (hls/progressive/localFile)
        + PlaybackKind (directPlay/directStream/transcode) + the negotiated
        rendition + this source's ladder + a needs_transcoding flag derived
        in Rust so the rule is answered once. Both enums are serde-tagged
        so the frontend matches a discriminant, not a substring. The paths
        that never negotiate get the same shape from Rust rather than
        assembling one — media_local_selection for a downloaded file,
        LiveStreamInfo.transport for a live channel — so there is no second
        place where a transport is decided.

DR-220  The ceiling becomes two levels: a durable device default (Settings,
        persisted) and a per-playback override the in-player picker sets.
        The picker had called itself a "this film, this connection" control
        since it was written but wrote the process-wide default, so dropping
        one awkward film to 2 Mbps silently capped every video played
        afterwards for the rest of the process, with Settings still showing
        the old value. The override is cleared whenever playback moves to a
        new item, which stops it surviving into an autoplayed next episode.
        effective_streaming_quality() is the single resolution point.

DR-221  The quality picker is filled from what this media source can offer.
        Rust marks a rung exceeds_source when its ceiling is at or above the
        source's own bitrate — such a rung is another way to spell Original
        — and the frontend does not draw those. Original is never marked; a
        source whose bitrate the server does not report marks nothing, which
        keeps every rung offered.

DR-222  Direct play and direct stream are negotiated, with two client-side
        overrides on top because the server's answer is right about the file
        and wrong about what this app will do with it: undecodable audio
        (Jellyfin 10.11.5 honours a DirectPlayProfile's container and video
        codec but ignores its audio codec, so it offers direct play for an
        E-AC-3 track the webview renders in silence) and a viewer-pinned
        audio track the file does not default to. A direct stream is a remux
        and is deliberately not counted as transcoding.

DR-223  Dropped on measurement, not deferred. A master playlist from this
        server carries exactly one EXT-X-STREAM-INF: Jellyfin builds it from
        the single rendition the request asked for rather than publishing a
        ladder. So there is no adaptation for hls.js to be preserving and
        none mpv would lose — the claim that there was, in
        playback-backend-unification.md, does not hold. Recorded rather than
        deleted because it is a measurement: a server that does publish a
        ladder would change the answer.

DR-224  Every backend consumes the same selection. The queue item carries
        the transport, so player_seek_video picks its seek strategy from the
        backend's decision instead of the last stream_url.contains(".m3u8")
        in the codebase. Items queued by a path that never negotiated carry
        None and fall back to needs_transcoding, which is exact rather than
        a guess because every transcode this app requests is HLS (DR-140).

The frontend loader decision moves to streamTransport.ts so it can be
tested: the two cases that pin it are the ones that failed against the old
implementation — a progressive stream whose URL contains ".m3u8" must not
get an HLS loader, and an HLS stream whose URL contains none must.

Also verified the URL the direct-play branch builds actually serves playable
bytes: 206, video/mp4, valid ISO-BMFF, and a mid-file range works, so
seeking a direct play works.

The spec is folded into docs/architecture/{01,02,03} and deleted, per the
rule that docs/specs holds only work that has not shipped. DR-121 leaves
read-through-media-cache.md with a pointer; that spec keeps its capture half.

Not verified: real playback on a device. Direct play changes what actually
gets played, and neither fixtures nor curl prove the WebKitGTK and ExoPlayer
paths render it.
2026-08-21 22:44:13 +02:00

268 lines
10 KiB
Markdown

# Data Flow
## Repository Query Flow (Cache-First)
```mermaid
sequenceDiagram
participant UI as Svelte Component
participant Client as RepositoryClient (TS)
participant Rust as Tauri Command
participant Hybrid as HybridRepository
participant Cache as OfflineRepository (SQLite)
participant Server as OnlineRepository (HTTP)
participant Conn as ConnectivityMonitor
UI->>Client: getItems(parentId)
Client->>Rust: invoke("repository_get_items", {handle, parentId})
Rust->>Hybrid: get_items()
par Parallel Racing
Hybrid->>Cache: get_items() with 100ms timeout
Hybrid->>Server: get_items() (no timeout)
end
Note over Server,Conn: Every server request reports its outcome
alt Server succeeds (or answers with 4xx/5xx)
Server->>Conn: mark_reachable() (server is up)
else Network failure / timeout
Server->>Conn: mark_unreachable() (debounced)
end
alt Cache returns with content
Cache-->>Hybrid: Result with items
Hybrid-->>Rust: Return cache result
else Cache timeout or empty
Server-->>Hybrid: Fresh result
Hybrid-->>Rust: Return server result
end
Rust-->>Client: SearchResult
Client-->>UI: items[]
Note over UI: Reactive update
```
**Key Points:**
- Cache queries have 100ms timeout for responsiveness
- Server queries always run for fresh data
- Cache wins if it has meaningful content
- Automatic fallback to server if cache is empty/stale
- Background cache updates (planned)
- **Connectivity side-effect**: each server request feeds the `ConnectivityMonitor`, which is the source of truth for the offline/online banner (see [07-connectivity.md](07-connectivity.md)). A server-answered error (401/404/5xx) still counts as *reachable* — only network failures, sustained past a debounce window, flip the app to offline.
## Search Flow (Locally Indexed)
**TRACES**: UR-065 | DR-108 … DR-111, IR-030
Search does not depend on a per-keystroke round trip to Jellyfin. The instant leg
reads the **local SQLite catalog**, which is already synced and already
FTS5-indexed, so results appear as fast as SQLite can answer — online or offline.
The server query stays, demoted to a background reconciliation that merges in
late results.
```mermaid
sequenceDiagram
participant UI as Search UI
participant Rust as repository_search
participant Cache as Local catalog (FTS5)
participant Server as Jellyfin
participant Indexer as spawn_catalog_indexer
UI->>Rust: search(query, scope)
Rust->>Cache: FTS5 query, scope expanded by SearchScope::item_types()
Cache-->>UI: instant results
Rust->>Server: reconciliation query (background)
Server-->>UI: search-event with late/merged results
Note over Indexer,Cache: Independent of any query:<br/>scheduled crawl keeps the index fresh,<br/>prunes items deleted on the server
```
**Key points:**
- The **scope is opaque on the wire**. The frontend sends a `SearchScope`
variant; Rust expands it to item types
([01-rust-backend.md](01-rust-backend.md#search-scope-and-the-taxonomy-boundary)).
- **Index freshness is a Rust policy**, not a frontend startup call — a scheduled
background pass, not "whatever was synced when the app last launched"
(DR-109). See
[Background workers](01-rust-backend.md#background-workers).
- **Index hygiene matters as much as freshness**: the catalog save path uses
`INSERT OR REPLACE` and the crawl prunes rows for content deleted on the
server, or search keeps returning items that no longer exist (DR-110).
- The index covers **exactly the types the result groups render** (DR-111) —
including Artists, which the crawl must reach or the Artists group is silently
always empty.
**Deliberately not done, with reasons:**
- **Incremental indexing** (Jellyfin's `MinDateLastSaved`). A *full* crawl is
what makes the deletion sweep sound — it yields the authoritative id set per
library, and an incremental pass cannot detect deletions. Worth revisiting if
full crawls prove slow on large libraries; measure first.
- **Removing the server leg.** The reconciliation query stays.
> ⚠️ Two dead search implementations still exist: `storage_search_items`
> (`commands/storage/mod.rs`) and `offline_search` (`commands/offline.rs`). Both
> are registered in `lib.rs` and exported to `bindings.ts`; neither is called
> from the frontend. Deleting them is correct and unclaimed.
## Playback Initiation Flow
```mermaid
sequenceDiagram
participant User
participant AudioPlayer
participant Tauri as Tauri IPC
participant Command as player_play_item()
participant Controller as PlayerController
participant Backend as PlayerBackend
participant Store as Frontend Store
User->>AudioPlayer: clicks play
AudioPlayer->>Tauri: invoke("player_play_item", {item})
Tauri->>Command: player_play_item()
Command->>Command: Convert PlayItemRequest -> MediaItem
Command->>Controller: play_item(item)
Controller->>Backend: load(item)
Note over Backend: State -> Loading
Controller->>Backend: play()
Note over Backend: State -> Playing
Controller-->>Command: Ok(())
Command-->>Tauri: PlayerStatus {state, position, duration, volume}
Tauri-->>AudioPlayer: status
AudioPlayer->>Store: player.setPlaying(media, position, duration)
Note over Store: UI updates reactively
```
## Video Stream Selection Flow
**TRACES: UR-070, UR-079 | DR-224, DR-226, DR-227**
Before a video plays, Rust decides *what stream* — direct play, remux or
transcode, over which transport — and hands the player one self-describing
`StreamSelection`. The page no longer inspects the URL to work any of this out.
```mermaid
sequenceDiagram
participant Page as player/[id]/+page.svelte
participant Repo as HybridRepository
participant Online as OnlineRepository
participant Server as Jellyfin
participant VP as VideoPlayer.svelte
Page->>Repo: playerLocalMediaPath(id)
alt a completed download exists
Page->>Repo: mediaLocalSelection(path)
Note over Page: LocalFile / DirectPlay, no ladder —<br/>nothing about a file on disk re-negotiates
else stream from the server
Page->>Repo: getStreamSelection(id, mediaSourceId)
Repo->>Online: get_stream_selection()
Online->>Online: effective_streaming_quality()
Note over Online: per-playback override, else device default
Online->>Server: POST /Items/{id}/PlaybackInfo<br/>(device profile + ceiling)
Server-->>Online: MediaSource {supportsDirectPlay,<br/>supportsDirectStream, transcodingUrl, bitrate}
Online->>Online: decide_playback_kind()
alt Transcode
Online->>Online: adopt/stop prior play session,<br/>build HLS URL
Note over Online: Transport::Hls
else DirectPlay / DirectStream
Online->>Online: /Videos/{id}/stream?static=true
Note over Online: Transport::Progressive,<br/>rendition = None (it IS the source)
end
Online->>Online: quality_options_for_source(bitrate)
Online-->>Page: StreamSelection
end
Page->>VP: selection
VP->>VP: videoLoaderFor(selection, caps)
Note over VP: hls.js / native HLS / direct —<br/>from the tag, never from the URL
```
The selection travels with the stream from then on. A reload — a quality change,
an audio-track switch, a transcoded seek — returns a *new* selection through the
same tagged `strategy` response, so transport and URL can never disagree; and the
queue item carries the transport so `player_seek_video` picks its seek strategy
from the backend's decision rather than from the URL string.
## Playback Mode Transfer Flow
```mermaid
sequenceDiagram
participant UI as Cast Button
participant Store as playbackMode store
participant Rust as Tauri Command
participant Manager as PlaybackModeManager
participant Player as PlayerController
participant Jellyfin as Jellyfin API
UI->>Store: transferToRemote(sessionId)
Store->>Rust: invoke("playback_mode_transfer_to_remote", {sessionId})
Rust->>Manager: transfer_to_remote()
Manager->>Player: Get current queue
Player-->>Manager: Vec<MediaItem>
Manager->>Manager: Extract Jellyfin IDs
Manager->>Jellyfin: POST /Sessions/{id}/Playing<br/>{itemIds, startIndex}
Jellyfin-->>Manager: 200 OK
Manager->>Jellyfin: POST /Sessions/{id}/Playing/Seek<br/>{positionTicks}
Jellyfin-->>Manager: 200 OK
Manager->>Player: stop()
Manager->>Manager: mode = Remote {sessionId}
Manager-->>Rust: Ok(())
Rust-->>Store: PlaybackMode
Store->>UI: Update cast icon
```
## Queue Navigation Flow
```mermaid
flowchart TB
User["User clicks Next"] --> Invoke["invoke('player_next')"]
Invoke --> ControllerNext["controller.next()"]
ControllerNext --> QueueNext["queue.next()<br/>- Check repeat mode<br/>- Check shuffle<br/>- Update history"]
QueueNext --> None["None<br/>(at end)"]
QueueNext --> Some["Some(next)"]
QueueNext --> Same["Same<br/>(repeat one)"]
Some --> PlayItem["play_item(next)<br/>Returns new status"]
```
## Volume Control Flow
```mermaid
sequenceDiagram
participant User
participant Slider as Volume Slider
participant Handler as handleVolumeChange()
participant Tauri as Tauri IPC
participant Command as player_set_volume
participant Controller as PlayerController
participant Backend as MpvBackend/NullBackend
participant Events as playerEvents.ts
participant Store as Player Store
participant UI
User->>Slider: adjusts (0-100)
Slider->>Handler: oninput event
Handler->>Handler: Convert 0-100 -> 0.0-1.0
Handler->>Tauri: invoke("player_set_volume", {volume})
Tauri->>Command: player_set_volume
Command->>Controller: set_volume(volume)
Controller->>Backend: set_volume(volume)
Backend->>Backend: Clamp to 0.0-1.0
Note over Backend: MpvBackend: Send to MPV loop
Backend-->>Tauri: emit "player-event"
Tauri-->>Events: VolumeChanged event
Events->>Store: player.setVolume(volume)
Store-->>UI: Reactive update
Note over UI: Both AudioPlayer and<br/>MiniPlayer stay in sync
```
**Key Implementation Details:**
- Volume is stored in the backend (NullBackend/MpvBackend)
- `PlayerController.volume()` delegates to backend
- `get_player_status()` returns `controller.volume()` (not hardcoded)
- Frontend uses normalized 0.0-1.0 scale, UI shows 0-100