Files
jellytau/docs/architecture/03-data-flow.md
T
dtourolle 1677f5f299 refactor(player): delete the webview video path; mpv selects its own tracks
DR-235 phase 3. Every video renderer is native now: mpv on Linux and
Windows, ExoPlayer on Android, all drawing behind the transparent
webview. The HTML5 <video> path is gone, not bypassed:

- Frontend: hls.js, Html5PlayerAdapter and its compatibility shim, the
  createAdapter factory, streamTransport, hlsRecovery, timeTracking,
  videoFit, the <video>/<track> markup and every element handler in
  VideoPlayer (3277 -> 2144 lines), the experimentalNativeVideo store
  and its Settings toggle, webviewVideoFallback/supportsNativeVideo, and
  the setHtml5VideoState PiP bridge call. NativePlayerAdapter is the one
  video adapter; webview audio gets its own adapter kind.
- Rust: use_html5 dropped from player_seek_video,
  player_switch_audio_track and player_set_stream_quality with the
  Html5* strategies and ReloadStream responses; use_html5_element and
  VideoBackend dropped from PlayerStatus; player_play_item always loads
  the backend (set_current_item removed); Capabilities::webview removed;
  the WebKitGTK GStreamer/VAAPI setup (and its gst-inspect spawn) removed.
- Android: the HTML5 video state in PictureInPictureManager and
  ScreenWakeManager, and the bridge method feeding it.
- CSP: connect-src loses http:/https: and worker-src loses blob: -
  both existed for hls.js; with it gone they were only an exfiltration
  channel and a blob worker for injected script. A test now keeps them
  out.

mpv takes over what the <video> element did (mpv_tracks, UT-275):
subtitles are the WebVTT list the play request carries, queued on
sub-files and selected by position in that list, starting off; audio
tracks are selected by position in the file; sid/aid are reset before
each load. Without this, Linux video had no subtitle selection and a
direct-play audio switch failed since mpv became its renderer.

Verified: Rust 948 passing, and the same 948 cross-compiled for Windows
under wine against the shipped DLL (track tests included); frontend
1111 passing; aarch64 debug APK builds. Lint warnings 158 -> 146, CI
ratchet tightened to match. Not yet seen on Windows hardware.
2026-09-24 23:11:17 -04:00

13 KiB

Data Flow

Repository Query Flow (Cache-First)

sequenceDiagram
    participant UI as Svelte Component
    participant Client as RepositoryClient (TS)
    participant Rust as Tauri Command
    participant Hybrid as HybridRepository
    participant Cache as OfflineRepository (SQLite)
    participant Server as OnlineRepository (HTTP)
    participant Conn as ConnectivityMonitor

    UI->>Client: getItems(parentId)
    Client->>Rust: invoke("repository_get_items", {handle, parentId})
    Rust->>Hybrid: get_items()

    par Parallel Racing
        Hybrid->>Cache: get_items() with 100ms timeout
        Hybrid->>Server: get_items() (no timeout)
    end

    Note over Server,Conn: Every server request reports its outcome
    alt Server succeeds (or answers with 4xx/5xx)
        Server->>Conn: mark_reachable() (server is up)
    else Network failure / timeout
        Server->>Conn: mark_unreachable() (debounced)
    end

    alt Cache answers first with content (inside 100ms, or later but before the server)
        Cache-->>Hybrid: Result with items
        Hybrid-->>Rust: Return cache result
        Server-->>Hybrid: Fresh result (later)
        Hybrid->>Cache: save_to_cache() in background
    else Server answers first, or cache is empty
        Server-->>Hybrid: Fresh result
        Hybrid-->>Rust: Return server result
    else Server fails
        Cache-->>Hybrid: Whatever the cache has (waited for)
        Hybrid-->>Rust: Return cache result, else the server error
    end

    Rust-->>Client: SearchResult
    Client-->>UI: items[]
    Note over UI: Reactive update

Key Points:

  • Both legs start together. A cache answer with content inside 100 ms (CACHE_FAST_PATH) returns at once.
  • The deadline does not decide the race. A cache read still running at 100 ms is raced against the server (HybridRepository::race_slow_cache), and whichever answers first with content wins. It used to be that a read past the deadline was only consulted if the server failed, so a page whose cache read took 150 ms always paid the full server round trip — about a second on a phone, on every visit.
  • An empty or failed cache answer is not a win; the server decides. A failed server falls back to whatever the cache said, waiting for it if necessary.
  • On a cache win the server's page is still cached in the background when it arrives, so per-user state (positions, favourites) keeps up.
  • A get_items leg that takes 250 ms or more is logged at INFO with its row count, so a slow page can be attributed to the cache or the server from a device log alone.
  • Connectivity side-effect: each server request feeds the ConnectivityMonitor, which is the source of truth for the offline/online banner (see 07-connectivity.md). A server-answered error (401/404/5xx) still counts as reachable — only network failures, sustained past a debounce window, flip the app to offline.

Listing order is decided in Rust

TRACES: UR-007 | DR-257

A browse call names the container (GetItemsOptions.parentKind, the neutral MediaKind the caller already holds) and not a sort field. default_listing_sort in repository/types.rs turns that kind into the order:

Container kind Order
channelFolder — one podcast inside a plugin channel PremiereDate descending
any other container SortName ascending
none given no SortBy — the server's own order stands

Both legs of the race apply it, so the cached list does not flash in name order before the server's arrives. An explicit sortBy from the caller always wins; the default only fills the gap.

This is a domain rule, not a display preference, which is why it is not in the frontend: the store that asks for a podcast's episodes has no business knowing that podcasts are read newest-first. MediaKind::ChannelFolder exists for the same reason — Jellyfin gives a channel container and an ordinary folder the same item type (ChannelFolderItem), and while both mapped to Folder there was nothing to key the rule on. The defect this prevents: every Jellypod podcast listed alphabetically, which discarded the release order and clumped every [Played] … episode at the top of the list.

Search Flow (Locally Indexed)

TRACES: UR-065 | DR-108 … DR-111, IR-030

Search does not depend on a per-keystroke round trip to Jellyfin. The instant leg reads the local SQLite catalog, which is already synced and already FTS5-indexed, so results appear as fast as SQLite can answer — online or offline. The server query stays, demoted to a background reconciliation that merges in late results.

sequenceDiagram
    participant UI as Search UI
    participant Rust as repository_search
    participant Cache as Local catalog (FTS5)
    participant Server as Jellyfin
    participant Indexer as spawn_catalog_indexer

    UI->>Rust: search(query, scope)
    Rust->>Cache: FTS5 query, scope expanded by SearchScope::item_types()
    Cache-->>UI: instant results
    Rust->>Server: reconciliation query (background)
    Server-->>UI: search-event with late/merged results
    Note over Indexer,Cache: Independent of any query:<br/>scheduled crawl keeps the index fresh,<br/>prunes items deleted on the server

Key points:

  • The scope is opaque on the wire. The frontend sends a SearchScope variant; Rust expands it to item types (01-rust-backend.md).
  • Index freshness is a Rust policy, not a frontend startup call — a scheduled background pass, not "whatever was synced when the app last launched" (DR-109). See Background workers.
  • Index hygiene matters as much as freshness: the catalog save path uses INSERT OR REPLACE and the crawl prunes rows for content deleted on the server, or search keeps returning items that no longer exist (DR-110).
  • The index covers exactly the types the result groups render (DR-111) — including Artists, which the crawl must reach or the Artists group is silently always empty.

Deliberately not done, with reasons:

  • Incremental indexing (Jellyfin's MinDateLastSaved). A full crawl is what makes the deletion sweep sound — it yields the authoritative id set per library, and an incremental pass cannot detect deletions. Worth revisiting if full crawls prove slow on large libraries; measure first.
  • Removing the server leg. The reconciliation query stays.

⚠️ Two dead search implementations still exist: storage_search_items (commands/storage/mod.rs) and offline_search (commands/offline.rs). Both are registered in lib.rs and exported to bindings.ts; neither is called from the frontend. Deleting them is correct and unclaimed.

Playback Initiation Flow

sequenceDiagram
    participant User
    participant AudioPlayer
    participant Tauri as Tauri IPC
    participant Command as player_play_item()
    participant Controller as PlayerController
    participant Backend as PlayerBackend
    participant Store as Frontend Store

    User->>AudioPlayer: clicks play
    AudioPlayer->>Tauri: invoke("player_play_item", {item})
    Tauri->>Command: player_play_item()
    Command->>Command: Convert PlayItemRequest -> MediaItem
    Command->>Controller: play_item(item)
    Controller->>Backend: load(item)
    Note over Backend: State -> Loading
    Controller->>Backend: play()
    Note over Backend: State -> Playing
    Controller-->>Command: Ok(())
    Command-->>Tauri: PlayerStatus {state, position, duration, volume}
    Tauri-->>AudioPlayer: status
    AudioPlayer->>Store: player.setPlaying(media, position, duration)
    Note over Store: UI updates reactively

Video Stream Selection Flow

TRACES: UR-070, UR-079 | DR-225, DR-227, DR-228

Before a video plays, Rust decides what stream — direct play, remux or transcode, over which transport — and hands the player one self-describing StreamSelection. The page no longer inspects the URL to work any of this out.

sequenceDiagram
    participant Page as player/[id]/+page.svelte
    participant Repo as HybridRepository
    participant Online as OnlineRepository
    participant Server as Jellyfin
    participant VP as VideoPlayer.svelte

    Page->>Repo: playerLocalMediaPath(id)
    alt a completed download exists
        Page->>Repo: mediaLocalSelection(path)
        Note over Page: LocalFile / DirectPlay, no ladder —<br/>nothing about a file on disk re-negotiates
    else stream from the server
        Page->>Repo: getStreamSelection(id, mediaSourceId)
        Repo->>Online: get_stream_selection()
        Online->>Online: effective_streaming_quality()
        Note over Online: per-playback override, else device default
        Online->>Server: POST /Items/{id}/PlaybackInfo<br/>(device profile + ceiling)
        Server-->>Online: MediaSource {supportsDirectPlay,<br/>supportsDirectStream, transcodingUrl, bitrate}
        Online->>Online: decide_playback_kind()
        alt Transcode
            Online->>Online: adopt/stop prior play session,<br/>build HLS URL
            Note over Online: Transport::Hls
        else DirectPlay / DirectStream
            Online->>Online: /Videos/{id}/stream?static=true
            Note over Online: Transport::Progressive,<br/>rendition = None (it IS the source)
        end
        Online->>Online: quality_options_for_source(bitrate)
        Online-->>Page: StreamSelection
    end
    Page->>VP: selection
    VP->>VP: player_play_item(selection.url, transport)
    Note over VP: the native player opens it —<br/>transport from the tag, never from the URL

The selection travels with the stream from then on. A reload — a quality change, an audio-track switch, a transcoded seek — returns a new selection through the same tagged strategy response, so transport and URL can never disagree; and the queue item carries the transport so player_seek_video picks its seek strategy from the backend's decision rather than from the URL string.

Playback Mode Transfer Flow

sequenceDiagram
    participant UI as Cast Button
    participant Store as playbackMode store
    participant Rust as Tauri Command
    participant Manager as PlaybackModeManager
    participant Player as PlayerController
    participant Jellyfin as Jellyfin API

    UI->>Store: transferToRemote(sessionId)
    Store->>Rust: invoke("playback_mode_transfer_to_remote", {sessionId})
    Rust->>Manager: transfer_to_remote()

    Manager->>Player: Get current queue
    Player-->>Manager: Vec<MediaItem>
    Manager->>Manager: Extract Jellyfin IDs

    Manager->>Jellyfin: POST /Sessions/{id}/Playing<br/>{itemIds, startIndex}
    Jellyfin-->>Manager: 200 OK

    Manager->>Jellyfin: POST /Sessions/{id}/Playing/Seek<br/>{positionTicks}
    Jellyfin-->>Manager: 200 OK

    Manager->>Player: stop()
    Manager->>Manager: mode = Remote {sessionId}

    Manager-->>Rust: Ok(())
    Rust-->>Store: PlaybackMode
    Store->>UI: Update cast icon

Queue Navigation Flow

flowchart TB
    User["User clicks Next"] --> Invoke["invoke('player_next')"]
    Invoke --> ControllerNext["controller.next()"]
    ControllerNext --> QueueNext["queue.next()<br/>- Check repeat mode<br/>- Check shuffle<br/>- Update history"]

    QueueNext --> None["None<br/>(at end)"]
    QueueNext --> Some["Some(next)"]
    QueueNext --> Same["Same<br/>(repeat one)"]

    Some --> PlayItem["play_item(next)<br/>Returns new status"]

Volume Control Flow

sequenceDiagram
    participant User
    participant Slider as Volume Slider
    participant Handler as handleVolumeChange()
    participant Tauri as Tauri IPC
    participant Command as player_set_volume
    participant Controller as PlayerController
    participant Backend as MpvBackend/NullBackend
    participant Events as playerEvents.ts
    participant Store as Player Store
    participant UI

    User->>Slider: adjusts (0-100)
    Slider->>Handler: oninput event
    Handler->>Handler: Convert 0-100 -> 0.0-1.0
    Handler->>Tauri: invoke("player_set_volume", {volume})
    Tauri->>Command: player_set_volume
    Command->>Controller: set_volume(volume)
    Controller->>Backend: set_volume(volume)
    Backend->>Backend: Clamp to 0.0-1.0
    Note over Backend: MpvBackend: Send to MPV loop
    Backend-->>Tauri: emit "player-event"
    Tauri-->>Events: VolumeChanged event
    Events->>Store: player.setVolume(volume)
    Store-->>UI: Reactive update
    Note over UI: Both AudioPlayer and<br/>MiniPlayer stay in sync

Key Implementation Details:

  • Volume is stored in the backend (NullBackend/MpvBackend)
  • PlayerController.volume() delegates to backend
  • get_player_status() returns controller.volume() (not hardcoded)
  • Frontend uses normalized 0.0-1.0 scale, UI shows 0-100