jellytau/docs/architecture/08-database-design.md

Offline Database Design

Entity Relationship Diagram

erDiagram
    servers ||--o{ users : "has"
    servers ||--o{ libraries : "has"
    libraries ||--o{ items : "contains"
    items ||--o{ items : "parent_of"
    items ||--o{ user_data : "has"
    items ||--o{ downloads : "has"
    items ||--o{ media_streams : "has"
    items ||--o{ thumbnails : "has"
    users ||--o{ user_data : "owns"
    users ||--o{ downloads : "owns"
    users ||--o{ sync_queue : "owns"

    servers {
        int id PK
        string jellyfin_id UK
        string name
        string url
        string version
        datetime last_sync
    }

    users {
        int id PK
        string jellyfin_id
        int server_id FK
        string name
        boolean is_active
    }

    libraries {
        int id PK
        string jellyfin_id
        int server_id FK
        string name
        string collection_type
        string image_tag
    }

    items {
        int id PK
        string jellyfin_id
        int server_id FK
        int library_id FK
        int parent_id FK
        string type
        string name
        string sort_name
        string overview
        int production_year
        float community_rating
        string official_rating
        int runtime_ticks
        string primary_image_tag
        string backdrop_image_tag
        string album_id
        string album_name
        string album_artist
        json artists
        json genres
        int index_number
        int parent_index_number
        string premiere_date
        json metadata_json
        datetime created_at
        datetime updated_at
        datetime last_sync
    }

    user_data {
        int id PK
        int item_id FK
        int user_id FK
        int position_ticks
        int play_count
        boolean is_favorite
        boolean played
        datetime last_played
        datetime updated_at
        datetime synced_at
    }

    downloads {
        int id PK
        int item_id FK
        int user_id FK
        string file_path
        int file_size
        string status
        float progress
        int priority
        string error_message
        datetime created_at
        datetime completed_at
    }

    media_streams {
        int id PK
        int item_id FK
        int stream_index
        string type
        string codec
        string language
        string display_title
        boolean is_default
        boolean is_forced
        boolean is_external
    }

    sync_queue {
        int id PK
        int user_id FK
        string operation
        string entity_type
        string entity_id
        json payload
        datetime created_at
        int attempts
        datetime last_attempt
        string status
    }

    thumbnails {
        int id PK
        int item_id FK
        string image_type
        string image_tag
        string file_path
        int width
        int height
        datetime cached_at
    }

Table Definitions

servers

Stores connected Jellyfin server information.

CREATE TABLE servers (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    jellyfin_id     TEXT NOT NULL UNIQUE,
    name            TEXT NOT NULL,
    url             TEXT NOT NULL,
    version         TEXT,
    last_sync       DATETIME,
    created_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at      DATETIME DEFAULT CURRENT_TIMESTAMP
);

users

Stores user accounts per server. Access tokens are stored separately in secure storage (see 09-security.md).

CREATE TABLE users (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    jellyfin_id     TEXT NOT NULL,
    server_id       INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    name            TEXT NOT NULL,
    is_active       BOOLEAN DEFAULT 0,
    created_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    UNIQUE(jellyfin_id, server_id)
);

libraries

Stores library/collection metadata.

CREATE TABLE libraries (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    jellyfin_id     TEXT NOT NULL,
    server_id       INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    name            TEXT NOT NULL,
    collection_type TEXT,
    image_tag       TEXT,
    sort_order      INTEGER DEFAULT 0,
    last_sync       DATETIME,
    created_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    UNIQUE(jellyfin_id, server_id)
);

CREATE INDEX idx_libraries_server ON libraries(server_id);

items

Main table for all media items (movies, episodes, albums, songs, etc.).

CREATE TABLE items (
    id                   INTEGER PRIMARY KEY AUTOINCREMENT,
    jellyfin_id          TEXT NOT NULL,
    server_id            INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    library_id           INTEGER REFERENCES libraries(id) ON DELETE SET NULL,
    parent_id            INTEGER REFERENCES items(id) ON DELETE CASCADE,

    -- Basic metadata
    type                 TEXT NOT NULL,
    name                 TEXT NOT NULL,
    sort_name            TEXT,
    overview             TEXT,

    -- Media info
    production_year      INTEGER,
    community_rating     REAL,
    official_rating      TEXT,
    runtime_ticks        INTEGER,

    -- Images
    primary_image_tag    TEXT,
    backdrop_image_tag   TEXT,

    -- Audio-specific
    album_id             TEXT,
    album_name           TEXT,
    album_artist         TEXT,
    artists              TEXT,                   -- JSON array

    -- Series/Season-specific
    index_number         INTEGER,
    parent_index_number  INTEGER,
    series_id            TEXT,
    series_name          TEXT,
    season_id            TEXT,

    -- Additional
    genres               TEXT,                   -- JSON array
    premiere_date        TEXT,
    metadata_json        TEXT,

    -- Sync tracking
    created_at           DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at           DATETIME DEFAULT CURRENT_TIMESTAMP,
    last_sync            DATETIME,

    UNIQUE(jellyfin_id, server_id)
);

-- Performance indexes
CREATE INDEX idx_items_server ON items(server_id);
CREATE INDEX idx_items_library ON items(library_id);
CREATE INDEX idx_items_parent ON items(parent_id);
CREATE INDEX idx_items_type ON items(type);
CREATE INDEX idx_items_album ON items(album_id);
CREATE INDEX idx_items_series ON items(series_id);
CREATE INDEX idx_items_name ON items(name COLLATE NOCASE);

-- Full-text search
CREATE VIRTUAL TABLE items_fts USING fts5(
    name,
    overview,
    artists,
    album_name,
    album_artist,
    content='items',
    content_rowid='id'
);

-- Triggers to keep FTS in sync
CREATE TRIGGER items_ai AFTER INSERT ON items BEGIN
    INSERT INTO items_fts(rowid, name, overview, artists, album_name, album_artist)
    VALUES (new.id, new.name, new.overview, new.artists, new.album_name, new.album_artist);
END;

CREATE TRIGGER items_ad AFTER DELETE ON items BEGIN
    INSERT INTO items_fts(items_fts, rowid, name, overview, artists, album_name, album_artist)
    VALUES ('delete', old.id, old.name, old.overview, old.artists, old.album_name, old.album_artist);
END;

CREATE TRIGGER items_au AFTER UPDATE ON items BEGIN
    INSERT INTO items_fts(items_fts, rowid, name, overview, artists, album_name, album_artist)
    VALUES ('delete', old.id, old.name, old.overview, old.artists, old.album_name, old.album_artist);
    INSERT INTO items_fts(rowid, name, overview, artists, album_name, album_artist)
    VALUES (new.id, new.name, new.overview, new.artists, new.album_name, new.album_artist);
END;

media_streams

Stores subtitle and audio track information for items.

CREATE TABLE media_streams (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    item_id         INTEGER NOT NULL REFERENCES items(id) ON DELETE CASCADE,
    stream_index    INTEGER NOT NULL,
    type            TEXT NOT NULL,
    codec           TEXT,
    language        TEXT,
    display_title   TEXT,
    is_default      BOOLEAN DEFAULT 0,
    is_forced       BOOLEAN DEFAULT 0,
    is_external     BOOLEAN DEFAULT 0,
    path            TEXT,
    UNIQUE(item_id, stream_index)
);

CREATE INDEX idx_media_streams_item ON media_streams(item_id);

user_data

Stores per-user data for items (favorites, progress, play count).

CREATE TABLE user_data (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    item_id         INTEGER NOT NULL REFERENCES items(id) ON DELETE CASCADE,
    user_id         INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,

    -- Playback state
    position_ticks  INTEGER DEFAULT 0,
    play_count      INTEGER DEFAULT 0,
    played          BOOLEAN DEFAULT 0,
    last_played     DATETIME,

    -- User preferences
    is_favorite     BOOLEAN DEFAULT 0,
    user_rating     REAL,

    -- Sync tracking
    updated_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    synced_at       DATETIME,
    needs_sync      BOOLEAN DEFAULT 0,

    UNIQUE(item_id, user_id)
);

CREATE INDEX idx_user_data_item ON user_data(item_id);
CREATE INDEX idx_user_data_user ON user_data(user_id);
CREATE INDEX idx_user_data_needs_sync ON user_data(needs_sync) WHERE needs_sync = 1;
CREATE INDEX idx_user_data_favorites ON user_data(user_id, is_favorite) WHERE is_favorite = 1;
CREATE INDEX idx_user_data_in_progress ON user_data(user_id, position_ticks)
    WHERE position_ticks > 0 AND played = 0;

downloads

Tracks downloaded media files.

CREATE TABLE downloads (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    item_id         INTEGER NOT NULL REFERENCES items(id) ON DELETE CASCADE,
    user_id         INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,

    file_path       TEXT,
    file_size       INTEGER,
    file_hash       TEXT,

    status          TEXT NOT NULL DEFAULT 'pending',
    progress        REAL DEFAULT 0,
    bytes_downloaded INTEGER DEFAULT 0,

    transcode_profile TEXT,

    priority        INTEGER DEFAULT 0,
    error_message   TEXT,
    retry_count     INTEGER DEFAULT 0,

    created_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    started_at      DATETIME,
    completed_at    DATETIME,
    expires_at      DATETIME,

    UNIQUE(item_id, user_id)
);

CREATE INDEX idx_downloads_status ON downloads(status);
CREATE INDEX idx_downloads_user ON downloads(user_id);
CREATE INDEX idx_downloads_queue ON downloads(status, priority DESC, created_at ASC)
    WHERE status IN ('pending', 'downloading');

sync_queue

Stores mutations to sync back to server when online.

CREATE TABLE sync_queue (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    user_id         INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,

    operation       TEXT NOT NULL,
    entity_type     TEXT NOT NULL,
    entity_id       TEXT NOT NULL,
    payload         TEXT,

    status          TEXT DEFAULT 'pending',
    attempts        INTEGER DEFAULT 0,
    max_attempts    INTEGER DEFAULT 5,
    last_attempt    DATETIME,
    error_message   TEXT,

    created_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    completed_at    DATETIME
);

CREATE INDEX idx_sync_queue_status ON sync_queue(status, created_at ASC)
    WHERE status = 'pending';
CREATE INDEX idx_sync_queue_user ON sync_queue(user_id);

thumbnails

Caches downloaded artwork.

CREATE TABLE thumbnails (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    item_id         INTEGER NOT NULL REFERENCES items(id) ON DELETE CASCADE,
    image_type      TEXT NOT NULL,
    image_tag       TEXT,
    file_path       TEXT NOT NULL,
    width           INTEGER,
    height          INTEGER,
    file_size       INTEGER,
    cached_at       DATETIME DEFAULT CURRENT_TIMESTAMP,
    last_accessed   DATETIME DEFAULT CURRENT_TIMESTAMP,
    UNIQUE(item_id, image_type, width)
);

CREATE INDEX idx_thumbnails_item ON thumbnails(item_id);
CREATE INDEX idx_thumbnails_lru ON thumbnails(last_accessed ASC);

playlists (for local/synced playlists)

CREATE TABLE playlists (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    jellyfin_id     TEXT,
    user_id         INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
    name            TEXT NOT NULL,
    description     TEXT,
    is_local_only   BOOLEAN DEFAULT 0,
    created_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    synced_at       DATETIME,
    needs_sync      BOOLEAN DEFAULT 0
);

CREATE TABLE playlist_items (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    playlist_id     INTEGER NOT NULL REFERENCES playlists(id) ON DELETE CASCADE,
    item_id         INTEGER NOT NULL REFERENCES items(id) ON DELETE CASCADE,
    sort_order      INTEGER NOT NULL,
    added_at        DATETIME DEFAULT CURRENT_TIMESTAMP,
    UNIQUE(playlist_id, item_id)
);

CREATE INDEX idx_playlist_items_playlist ON playlist_items(playlist_id, sort_order);

Key Queries

Get items for offline library browsing

-- Get all albums in a music library
SELECT * FROM items
WHERE library_id = ? AND type = 'MusicAlbum'
ORDER BY sort_name;

-- Get tracks for an album
SELECT * FROM items
WHERE album_id = ? AND type = 'Audio'
ORDER BY parent_index_number, index_number;

Resume / Continue Watching

SELECT i.*, ud.position_ticks, ud.last_played
FROM items i
JOIN user_data ud ON ud.item_id = i.id
WHERE ud.user_id = ?
  AND ud.position_ticks > 0
  AND ud.played = 0
ORDER BY ud.last_played DESC
LIMIT 20;

Offline search

SELECT i.* FROM items i
JOIN items_fts fts ON fts.rowid = i.id
WHERE items_fts MATCH ?
ORDER BY rank;

Download queue management

-- Get next item to download
SELECT d.*, i.name, i.type
FROM downloads d
JOIN items i ON i.id = d.item_id
WHERE d.status = 'pending'
ORDER BY d.priority DESC, d.created_at ASC
LIMIT 1;

-- Get download progress for UI
SELECT
    d.status,
    COUNT(*) as count,
    SUM(d.file_size) as total_size,
    SUM(d.bytes_downloaded) as downloaded
FROM downloads d
WHERE d.user_id = ?
GROUP BY d.status;

Sync queue processing

-- Get pending sync operations (oldest first)
SELECT * FROM sync_queue
WHERE status = 'pending'
  AND attempts < max_attempts
ORDER BY created_at ASC
LIMIT 10;

-- Mark operation complete
UPDATE sync_queue
SET status = 'completed', completed_at = CURRENT_TIMESTAMP
WHERE id = ?;

Data Flow

Online Mode

flowchart TB
    subgraph OnlineMode["Online Mode"]
        JellyfinServer["Jellyfin Server"]
        OnlineRepo["OnlineRepo"]
        SQLite["SQLite"]
        HybridRepo["HybridRepository"]
        UI["UI / Stores"]

        JellyfinServer -->|"API Response"| OnlineRepo
        OnlineRepo -->|"Cache"| SQLite
        SQLite -->|"Sync"| JellyfinServer
        OnlineRepo -->|"Response"| HybridRepo
        SQLite -->|"Fallback"| HybridRepo
        HybridRepo --> UI
    end

Offline Mode

flowchart TB
    subgraph OfflineMode["Offline Mode"]
        OfflineRepo["OfflineRepo"]
        SQLite2["SQLite"]
        SyncQueue["sync_queue<br/>(Queued for later)"]
        HybridRepo2["HybridRepository"]
        UI2["UI / Stores"]

        OfflineRepo <-->|"Query"| SQLite2
        SQLite2 -->|"Mutations"| SyncQueue
        OfflineRepo --> HybridRepo2
        HybridRepo2 --> UI2
    end

Sync on Reconnect

flowchart LR
    NetworkRestored["Network restored"]
    SyncService["SyncService"]
    SyncQueue2["sync_queue"]
    JellyfinAPI["Jellyfin API"]
    MarkSynced["Mark synced"]

    NetworkRestored --> SyncService
    SyncService -->|"Read"| SyncQueue2
    SyncQueue2 -->|"Send"| JellyfinAPI
    JellyfinAPI -->|"Success"| MarkSynced
    MarkSynced --> SyncService

Storage Estimates

Content Type	Metadata Size	Thumbnail Size	Media Size
Song	~2 KB	~50 KB (300px)	5-15 MB
Album (12 tracks)	~30 KB	~100 KB	60-180 MB
Movie	~5 KB	~200 KB	1-8 GB
Episode	~3 KB	~100 KB	300 MB - 2 GB
Full music library (5000 songs)	~10 MB	~250 MB	25-75 GB

Connection ownership

storage::Database is opened once at startup and owns the database for the life of the app; nothing else opens the file. Everything goes through Database::service(), which returns a clone of one RusqliteService (storage/db_service.rs) — callers never hold a Connection.

                    ┌─────────────────────────────┐
 execute / insert / │ writer thread ("db-writer") │  one read-write connection,
 transaction / ────►│ jobs run in arrival order   │  foreign_keys = ON
 execute_detached   └─────────────────────────────┘
                    ┌─────────────────────────────┐
 query_one /        │ reader pool (3 connections) │  query_only = ON; WAL gives
 query_optional / ─►│ via spawn_blocking          │  each the last committed
 query_many         └─────────────────────────────┘  snapshot

Pragmas: journal_mode = WAL, synchronous = NORMAL, busy_timeout = 5 s on every connection.

Why. It used to be one connection behind one std::sync::Mutex. WAL was on, but with a single connection its one benefit — readers running beside a writer — never applied: every read queued behind every write. A series page's background refresh cached hundreds of episodes one autocommit (and one fsync) at a time, and library pages, thumbnail lookups and settings reads all waited behind the pile; the same page measured 1.7 s or 3.95 s depending on what was queued.

Invariants a change must keep:

• Writes go to the writer, reads to the pool. A reader is query_only, so a write sent through query_* fails loudly rather than racing the writer. A read that must see a write in the same unit of work belongs inside the transaction closure, which runs on the writer.
• Connection-wide state is set inside one writer job. PRAGMA foreign_keys is per connection and ignored inside a transaction. save_to_cache used to switch it off with one execute and back on with another, so it stayed off for every other write that ran across the save's awaits (they did — nearly all of 1,600 FK-violating writes got through in the regression test). transaction_without_foreign_keys flips it around BEGIN/COMMIT in a single job. Any future pragma toggle must work the same way.
• An insert's rowid comes from the same job. insert() returns it; there is deliberately no standalone last_insert_rowid(), which returned whichever row the last anyone inserted.
• Batch writes into one transaction. A commit is a queue slot on the writer; a page of items is one transaction, not one execute per row.
• Best-effort bookkeeping does not wait. execute_detached queues a write (the thumbnail LRU access time) in order with the rest and returns immediately; failures are only logged.
• The writer survives a panicking job. The job's caller gets an error; the loop rolls back any transaction the job left open and restores foreign_keys = ON, then carries on.

synchronous = NORMAL is corruption-safe in WAL mode and survives an app crash; only a power cut can lose the last few commits. Everything stored is either re-fetchable from the server or (the sync queue, local positions) recoverable at that granularity.

In-memory databases (tests) cannot be shared between connections, so RusqliteService::new has no pool and routes reads through the writer — the old serialized behaviour.

Not done, and why: grouping consecutive small execute jobs into one commit on the writer. With synchronous = NORMAL a commit no longer fsyncs, so the gain is small, and it would change a failed statement's error semantics for the jobs grouped with it.

Listing query shape

OfflineRepository::get_items (items_listing_sql) is the hot read: every library, series, season and album page goes through it, and it must fit the 100 ms cache fast path on a phone.

• Availability is checked per row, with EXISTS (cached for browsing, downloaded, or a container with a downloaded child). It used to be a CTE that built the id of every available item in the database before filtering to the parent: ~80 ms on a desktop for a 100k-item cache, whatever the parent.
• A parent that is not a library is matched on the hierarchy columns alone (parent_id/album_id/season_id/series_id), which SQLite answers with a multi-index OR. Whether the parent is a library is looked up first; the library clause can only match for a library, and inside the same OR it forced a full scan.
• +i.server_id: the app never runs ANALYZE, and without statistics the planner prefers the server_id index — which every row shares — over the hierarchy indexes. The unary + takes it out of consideration.
• User data is fetched in batches (with_user_data, one IN (…) query per 500 rows), not once per row.

Result: 80 ms → 1.5 ms on the desktop benchmark; listing_a_non_library_parent_uses_the_hierarchy_indexes asserts the plan. storage::tests::write_bench_database (ignored) writes a phone-sized catalogue for timing queries with the sqlite3 CLI.

Rust Module Structure

src-tauri/src/storage/
├── mod.rs              # Module exports, Database struct
├── schema.rs           # Table definitions, migrations
├── models.rs           # Rust structs matching tables
├── queries/
│   ├── mod.rs
│   ├── items.rs        # Item CRUD operations
│   ├── user_data.rs    # User data operations
│   ├── downloads.rs    # Download queue operations
│   └── sync.rs         # Sync queue operations
└── sync/
    ├── mod.rs          # SyncService
    ├── manager.rs      # Background sync manager
    └── operations.rs   # Individual sync operation handlers