jellytau/docs/architecture/08-database-design.md

# Offline Database Design

## Entity Relationship Diagram

```mermaid
erDiagram
    servers ||--o{ users : "has"
    servers ||--o{ libraries : "has"
    libraries ||--o{ items : "contains"
    items ||--o{ items : "parent_of"
    items ||--o{ user_data : "has"
    items ||--o{ downloads : "has"
    items ||--o{ media_streams : "has"
    items ||--o{ thumbnails : "has"
    users ||--o{ user_data : "owns"
    users ||--o{ downloads : "owns"
    users ||--o{ sync_queue : "owns"

    servers {
        int id PK
        string jellyfin_id UK
        string name
        string url
        string version
        datetime last_sync
    }

    users {
        int id PK
        string jellyfin_id
        int server_id FK
        string name
        boolean is_active
    }

    libraries {
        int id PK
        string jellyfin_id
        int server_id FK
        string name
        string collection_type
        string image_tag
    }

    items {
        int id PK
        string jellyfin_id
        int server_id FK
        int library_id FK
        int parent_id FK
        string type
        string name
        string sort_name
        string overview
        int production_year
        float community_rating
        string official_rating
        int runtime_ticks
        string primary_image_tag
        string backdrop_image_tag
        string album_id
        string album_name
        string album_artist
        json artists
        json genres
        int index_number
        int parent_index_number
        string premiere_date
        json metadata_json
        datetime created_at
        datetime updated_at
        datetime last_sync
    }

    user_data {
        int id PK
        int item_id FK
        int user_id FK
        int position_ticks
        int play_count
        boolean is_favorite
        boolean played
        datetime last_played
        datetime updated_at
        datetime synced_at
    }

    downloads {
        int id PK
        int item_id FK
        int user_id FK
        string file_path
        int file_size
        string status
        float progress
        int priority
        string error_message
        datetime created_at
        datetime completed_at
    }

    media_streams {
        int id PK
        int item_id FK
        int stream_index
        string type
        string codec
        string language
        string display_title
        boolean is_default
        boolean is_forced
        boolean is_external
    }

    sync_queue {
        int id PK
        int user_id FK
        string operation
        string entity_type
        string entity_id
        json payload
        datetime created_at
        int attempts
        datetime last_attempt
        string status
    }

    thumbnails {
        int id PK
        int item_id FK
        string image_type
        string image_tag
        string file_path
        int width
        int height
        datetime cached_at
    }
```

## Table Definitions

### servers
Stores connected Jellyfin server information.

```sql
CREATE TABLE servers (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    jellyfin_id     TEXT NOT NULL UNIQUE,
    name            TEXT NOT NULL,
    url             TEXT NOT NULL,
    version         TEXT,
    last_sync       DATETIME,
    created_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at      DATETIME DEFAULT CURRENT_TIMESTAMP
);
```

### users
Stores user accounts per server. Access tokens are stored separately in secure storage (see [09-security.md](09-security.md)).

```sql
CREATE TABLE users (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    jellyfin_id     TEXT NOT NULL,
    server_id       INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    name            TEXT NOT NULL,
    is_active       BOOLEAN DEFAULT 0,
    created_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    UNIQUE(jellyfin_id, server_id)
);
```

### libraries
Stores library/collection metadata.

```sql
CREATE TABLE libraries (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    jellyfin_id     TEXT NOT NULL,
    server_id       INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    name            TEXT NOT NULL,
    collection_type TEXT,
    image_tag       TEXT,
    sort_order      INTEGER DEFAULT 0,
    last_sync       DATETIME,
    created_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    UNIQUE(jellyfin_id, server_id)
);

CREATE INDEX idx_libraries_server ON libraries(server_id);
```

### items
Main table for all media items (movies, episodes, albums, songs, etc.).

```sql
CREATE TABLE items (
    id                   INTEGER PRIMARY KEY AUTOINCREMENT,
    jellyfin_id          TEXT NOT NULL,
    server_id            INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    library_id           INTEGER REFERENCES libraries(id) ON DELETE SET NULL,
    parent_id            INTEGER REFERENCES items(id) ON DELETE CASCADE,

    -- Basic metadata
    type                 TEXT NOT NULL,
    name                 TEXT NOT NULL,
    sort_name            TEXT,
    overview             TEXT,

    -- Media info
    production_year      INTEGER,
    community_rating     REAL,
    official_rating      TEXT,
    runtime_ticks        INTEGER,

    -- Images
    primary_image_tag    TEXT,
    backdrop_image_tag   TEXT,

    -- Audio-specific
    album_id             TEXT,
    album_name           TEXT,
    album_artist         TEXT,
    artists              TEXT,                   -- JSON array

    -- Series/Season-specific
    index_number         INTEGER,
    parent_index_number  INTEGER,
    series_id            TEXT,
    series_name          TEXT,
    season_id            TEXT,

    -- Additional
    genres               TEXT,                   -- JSON array
    premiere_date        TEXT,
    metadata_json        TEXT,

    -- Sync tracking
    created_at           DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at           DATETIME DEFAULT CURRENT_TIMESTAMP,
    last_sync            DATETIME,

    UNIQUE(jellyfin_id, server_id)

    -- Logical container (migration 027): episode → season/series, season →
    -- series, track → album, else parent. See "Listing query shape".
    -- container_id TEXT GENERATED ALWAYS AS (CASE item_type … END) VIRTUAL
);

-- Performance indexes
CREATE INDEX idx_items_container ON items(container_id, sort_name, name);
CREATE INDEX idx_items_server ON items(server_id);
CREATE INDEX idx_items_library ON items(library_id);
CREATE INDEX idx_items_parent ON items(parent_id);
CREATE INDEX idx_items_type ON items(type);
CREATE INDEX idx_items_album ON items(album_id);
CREATE INDEX idx_items_series ON items(series_id);
CREATE INDEX idx_items_name ON items(name COLLATE NOCASE);

-- Full-text search
CREATE VIRTUAL TABLE items_fts USING fts5(
    name,
    overview,
    artists,
    album_name,
    album_artist,
    content='items',
    content_rowid='id'
);

-- Triggers to keep FTS in sync
CREATE TRIGGER items_ai AFTER INSERT ON items BEGIN
    INSERT INTO items_fts(rowid, name, overview, artists, album_name, album_artist)
    VALUES (new.id, new.name, new.overview, new.artists, new.album_name, new.album_artist);
END;

CREATE TRIGGER items_ad AFTER DELETE ON items BEGIN
    INSERT INTO items_fts(items_fts, rowid, name, overview, artists, album_name, album_artist)
    VALUES ('delete', old.id, old.name, old.overview, old.artists, old.album_name, old.album_artist);
END;

CREATE TRIGGER items_au AFTER UPDATE ON items BEGIN
    INSERT INTO items_fts(items_fts, rowid, name, overview, artists, album_name, album_artist)
    VALUES ('delete', old.id, old.name, old.overview, old.artists, old.album_name, old.album_artist);
    INSERT INTO items_fts(rowid, name, overview, artists, album_name, album_artist)
    VALUES (new.id, new.name, new.overview, new.artists, new.album_name, new.album_artist);
END;
```

### media_streams
Stores subtitle and audio track information for items.

```sql
CREATE TABLE media_streams (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    item_id         INTEGER NOT NULL REFERENCES items(id) ON DELETE CASCADE,
    stream_index    INTEGER NOT NULL,
    type            TEXT NOT NULL,
    codec           TEXT,
    language        TEXT,
    display_title   TEXT,
    is_default      BOOLEAN DEFAULT 0,
    is_forced       BOOLEAN DEFAULT 0,
    is_external     BOOLEAN DEFAULT 0,
    path            TEXT,
    UNIQUE(item_id, stream_index)
);

CREATE INDEX idx_media_streams_item ON media_streams(item_id);
```

### user_data
Stores per-user data for items (favorites, progress, play count).

```sql
CREATE TABLE user_data (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    item_id         INTEGER NOT NULL REFERENCES items(id) ON DELETE CASCADE,
    user_id         INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,

    -- Playback state
    position_ticks  INTEGER DEFAULT 0,
    play_count      INTEGER DEFAULT 0,
    played          BOOLEAN DEFAULT 0,
    last_played     DATETIME,

    -- User preferences
    is_favorite     BOOLEAN DEFAULT 0,
    user_rating     REAL,

    -- Sync tracking
    updated_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    synced_at       DATETIME,
    needs_sync      BOOLEAN DEFAULT 0,

    UNIQUE(item_id, user_id)
);

CREATE INDEX idx_user_data_item ON user_data(item_id);
CREATE INDEX idx_user_data_user ON user_data(user_id);
CREATE INDEX idx_user_data_needs_sync ON user_data(needs_sync) WHERE needs_sync = 1;
CREATE INDEX idx_user_data_favorites ON user_data(user_id, is_favorite) WHERE is_favorite = 1;
CREATE INDEX idx_user_data_in_progress ON user_data(user_id, position_ticks)
    WHERE position_ticks > 0 AND played = 0;
```

### downloads
Tracks downloaded media files.

```sql
CREATE TABLE downloads (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    item_id         INTEGER NOT NULL REFERENCES items(id) ON DELETE CASCADE,
    user_id         INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,

    file_path       TEXT,
    file_size       INTEGER,
    file_hash       TEXT,

    status          TEXT NOT NULL DEFAULT 'pending',
    progress        REAL DEFAULT 0,
    bytes_downloaded INTEGER DEFAULT 0,

    transcode_profile TEXT,

    priority        INTEGER DEFAULT 0,
    error_message   TEXT,
    retry_count     INTEGER DEFAULT 0,

    created_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    started_at      DATETIME,
    completed_at    DATETIME,
    expires_at      DATETIME,

    UNIQUE(item_id, user_id)
);

CREATE INDEX idx_downloads_status ON downloads(status);
CREATE INDEX idx_downloads_user ON downloads(user_id);
CREATE INDEX idx_downloads_queue ON downloads(status, priority DESC, created_at ASC)
    WHERE status IN ('pending', 'downloading');
```

### sync_queue
Stores mutations to sync back to server when online.

```sql
CREATE TABLE sync_queue (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    user_id         INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,

    operation       TEXT NOT NULL,
    entity_type     TEXT NOT NULL,
    entity_id       TEXT NOT NULL,
    payload         TEXT,

    status          TEXT DEFAULT 'pending',
    attempts        INTEGER DEFAULT 0,
    max_attempts    INTEGER DEFAULT 5,
    last_attempt    DATETIME,
    error_message   TEXT,

    created_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    completed_at    DATETIME
);

CREATE INDEX idx_sync_queue_status ON sync_queue(status, created_at ASC)
    WHERE status = 'pending';
CREATE INDEX idx_sync_queue_user ON sync_queue(user_id);
```

### thumbnails
Caches downloaded artwork.

```sql
CREATE TABLE thumbnails (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    item_id         INTEGER NOT NULL REFERENCES items(id) ON DELETE CASCADE,
    image_type      TEXT NOT NULL,
    image_tag       TEXT,
    file_path       TEXT NOT NULL,
    width           INTEGER,
    height          INTEGER,
    file_size       INTEGER,
    cached_at       DATETIME DEFAULT CURRENT_TIMESTAMP,
    last_accessed   DATETIME DEFAULT CURRENT_TIMESTAMP,
    UNIQUE(item_id, image_type, width)
);

CREATE INDEX idx_thumbnails_item ON thumbnails(item_id);
CREATE INDEX idx_thumbnails_lru ON thumbnails(last_accessed ASC);
```

### playlists (for local/synced playlists)

```sql
CREATE TABLE playlists (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    jellyfin_id     TEXT,
    user_id         INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
    name            TEXT NOT NULL,
    description     TEXT,
    is_local_only   BOOLEAN DEFAULT 0,
    created_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    synced_at       DATETIME,
    needs_sync      BOOLEAN DEFAULT 0
);

CREATE TABLE playlist_items (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    playlist_id     INTEGER NOT NULL REFERENCES playlists(id) ON DELETE CASCADE,
    item_id         INTEGER NOT NULL REFERENCES items(id) ON DELETE CASCADE,
    sort_order      INTEGER NOT NULL,
    added_at        DATETIME DEFAULT CURRENT_TIMESTAMP,
    UNIQUE(playlist_id, item_id)
);

CREATE INDEX idx_playlist_items_playlist ON playlist_items(playlist_id, sort_order);
```

## Key Queries

### Get items for offline library browsing
```sql
-- Get all albums in a music library
SELECT * FROM items
WHERE library_id = ? AND type = 'MusicAlbum'
ORDER BY sort_name;

-- Get tracks for an album
SELECT * FROM items
WHERE album_id = ? AND type = 'Audio'
ORDER BY parent_index_number, index_number;
```

### Resume / Continue Watching
```sql
SELECT i.*, ud.position_ticks, ud.last_played
FROM items i
JOIN user_data ud ON ud.item_id = i.id
WHERE ud.user_id = ?
  AND ud.position_ticks > 0
  AND ud.played = 0
ORDER BY ud.last_played DESC
LIMIT 20;
```

### Offline search
```sql
SELECT i.* FROM items i
JOIN items_fts fts ON fts.rowid = i.id
WHERE items_fts MATCH ?
ORDER BY rank;
```

### Download queue management
```sql
-- Get next item to download
SELECT d.*, i.name, i.type
FROM downloads d
JOIN items i ON i.id = d.item_id
WHERE d.status = 'pending'
ORDER BY d.priority DESC, d.created_at ASC
LIMIT 1;

-- Get download progress for UI
SELECT
    d.status,
    COUNT(*) as count,
    SUM(d.file_size) as total_size,
    SUM(d.bytes_downloaded) as downloaded
FROM downloads d
WHERE d.user_id = ?
GROUP BY d.status;
```

### Sync queue processing
```sql
-- Get pending sync operations (oldest first)
SELECT * FROM sync_queue
WHERE status = 'pending'
  AND attempts < max_attempts
ORDER BY created_at ASC
LIMIT 10;

-- Mark operation complete
UPDATE sync_queue
SET status = 'completed', completed_at = CURRENT_TIMESTAMP
WHERE id = ?;
```

## Data Flow

### Online Mode

```mermaid
flowchart TB
    subgraph OnlineMode["Online Mode"]
        JellyfinServer["Jellyfin Server"]
        OnlineRepo["OnlineRepo"]
        SQLite["SQLite"]
        HybridRepo["HybridRepository"]
        UI["UI / Stores"]

        JellyfinServer -->|"API Response"| OnlineRepo
        OnlineRepo -->|"Cache"| SQLite
        SQLite -->|"Sync"| JellyfinServer
        OnlineRepo -->|"Response"| HybridRepo
        SQLite -->|"Fallback"| HybridRepo
        HybridRepo --> UI
    end
```

### Offline Mode

```mermaid
flowchart TB
    subgraph OfflineMode["Offline Mode"]
        OfflineRepo["OfflineRepo"]
        SQLite2["SQLite"]
        SyncQueue["sync_queue<br/>(Queued for later)"]
        HybridRepo2["HybridRepository"]
        UI2["UI / Stores"]

        OfflineRepo <-->|"Query"| SQLite2
        SQLite2 -->|"Mutations"| SyncQueue
        OfflineRepo --> HybridRepo2
        HybridRepo2 --> UI2
    end
```

### Sync on Reconnect

```mermaid
flowchart LR
    NetworkRestored["Network restored"]
    SyncService["SyncService"]
    SyncQueue2["sync_queue"]
    JellyfinAPI["Jellyfin API"]
    MarkSynced["Mark synced"]

    NetworkRestored --> SyncService
    SyncService -->|"Read"| SyncQueue2
    SyncQueue2 -->|"Send"| JellyfinAPI
    JellyfinAPI -->|"Success"| MarkSynced
    MarkSynced --> SyncService
```

## Storage Estimates

| Content Type | Metadata Size | Thumbnail Size | Media Size |
|--------------|---------------|----------------|------------|
| Song | ~2 KB | ~50 KB (300px) | 5-15 MB |
| Album (12 tracks) | ~30 KB | ~100 KB | 60-180 MB |
| Movie | ~5 KB | ~200 KB | 1-8 GB |
| Episode | ~3 KB | ~100 KB | 300 MB - 2 GB |
| Full music library (5000 songs) | ~10 MB | ~250 MB | 25-75 GB |

## Connection ownership

`storage::Database` is opened once at startup and owns the database for the
life of the app; nothing else opens the file. Everything goes through
`Database::service()`, which returns a clone of one `RusqliteService`
(`storage/db_service.rs`) — callers never hold a `Connection`.

```
                    ┌─────────────────────────────┐
 execute / insert / │ writer thread ("db-writer") │  one read-write connection,
 transaction / ────►│ jobs run in arrival order   │  foreign_keys = ON
 execute_detached   └─────────────────────────────┘
                    ┌─────────────────────────────┐
 query_one /        │ reader pool (3 connections) │  query_only = ON; WAL gives
 query_optional / ─►│ via spawn_blocking          │  each the last committed
 query_many         └─────────────────────────────┘  snapshot
```

Pragmas: `journal_mode = WAL`, `synchronous = NORMAL`, `busy_timeout = 5 s` on
every connection. `PRAGMA optimize` runs once at open, after migrations, so the planner has
statistics (see "Listing query shape").

**Why.** It used to be one connection behind one `std::sync::Mutex`. WAL was on,
but with a single connection its one benefit — readers running beside a writer —
never applied: every read queued behind every write. A series page's background
refresh cached hundreds of episodes one autocommit (and one fsync) at a time,
and library pages, thumbnail lookups and settings reads all waited behind the
pile; the same page measured 1.7 s or 3.95 s depending on what was queued.

**Invariants a change must keep:**

- **Writes go to the writer, reads to the pool.** A reader is `query_only`, so a
  write sent through `query_*` fails loudly rather than racing the writer. A
  read that must see a write *in the same unit of work* belongs inside the
  `transaction` closure, which runs on the writer.
- **Connection-wide state is set inside one writer job.** `PRAGMA foreign_keys`
  is per connection and ignored inside a transaction. `save_to_cache` used to
  switch it off with one `execute` and back on with another, so it stayed off
  for every other write that ran across the save's awaits (they did — nearly
  all of 1,600 FK-violating writes got through in the regression test).
  `transaction_without_foreign_keys` flips it around `BEGIN`/`COMMIT` in a single
  job. Any future pragma toggle must work the same way.
- **An insert's rowid comes from the same job.** `insert()` returns it; there is
  deliberately no standalone `last_insert_rowid()`, which returned whichever row
  the last *anyone* inserted.
- **Batch writes into one transaction.** A commit is a queue slot on the
  writer; a page of items is one `transaction`, not one `execute` per row.
- **Best-effort bookkeeping does not wait.** `execute_detached` queues a write
  (the thumbnail LRU access time) in order with the rest and returns
  immediately; failures are only logged.
- **The writer survives a panicking job.** The job's caller gets an error; the
  loop rolls back any transaction the job left open and restores
  `foreign_keys = ON`, then carries on.

**`synchronous = NORMAL`** is corruption-safe in WAL mode and survives an app
crash; only a power cut can lose the last few commits. Everything stored is
either re-fetchable from the server or (the sync queue, local positions)
recoverable at that granularity.

**In-memory databases** (tests) cannot be shared between connections, so
`RusqliteService::new` has no pool and routes reads through the writer — the
old serialized behaviour.

**Not done, and why:** grouping consecutive small `execute` jobs into one
commit on the writer. With `synchronous = NORMAL` a commit no longer fsyncs, so
the gain is small, and it would change a failed statement's error semantics
for the jobs grouped with it.

### Listing query shape

`OfflineRepository::get_items` (`items_listing_sql`) is the hot read: every
library, series, season and album page goes through it, and it must fit the
100 ms cache fast path on a phone. Every other cached read follows the same
rules.

- **Children are matched on `items.container_id`** (migration 027), a VIRTUAL
  generated column holding the item's *logical* container: an episode's season
  (else series, else parent), a season's series, a track's album, otherwise the
  parent. Jellyfin's `ParentId` is the storage parent, not the logical one — in
  a series without season folders an episode's `ParentId` is the series while
  its `SeasonId` names a virtual season — so listings used to match on four
  columns at once. That `OR` defeated the planner into walking the whole table,
  and it was wrong: every episode carries its series id, so a series listed all
  its episodes beside its seasons. Being generated, the column covers every
  write path (cache, downloads, catalog crawl) without any of them knowing, and
  cannot drift from the columns it is computed from.
- **`idx_items_container (container_id, sort_name, name)`** serves a listing as
  one index range already in display order — no sort step. `sort_name` is
  usually NULL in the cache (the cache never writes it), hence `name` in the
  index too.
- **Containers exist even when never browsed.** A series lists its seasons, so
  an episode whose season row was never cached (it arrived through Next Up or
  Latest) would be unreachable from its series offline. `save_to_cache` — and
  migration 027 for rows already on disk — inserts placeholders named from the
  child's own fields (`season_name`, `series_name`, `album_name`) with
  `synced_at` NULL, so they show only when a download makes them available; the
  server's real row replaces them wholesale on the next browse.
- **Availability is a per-row `EXISTS`** (`downloaded_sql` / `available_sql`):
  cached for browsing (only with the catalog-browse flag), downloaded, or a
  container with a downloaded *descendant* — which is why that one check still
  looks at all four link columns (a series is available through an episode two
  levels down). It used to be a CTE that built the id of every available item
  in the database before filtering, paying for the whole table on every call.
- **The library clause is added only for a library parent** (`is_library`),
  never `OR`ed into an ordinary listing, where it forces a full scan.
- **`+i.server_id`** keeps the planner off the server index, which every row
  shares. `PRAGMA optimize` at open (`analysis_limit = 400`) gives the planner
  statistics, but even with them it chose that index for the old `OR`; the `+`
  is the guarantee.
- **User data is fetched in batches** (`with_user_data`, one `IN (…)` query per
  500 rows), not once per row.

Measured on a ~110k-item benchmark catalogue (desktop): a series listing went
from ~80 ms to under 1 ms; migration 027 upgrades an existing database of that
size in ~0.1 s. `listing_a_non_library_parent_uses_the_container_index` and
migration 027's tests assert the plans; `storage::tests::write_bench_database`
(ignored; set `JELLYTAU_BENCH_DB`) writes the benchmark catalogue for the
`sqlite3` CLI.

## Rust Module Structure

```
src-tauri/src/storage/
├── mod.rs              # Module exports, Database struct
├── schema.rs           # Table definitions, migrations
├── models.rs           # Rust structs matching tables
├── queries/
│   ├── mod.rs
│   ├── items.rs        # Item CRUD operations
│   ├── user_data.rs    # User data operations
│   ├── downloads.rs    # Download queue operations
│   └── sync.rs         # Sync queue operations
└── sync/
    ├── mod.rs          # SyncService
    ├── manager.rs      # Background sync manager
    └── operations.rs   # Individual sync operation handlers
```