Files
jellytau/docs/architecture/06-downloads-and-offline.md
T
dtourolle 5259b47cf3 fix(offline): offline mode no longer needs the network
Three defects made "offline" depend on a server it could not reach.

A downloaded film would not play offline. The player found the file on disk,
then asked the server for the item's PlaybackInfo only to read its
media-source id; with no network that retried for seven seconds and failed,
and the file was never opened. A completed download now answers playback
info from its download row — local path, direct play, item id as media
source — and the hybrid repository consults it before the network.

"More info" on a downloaded show failed with "Failed to load item". The
cache is one SQLite connection behind one mutex, so any write in progress
(the catalog sync at every launch, a download finishing) pushes a read past
the 100 ms fast path — and get_items, the library list, genres and playlist
items discarded such a read, waited on the server, and returned its error
over data sitting on disk. They now keep the read running and wait for it
when the server fails; the cache-only reads (search, favourites) simply
await the cache, having no server to fall back from.

Next Up went only to the server, and the TV landing page loads it in one
Promise.all with its other rows, so offline it blanked the whole page. It now
falls back to the cache.

Each fix has a test that failed first against an unreachable server (and, for
the cache, a database held past the fast path).

DR-294, UT-260, UT-261, UT-263.
2026-09-22 22:22:14 -04:00

396 lines
14 KiB
Markdown

# Download Manager & Offline Architecture
## Overview
**Location**: `src-tauri/src/download/`
The download manager provides offline media support with priority-based queue management, progress tracking, retry logic, and smart caching.
```mermaid
flowchart TB
subgraph Frontend["Frontend"]
DownloadButton["DownloadButton.svelte"]
DownloadsPage["/downloads"]
DownloadsStore["downloads.ts store"]
end
subgraph Backend["Rust Backend"]
Commands["Download Commands"]
DownloadManager["DownloadManager"]
DownloadWorker["DownloadWorker"]
SmartCache["SmartCache Engine"]
end
subgraph Storage["Storage"]
SQLite[("SQLite DB")]
MediaFiles[("Downloaded Files")]
end
DownloadButton -->|"invoke('download_item')"| Commands
DownloadsPage -->|"invoke('get_downloads')"| Commands
Commands --> DownloadManager
DownloadManager --> DownloadWorker
DownloadManager --> SmartCache
DownloadWorker -->|"HTTP Stream"| MediaFiles
DownloadWorker -->|"Events"| DownloadsStore
Commands <--> SQLite
SmartCache <--> SQLite
```
## Download Worker
**Location**: `src-tauri/src/download/worker.rs`
The download worker handles HTTP streaming with retry logic and resume support:
```rust
pub struct DownloadWorker {
client: reqwest::Client,
max_retries: u32,
}
pub struct DownloadTask {
pub id: i64,
pub item_id: String,
pub user_id: String,
pub priority: i32,
pub url: String,
pub target_path: PathBuf,
pub mime_type: Option<String>,
pub expected_size: Option<i64>,
}
```
**Retry Strategy**:
- Exponential backoff: 5s, 15s, 45s
- Maximum 3 retry attempts
- HTTP Range requests for resume support
- Progress events emitted every 1MB
**Download Flow**:
```mermaid
sequenceDiagram
participant UI
participant Command as download_item
participant DB as SQLite
participant Worker as DownloadWorker
participant Jellyfin as Jellyfin Server
participant Store as downloads store
UI->>Command: download_item(itemId, userId)
Command->>DB: INSERT INTO downloads
Command->>Worker: Start download task
Worker->>Jellyfin: GET /Items/{id}/Download
loop Progress Updates
Jellyfin->>Worker: Stream chunks
Worker->>Worker: Write to .part file
Worker->>Store: Emit progress event
Store->>UI: Update progress bar
end
Worker->>Worker: Rename .part to final
Worker->>DB: UPDATE status='completed'
Worker->>Store: Emit completed event
Store->>UI: Show completed
```
## Smart Caching Engine
**Location**: `src-tauri/src/download/cache.rs`
The smart caching system provides predictive downloads based on listening patterns:
```rust
pub struct SmartCache {
config: Arc<Mutex<CacheConfig>>,
album_play_history: Arc<Mutex<HashMap<String, Vec<String>>>>,
}
pub struct CacheConfig {
pub queue_precache_enabled: bool,
pub queue_precache_count: usize, // Default: 5
pub album_affinity_enabled: bool,
pub album_affinity_threshold: usize, // Default: 3
pub storage_limit: u64, // Default: 10GB
pub wifi_only: bool, // Default: true
}
```
**Caching Strategies**:
1. **Queue Pre-caching**: Auto-download next 5 tracks when playing (WiFi only)
2. **Album Affinity**: If user plays 3+ tracks from album, cache entire album
3. **LRU Eviction**: Remove least recently accessed when storage limit reached
```mermaid
flowchart TB
Play["Track Played"] --> CheckQueue{"Queue<br/>Pre-cache?"}
CheckQueue -->|"Yes"| CacheNext5["Download<br/>Next 5 Tracks"]
Play --> TrackHistory["Track Play History"]
TrackHistory --> CheckAlbum{"3+ Tracks<br/>from Album?"}
CheckAlbum -->|"Yes"| CacheAlbum["Download<br/>Full Album"]
CacheNext5 --> CheckStorage{"Storage<br/>Limit?"}
CacheAlbum --> CheckStorage
CheckStorage -->|"Exceeded"| EvictLRU["Evict LRU Items"]
CheckStorage -->|"OK"| Download["Queue Download"]
```
## One Storage Model: Cache Entries Are Downloads
**TRACES**: UR-071 | DR-126, DR-127
A cache entry **is** a download with a shorter life: the same `downloads` row and
the same file handling, distinguished by `download_source` plus an expiry. There
is one storage model rather than a cache and a download library that can
disagree about what is on disk.
| `download_source` | Life | Reclaimed by |
|-------------------|------|--------------|
| `'auto'` (temporary) | Expiry, or eviction under space pressure | Both |
| `'user'` (permanent) | No expiry | Neither |
**Eviction only reclaims the temporary tier.** `evict_lru_async` originally
selected every completed download ordered by `completed_at ASC` with no source
filter, so hitting the storage limit deleted the *oldest* download — typically a
film saved deliberately for offline — to make room for a newly precached track.
It now evicts only `COALESCE(download_source, 'user') = 'auto'` rows.
`COALESCE` rather than a bare equality is load-bearing: rows predating the
migration can be NULL, and **unknown provenance must be treated as the user's,
never as disposable**. Freeing less than requested is the correct outcome when
only user downloads remain — the caller reports "unable to free enough".
A temporary row can be **promoted** to permanent when the user chooses to keep
it. That only clears the expiry and flips the source; the bytes never move.
## Offline Catalog Visibility
**TRACES**: UR-052 | DR-078, DR-079, DR-080
Offline, a library page shows **only media on the device**. A "Show all server
media" toggle additionally reveals the cached server catalog, greyed out and
queueable for download on reconnect.
The gate is a process-global `INCLUDE_CATALOG_BROWSE` in
`repository/offline.rs`, written by the `set_show_server_catalog` command. It
gates the synced-catalog leg of `get_items`; without it the toggle rendered but
every server item still appeared, which is the defect the spec was written for.
`isConnected` derives from backend-reported reachability alone (DR-079) — see
[07-connectivity.md](07-connectivity.md).
Per-item disk usage comes from `repository_get_download_disk_usage`
(`DownloadDiskUsage`), aggregated from `downloads.file_size` — used by the
Downloaded browse cards, detail pages, the device total and the remove
confirmation (DR-085).
## What a Video Download Fetches
**TRACES**: UR-071, UR-004 | DR-171, DR-293
An `original`-quality download is the server's untouched file (`Static=true`)
unless its audio cannot be decoded by **the renderer that will play it**
`renderer_can_decode_audio`, DR-234's per-platform answer. Only then is the
server asked to re-encode the audio on the way down (`allowVideoStreamCopy`
keeps the picture byte-for-byte).
The distinction matters because a transcode is generated as it is sent: no
`Content-Length`, `Range` ignored. It measured ~1 MB/s and restarted from byte
zero on every network blip, against a direct copy that moved a 910 MB episode in
94 s with no retries. On Android the renderer is ExoPlayer with the FFmpeg
extension ([05-platform-backends.md](05-platform-backends.md)), which decodes
AC-3/E-AC-3/DTS/TrueHD, so Android downloads are always the direct copy. On
Linux the webview still renders video and the transcode still applies.
The policy used to judge against the *webview's* codec list on every platform
(DR-171), because a download outlives the native-video setting that was active
when it arrived. That reasoning is why Android's webview video path was removed
rather than merely defaulted off: a file downloaded as the original must never
meet a renderer that cannot decode it.
## Offline Means No Network
**TRACES**: UR-002, UR-071 | DR-294
Three defects made "offline" depend on the network; the invariants that replace
them:
- **A download plays without the server.** Playing a downloaded item asked the
server for its `PlaybackInfo` only to read the media-source id; offline that
retried for seven seconds, failed, and the file was never opened.
`OfflineRepository::local_playback_info` answers for any completed download of
the current user — local path, direct play, item id as media source (a download
names no source, so the server served its default, which carries the item's id)
— and `HybridRepository::get_playback_info` consults it **first**.
- **A slow cache read is waited for, never discarded.** The cache is one SQLite
connection behind one mutex, so any write in progress (the catalog sync that
starts at every launch, a download finishing) pushes a read past the 100 ms fast
path. `get_items`, the library list, genres and playlist items used to discard
such a read, wait for the server, and — offline — return its error over data on
disk; "More info" on a downloaded show failed that way. They now start the read
with `cache_try` (which keeps it running) and `settle` on it when the server
fails. Cache-only reads (search, favourites) have no server to fall back from,
so they simply await the cache.
- **Server-only sections degrade, they do not fail a page.** Next Up went only to
the server, and the TV landing page loads it in one `Promise.all`, so offline it
blanked the whole page. It now falls back to the cache when the server cannot
answer.
What still needs the server, deliberately: streaming anything not downloaded,
live TV and channels, reporting playback, and edits (favourites, playlists,
played state).
## Download Commands
**Location**: `src-tauri/src/commands/download/``mod.rs` (the commands below), `pinning.rs`, `smart_cache.rs`
| Command | Parameters | Description |
|---------|------------|-------------|
| `download_item` | `item_id, user_id, file_path` | Queue single item download |
| `download_album` | `album_id, user_id` | Queue all tracks in album |
| `get_downloads` | `user_id, status_filter` | Get download list |
| `pause_download` | `download_id` | Pause active download |
| `resume_download` | `download_id` | Resume paused download |
| `cancel_download` | `download_id` | Cancel and delete partial |
| `delete_download` | `download_id` | Delete completed download |
| `download_video` / `download_series` / `download_season` | item ids | Queue video content |
| `get_download_storage_stats` | `user_id` | Device totals for the downloads screen |
| `delete_album_downloads` / `delete_downloads_under` / `delete_all_downloads` | container id | Bulk removal |
| `pin_item` / `unpin_item` / `is_item_pinned` | `item_id` | Protect metadata from a cache clear |
| `set_max_concurrent_downloads` | `max` | Worker concurrency (3 by default) |
## Offline Commands
**Location**: `src-tauri/src/commands/offline.rs`
| Command | Parameters | Description |
|---------|------------|-------------|
| `offline_is_available` | `item_id` | Check if item downloaded |
| `offline_get_items` | `user_id` | Get all offline items |
| `offline_search` | `user_id, query` | Search downloaded items |
## Player Integration
**Location**: `src-tauri/src/commands/player.rs` (modified)
The player checks for local downloads before streaming:
```rust
fn create_media_item(req: PlayItemRequest, db: Option<&DatabaseWrapper>) -> MediaItem {
let local_path = db.and_then(|db_wrapper| {
check_for_local_download(db_wrapper, &jellyfin_id).ok().flatten()
});
let source = if let Some(path) = local_path {
MediaSource::Local {
file_path: PathBuf::from(path),
jellyfin_item_id: Some(jellyfin_id.clone())
}
} else {
MediaSource::Remote {
stream_url: req.stream_url,
jellyfin_item_id: jellyfin_id.clone()
}
};
MediaItem { source, /* ... */ }
}
```
## Frontend Downloads Store
**Location**: `src/lib/stores/downloads.ts`
```typescript
interface DownloadsState {
downloads: Record<number, DownloadInfo>;
activeCount: number;
queuedCount: number;
}
const downloads = createDownloadsStore();
// Actions
downloads.downloadItem(itemId, userId, filePath)
downloads.downloadAlbum(albumId, userId)
downloads.pause(downloadId)
downloads.resume(downloadId)
downloads.cancel(downloadId)
downloads.delete(downloadId)
downloads.refresh(userId, statusFilter)
// Derived stores
export const activeDownloads = derived(downloads, ($d) =>
Object.values($d.downloads).filter((d) => d.status === 'downloading')
);
```
**Event Handling**:
The store listens to Tauri events for real-time updates:
```typescript
listen<DownloadEvent>('download-event', (event) => {
const payload = event.payload;
switch (payload.type) {
case 'started':
// Update status to 'downloading'
case 'progress':
// Update progress and bytes_downloaded
case 'completed':
// Update status to 'completed', progress to 1.0
case 'failed':
// Update status to 'failed', store error message
}
});
```
## Download UI Components
**DownloadButton** (`src/lib/components/library/DownloadButton.svelte`):
- Multiple states: available, downloading, completed, failed, paused
- Circular progress ring during download
- Size variants: sm, md, lg
- Integrated into TrackList with `showDownload={true}` prop
**DownloadItem** (`src/lib/components/downloads/DownloadItem.svelte`):
- Individual download list item with progress bar
- Action buttons: pause, resume, cancel, delete
- Status indicators with color coding
**Downloads Page** (`src/routes/downloads/+page.svelte`):
- Active/Completed tabs
- Bulk actions: Pause All, Resume All, Clear Completed
- Empty states with helpful instructions
## Database Schema
**downloads table**:
```sql
CREATE TABLE downloads (
id INTEGER PRIMARY KEY AUTOINCREMENT,
item_id TEXT NOT NULL,
user_id TEXT NOT NULL,
file_path TEXT,
file_size INTEGER,
mime_type TEXT,
status TEXT DEFAULT 'pending', -- pending, downloading, completed, failed, paused
progress REAL DEFAULT 0.0,
bytes_downloaded INTEGER DEFAULT 0,
priority INTEGER DEFAULT 0,
error_message TEXT,
retry_count INTEGER DEFAULT 0,
queued_at TEXT DEFAULT CURRENT_TIMESTAMP,
started_at TEXT,
completed_at TEXT
);
CREATE INDEX idx_downloads_queue
ON downloads(status, priority DESC, queued_at ASC)
WHERE status IN ('pending', 'downloading');
```