feat(search): answer search from a local index; tier downloads by lifetime

Search's instant leg read only downloaded items, so with no downloads it
returned nothing and every keystroke fell through to a full Recursive=true
server query. It now reads the whole synced catalog through the same
availability CTE get_items uses, gated on the same include_catalog_browse
flag so search and browse cannot diverge. (UR-065, DR-108)

Also fixes three defects found while confirming that:

- items_fts grew by a full duplicate index every catalog pass. INSERT OR
  REPLACE fires no AFTER DELETE trigger without recursive_triggers, so the
  old index row was orphaned, and a TEXT PRIMARY KEY meant the replacement
  took a fresh rowid and inserted a second entry. Now a real upsert, with
  migration 021 rebuilding existing indexes. (DR-110)
- DELETE FROM items existed nowhere, so server-side deletions never
  propagated. Adds a post-crawl mark-and-sweep, scoped to crawled types,
  skipping downloaded items, and refusing to run after a partial crawl
  because items.parent_id cascades. (DR-110)
- The index omitted MusicArtist, Playlist and People, which search groups
  results by. Adds them plus people_fts (migration 022). (DR-111)

Re-indexing moves from a frontend startup call to a Rust background task
with a 6h TTL, so a long session no longer searches a stale catalog and a
restart no longer forces a crawl regardless of freshness. (DR-109, IR-030)

Downloads gain a lifetime tier. Eviction selected every completed row by
age with no download_source filter, so hitting the storage limit deleted
the oldest download -- typically one saved deliberately for offline -- to
make room for a precached track. It now reclaims only 'auto' rows, and
expired ones are reclaimed first, before live cache is evicted.
(DR-126, DR-127)

Downloaded video and audio-only handoffs now play from disk instead of
streaming; the video path had never consulted downloads at all. No
transcode is involved: MPV runs video=no and ExoPlayer has no surface for
an Audio item. (DR-123 in part, DR-128)

FTS queries are built as quoted phrases so apostrophes, hyphens and
slashes are data rather than operator syntax, and the item-type filter is
bound rather than interpolated.

Specs: docs/specs/catalog-index-search.md,
docs/specs/read-through-media-cache.md

Includes concurrently-developed favourites browsing and background-audio
stream-end handling; the two workstreams share offline.rs, lib.rs and
online.rs, so no subset of files builds independently.
This commit is contained in:
2026-08-04 17:35:17 +02:00
parent c55ff45692
commit 62873cab3d
52 changed files with 6110 additions and 191 deletions
+91
View File
@@ -25,6 +25,9 @@ pub const MIGRATIONS: &[(&str, &str)] = &[
("018_items_is_folder", MIGRATION_018),
("019_genres_cache", MIGRATION_019),
("020_items_season_index", MIGRATION_020),
("021_rebuild_items_fts", MIGRATION_021),
("022_people_fts", MIGRATION_022),
("023_downloads_expiry", MIGRATION_023),
];
/// Initial schema migration
@@ -728,3 +731,91 @@ CREATE INDEX IF NOT EXISTS idx_genres_scope ON genres(server_id, library_id);
const MIGRATION_020: &str = r#"
CREATE INDEX IF NOT EXISTS idx_items_season ON items(season_id);
"#;
/// Discard and rebuild the FTS index from the `items` table.
///
/// Until DR-110, `save_to_cache` used `INSERT OR REPLACE INTO items`. REPLACE
/// deletes the conflicting row and inserts a new one, but SQLite only fires
/// `AFTER DELETE` triggers on that implicit delete when `recursive_triggers` is
/// enabled — it is not (storage/mod.rs sets only `foreign_keys` and
/// `journal_mode`), so `items_ad` never ran and the old index row was orphaned.
/// Worse, `items.id` is a `TEXT PRIMARY KEY`, so the replacement row also took a
/// *fresh rowid* and `items_ai` appended a second entry. Every catalog pass
/// therefore left another duplicate behind, and existing installs carry one
/// stale entry per item per sync since the database was created.
///
/// This was invisible in results — the `JOIN items_fts fts ON fts.rowid =
/// i.rowid` drops rowids that no longer exist — but it degrades `MATCH`
/// permanently, and it becomes a *correctness* problem the moment rowids are
/// freed and reused: a new item landing on a freed rowid inherits the orphan's
/// index entry and matches queries for the deleted item's title. The DR-110
/// deletion sweep frees rowids, so this rebuild must run before it.
///
/// `'rebuild'` is the FTS5 command for exactly this: it truncates the index and
/// repopulates it from the external content table.
///
/// TRACES: UR-065 | DR-110
const MIGRATION_021: &str = r#"
INSERT INTO items_fts(items_fts) VALUES('rebuild');
"#;
/// Full-text index over `people`, mirroring `items_fts`.
///
/// People live in their own table (migration 009) rather than in `items`, and
/// had no FTS index at all — so the People group UR-060 requires could only ever
/// be filled by the server leg of search. With the local index now answering
/// first, an actor's name has to be findable offline too.
///
/// TRACES: UR-065, UR-060 | DR-111
const MIGRATION_022: &str = r#"
CREATE VIRTUAL TABLE IF NOT EXISTS people_fts USING fts5(
name,
overview,
content='people',
content_rowid='rowid'
);
CREATE TRIGGER IF NOT EXISTS people_ai AFTER INSERT ON people BEGIN
INSERT INTO people_fts(rowid, name, overview)
VALUES (new.rowid, new.name, new.overview);
END;
CREATE TRIGGER IF NOT EXISTS people_ad AFTER DELETE ON people BEGIN
INSERT INTO people_fts(people_fts, rowid, name, overview)
VALUES('delete', old.rowid, old.name, old.overview);
END;
CREATE TRIGGER IF NOT EXISTS people_au AFTER UPDATE ON people BEGIN
INSERT INTO people_fts(people_fts, rowid, name, overview)
VALUES('delete', old.rowid, old.name, old.overview);
INSERT INTO people_fts(rowid, name, overview)
VALUES (new.rowid, new.name, new.overview);
END;
-- Backfill for rows cached before this index existed.
INSERT INTO people_fts(people_fts) VALUES('rebuild');
"#;
/// Give temporary downloads a life limit.
///
/// A cache entry is not a different kind of object from a download — it is a
/// download with a shorter life. Modelling it as one `downloads` row with an
/// expiry (rather than a parallel cache store) means there is a single storage
/// accounting, a single eviction path, and no way for a cache and a download
/// library to disagree about what is on disk.
///
/// `expires_at` is NULL for permanent rows, which is every row that exists
/// today: `download_source` defaults to `'user'`, and a user's own download
/// never expires. Only `'auto'` rows get a timestamp, and they are reclaimed by
/// whichever comes first — the expiry passing, or LRU eviction under space
/// pressure (DR-126).
///
/// TRACES: UR-071 | DR-127
const MIGRATION_023: &str = r#"
ALTER TABLE downloads ADD COLUMN expires_at TEXT;
-- Reclaim scans filter on expiry among 'auto' rows; index both so the sweep
-- stays cheap as the cache tier grows.
CREATE INDEX IF NOT EXISTS idx_downloads_expiry
ON downloads(download_source, expires_at);
"#;