dtourolle/jellytau
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 39 Wiki Activity

feat(search): answer search from a local index; tier downloads by lifetime

Browse Source 
Search's instant leg read only downloaded items, so with no downloads it
returned nothing and every keystroke fell through to a full Recursive=true
server query. It now reads the whole synced catalog through the same
availability CTE get_items uses, gated on the same include_catalog_browse
flag so search and browse cannot diverge. (UR-065, DR-108)

Also fixes three defects found while confirming that:

- items_fts grew by a full duplicate index every catalog pass. INSERT OR
  REPLACE fires no AFTER DELETE trigger without recursive_triggers, so the
  old index row was orphaned, and a TEXT PRIMARY KEY meant the replacement
  took a fresh rowid and inserted a second entry. Now a real upsert, with
  migration 021 rebuilding existing indexes. (DR-110)
- DELETE FROM items existed nowhere, so server-side deletions never
  propagated. Adds a post-crawl mark-and-sweep, scoped to crawled types,
  skipping downloaded items, and refusing to run after a partial crawl
  because items.parent_id cascades. (DR-110)
- The index omitted MusicArtist, Playlist and People, which search groups
  results by. Adds them plus people_fts (migration 022). (DR-111)

Re-indexing moves from a frontend startup call to a Rust background task
with a 6h TTL, so a long session no longer searches a stale catalog and a
restart no longer forces a crawl regardless of freshness. (DR-109, IR-030)

Downloads gain a lifetime tier. Eviction selected every completed row by
age with no download_source filter, so hitting the storage limit deleted
the oldest download -- typically one saved deliberately for offline -- to
make room for a precached track. It now reclaims only 'auto' rows, and
expired ones are reclaimed first, before live cache is evicted.
(DR-126, DR-127)

Downloaded video and audio-only handoffs now play from disk instead of
streaming; the video path had never consulted downloads at all. No
transcode is involved: MPV runs video=no and ExoPlayer has no surface for
an Audio item. (DR-123 in part, DR-128)

FTS queries are built as quoted phrases so apostrophes, hyphens and
slashes are data rather than operator syntax, and the item-type filter is
bound rather than interpolated.

Specs: docs/specs/catalog-index-search.md,
docs/specs/read-through-media-cache.md

Includes concurrently-developed favourites browsing and background-audio
stream-end handling; the two workstreams share offline.rs, lib.rs and
online.rs, so no subset of files builds independently.
This commit is contained in:
dtourolle
2026-08-04 17:35:17 +02:00
parent c55ff45692
commit 62873cab3d
52 changed files with 6110 additions and 191 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src-tauri/src/repository/online.rs
+307 -48
View File
@@ -48,6 +48,12 @@ pub struct OnlineRepository {
}
impl OnlineRepository {
/// The signed-in user these requests are made as. Needed by the favourites
/// drain, which reads this user's queued rows. TRACES: UR-069 | DR-120
pub fn user_id(&self) -> &str {
&self.user_id
}
pub fn new(
http_client: Arc<HttpClient>,
server_url: String,
@@ -560,6 +566,138 @@ struct JellyfinItem {
media_streams: Option<Vec<JellyfinMediaStream>>,
media_sources: Option<Vec<JellyfinMediaSource>>,
people: Option<Vec<crate::repository::types::Person>>,
user_data: Option<JellyfinUserData>,
}
/// Per-user state Jellyfin attaches to an item (favourite, played, resume).
///
/// Returned on every `/Users/{uid}/Items*` response; we additionally name
/// `UserData` in the `Fields=` list so the shape is explicit rather than
/// dependent on the server's default field set.
///
/// TRACES: UR-069 | DR-113, JA-034 | UT-099
#[derive(Debug, Deserialize, Clone)]
#[serde(rename_all = "PascalCase")]
struct JellyfinUserData {
playback_position_ticks: Option<i64>,
#[serde(rename = "Played")]
is_played: Option<bool>,
is_favorite: Option<bool>,
play_count: Option<i32>,
last_played_date: Option<String>,
}
impl From<JellyfinUserData> for UserData {
fn from(jf: JellyfinUserData) -> Self {
UserData {
playback_position_ticks: jf.playback_position_ticks,
playback_position_ms: jf.playback_position_ticks.map(crate::domain::ticks_to_ms),
is_played: jf.is_played,
is_favorite: jf.is_favorite,
play_count: jf.play_count,
last_played_date: jf.last_played_date,
playback_context_type: None,
playback_context_id: None,
}
}
}
/// Build the Jellyfin endpoint for a folder listing.
///
/// Extracted from `get_items` so the query it produces — in particular the
/// favourites filter — can be asserted without standing up an HTTP server.
///
/// TRACES: UR-007, UR-067 | DR-116 | UT-104
fn build_get_items_endpoint(
user_id: &str,
parent_id: &str,
options: Option<&GetItemsOptions>,
) -> String {
let mut endpoint = format!("/Users/{}/Items?ParentId={}", user_id, parent_id);
if let Some(opts) = options {
if let Some(limit) = opts.limit {
endpoint.push_str(&format!("&Limit={}", limit));
}
if let Some(start_index) = opts.start_index {
endpoint.push_str(&format!("&StartIndex={}", start_index));
}
if let Some(types) = &opts.include_item_types {
endpoint.push_str(&format!("&IncludeItemTypes={}", types.join(",")));
}
if let Some(sort_by) = &opts.sort_by {
endpoint.push_str(&format!("&SortBy={}", sort_by));
}
if let Some(sort_order) = &opts.sort_order {
endpoint.push_str(&format!("&SortOrder={}", sort_order));
}
if let Some(recursive) = opts.recursive {
endpoint.push_str(&format!("&Recursive={}", recursive));
}
if let Some(genres) = &opts.genres {
if !genres.is_empty() {
// Genre names may contain spaces/ampersands, so percent-encode each.
let encoded: Vec<String> = genres
.iter()
.map(|g| urlencoding::encode(g).into_owned())
.collect();
endpoint.push_str(&format!("&Genres={}", encoded.join("|")));
}
}
// TRACES: UR-067 | DR-116 | UT-104
if opts.favorites_only == Some(true) {
endpoint.push_str("&Filters=IsFavorite");
}
}
// Request image fields for list views (People only needed in get_item
// detail view). Genres is needed so cached items carry their genres,
// which lets the offline store derive genre lists + per-genre counts.
endpoint
.push_str("&Fields=BackdropImageTags,ParentBackdropImageTags,Genres,PremiereDate,UserData");
endpoint
}
/// Build the Jellyfin endpoint for a favourites listing.
///
/// Pulled out of `get_favorites` so the query can be asserted without an HTTP
/// server. `scope` is expanded here — `SearchScope::All` yields `None`, and the
/// `IncludeItemTypes` filter is then **omitted entirely** rather than sent as a
/// union, which would silently drop every type nobody enumerated (see
/// `SearchScope::item_types`).
///
/// TRACES: UR-067 | DR-115, JA-033 | UT-100
fn build_favorites_endpoint(
user_id: &str,
scope: SearchScope,
options: Option<&GetItemsOptions>,
) -> String {
let mut endpoint = format!("/Users/{}/Items?Filters=IsFavorite&Recursive=true", user_id);
if let Some(types) = scope.item_types() {
endpoint.push_str(&format!("&IncludeItemTypes={}", types.join(",")));
}
// Jellyfin has no "date favourited", so name order is the only stable sort
// available; callers may still override it.
let sort_by = options
.and_then(|o| o.sort_by.as_deref())
.unwrap_or("SortName");
let sort_order = options
.and_then(|o| o.sort_order.as_deref())
.unwrap_or("Ascending");
endpoint.push_str(&format!("&SortBy={}&SortOrder={}", sort_by, sort_order));
if let Some(limit) = options.and_then(|o| o.limit) {
endpoint.push_str(&format!("&Limit={}", limit));
}
if let Some(start_index) = options.and_then(|o| o.start_index) {
endpoint.push_str(&format!("&StartIndex={}", start_index));
}
endpoint
.push_str("&Fields=BackdropImageTags,ParentBackdropImageTags,Genres,PremiereDate,UserData");
endpoint
}
// ImageTags from Jellyfin API - can be a HashMap with various image type keys
@@ -653,7 +791,9 @@ impl JellyfinItem {
series_name: self.series_name,
season_id: self.season_id,
season_name: self.season_name,
user_data: None, // User data not included in basic item responses
// Favourite/played/resume state as the server sees it. TRACES:
// UR-069 | DR-113, JA-034
user_data: self.user_data.map(UserData::from),
media_streams: self.media_streams.map(|streams| {
streams
.into_iter()
@@ -728,43 +868,7 @@ impl MediaRepository for OnlineRepository {
parent_id: &str,
options: Option<GetItemsOptions>,
) -> Result<SearchResult, RepoError> {
let mut endpoint = format!("/Users/{}/Items?ParentId={}", self.user_id, parent_id);
if let Some(opts) = options {
if let Some(limit) = opts.limit {
endpoint.push_str(&format!("&Limit={}", limit));
}
if let Some(start_index) = opts.start_index {
endpoint.push_str(&format!("&StartIndex={}", start_index));
}
if let Some(types) = opts.include_item_types {
endpoint.push_str(&format!("&IncludeItemTypes={}", types.join(",")));
}
if let Some(sort_by) = opts.sort_by {
endpoint.push_str(&format!("&SortBy={}", sort_by));
}
if let Some(sort_order) = opts.sort_order {
endpoint.push_str(&format!("&SortOrder={}", sort_order));
}
if let Some(recursive) = opts.recursive {
endpoint.push_str(&format!("&Recursive={}", recursive));
}
if let Some(genres) = opts.genres {
if !genres.is_empty() {
// Genre names may contain spaces/ampersands, so percent-encode each.
let encoded: Vec<String> = genres
.iter()
.map(|g| urlencoding::encode(g).into_owned())
.collect();
endpoint.push_str(&format!("&Genres={}", encoded.join("|")));
}
}
}
// Request image fields for list views (People only needed in get_item
// detail view). Genres is needed so cached items carry their genres,
// which lets the offline store derive genre lists + per-genre counts.
endpoint.push_str("&Fields=BackdropImageTags,ParentBackdropImageTags,Genres,PremiereDate");
let endpoint = build_get_items_endpoint(&self.user_id, parent_id, options.as_ref());
let response: ItemsResponse = self.get_json(&endpoint).await?;
@@ -779,7 +883,7 @@ impl MediaRepository for OnlineRepository {
}
async fn get_item(&self, item_id: &str) -> Result<MediaItem, RepoError> {
let endpoint = format!("/Users/{}/Items/{}?Fields=BackdropImageTags,ParentBackdropImageTags,People,MediaStreams,MediaSources,PremiereDate", self.user_id, item_id);
let endpoint = format!("/Users/{}/Items/{}?Fields=BackdropImageTags,ParentBackdropImageTags,People,MediaStreams,MediaSources,PremiereDate,UserData", self.user_id, item_id);
let item: JellyfinItem = self.get_json(&endpoint).await?;
let media_item = item.to_media_item(self.user_id.clone());
@@ -794,7 +898,7 @@ impl MediaRepository for OnlineRepository {
) -> Result<Vec<MediaItem>, RepoError> {
let limit_str = limit.unwrap_or(16);
let endpoint = format!(
"/Users/{}/Items/Latest?ParentId={}&Limit={}&Fields=BackdropImageTags,ParentBackdropImageTags",
"/Users/{}/Items/Latest?ParentId={}&Limit={}&Fields=BackdropImageTags,ParentBackdropImageTags,UserData",
self.user_id, parent_id, limit_str
);
@@ -812,7 +916,7 @@ impl MediaRepository for OnlineRepository {
) -> Result<Vec<MediaItem>, RepoError> {
let limit_str = limit.unwrap_or(16);
let mut endpoint = format!(
"/Users/{}/Items/Resume?Limit={}&MediaTypes=Video&Fields=BackdropImageTags,ParentBackdropImageTags",
"/Users/{}/Items/Resume?Limit={}&MediaTypes=Video&Fields=BackdropImageTags,ParentBackdropImageTags,UserData",
self.user_id, limit_str
);
@@ -835,7 +939,7 @@ impl MediaRepository for OnlineRepository {
) -> Result<Vec<MediaItem>, RepoError> {
let limit_str = limit.unwrap_or(16);
let mut endpoint = format!(
"/Shows/NextUp?UserId={}&Limit={}&Fields=BackdropImageTags,ParentBackdropImageTags",
"/Shows/NextUp?UserId={}&Limit={}&Fields=BackdropImageTags,ParentBackdropImageTags,UserData",
self.user_id, limit_str
);
@@ -859,7 +963,7 @@ impl MediaRepository for OnlineRepository {
// Fetch more items to account for grouping reducing the count
let fetch_limit = limit_val * 3;
let endpoint = format!(
"/Users/{}/Items?SortBy=DatePlayed&SortOrder=Descending&IncludeItemTypes=Audio&Limit={}&Recursive=true&Filters=IsPlayed&Fields=BackdropImageTags,ParentBackdropImageTags",
"/Users/{}/Items?SortBy=DatePlayed&SortOrder=Descending&IncludeItemTypes=Audio&Limit={}&Recursive=true&Filters=IsPlayed&Fields=BackdropImageTags,ParentBackdropImageTags,UserData",
self.user_id, fetch_limit
);
@@ -993,7 +1097,7 @@ impl MediaRepository for OnlineRepository {
// Filters=IsPlayed keeps only albums the user has actually listened to,
// and SortBy=DatePlayed ascending surfaces the ones they've neglected.
let mut endpoint = format!(
"/Users/{}/Items?SortBy=DatePlayed&SortOrder=Ascending&IncludeItemTypes=MusicAlbum&Limit={}&Recursive=true&Filters=IsPlayed&Fields=BackdropImageTags,ParentBackdropImageTags",
"/Users/{}/Items?SortBy=DatePlayed&SortOrder=Ascending&IncludeItemTypes=MusicAlbum&Limit={}&Recursive=true&Filters=IsPlayed&Fields=BackdropImageTags,ParentBackdropImageTags,UserData",
self.user_id, limit_val
);
@@ -1012,7 +1116,7 @@ impl MediaRepository for OnlineRepository {
async fn get_resume_movies(&self, limit: Option<usize>) -> Result<Vec<MediaItem>, RepoError> {
let limit_str = limit.unwrap_or(16);
let endpoint = format!(
"/Users/{}/Items/Resume?Limit={}&MediaTypes=Video&IncludeItemTypes=Movie&Fields=BackdropImageTags,ParentBackdropImageTags",
"/Users/{}/Items/Resume?Limit={}&MediaTypes=Video&IncludeItemTypes=Movie&Fields=BackdropImageTags,ParentBackdropImageTags,UserData",
self.user_id, limit_str
);
@@ -1113,7 +1217,9 @@ impl MediaRepository for OnlineRepository {
// Request image fields for list views (plus Genres so cached items
// carry genres for offline genre lists/counts).
endpoint.push_str("&Fields=BackdropImageTags,ParentBackdropImageTags,Genres,PremiereDate");
endpoint.push_str(
"&Fields=BackdropImageTags,ParentBackdropImageTags,Genres,PremiereDate,UserData",
);
let response: ItemsResponse = self.get_json(&endpoint).await?;
Ok(SearchResult {
@@ -1654,6 +1760,25 @@ impl MediaRepository for OnlineRepository {
self.post_json(&endpoint, &serde_json::json!({})).await
}
/// TRACES: UR-067 | DR-115, JA-033 | UT-100
async fn get_favorites(
&self,
scope: SearchScope,
options: Option<GetItemsOptions>,
) -> Result<SearchResult, RepoError> {
let endpoint = build_favorites_endpoint(&self.user_id, scope, options.as_ref());
let response: ItemsResponse = self.get_json(&endpoint).await?;
Ok(SearchResult {
items: response
.items
.into_iter()
.map(|item| item.to_media_item(self.user_id.clone()))
.collect(),
total_record_count: response.total_record_count,
})
}
async fn unmark_favorite(&self, item_id: &str) -> Result<(), RepoError> {
let endpoint = format!("/Users/{}/FavoriteItems/{}", self.user_id, item_id);
let url = format!("{}{}", self.server_url, endpoint);
@@ -1747,7 +1872,7 @@ impl MediaRepository for OnlineRepository {
let limit = options.as_ref().and_then(|o| o.limit).unwrap_or(100);
let mut endpoint = format!(
"/Users/{}/Items?PersonIds={}&Limit={}&Recursive=true&Fields=BackdropImageTags,ParentBackdropImageTags",
"/Users/{}/Items?PersonIds={}&Limit={}&Recursive=true&Fields=BackdropImageTags,ParentBackdropImageTags,UserData",
self.user_id, person_id, limit
);
@@ -1781,7 +1906,7 @@ impl MediaRepository for OnlineRepository {
// Try the /Similar endpoint which works for most items
let endpoint = format!(
"/Items/{}/Similar?UserId={}&Limit={}&Fields=BackdropImageTags,ParentBackdropImageTags",
"/Items/{}/Similar?UserId={}&Limit={}&Fields=BackdropImageTags,ParentBackdropImageTags,UserData",
item_id, self.user_id, limit_str
);
@@ -2369,6 +2494,140 @@ mod tests {
);
}
/// UT-100 — the favourites endpoint asks the server for favourites, scoped.
///
/// TRACES: UR-067 | DR-115, JA-033 | UT-100
#[test]
fn test_build_favorites_endpoint_scopes_and_filters() {
let movies = build_favorites_endpoint("u1", SearchScope::Movies, None);
assert!(movies.starts_with("/Users/u1/Items?Filters=IsFavorite&Recursive=true"));
assert!(movies.contains("&IncludeItemTypes=Movie"));
// Jellyfin has no favourite timestamp, so name order is the default.
assert!(movies.contains("&SortBy=SortName&SortOrder=Ascending"));
// Hearts must render on the returned cards.
assert!(movies.contains("UserData"));
// Tv covers both the show and any individually favourited episode.
let tv = build_favorites_endpoint("u1", SearchScope::Tv, None);
assert!(tv.contains("&IncludeItemTypes=Series,Episode"));
let music = build_favorites_endpoint("u1", SearchScope::Music, None);
assert!(music.contains("&IncludeItemTypes=MusicAlbum,MusicArtist,Audio,Playlist"));
}
/// `All` must omit the type filter entirely rather than send a union, which
/// would silently drop every type nobody enumerated.
///
/// TRACES: UR-067 | DR-115 | UT-100
#[test]
fn test_build_favorites_endpoint_all_scope_omits_type_filter() {
let all = build_favorites_endpoint("u1", SearchScope::All, None);
assert!(!all.contains("IncludeItemTypes"));
}
/// Paging and an explicit sort still reach the server.
///
/// TRACES: UR-067 | DR-115 | UT-100
#[test]
fn test_build_favorites_endpoint_honours_paging_and_sort() {
let endpoint = build_favorites_endpoint(
"u1",
SearchScope::All,
Some(&GetItemsOptions {
limit: Some(20),
start_index: Some(40),
sort_by: Some("Random".to_string()),
sort_order: Some("Descending".to_string()),
..Default::default()
}),
);
assert!(endpoint.contains("&Limit=20"));
assert!(endpoint.contains("&StartIndex=40"));
assert!(endpoint.contains("&SortBy=Random&SortOrder=Descending"));
}
/// UT-104 — the in-library favourites toggle reaches the server as
/// `Filters=IsFavorite`, and is absent unless asked for.
///
/// TRACES: UR-067 | DR-116 | UT-104
#[test]
fn test_get_items_endpoint_applies_favorites_only() {
let plain = build_get_items_endpoint("u1", "lib-1", None);
assert!(!plain.contains("Filters=IsFavorite"));
let filtered = build_get_items_endpoint(
"u1",
"lib-1",
Some(&GetItemsOptions {
favorites_only: Some(true),
include_item_types: Some(vec!["Movie".to_string()]),
..Default::default()
}),
);
assert!(filtered.contains("&Filters=IsFavorite"));
// Composes with the filters already there rather than replacing them.
assert!(filtered.contains("&IncludeItemTypes=Movie"));
assert!(filtered.contains("ParentId=lib-1"));
// Explicitly false is not a request to filter.
let off = build_get_items_endpoint(
"u1",
"lib-1",
Some(&GetItemsOptions {
favorites_only: Some(false),
..Default::default()
}),
);
assert!(!off.contains("Filters=IsFavorite"));
}
/// UT-099 — a Jellyfin item's `UserData` reaches `MediaItem.user_data`.
///
/// Before DR-113 this mapping was hardcoded to `None`, so nothing outside
/// the mini player could know an item was favourited.
///
/// TRACES: UR-069 | DR-113, JA-034 | UT-099
#[test]
fn test_jellyfin_item_maps_user_data_favorite() {
let json = r#"{
"Id": "movie123",
"Name": "Test Movie",
"Type": "Movie",
"UserData": {
"PlaybackPositionTicks": 6000000000,
"Played": false,
"IsFavorite": true,
"PlayCount": 2,
"LastPlayedDate": "2026-08-01T12:00:00Z"
}
}"#;
let item: JellyfinItem = serde_json::from_str(json).expect("item should deserialize");
let media = item.to_media_item("server1".to_string());
let user_data = media.user_data.expect("user data should be mapped");
assert_eq!(user_data.is_favorite, Some(true));
assert_eq!(user_data.is_played, Some(false));
assert_eq!(user_data.play_count, Some(2));
assert_eq!(user_data.playback_position_ticks, Some(6_000_000_000));
// Ticks are converted for the frontend, which never divides them itself.
assert_eq!(user_data.playback_position_ms, Some(600_000));
}
/// An item without `UserData` still maps — the field is optional, and every
/// non-user-scoped endpoint omits it.
///
/// TRACES: UR-069 | DR-113 | UT-099
#[test]
fn test_jellyfin_item_without_user_data_maps_to_none() {
let json = r#"{"Id": "x", "Name": "No User Data", "Type": "Movie"}"#;
let item: JellyfinItem = serde_json::from_str(json).expect("item should deserialize");
let media = item.to_media_item("server1".to_string());
assert!(media.user_data.is_none());
}
#[test]
fn test_jellyfin_item_deserialize_with_artist_items() {
// Test that ArtistItems with PascalCase fields deserialize correctly