Page the grid along an index instead of sorting the library each time

Scrolling jittered, and this was the largest single reason. Every window the
grid loads is `ORDER BY ... LIMIT n OFFSET k`, and neither half of that was
being answered the cheap way.

**The sort.** `GRID_ORDER` leads with `captured_at IS NULL`, so undated frames
fall to the end. No ordinary index answers that — the leading term is an
expression, not a column — so SQLite sorted the whole library into a temp
b-tree on every window read, then threw away the first `k` rows of it. Schema
V7 indexes the expression exactly as the query writes it, partial on the same
`shadowed_by IS NULL AND trashed_at IS NULL` the grid filters by, so the read
becomes a walk along the index.

**The join.** `LEFT JOIN remote` was paged *after* it was joined, so reading
280 cells at offset 20,000 first seeked into `remote` for all 24,000 rows and
then discarded 23,720 of them. The file ids are now fetched for the 280 rows
that survived — the shape the badge and rating reads already use, one query for
the window rather than one per cell.

Measured together on 24,000 images at offset 20,000: **15.2 ms → 0.36 ms**,
inside a scroll handler that has 16.7 ms to draw a frame.

The test asserts on the query plan rather than on a duration, because there is
no other symptom. A `GRID_ORDER` edited out of step with the index, or a column
added back that drags `remote` in again, both still return exactly the right
cells — just after sorting the library — and the jitter would come back with
nothing to point at.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-25 20:01:04 +02:00
co-authored by Claude Opus 5
parent 940058c78a
commit 6d6ef8d34b
3 changed files with 254 additions and 64 deletions
+58 -1
View File
@@ -15,7 +15,7 @@ use rusqlite::Connection;
use crate::error::CatalogError;
/// Schema version this build writes and understands.
pub const SCHEMA_VERSION: i64 = 6;
pub const SCHEMA_VERSION: i64 = 7;
/// Apply migrations up to [`SCHEMA_VERSION`].
///
@@ -72,6 +72,12 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
tx.pragma_update(None, "user_version", 6)?;
tx.commit()?;
}
if from < 7 {
let tx = conn.unchecked_transaction()?;
tx.execute_batch(V7)?;
tx.pragma_update(None, "user_version", 7)?;
tx.commit()?;
}
Ok(from)
}
@@ -173,6 +179,13 @@ pub fn v1_for_attached(schema_name: &str) -> String {
/// `ALTER TABLE ... ADD COLUMN`, and the columns they add are local index
/// state — shadowing, trashing, cache pinning — that a merge never reads
/// across the attachment.
///
/// V7 creates an object and is still excluded, which is the one exception to
/// that rule and not an oversight: it indexes `shadowed_by` and `trashed_at`,
/// the very columns V2 through V5 add and this function leaves out, so
/// creating it over there would fail on columns that are not there. Nothing is
/// lost by its absence — it exists to make the *grid* page quickly, and the
/// grid never reads across an attachment.
pub fn for_attached(schema_name: &str) -> String {
format!(
"{}\n{}",
@@ -276,6 +289,50 @@ fn stem_of(path: &str) -> &str {
}
}
/// TRACES: FR-CAT-4 | NFR-P5
/// The order the grid reads in, as an index.
///
/// # What this is for
///
/// Every window the grid loads is `ORDER BY ... LIMIT n OFFSET k`, and without
/// an index that matches the ordering SQLite answers it by sorting the whole
/// library into a temp b-tree and then discarding the first `k` rows. Measured
/// on 24,000 images at offset 20,000, one window read cost 15 ms — a frame
/// budget of 16.7 ms, spent inside the scroll handler, several times per
/// screenful. That is the jitter.
///
/// With this index the same read is a walk along it: 0.36 ms.
///
/// # Why the shape is what it is
///
/// `captured_at IS NULL` leads, because [`crate::library`]'s `GRID_ORDER` does
/// — undated images sort last, and an ordinary index on `captured_at` cannot
/// answer that, since the expression is not a column. SQLite indexes
/// expressions, so it is spelled out here exactly as the query spells it; the
/// two must stay identical or the planner silently falls back to sorting and
/// the cost comes back with no other symptom.
///
/// `source_ref` is included because it breaks ties in the same ordering, and an
/// index that stopped at `captured_at` would leave a sort for the ties.
///
/// # Partial, on the same predicate the grid filters by
///
/// The grid never lists shadowed or trashed rows, so an index carrying them
/// would be larger than the question ever asks about, and — more to the point —
/// a partial index is only usable when its `WHERE` is implied by the query's,
/// which is what makes this one apply to the grid's reads and to nothing else.
///
/// # It does not cover the rating filter or a collection scope
///
/// Both narrow the walk rather than reorder it, so the index still supplies the
/// ordering and SQLite tests the extra predicate per row. That is the cheap
/// direction: the expensive part was never the filtering, it was the sort.
const V7: &str = r#"
CREATE INDEX images_grid_order
ON images(captured_at IS NULL, captured_at, source_ref)
WHERE shadowed_by IS NULL AND trashed_at IS NULL;
"#;
const V6: &str = r#"
-- TRACES: FR-CAT-5 | FR-CAT-6 | FR-NC-9
-- Keywords gain an identity, so that renaming and deleting one can cross