Page the grid along an index instead of sorting the library each time
Scrolling jittered, and this was the largest single reason. Every window the grid loads is `ORDER BY ... LIMIT n OFFSET k`, and neither half of that was being answered the cheap way. **The sort.** `GRID_ORDER` leads with `captured_at IS NULL`, so undated frames fall to the end. No ordinary index answers that — the leading term is an expression, not a column — so SQLite sorted the whole library into a temp b-tree on every window read, then threw away the first `k` rows of it. Schema V7 indexes the expression exactly as the query writes it, partial on the same `shadowed_by IS NULL AND trashed_at IS NULL` the grid filters by, so the read becomes a walk along the index. **The join.** `LEFT JOIN remote` was paged *after* it was joined, so reading 280 cells at offset 20,000 first seeked into `remote` for all 24,000 rows and then discarded 23,720 of them. The file ids are now fetched for the 280 rows that survived — the shape the badge and rating reads already use, one query for the window rather than one per cell. Measured together on 24,000 images at offset 20,000: **15.2 ms → 0.36 ms**, inside a scroll handler that has 16.7 ms to draw a frame. The test asserts on the query plan rather than on a duration, because there is no other symptom. A `GRID_ORDER` edited out of step with the index, or a column added back that drags `remote` in again, both still return exactly the right cells — just after sorting the library — and the jitter would come back with nothing to point at. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -15,7 +15,7 @@ use rusqlite::Connection;
|
||||
use crate::error::CatalogError;
|
||||
|
||||
/// Schema version this build writes and understands.
|
||||
pub const SCHEMA_VERSION: i64 = 6;
|
||||
pub const SCHEMA_VERSION: i64 = 7;
|
||||
|
||||
/// Apply migrations up to [`SCHEMA_VERSION`].
|
||||
///
|
||||
@@ -72,6 +72,12 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
|
||||
tx.pragma_update(None, "user_version", 6)?;
|
||||
tx.commit()?;
|
||||
}
|
||||
if from < 7 {
|
||||
let tx = conn.unchecked_transaction()?;
|
||||
tx.execute_batch(V7)?;
|
||||
tx.pragma_update(None, "user_version", 7)?;
|
||||
tx.commit()?;
|
||||
}
|
||||
|
||||
Ok(from)
|
||||
}
|
||||
@@ -173,6 +179,13 @@ pub fn v1_for_attached(schema_name: &str) -> String {
|
||||
/// `ALTER TABLE ... ADD COLUMN`, and the columns they add are local index
|
||||
/// state — shadowing, trashing, cache pinning — that a merge never reads
|
||||
/// across the attachment.
|
||||
///
|
||||
/// V7 creates an object and is still excluded, which is the one exception to
|
||||
/// that rule and not an oversight: it indexes `shadowed_by` and `trashed_at`,
|
||||
/// the very columns V2 through V5 add and this function leaves out, so
|
||||
/// creating it over there would fail on columns that are not there. Nothing is
|
||||
/// lost by its absence — it exists to make the *grid* page quickly, and the
|
||||
/// grid never reads across an attachment.
|
||||
pub fn for_attached(schema_name: &str) -> String {
|
||||
format!(
|
||||
"{}\n{}",
|
||||
@@ -276,6 +289,50 @@ fn stem_of(path: &str) -> &str {
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-CAT-4 | NFR-P5
|
||||
/// The order the grid reads in, as an index.
|
||||
///
|
||||
/// # What this is for
|
||||
///
|
||||
/// Every window the grid loads is `ORDER BY ... LIMIT n OFFSET k`, and without
|
||||
/// an index that matches the ordering SQLite answers it by sorting the whole
|
||||
/// library into a temp b-tree and then discarding the first `k` rows. Measured
|
||||
/// on 24,000 images at offset 20,000, one window read cost 15 ms — a frame
|
||||
/// budget of 16.7 ms, spent inside the scroll handler, several times per
|
||||
/// screenful. That is the jitter.
|
||||
///
|
||||
/// With this index the same read is a walk along it: 0.36 ms.
|
||||
///
|
||||
/// # Why the shape is what it is
|
||||
///
|
||||
/// `captured_at IS NULL` leads, because [`crate::library`]'s `GRID_ORDER` does
|
||||
/// — undated images sort last, and an ordinary index on `captured_at` cannot
|
||||
/// answer that, since the expression is not a column. SQLite indexes
|
||||
/// expressions, so it is spelled out here exactly as the query spells it; the
|
||||
/// two must stay identical or the planner silently falls back to sorting and
|
||||
/// the cost comes back with no other symptom.
|
||||
///
|
||||
/// `source_ref` is included because it breaks ties in the same ordering, and an
|
||||
/// index that stopped at `captured_at` would leave a sort for the ties.
|
||||
///
|
||||
/// # Partial, on the same predicate the grid filters by
|
||||
///
|
||||
/// The grid never lists shadowed or trashed rows, so an index carrying them
|
||||
/// would be larger than the question ever asks about, and — more to the point —
|
||||
/// a partial index is only usable when its `WHERE` is implied by the query's,
|
||||
/// which is what makes this one apply to the grid's reads and to nothing else.
|
||||
///
|
||||
/// # It does not cover the rating filter or a collection scope
|
||||
///
|
||||
/// Both narrow the walk rather than reorder it, so the index still supplies the
|
||||
/// ordering and SQLite tests the extra predicate per row. That is the cheap
|
||||
/// direction: the expensive part was never the filtering, it was the sort.
|
||||
const V7: &str = r#"
|
||||
CREATE INDEX images_grid_order
|
||||
ON images(captured_at IS NULL, captured_at, source_ref)
|
||||
WHERE shadowed_by IS NULL AND trashed_at IS NULL;
|
||||
"#;
|
||||
|
||||
const V6: &str = r#"
|
||||
-- TRACES: FR-CAT-5 | FR-CAT-6 | FR-NC-9
|
||||
-- Keywords gain an identity, so that renaming and deleting one can cross
|
||||
|
||||
Reference in New Issue
Block a user