Count the outstanding repairs from the faces, on partial indexes
"How many images still owe a quality reading" was a correlated EXISTS per image over `faces`, and the face row is 8 KB of embedding and crop before the column it looks at, so each count opened every row. Six such counts run on every open of the Identity screen and at the end of every sweep: 160 ms on the reference library. V19 adds three partial indexes holding only the faces still owing each pass, keyed on the image and carrying the model id the predicate reads, and replaces `faces_image` with `(image_id, model_id)` so "does this image hold this embedder's faces" is answered from the index too. The planner takes a partial index when the count is driven from `faces` and ignores it inside the EXISTS, so `Needs::Face` carries the per-face fragment and `repairs::count` spells the query from the faces' side; the list and the per-image check keep the EXISTS. A test holds the two spellings to the same answer for every repair.
This commit is contained in:
@@ -15,7 +15,7 @@ use rusqlite::Connection;
|
||||
use crate::error::CatalogError;
|
||||
|
||||
/// Schema version this build writes and understands.
|
||||
pub const SCHEMA_VERSION: i64 = 18;
|
||||
pub const SCHEMA_VERSION: i64 = 19;
|
||||
|
||||
/// Apply migrations up to [`SCHEMA_VERSION`].
|
||||
///
|
||||
@@ -181,6 +181,13 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
|
||||
tx.commit()?;
|
||||
}
|
||||
|
||||
if from < 19 {
|
||||
let tx = conn.unchecked_transaction()?;
|
||||
tx.execute_batch(V19)?;
|
||||
tx.pragma_update(None, "user_version", 19)?;
|
||||
tx.commit()?;
|
||||
}
|
||||
|
||||
Ok(from)
|
||||
}
|
||||
|
||||
@@ -766,6 +773,44 @@ CREATE TABLE IF NOT EXISTS xmp_conflicts (
|
||||
);
|
||||
"#;
|
||||
|
||||
// V19 -- TRACES: NFR-P9
|
||||
//
|
||||
// The indexes the repair counts are served from, and V17's lesson applied
|
||||
// to the rest of the face columns.
|
||||
//
|
||||
// "How many images still owe a quality reading" was answered per image: a
|
||||
// correlated EXISTS over `faces` that had to open each face's row to look
|
||||
// at one nullable column -- the row being eight kilobytes of embedding and
|
||||
// crop. Six such counts run every time the Identity screen opens and every
|
||||
// time a sweep ends, 160 ms of them on the reference library. Three
|
||||
// partial indexes hold only the faces still owing each pass, keyed by the
|
||||
// image and carrying the model id the predicate also reads, so the count
|
||||
// walks a few thousand index entries and touches no row at all -- and each
|
||||
// index shrinks to nothing as its pass completes. The planner takes them
|
||||
// when the count is driven from `faces` (`repairs::count`) and ignores
|
||||
// them inside the per-image EXISTS, which is why that function has two
|
||||
// spellings of the same predicate.
|
||||
//
|
||||
// `faces_image_model` replaces `faces_image`: the same key with the model
|
||||
// id beside it, so "does this image hold this embedder's faces" -- asked in
|
||||
// the audit, the proxy repair and the outstanding-detection count -- is an
|
||||
// index-only probe where it used to read the row for the model id. Every
|
||||
// lookup that used `faces_image` is served by its prefix.
|
||||
//
|
||||
// Not applied to attached catalogs, like V7 and V17: an index is a local
|
||||
// concern, and a merge never runs these queries across an attachment.
|
||||
|
||||
const V19: &str = r#"
|
||||
CREATE INDEX IF NOT EXISTS faces_image_model ON faces(image_id, model_id);
|
||||
DROP INDEX IF EXISTS faces_image;
|
||||
CREATE INDEX IF NOT EXISTS faces_owed_quality ON faces(image_id, model_id)
|
||||
WHERE quality IS NULL;
|
||||
CREATE INDEX IF NOT EXISTS faces_owed_crop ON faces(image_id, model_id)
|
||||
WHERE crop IS NULL;
|
||||
CREATE INDEX IF NOT EXISTS faces_owed_eyes ON faces(image_id, model_id)
|
||||
WHERE eye_right IS NULL OR landmarks_dense IS NULL;
|
||||
"#;
|
||||
|
||||
// V18 -- TRACES: FR-CULL-8a | FR-CULL-12
|
||||
//
|
||||
// The 106 dense landmarks the eye pass reads its eye boxes from, kept beside
|
||||
|
||||
Reference in New Issue
Block a user