Adopt faces a hundred images per commit, and hold one generation per image in the shards
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m16s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 4h42m34s
Build and test / Layer separation (push) Successful in 59s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 50s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Failing after 0s
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m16s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 4h42m34s
Build and test / Layer separation (push) Successful in 59s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 50s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Failing after 0s
The shard import recorded each adopted image in its own transaction: fourteen thousand commits, and fourteen thousand turns at the write lock that every read on the UI thread queued behind — the sync was felt as a laggy grid and as "database is locked" from whichever writer lost the wait. `record_detections_within` takes the caller's transaction, and the import commits every hundred images. The store carried every detector generation of an image — 24,123 entries for 19,089 images on the reference library, a third of its 293 MB — when only the strongest is ever adopted. A put now skips a pass a held one outranks, and retires the passes it outranks from the index; sealed shards keep their bytes, but nothing is written twice from here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -279,7 +279,25 @@ pub fn record_detections(
|
||||
faces: &[DetectedFace],
|
||||
) -> Result<Vec<FaceId>, CatalogError> {
|
||||
let tx = conn.unchecked_transaction()?;
|
||||
let ids = record_detections_within(&tx, image_id, model_id, source_edge, faces)?;
|
||||
tx.commit()?;
|
||||
Ok(ids)
|
||||
}
|
||||
|
||||
/// [`record_detections`] inside a transaction the caller owns.
|
||||
///
|
||||
/// For a caller recording many images at once — the shard import adopts
|
||||
/// fourteen thousand in one pass — where a commit per image is fourteen
|
||||
/// thousand fsyncs and fourteen thousand turns at the write lock that every
|
||||
/// read on the UI thread queues behind. `unchecked_transaction` cannot nest,
|
||||
/// so the batching has to be offered here rather than wrapped from above.
|
||||
pub fn record_detections_within(
|
||||
tx: &Connection,
|
||||
image_id: ImageId,
|
||||
model_id: &str,
|
||||
source_edge: u32,
|
||||
faces: &[DetectedFace],
|
||||
) -> Result<Vec<FaceId>, CatalogError> {
|
||||
// Everything the old faces knew, so it can be carried across the
|
||||
// replacement. Read only when there is something to carry it onto: a
|
||||
// pass that found nothing has nothing to match, and decoding a vector
|
||||
@@ -287,7 +305,7 @@ pub fn record_detections(
|
||||
let prior = if faces.is_empty() {
|
||||
Vec::new()
|
||||
} else {
|
||||
read_priors(&tx, image_id)?
|
||||
read_priors(tx, image_id)?
|
||||
};
|
||||
|
||||
tx.execute("DELETE FROM faces WHERE image_id = ?1", [image_id.0 as i64])?;
|
||||
@@ -389,7 +407,6 @@ pub fn record_detections(
|
||||
],
|
||||
)?;
|
||||
|
||||
tx.commit()?;
|
||||
Ok(ids)
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user