Compare commits

..
5 Commits
Author SHA1 Message Date
dtourolle 301e6f3828 Release 0.13.3
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m21s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Failing after 39s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m21s
Build and test / Windows (x86_64, cross) (push) Failing after 3m8s
2026-09-20 13:01:41 +02:00
dtourolle a437363bd6 Schema V20: put the mis-spelled run markers right
The markers the previous commit stops writing are already in the
catalogs — 2 on the desktop, 429 on the tablet — and in the shards
both have exchanged. Renaming them to the faces' own id with a fresh
time is what makes the export send each image again, under an entry
newer than the empty one `held_model` would otherwise pick. Where the
old write had inserted its marker beside the right one, the wrong one
goes and the right one is refreshed for the same reason: its entry in
the shards is older than the empty one.

Images V14 left with faces and no marker are not touched. That state is
the quality pass's cue, and the fixed write marks them correctly when
it reaches them.

Checked against copies of both real catalogs: the desktop renames 2,
the tablet deletes 429, both in under 200 ms.
2026-09-20 13:00:59 +02:00
dtourolle 065872bec5 Keep a run marker under the detector that found the faces
`record_updates` — the write behind the quality, eye and crop passes —
re-marked the image as indexed under the pipeline the pass ran as, and
left the faces it had updated under the id of the detector that found
them. On a desktop set to Thorough that put `scrfd_10g+w600k_mbf` over
faces spelled `w600k_mbf`; on the tablet, `scrfd_10g_i8+w600k_mbf` over
faces it had adopted from the desktop's thorough pass.

Every reader takes the marker and the faces to agree. `marker_under`
reads the marker as the detector having examined the image, so the
upgrade repair never revisits it. The shard store keys each face by
its pipeline id, so `export_to_shards` selects an image's faces by the
marker's id, finds none, and sends an entry that says the thorough
detector looked and found nothing — over photographs with named faces
on them. The desktop's shard index holds 54 such entries beside real
faces; the tablet's eye pass over the faces it had adopted made 430
more, and both devices have exchanged them. `held_model` takes the
newest entry for an image, which is the empty one. Nothing has been
lost yet only because the two spellings of the thorough detector rank
equal and neither side adopts the other's; a third device, or either
one after a reinstall, would adopt "nothing here" for 484 images. And
the desktop's eye pass is 4,739 images from doing the same to every
face from before V14 — which are the ones that only exist on the
desktop, and would then never reach anywhere.

The marker now takes the id the faces carry; the pass's own id is used
only when it dropped the last of them and there is no detector left to
name. A stale marker under another spelling of the same embedder is
removed in the same transaction, so one embedder has one marker.
2026-09-20 13:00:58 +02:00
dtourolle 0ed38ada28 Adopt a peer's unmeasured faces instead of refusing them
The tablet showed a fraction of each person: 681 of the desktop's 3,851
confirmations, and none of Ian's 746, Catherine's 626 or my own 480.
Every face that existed on both devices agreed on who it was, and the
people rows were identical — the merge was fine. The missing 3,170
confirmations were on faces the tablet did not hold at all: the
desktop's 16,080 faces from the original detector, on 4,310 images,
detected before schema V14 kept the quality reading.

Those faces were in shards the tablet had already downloaded, in
August's export. `import_from_shards` looked at them on every sync pass
and declined each one, because a face without a quality reading was
"work this device cannot finish": adopting it would write the run
marker, and the marker was what stopped an image being looked at again.
That was true when it was written and has not been since the quality
repair existed — that pass lists its work by `f.quality IS NULL`, not by
the marker, exactly as the eye pass does, and faces without an eye
reading were already adopted on that reasoning.

The refusal had no exit. V14 had deleted the markers of every image
holding such faces so the quality pass would find them, and
`export_to_shards` walks the markers, so the desktop never re-exported
them either; the unmeasured August copies were the only ones there
would ever be. The tablet's answer was to queue all 17,727 images for a
re-detection of its own, a fetch of the whole library, while holding
the faces on disk.

Adopt them. The receiving device's quality pass measures them when it
reaches them, and the desktop's confirmations match onto them by box
overlap on the next catalog merge. The test that asserted the refusal
now asserts the adoption and that the image is still owed to the pass.
2026-09-20 12:58:41 +02:00
dtourolle 695d5ec304 Correct four claims in the README against the tree
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m26s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 28s
Traceability / Requirement traces (push) Failing after 40s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Build and test / Android (aarch64) (push) Failing after 2m21s
Build and test / Windows (x86_64, cross) (push) Failing after 3m5s
Eighteen declared operations, not fifteen; JPEG XL is an export format;
the grid does not filter by keyword, only the catalog's query can; and a
panorama's provenance is a sidecar beside the composite, not a history
step in it.
2026-09-20 11:57:17 +02:00
8 changed files with 371 additions and 85 deletions
Generated
+25 -25
View File
@@ -1221,7 +1221,7 @@ checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f"
[[package]]
name = "darkroom-android"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"android_logger",
"dr-plat",
@@ -1234,7 +1234,7 @@ dependencies = [
[[package]]
name = "darkroom-desktop"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"anyhow",
"dr-plat",
@@ -1408,7 +1408,7 @@ checksum = "d8b14ccef22fc6f5a8f4d7d768562a182c04ce9a3b3157b91390b52ddfdf1a76"
[[package]]
name = "dr-bench"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"anyhow",
"dr-catalog",
@@ -1425,7 +1425,7 @@ dependencies = [
[[package]]
name = "dr-catalog"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"dr-face",
"dr-plat",
@@ -1440,7 +1440,7 @@ dependencies = [
[[package]]
name = "dr-decode"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"dr-types",
"env_logger",
@@ -1454,7 +1454,7 @@ dependencies = [
[[package]]
name = "dr-export"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"dr-decode",
"dr-gpu",
@@ -1473,7 +1473,7 @@ dependencies = [
[[package]]
name = "dr-face"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"dr-inference-engine",
"env_logger",
@@ -1486,7 +1486,7 @@ dependencies = [
[[package]]
name = "dr-film"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"log",
"serde",
@@ -1495,7 +1495,7 @@ dependencies = [
[[package]]
name = "dr-gpu"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"bytemuck",
"dr-decode",
@@ -1513,7 +1513,7 @@ dependencies = [
[[package]]
name = "dr-inference-engine"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"libloading",
"log",
@@ -1527,7 +1527,7 @@ dependencies = [
[[package]]
name = "dr-ingest"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"dr-plat",
"dr-types",
@@ -1539,7 +1539,7 @@ dependencies = [
[[package]]
name = "dr-lens"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"lensfun",
"log",
@@ -1547,7 +1547,7 @@ dependencies = [
[[package]]
name = "dr-pano"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"dr-decode",
"dr-inference-engine",
@@ -1561,7 +1561,7 @@ dependencies = [
[[package]]
name = "dr-pipeline"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"dr-types",
"log",
@@ -1570,7 +1570,7 @@ dependencies = [
[[package]]
name = "dr-plat"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"android-native-keyring-store",
"dr-types",
@@ -1586,7 +1586,7 @@ dependencies = [
[[package]]
name = "dr-preset-xmp"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"dr-pipeline",
"log",
@@ -1596,7 +1596,7 @@ dependencies = [
[[package]]
name = "dr-segment"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"dr-inference-engine",
"env_logger",
@@ -1609,7 +1609,7 @@ dependencies = [
[[package]]
name = "dr-sync"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"async-trait",
"dr-plat",
@@ -1623,7 +1623,7 @@ dependencies = [
[[package]]
name = "dr-sync-folder"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"async-trait",
"dr-sync",
@@ -1635,7 +1635,7 @@ dependencies = [
[[package]]
name = "dr-sync-nextcloud"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"async-trait",
"dr-decode",
@@ -1657,7 +1657,7 @@ dependencies = [
[[package]]
name = "dr-thumbs"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"dr-types",
"jpeg-encoder",
@@ -1669,7 +1669,7 @@ dependencies = [
[[package]]
name = "dr-types"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"serde",
"serde_json",
@@ -1678,7 +1678,7 @@ dependencies = [
[[package]]
name = "dr-ui"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"anyhow",
"async-trait",
@@ -1720,7 +1720,7 @@ dependencies = [
[[package]]
name = "dr-xmp"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"dr-types",
"log",
@@ -7021,7 +7021,7 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "traceability"
version = "0.13.2"
version = "0.13.3"
dependencies = [
"anyhow",
"serde",
+1 -1
View File
@@ -29,7 +29,7 @@ members = [
]
[workspace.package]
version = "0.13.2"
version = "0.13.3"
edition = "2021"
rust-version = "1.92"
license = "GPL-3.0-or-later"
+6 -6
View File
@@ -16,12 +16,12 @@ is still missing.
or one a Nextcloud client keeps in virtual-files mode, where a placeholder
is treated as the photograph rather than as a one-byte file — or at a
Nextcloud account directly. The grid is virtualised, ordered by capture
time with a timeline beside it, and filtered by rating, flag, keyword,
person and whether the file is here. Ratings, keywords, collections and a
time with a timeline beside it, and filtered by rating, flag, person and
whether the file is here. Ratings, keywords, collections and a
trash that survives a crash mid-operation. Card ingest. Bursts fold. Face
detection and identity, with the index syncing between devices.
**Developing.** Fifteen declared operations fused into one compute
**Developing.** Eighteen declared operations fused into one compute
dispatch, plus the neighbourhood work that cannot be: clarity, texture,
capture sharpening, noise reduction, lens correction, spectral film
simulation. Crop and straighten, spot repair, and local adjustments over
@@ -33,11 +33,11 @@ judging what is recoverable. Named presets; XMP sidecars other editors read.
**Panoramas.** Select the frames, align, choose a projection, fill the
ragged border rather than crop it, and the composite lands beside its
sources as a DNG with the merge as the first step in its history.
sources as a DNG, with a sidecar recording what it was merged from.
[![Twelve hand-held frames aligned on a cylinder](docs/manual/media/panorama-aligned.png)](docs/manual/README.md#merging-a-panorama)
**Export.** JPEG, PNG, AVIF, 8- and 16-bit TIFF, with resize, output
**Export.** JPEG, PNG, AVIF, JPEG XL, 8- and 16-bit TIFF, with resize, output
sharpening, a naming template and a colour space — to a folder here or back
into the library.
@@ -76,7 +76,7 @@ controls, its place in the chain and its tests.
## Where it stands
**0.13.2**, sixteen tagged releases in. 184 numbered requirements in
**0.13.3**, seventeen tagged releases in. 184 numbered requirements in
scope, 84% of them claimed by code and [traced to it](docs/traceability.md);
the rest are written down rather than merely absent.
+31 -23
View File
@@ -820,20 +820,22 @@ pub fn import_from_shards(
let Some((faces, edge)) = store.get_image(file_id as u64, &held)? else {
continue;
};
// A peer that embedded before the quality was kept has done work this
// device cannot finish: the number exists only at embedding time, and
// adopting the faces would write the run marker that keeps them from
// ever being measured (schema V14). Left for this device's own pass —
// or for the peer's, whose re-export replaces these.
// A face the peer embedded before its quality was kept (schema V14)
// is adopted with the reading missing, exactly as one without an eye
// reading is. The measuring passes find their work by the NULL
// column, not by the run marker (`dr_ui::repairs`, `faces_needing`),
// so adopting costs the reading nothing and this device's own pass
// fills it.
//
// A missing *eye* reading is not the same case and is adopted. The
// measuring pass finds those by the NULL, not by the marker, so
// adopting the faces costs the reading nothing (schema V16) — and a
// peer that has no eye models may be the only one that has done the
// detection at all.
if faces.iter().any(|f| f.quality.is_none()) {
continue;
}
// This used to refuse such faces, on the reasoning that the marker
// would stop them ever being measured — true before the quality
// repair existed, and wrong after. What it cost: V14 had dropped the
// markers of every image holding such faces, so the peer never
// re-exported them, and the only copies in the shards were the
// unmeasured ones. A tablet holding shards with 4,310 of the
// desktop's images and 3,170 of its confirmations declined every one
// of them, showed a fraction of each person, and queued the whole
// library for a re-detection of its own instead.
let local: Vec<crate::faces::DetectedFace> = faces
.into_iter()
.map(|f| crate::faces::DetectedFace {
@@ -1365,11 +1367,12 @@ mod catalog_round_trip {
);
}
/// A face a peer embedded without measuring it is work this device
/// cannot finish, and adopting it would write the marker that stops it
/// ever being measured. The image stays outstanding instead.
/// A face a peer embedded without measuring it is adopted all the same,
/// and left on this device's quality pass by its missing reading. Refusing
/// it was what stranded every confirmation the desktop had made on faces
/// from before V14: the tablet held the shards and would not use them.
#[test]
fn a_peers_unmeasured_faces_are_left_for_this_device_to_index() {
fn a_peers_unmeasured_faces_are_adopted_and_left_for_the_quality_pass() {
let b = device(&[(90, 5001), (91, 5002)]);
let mut store = FaceShardStore::open(&tempdir("unmeasured")).unwrap();
let shared = |file_id: u64, quality: Option<f32>| SharedFace {
@@ -1395,13 +1398,18 @@ mod catalog_round_trip {
.put_image(5002, "w600k_mbf", 2560, &[shared(5002, Some(19.0))])
.unwrap();
assert_eq!(import_from_shards(&b, &store, "w600k_mbf").unwrap(), 1);
assert_eq!(import_from_shards(&b, &store, "w600k_mbf").unwrap(), 2);
let cov = faces::coverage(&b, "w600k_mbf").unwrap();
assert_eq!(cov.indexed, 1);
assert_eq!(cov.outstanding(), 1, "the unmeasured image was adopted");
assert!(faces::for_image(&b, dr_types::ImageId(90))
.unwrap()
.is_empty());
assert_eq!(cov.indexed, 2);
assert_eq!(cov.outstanding(), 0, "the unmeasured image was refused");
let got = faces::for_image(&b, dr_types::ImageId(90)).unwrap();
assert_eq!(got.len(), 1);
assert_eq!(got[0].quality, None, "a reading was invented");
// Still owed to the measuring pass, which lists by the column.
assert_eq!(
faces::count_needing(&b, "w600k_mbf", "f.quality IS NULL").unwrap(),
1
);
}
#[test]
+110 -5
View File
@@ -556,6 +556,19 @@ impl FaceUpdate {
/// bookkeeping: `face_shard::export_to_shards` re-exports an image whose
/// marker is newer than the store's copy, which is how what was written
/// here reaches the other devices.
///
/// It is re-written under the pipeline id the **faces carry**, not the one
/// this pass ran as. `model_id` names the pass only through its embedder;
/// the detector half of a marker is a statement about who drew the boxes,
/// and this pass drew none. Every reader takes the two to agree: the export
/// selects an image's faces by the marker's id, `marker_under` takes a
/// marker as proof the detector has been over the image, and the shard
/// store keys each face by it. When the marker was written as
/// `scrfd_10g+w600k_mbf` over faces still spelled `w600k_mbf`, the export
/// found no faces under it and sent the other devices an entry saying the
/// thorough detector had looked and found nothing — over photographs with
/// named faces on them. With no faces left, the pass's own id is the only
/// one there is, and the marker says so.
pub fn record_updates(
conn: &Connection,
image_id: ImageId,
@@ -602,13 +615,25 @@ pub fn record_updates(
for f in dropped {
tx.execute("DELETE FROM faces WHERE id = ?1", [f.0 as i64])?;
}
let remaining: i64 = tx.query_row(
let (remaining, found_by): (i64, Option<String>) = tx.query_row(
&format!(
"SELECT COUNT(*) FROM faces WHERE image_id = ?1 AND {} = ?2",
"SELECT COUNT(*), MIN(model_id) FROM faces WHERE image_id = ?1 AND {} = ?2",
embedder_sql("model_id")
),
rusqlite::params![image_id.0 as i64, embedder_of(model_id)],
|r| r.get(0),
|r| Ok((r.get(0)?, r.get(1)?)),
)?;
let marker = found_by.as_deref().unwrap_or(model_id);
// One marker per embedder: a stale one under another spelling would
// keep saying that detector had been here, which is the claim the
// faces' own id is now making in its place.
tx.execute(
&format!(
"DELETE FROM face_index
WHERE image_id = ?1 AND model_id != ?2 AND {} = ?3",
embedder_sql("model_id")
),
rusqlite::params![image_id.0 as i64, marker, embedder_of(model_id)],
)?;
tx.execute(
"INSERT INTO face_index (image_id, model_id, indexed_at, faces_found, source_edge)
@@ -619,7 +644,7 @@ pub fn record_updates(
source_edge = excluded.source_edge",
rusqlite::params![
image_id.0 as i64,
model_id,
marker,
now_secs(),
remaining,
source_edge as i64,
@@ -1557,7 +1582,7 @@ fn iou(a: (f32, f32, f32, f32), b: (f32, f32, f32, f32)) -> f32 {
}
}
fn now_secs() -> i64 {
pub(crate) fn now_secs() -> i64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs() as i64)
@@ -1890,6 +1915,86 @@ mod tests {
assert!(at >= marked_at, "the marker was not refreshed");
}
/// The marker a per-face pass leaves names the detector that drew the
/// boxes, whatever pipeline the pass itself ran as. A marker under the
/// pass's id over faces spelled another way is one the export finds no
/// faces under — and it sent every other device "nothing here".
#[test]
fn an_update_keeps_the_marker_under_the_detector_that_found_the_faces() {
let c = db();
let img = image(&c, 1);
let ids = record_detections(
&c,
img,
"w600k_mbf",
1024,
&[DetectedFace {
quality: None,
..face(1)
}],
)
.unwrap();
// The state V14 leaves: the faces, and no marker at all.
c.execute("DELETE FROM face_index", []).unwrap();
record_updates(
&c,
img,
"scrfd_10g+w600k_mbf",
6000,
&[FaceUpdate {
embedding: Some((vec![9; 1024], 21.5)),
..FaceUpdate::for_face(ids[0])
}],
&[],
)
.unwrap();
let markers: Vec<(String, i64)> = c
.prepare("SELECT model_id, faces_found FROM face_index")
.unwrap()
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))
.unwrap()
.map(Result::unwrap)
.collect();
assert_eq!(markers, vec![("w600k_mbf".to_string(), 1)]);
// A marker already there under the pass's own id is replaced, not
// kept beside the right one.
c.execute(
"INSERT INTO face_index(image_id, model_id, indexed_at, faces_found, source_edge)
VALUES (1, 'scrfd_10g+w600k_mbf', 0, 0, 6000)",
[],
)
.unwrap();
record_updates(
&c,
img,
"scrfd_10g+w600k_mbf",
6000,
&[FaceUpdate {
crop: Some(vec![1, 2, 3]),
..FaceUpdate::for_face(ids[0])
}],
&[],
)
.unwrap();
let n: i64 = c
.query_row("SELECT COUNT(*) FROM face_index", [], |r| r.get(0))
.unwrap();
assert_eq!(n, 1, "a second marker survived");
// With every face dropped there is no detector left to name, and
// the pass's own id records that it looked.
record_updates(&c, img, "scrfd_10g+w600k_mbf", 6000, &[], &ids).unwrap();
let marker: (String, i64) = c
.query_row("SELECT model_id, faces_found FROM face_index", [], |r| {
Ok((r.get(0)?, r.get(1)?))
})
.unwrap();
assert_eq!(marker, ("scrfd_10g+w600k_mbf".to_string(), 0));
}
/// Re-detection is coalesced per image, so it must replace rather than
/// append — otherwise every re-index doubles the library's face count.
#[test]
+174 -1
View File
@@ -15,7 +15,7 @@ use rusqlite::Connection;
use crate::error::CatalogError;
/// Schema version this build writes and understands.
pub const SCHEMA_VERSION: i64 = 19;
pub const SCHEMA_VERSION: i64 = 20;
/// Apply migrations up to [`SCHEMA_VERSION`].
///
@@ -188,9 +188,98 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
tx.commit()?;
}
if from < 20 {
let tx = conn.unchecked_transaction()?;
v20_markers_name_the_detector_that_found_the_faces(&tx)?;
tx.pragma_update(None, "user_version", 20)?;
tx.commit()?;
}
Ok(from)
}
// V20 -- TRACES: FR-CAT-7
//
// Run markers that named the wrong detector, put right.
//
// `faces::record_updates` -- the write behind the quality, eye and crop
// passes -- re-marked an image under the pipeline the pass ran as, while
// the faces it had updated kept the id of the detector that found them.
// A marker of `scrfd_10g+w600k_mbf` over faces spelled `w600k_mbf` reads,
// to every consumer, as the thorough detector having examined the image:
// the upgrade repair skips it, and `face_shard::export_to_shards` selects
// its faces by the marker's id, finds none, and tells every other device
// that the thorough detector found nothing there. The desktop's shard index
// held 54 such entries over photographs with named faces, and the tablet's
// eye pass over faces it had adopted from the desktop had made 430 more.
//
// The write is fixed to keep the marker under the faces' own id. This puts
// the markers already written right, with a fresh time so the export sends
// each image again under an entry newer than the empty one -- which is what
// `held_model` orders by. Where the right marker is still there beside the
// wrong one (the old write inserted rather than replaced), the wrong one
// goes and the right one is refreshed for the same reason: its entry in
// the shards is older than the empty one, and a device that has neither
// would take the empty one. An image V14 left with faces and no marker at
// all is not touched: that state is the quality pass's cue, and the fixed
// write marks it correctly when the pass reaches it.
//
// Restated in Rust rather than SQL because the embedder half of a pipeline
// id is `faces::embedder_sql`, which this must agree with.
fn v20_markers_name_the_detector_that_found_the_faces(tx: &Connection) -> Result<(), CatalogError> {
let fi = crate::faces::embedder_sql("face_index.model_id");
let f = crate::faces::embedder_sql("f.model_id");
// A marker is wrong when the image holds faces of its embedder under
// another id. First the wrong ones that sit beside a right one -- the
// update below would collide with it -- then the rest are renamed.
let wrong = format!(
"EXISTS (SELECT 1 FROM faces f
WHERE f.image_id = face_index.image_id
AND {f} = {fi}
AND f.model_id != face_index.model_id)"
);
let found_by = format!(
"(SELECT MIN(f.model_id) FROM faces f
WHERE f.image_id = face_index.image_id AND {f} = {fi})"
);
let now = crate::faces::now_secs();
tx.execute(
&format!(
"UPDATE face_index
SET indexed_at = ?1
WHERE model_id = {found_by}
AND EXISTS (SELECT 1 FROM face_index w
WHERE w.image_id = face_index.image_id
AND w.model_id != face_index.model_id
AND {} = {fi})",
crate::faces::embedder_sql("w.model_id")
),
[now],
)?;
tx.execute(
&format!(
"DELETE FROM face_index
WHERE {wrong}
AND EXISTS (SELECT 1 FROM face_index o
WHERE o.image_id = face_index.image_id
AND o.model_id = {found_by})"
),
[],
)?;
tx.execute(
&format!(
"UPDATE face_index
SET model_id = {found_by},
faces_found = (SELECT COUNT(*) FROM faces f
WHERE f.image_id = face_index.image_id AND {f} = {fi}),
indexed_at = ?1
WHERE {wrong}"
),
[now],
)?;
Ok(())
}
/// The seven columns V16 adds to `faces`, in the order the readers name them.
///
/// Named once because three places have to agree on them: this migration,
@@ -1854,6 +1943,90 @@ mod tests {
assert_eq!(faces, 2);
}
#[test]
fn v20_renames_markers_to_the_detector_that_found_the_faces() {
let c = mem();
c.pragma_update(None, "user_version", 0).unwrap();
migrate(&c).unwrap();
c.execute(
"INSERT INTO roots(id, kind, label) VALUES (1, 'local', 'test')",
[],
)
.unwrap();
c.execute(
"INSERT INTO images(id, root_id, source_ref, added_at)
VALUES (1,1,'a',0),(2,1,'b',0),(3,1,'c',0),(4,1,'d',0),(5,1,'e',0)",
[],
)
.unwrap();
// 1: the desktop's case -- old faces, re-marked as thorough.
// 2: the tablet's case -- adopted thorough faces, re-marked int8,
// and the right marker still beside it (refreshed, so it is
// exported again over the empty entry).
// 3: right already. 4: examined and empty. 5: V14's state, faces
// and no marker.
for (image, model) in [
(1, "scrfd_10g+w600k_mbf"),
(2, "scrfd_10g_i8+w600k_mbf"),
(2, "scrfd_10g+w600k_mbf"),
(3, "scrfd_10g+w600k_mbf"),
(4, "scrfd_10g+w600k_mbf"),
] {
c.execute(
"INSERT INTO face_index(image_id, model_id, indexed_at, faces_found, source_edge)
VALUES (?1, ?2, 100, 0, 6000)",
rusqlite::params![image, model],
)
.unwrap();
}
for (image, model) in [
(1, "w600k_mbf"),
(1, "w600k_mbf"),
(2, "scrfd_10g+w600k_mbf"),
(3, "scrfd_10g+w600k_mbf"),
(5, "w600k_mbf"),
] {
c.execute(
"INSERT INTO faces
(image_id, x, y, w, h, landmarks, detector_confidence, embedding,
crop_px, model_id, detected_at)
VALUES (?1, 0.1, 0.1, 0.2, 0.2, X'00', 0.9, X'00', 180.0, ?2, 0)",
rusqlite::params![image, model],
)
.unwrap();
}
c.pragma_update(None, "user_version", 19).unwrap();
migrate(&c).unwrap();
let markers: Vec<(i64, String, i64, bool)> = c
.prepare(
"SELECT image_id, model_id, faces_found, indexed_at > 100
FROM face_index ORDER BY image_id, model_id",
)
.unwrap()
.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)))
.unwrap()
.map(Result::unwrap)
.collect();
assert_eq!(
markers,
vec![
(1, "w600k_mbf".to_string(), 2, true),
(2, "scrfd_10g+w600k_mbf".to_string(), 0, true),
(3, "scrfd_10g+w600k_mbf".to_string(), 0, false),
(4, "scrfd_10g+w600k_mbf".to_string(), 0, false),
]
);
// Re-enterable: nothing left to rename.
c.pragma_update(None, "user_version", 19).unwrap();
migrate(&c).unwrap();
let n: i64 = c
.query_row("SELECT count(*) FROM face_index", [], |r| r.get(0))
.unwrap();
assert_eq!(n, 4);
}
#[test]
fn job_uniqueness_coalesces_rather_than_duplicating() {
let c = mem();
+23 -23
View File
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -4,7 +4,7 @@
# makes `makepkg -si` in this directory install what you are actually working
# on. Swap `source` for a tagged tarball when there is something to release.
pkgname=darkroom
pkgver=0.13.2
pkgver=0.13.3
# Back to 1 with the version: a new pkgver is a new archive name, so there is
# nothing for makepkg to reuse and nothing for a release number to disambiguate.
pkgrel=1