Develop opens the catalog up to five times per photograph, and each open runs the backfill #72

New Issue

2026-09-26T14:43:21Z

dtourolle commented

2026-09-26 14:43:21 +00:00

Landing on a photograph in develop runs Catalog::open up to five times: once in fetch_original, and once in holds_original (ui/dr-ui/src/library/thumbnails_fetch.rs) for each neighbour it prefetches. That function's own comment calls it "a row check".

Why

Every open runs the backfill: default versions, UUID alignment, RAW/JPEG pairing, orphaned keywords. v0.16.0 halved an open (26 → 12 ms), so a landing is now ~60 ms of CPU where it was ~130 ms. That is still ~60 ms of CPU spent on work the develop view never needed. ensure_default_versions (4–9 ms) and align_default_version_uuids (2–5 ms) still scan on every open, and the second could use a partial index.

Deliverable

The prefetch path asks its row question on a connection it already holds, or on one opened without the backfill.
The backfill runs once per process or catalog generation, not once per worker open.

Acceptance

One develop landing, measured on a copy of the reference catalog: number of opens and CPU, before and after
The backfill still runs on the first open after a migration or a pulled catalog (a test)

**Landing on a photograph in develop runs `Catalog::open` up to five times:** once in `fetch_original`, and once in `holds_original` (`ui/dr-ui/src/library/thumbnails_fetch.rs`) for each neighbour it prefetches. That function's own comment calls it "a row check". ## Why Every open runs the backfill: default versions, UUID alignment, RAW/JPEG pairing, orphaned keywords. v0.16.0 halved an open (26 → 12 ms), so a landing is now ~60 ms of CPU where it was ~130 ms. That is still ~60 ms of CPU spent on work the develop view never needed. `ensure_default_versions` (4–9 ms) and `align_default_version_uuids` (2–5 ms) still scan on every open, and the second could use a partial index. ## Deliverable - The prefetch path asks its row question on a connection it already holds, or on one opened without the backfill. - The backfill runs once per process or catalog generation, not once per worker open. ## Acceptance - [ ] One develop landing, measured on a copy of the reference catalog: number of opens and CPU, before and after - [ ] The backfill still runs on the first open after a migration or a pulled catalog (a test)

dtourolle added the develop catalog size:S performance labels 2026-09-26 14:43:21 +00:00

dtourolle commented

2026-09-26 20:20:16 +00:00

Done in 2226d54, ffdd640, faf52f6 and 02ddce8, released in v0.17.0. Catalog::open runs the backfill once per catalog state rather than on every open. The state is a stamp of the newest rows of images, versions and keywords (content as well as ids, since rowids are reused after a delete), plus the file's identity and user_version. A migration, a pull, or a restore or rebuild always backfills. The prefetch asks its cache questions on one held connection. On the reference catalog: one open ~17 ms → ~0 ms, a develop landing ~80 ms → ~2 ms CPU. No schema change.

Done in 2226d54, ffdd640, faf52f6 and 02ddce8, released in v0.17.0. `Catalog::open` runs the backfill once per catalog state rather than on every open. The state is a stamp of the newest rows of `images`, `versions` and `keywords` (content as well as ids, since rowids are reused after a delete), plus the file's identity and `user_version`. A migration, a pull, or a restore or rebuild always backfills. The prefetch asks its cache questions on one held connection. On the reference catalog: one open ~17 ms → ~0 ms, a develop landing ~80 ms → ~2 ms CPU. No schema change.

dtourolle closed this issue

2026-09-26 20:20:17 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dtourolle/DarkRoom#72