Say what the face sync is doing while it does it

The face pass set the status to "checking faces…" once and then said nothing
until it was finished. On a library whose first export after a re-index is 9,849
photographs and 85 MB of shards, that is eight minutes of a progress bar sitting
still — which is indistinguishable from a hang, and was reported as one twice.

Nothing was wrong with the sync. The only fault was that it was silent.

Three places now report, which are the three that take real time:

- **Preparing**, per image with a count, since this is the long one and the only
  one whose length the user cannot guess from anything on screen.
- **Sending**, per shard with its size, because a face shard carries crops and
  runs to tens of megabytes — one of them is a visible wait on any connection.
  Announced before the upload rather than after, since the wait *is* the upload.
- **Taking in** a peer's shard, which is a download and then a row-by-row merge.

The export reports every 25 images rather than every one, so the channel behind
it stays lost in the write it accompanies. `export_to_shards` keeps its old
signature and delegates, so the callers that do not want progress do not grow a
parameter for it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-28 21:56:16 +02:00
co-authored by Claude Opus 5
parent 35febf31dd
commit a1e8f494b9
2 changed files with 60 additions and 4 deletions
+33 -3
View File
@@ -158,7 +158,7 @@ async fn run(
sync_shards(backend, &base, thumbs_dir, scratch, &mut report).await?;
let _ = tx.send(SyncMessage::Status("checking faces…".into()));
sync_face_shards(backend, &base, catalog_path, scratch, &mut report).await?;
sync_face_shards(backend, &base, catalog_path, scratch, &mut report, tx).await?;
let _ = tx.send(SyncMessage::Status("checking collections…".into()));
sync_catalog(backend, &base, catalog_path, scratch, &mut report).await?;
@@ -328,6 +328,7 @@ async fn sync_face_shards(
catalog_path: &Path,
scratch: &Path,
report: &mut SyncReport,
tx: &std::sync::mpsc::Sender<SyncMessage>,
) -> Result<(), String> {
use dr_catalog::face_shard::{self, FaceShardStore};
@@ -351,7 +352,21 @@ async fn sync_face_shards(
// ---- everything this device has detected, into the shards ------------
if let Ok(catalog) = dr_catalog::Catalog::open(catalog_path) {
match face_shard::export_to_shards(catalog.connection(), &mut store, model) {
// The first export after a re-index walks the whole library and writes
// thousands of images. Saying so is the difference between a sync that
// looks slow and one that looks broken — this pass sat on "checking
// faces…" for eight minutes and was reported as a hang.
let mut say = |done: usize, total: usize| {
let _ = tx.send(SyncMessage::Status(format!(
"preparing faces to send: {done}/{total}"
)));
};
match face_shard::export_to_shards_reporting(
catalog.connection(),
&mut store,
model,
&mut say,
) {
Ok(0) => {}
Ok(n) => log::info!("face sync: {n} newly indexed image(s) ready to upload"),
Err(e) => log::warn!("face sync: exporting to shards: {e}"),
@@ -377,12 +392,21 @@ async fn sync_face_shards(
// ---- upload ----------------------------------------------------------
let local = store.shards().map_err(|e| e.to_string())?;
for shard in &local {
for (n, shard) in local.iter().enumerate() {
let path = store.shard_path(shard.id);
let Ok(bytes) = std::fs::read(&path) else {
continue;
};
let name = shard_name(&client, shard.id);
// Face shards carry crops and run to tens of megabytes each, so a
// single one is a visible wait on any connection. Announced before the
// put rather than after, because the wait is the upload.
let _ = tx.send(SyncMessage::Status(format!(
"sending faces: shard {}/{} ({} MB)",
n + 1,
local.len(),
bytes.len() / 1_048_576
)));
// Sealed and present means byte-identical, and the client is in the
// name so nobody else could have written it. The open shard goes up
@@ -411,6 +435,12 @@ async fn sync_face_shards(
if owner == client || store.has_adopted(name, *size) {
continue;
}
// Taking in a peer's shard means a download and then a row-by-row
// merge, both of which take a while on a shard carrying crops.
let _ = tx.send(SyncMessage::Status(format!(
"taking in faces from another device ({} MB)",
size / 1_048_576
)));
let source = RemotePath::new(format!("{}/{name}", face_base.as_str()));
let bytes = match backend.get(&RemoteId::Path(source), None).await {