Read the sidecars other editors write, and write them back on request

FR-CAT-13 asked for standard XMP and `core/dr-xmp` answered the file: it
has read and written `dc:subject`, `xmp:Rating`, `xmp:Label` and the IPTC
core since 5fa4c07, under an ownership rule that leaves everything else in
the document untouched. What nothing did was call it. No scan found an
`.xmp` beside a raw, no catalog row was filled from one, no judgement
wrote one back, and the "external modification detected, reload offered"
clause had no mechanism. A library imported from Lightroom came in and
could not go back out.

The scan collects `.xmp` beside `.drsc` from the listings it was already
paying for, and the pull reads each one whose ETag has moved. Both
namings resolve: darktable's `IMG_0001.CR3.xmp` names its file exactly,
Lightroom's `IMG_0001.xmp` names the stem, and under the stem the JPEG
beside a RAW is the same photograph and takes the same document, as
DarkRoom's own sidecar already does. Each is reconciled with the catalog
winning — keywords union, a rating or label taken only where the catalog
has none — because a standard XMP carries nothing that could say whether
its value is newer. A genuine disagreement is not resolved; it is written
to a table, and the settings page offers the sidecars' values against it.
That button is the reload the requirement asks to be offered, and the
ETag that moved is the detection it asks for: an `.xmp` edited elsewhere
is exactly a file the pull's ordinary incrementality re-reads.

Writing goes the other way behind a setting that starts off, since NFR-R4
makes writes beside somebody's originals theirs to switch on. With it on,
a judgement or a keyword rewrites the sidecar of whichever spelling
exists, or creates Lightroom's. The record is read from the catalog
whole at that moment rather than carried from the gesture, so a rating
and a keyword a second apart are two writes of one file that agree. And
the file's own title, caption, copyright and hierarchy come through the
rewrite: the catalog has no columns for them, `rewrite` replaces the
owned set wholesale, and a record that said nothing about them would have
deleted them from a Lightroom sidecar on every star.

The rating's two axes cross the format's one field both ways: a
rejection is Adobe's `-1` and stars are stars, and stars arriving on a
rejected frame lift the rejection, since the file said it was worth a
number. An unrated file says nothing and clears nothing, on the rule the
`.drsc` merge keeps. `versions.label` finally has a reader and a writer,
with the code table moved out of the query so the two cannot drift.

ui/dr-ui/src/lib.rs

@@ -59,6 +59,7 @@ mod settings_ui;
 mod sidecar_cache;
 mod spots_ui;
 mod trash;
+mod xmp_sync;
 
 use std::cell::{Cell, RefCell};
 use std::path::{Path, PathBuf};
@@ -1583,6 +1584,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
             let stored = settings.snapshot();
             library.set_cache_budget(stored.cache.original_budget_bytes);
             library.set_keep_opened_originals(stored.cache.keep_opened_originals);
+            library.set_write_xmp_sidecars(stored.library.write_xmp_sidecars);
             library.set_timeline_bars(stored.library.timeline_bars);
             library.set_face_model_id(stored.faces.detector.model_id());
         }
@@ -1714,6 +1716,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
                 // cache would sit over budget indefinitely.
                 lib.set_cache_budget(s.cache.original_budget_bytes);
                 lib.set_keep_opened_originals(s.cache.keep_opened_originals);
+                lib.set_write_xmp_sidecars(s.library.write_xmp_sidecars);
                 if let Some(w) = weak.upgrade() {
                     refresh_export_label(&w, &ctl);
                 }

ui/dr-ui/src/library.rs

@@ -1498,6 +1498,34 @@ async fn pull_sidecars(
         };
 
         let text = String::from_utf8_lossy(&bytes);
+
+        // TRACES: FR-CAT-13
+        // A standard XMP beside the photograph — Lightroom's, darktable's,
+        // anybody's — takes the other branch: reconciled field by field with
+        // the catalog winning, and a disagreement recorded for the reload
+        // the requirement asks to be offered. See `xmp_sync`.
+        if crate::xmp_sync::is_xmp(path) {
+            match crate::xmp_sync::take_in(conn, root_id, path, &text, now_secs()) {
+                Ok(taken) => {
+                    applied += taken.changed;
+                    if !taken.conflicts.is_empty() {
+                        log::info!(
+                            "xmp sidecar {path} disagrees with the catalog on {:?}; \
+                             a reload is offered in Settings",
+                            taken.conflicts
+                        );
+                    }
+                    // Recorded only once it reached a photograph: a sidecar
+                    // that arrived before its image is read again next time.
+                    if taken.described > 0 {
+                        record_sidecar_read(conn, root_id, path, &entry.validator);
+                    }
+                }
+                Err(e) => log::warn!("xmp sidecar at {path} is unreadable ({e})"),
+            }
+            continue;
+        }
+
         let mut sidecar = match dr_pipeline::Sidecar::parse(&text) {
             Ok(s) => s,
             // Unreadable is not empty. Recording the ETag would mean never
@@ -1534,6 +1562,191 @@ async fn pull_sidecars(
     Ok(applied)
 }
 
+/// TRACES: FR-CAT-13 | NFR-R4
+/// One image's ratings, label and keywords, on their way to the `.xmp`
+/// beside it.
+///
+/// Distinct from [`SidecarWrite`], which amends DarkRoom's own document
+/// through the cache and the outbox. This is best-effort in the other
+/// direction: the catalog and the `.drsc` are authoritative, the `.xmp` is a
+/// courtesy to whatever else reads the folder, and a write that cannot
+/// happen now is written again — from the catalog, whole — the next time
+/// anything about the photograph is judged. So nothing is queued.
+#[derive(Debug, Clone)]
+pub struct XmpWrite {
+    pub image_path: String,
+    pub record: dr_xmp::Xmp,
+}
+
+/// TRACES: FR-CAT-13 | NFR-R4
+/// Write each record into the XMP sidecar beside its image.
+///
+/// An existing sidecar of either spelling is rewritten in place, which is the
+/// whole point of `dr_xmp::rewrite`: only the properties DarkRoom owns move,
+/// and another application's settings, comments and namespaces come through
+/// byte for byte. A photograph with neither gets a new file under Lightroom's
+/// name. Reported once at the end, as the judgement writes are.
+pub fn spawn_xmp_writes(conn: Connection, writes: Vec<XmpWrite>) -> Receiver<XmpMessage> {
+    let (tx, rx) = std::sync::mpsc::channel();
+    std::thread::spawn(move || {
+        let mut written = 0usize;
+        let mut failed = 0usize;
+        let mut last_error = None;
+        let rt = match crate::net_runtime::build() {
+            Ok(rt) => rt,
+            Err(e) => {
+                let _ = tx.send(XmpMessage::Finished {
+                    written: 0,
+                    failed: writes.len(),
+                    last_error: Some(e.to_string()),
+                });
+                return;
+            }
+        };
+        rt.block_on(async {
+            let backend = match crate::remote::connect(&conn) {
+                Ok(b) => b,
+                Err(e) => {
+                    failed = writes.len();
+                    last_error = Some(e.to_string());
+                    return;
+                }
+            };
+            for w in &writes {
+                match write_one_xmp(&*backend, w).await {
+                    Ok(()) => written += 1,
+                    Err(e) => {
+                        log::warn!("xmp sidecar for {}: {e}", w.image_path);
+                        last_error = Some(e);
+                        failed += 1;
+                    }
+                }
+            }
+        });
+        let _ = tx.send(XmpMessage::Finished {
+            written,
+            failed,
+            last_error,
+        });
+    });
+    rx
+}
+
+/// The outcome of a batch of XMP writes.
+#[derive(Debug)]
+pub enum XmpMessage {
+    Finished {
+        written: usize,
+        failed: usize,
+        last_error: Option<String>,
+    },
+}
+
+/// Read-modify-write one image's XMP sidecar on the server.
+async fn write_one_xmp(backend: &dyn RemoteBackend, w: &XmpWrite) -> Result<(), String> {
+    let [darktable, lightroom] = crate::xmp_sync::candidate_paths(&w.image_path);
+    // Whichever exists is the one rewritten; neither existing means the
+    // Lightroom spelling is created. An existing file this build cannot
+    // parse is left alone rather than replaced — it is somebody else's
+    // document, and a refusal is recoverable where an overwrite is not.
+    let mut target = lightroom.clone();
+    let mut existing: Option<String> = None;
+    for path in [&darktable, &lightroom] {
+        if let Ok(bytes) = backend
+            .get(&RemoteId::Path(RemotePath::new(path.clone())), None)
+            .await
+        {
+            if !bytes.is_empty() {
+                target = path.clone();
+                existing = Some(String::from_utf8_lossy(&bytes).into_owned());
+                break;
+            }
+        }
+    }
+    let text = match existing {
+        Some(text) => {
+            // The file's caption, copyright and hierarchy come through: the
+            // catalog has nowhere to keep them, and a rewrite that said
+            // nothing about them would remove them.
+            let theirs = dr_xmp::Xmp::parse(&text)
+                .map_err(|e| format!("{target} is not a sidecar this build can read: {e}"))?;
+            let mut record = w.record.clone();
+            crate::xmp_sync::carry_through(&mut record, &theirs);
+            record
+                .rewrite(&text)
+                .map_err(|e| format!("{target} is not a sidecar this build can rewrite: {e}"))?
+        }
+        None => w.record.to_text(),
+    };
+    backend
+        .put(&RemotePath::new(target), text.into_bytes(), None)
+        .await
+        .map(|_| ())
+        .map_err(|e| e.to_string())
+}
+
+/// TRACES: FR-CAT-13
+/// The offered reload: re-read the sidecars a person chose to trust, with
+/// the sidecar winning. One fetch per path, the catalog opened on this
+/// thread as the scan opens it.
+pub fn spawn_xmp_reload(
+    conn: Connection,
+    root: String,
+    catalog_path: PathBuf,
+    paths: Vec<String>,
+) -> Receiver<XmpMessage> {
+    let (tx, rx) = std::sync::mpsc::channel();
+    std::thread::spawn(move || {
+        let mut written = 0usize;
+        let mut failed = 0usize;
+        let mut last_error = None;
+        let outcome: Result<(), String> = (|| {
+            let catalog = Catalog::open(&catalog_path).map_err(|e| e.to_string())?;
+            let root_id: i64 = catalog
+                .connection()
+                .query_row(
+                    "SELECT id FROM roots WHERE label = ?1 AND kind = 'remote'",
+                    [&root],
+                    |r| r.get(0),
+                )
+                .map_err(|e| e.to_string())?;
+            let rt = crate::net_runtime::build().map_err(|e| e.to_string())?;
+            rt.block_on(async {
+                let backend = crate::remote::connect(&conn).map_err(|e| e.to_string())?;
+                for path in &paths {
+                    let fetched = backend
+                        .get(&RemoteId::Path(RemotePath::new(path.clone())), None)
+                        .await
+                        .map_err(|e| e.to_string())
+                        .and_then(|bytes| {
+                            let text = String::from_utf8_lossy(&bytes);
+                            crate::xmp_sync::reload(catalog.connection(), root_id, path, &text)
+                        });
+                    match fetched {
+                        Ok(taken) => written += taken.changed,
+                        Err(e) => {
+                            log::warn!("reloading {path}: {e}");
+                            last_error = Some(e);
+                            failed += 1;
+                        }
+                    }
+                }
+                Ok(())
+            })
+        })();
+        if let Err(e) = outcome {
+            failed = paths.len();
+            last_error = Some(e);
+        }
+        let _ = tx.send(XmpMessage::Finished {
+            written,
+            failed,
+            last_error,
+        });
+    });
+    rx
+}
+
 /// What this device has already read, so a pull fetches only what changed.
 fn load_sidecar_etags(
     catalog: &Catalog,

ui/dr-ui/src/library_ui.rs

@@ -298,6 +298,9 @@ pub struct LibraryController {
     /// Drains the sidecar writer. Held so a second judgement replaces the
     /// timer rather than leaving two draining the same finished channel.
     sidecar_timer: RefCell<Option<slint::Timer>>,
+    /// TRACES: FR-CAT-13
+    /// The same, for a batch of XMP writes or a reload.
+    xmp_timer: RefCell<Option<slint::Timer>>,
     /// Which window load the model belongs to, bumped by [`load_window`].
     ///
     /// A thumbnail worker addresses cells by *row index into the window that
@@ -381,6 +384,10 @@ pub struct LibraryController {
     /// small budget still keeps a working set. A metered or small-disk device
     /// wants the first.
     keep_opened: std::cell::Cell<bool>,
+    /// TRACES: FR-CAT-13 | NFR-R4
+    /// Whether judgements and keywords also go to the `.xmp` beside the
+    /// original. Mirrors `LibrarySettings::write_xmp_sidecars`.
+    write_xmp: std::cell::Cell<bool>,
     /// TRACES: FR-CAT-6
     /// How many bars the capture-time axis is cut into, from the settings
     /// page.
@@ -450,6 +457,7 @@ impl LibraryController {
             current_bucket: RefCell::new(None),
             filter: RefCell::new(library::RatingFilter::default()),
             sidecar_timer: RefCell::new(None),
+            xmp_timer: RefCell::new(None),
             generation: std::cell::Cell::new(0),
             reachability: RefCell::new(dr_sync::Reachability::new()),
             root_lost: RefCell::new(None),
@@ -466,6 +474,9 @@ impl LibraryController {
             keep_opened: std::cell::Cell::new(
                 dr_types::CacheSettings::default().keep_opened_originals,
             ),
+            write_xmp: std::cell::Cell::new(
+                dr_types::LibrarySettings::default().write_xmp_sidecars,
+            ),
             timeline_bars: std::cell::Cell::new(dr_types::LibrarySettings::default().timeline_bars),
             face_model_id: RefCell::new(dr_types::FaceDetector::default().model_id().to_string()),
         })
@@ -505,6 +516,11 @@ impl LibraryController {
         self.keep_opened.set(keep);
     }
 
+    /// TRACES: FR-CAT-13 | NFR-R4
+    pub fn set_write_xmp_sidecars(&self, on: bool) {
+        self.write_xmp.set(on);
+    }
+
     /// TRACES: FR-NC-6a
     /// Set the ceiling on passively cached originals, and apply it now.
     ///
@@ -1266,6 +1282,11 @@ fn drain_scan(
                             log::info!("library folder is readable again");
                         }
                         refresh_offline(&w, ctl);
+                        // TRACES: FR-CAT-13
+                        // The pull may have found an `.xmp` that disagrees
+                        // with the catalog; the settings page offers the
+                        // reload, and this is what tells it how many.
+                        refresh_xmp_conflicts(&w, ctl);
 
                         // An incremental rescan lists almost nothing, so
                         // reporting the listed count would read as "0 images"
@@ -3086,6 +3107,7 @@ fn apply_keyword(window: &AppWindow, ctl: &Rc<LibraryController>, word: &str, as
     window.set_library_error(slint::SharedString::new());
     window.set_library_status(keyword_summary(&word, n, images.len(), assigning).into());
     refresh_keywords(window, ctl, &images);
+    start_xmp_writes(window, ctl, &images);
 
     // A filtered grid may no longer hold what was just keyworded — taking
     // "puffin" off an image while showing only puffins means it belongs
@@ -3194,6 +3216,7 @@ fn apply_judgement(
     }
 
     start_sidecar_writes(window, ctl, writes);
+    start_xmp_writes(window, ctl, images);
 }
 
 /// What the status line says about a judgement that just landed.
@@ -3385,6 +3408,186 @@ fn collect_sidecar_writes(
 }
 
 /// Push judgements out to sidecars on a worker, reporting once at the end.
+/// TRACES: FR-CAT-13 | NFR-R4
+/// Write these images' ratings, labels and keywords to the `.xmp` beside
+/// each, where the user has switched that on.
+///
+/// The record is read from the catalog *now*, whole, rather than carried
+/// from the gesture: a rating and a keyword typed a second apart are two
+/// writes of the same file, and the second must not carry a copy of the
+/// first taken before it landed.
+pub(crate) fn start_xmp_writes(
+    window: &AppWindow,
+    ctl: &Rc<LibraryController>,
+    images: &[dr_types::ImageId],
+) {
+    if !ctl.write_xmp.get() || images.is_empty() || ctl.is_offline() {
+        return;
+    }
+    let Some((conn, _)) = ctl.session.borrow().clone() else {
+        return;
+    };
+    let writes: Vec<library::XmpWrite> = {
+        let borrow = ctl.catalog.borrow();
+        let Some(catalog) = borrow.as_ref() else {
+            return;
+        };
+        let c = catalog.connection();
+        images
+            .iter()
+            .filter_map(|&image| {
+                let version = dr_catalog::rating::default_version_id(c, image).ok()?;
+                let image_path: String = c
+                    .query_row(
+                        "SELECT source_ref FROM images WHERE id = ?1",
+                        [image.0 as i64],
+                        |r| r.get(0),
+                    )
+                    .ok()?;
+                Some(library::XmpWrite {
+                    image_path,
+                    record: crate::xmp_sync::record_of(c, image, version),
+                })
+            })
+            .collect()
+    };
+    if writes.is_empty() {
+        return;
+    }
+
+    let count = writes.len();
+    let rx = library::spawn_xmp_writes(conn, writes);
+    let job = ctl.activity.begin(
+        crate::activity::Kind::Upload,
+        format!("Writing {count} XMP sidecar(s)"),
+    );
+    drain_xmp(window.as_weak(), ctl.clone(), rx, job, false);
+}
+
+/// TRACES: FR-CAT-13
+/// The offered reload: take the sidecars' values for every photograph the
+/// last pull found disagreeing with the catalog.
+pub(crate) fn start_xmp_reload(window: &AppWindow, ctl: &Rc<LibraryController>) {
+    let Some((conn, _)) = ctl.session.borrow().clone() else {
+        return;
+    };
+    let paths: Vec<String> = {
+        let borrow = ctl.catalog.borrow();
+        let Some(catalog) = borrow.as_ref() else {
+            return;
+        };
+        let Some(root_id) = root_id_of(catalog, &conn.account.root) else {
+            return;
+        };
+        crate::xmp_sync::conflicts(catalog, root_id)
+            .into_iter()
+            .map(|c| c.path)
+            .collect()
+    };
+    if paths.is_empty() {
+        return;
+    }
+    let count = paths.len();
+    let rx = library::spawn_xmp_reload(
+        conn.clone(),
+        conn.account.root.clone(),
+        library::catalog_path(&conn.account),
+        paths,
+    );
+    let job = ctl.activity.begin(
+        crate::activity::Kind::Download,
+        format!("Reloading {count} XMP sidecar(s)"),
+    );
+    drain_xmp(window.as_weak(), ctl.clone(), rx, job, true);
+}
+
+/// Wait for a batch of XMP work to report, then say what it did.
+fn drain_xmp(
+    weak: slint::Weak<AppWindow>,
+    ctl: Rc<LibraryController>,
+    rx: std::sync::mpsc::Receiver<library::XmpMessage>,
+    job: crate::activity::Activity,
+    reload: bool,
+) {
+    let timer = slint::Timer::default();
+    let job = RefCell::new(Some(job));
+    let ctl_cb = ctl.clone();
+    timer.start(
+        slint::TimerMode::Repeated,
+        std::time::Duration::from_millis(200),
+        move || {
+            let ctl = &ctl_cb;
+            let message = match rx.try_recv() {
+                Ok(m) => m,
+                Err(std::sync::mpsc::TryRecvError::Empty) => return,
+                Err(std::sync::mpsc::TryRecvError::Disconnected) => {
+                    stop(&ctl.xmp_timer);
+                    return;
+                }
+            };
+            stop(&ctl.xmp_timer);
+            let library::XmpMessage::Finished {
+                written,
+                failed,
+                last_error,
+            } = message;
+            let status = match (reload, failed, last_error) {
+                (false, 0, _) => format!("{written} XMP sidecar(s) written"),
+                (true, 0, _) => format!("{written} photograph(s) reloaded from XMP"),
+                (_, n, Some(e)) => format!("{n} XMP sidecar(s) failed: {e}"),
+                (_, n, None) => format!("{n} XMP sidecar(s) failed"),
+            };
+            if let Some(job) = job.borrow_mut().take() {
+                if failed > 0 {
+                    job.fail(status.clone());
+                } else {
+                    job.finish(status.clone());
+                }
+            }
+            if let Some(w) = weak.upgrade() {
+                w.set_library_status(status.into());
+                if reload {
+                    // The grid draws what the reload changed, and the
+                    // settings page stops offering what is settled.
+                    let visible = ctl.visible_ids();
+                    if let Some(catalog) = ctl.catalog.borrow().as_ref() {
+                        sync_ratings(&w, catalog, &visible);
+                        refresh_rating_counts(&w, catalog);
+                    }
+                    refresh_xmp_conflicts(&w, ctl);
+                }
+            }
+        },
+    );
+    *ctl.xmp_timer.borrow_mut() = Some(timer);
+}
+
+/// TRACES: FR-CAT-13
+/// How many sidecars the last pull found disagreeing with the catalog, for
+/// the settings page to offer the reload against.
+pub(crate) fn refresh_xmp_conflicts(window: &AppWindow, ctl: &Rc<LibraryController>) {
+    let count = (|| {
+        let (conn, _) = ctl.session.borrow().clone()?;
+        let borrow = ctl.catalog.borrow();
+        let catalog = borrow.as_ref()?;
+        let root_id = root_id_of(catalog, &conn.account.root)?;
+        Some(crate::xmp_sync::conflicts(catalog, root_id).len())
+    })()
+    .unwrap_or(0);
+    window.set_settings_xmp_conflicts(count as i32);
+}
+
+fn root_id_of(catalog: &Catalog, root: &str) -> Option<i64> {
+    catalog
+        .connection()
+        .query_row(
+            "SELECT id FROM roots WHERE label = ?1 AND kind = 'remote'",
+            [root],
+            |r| r.get(0),
+        )
+        .ok()
+}
+
 pub(crate) fn start_sidecar_writes(
     window: &AppWindow,
     ctl: &Rc<LibraryController>,
@@ -5503,6 +5706,18 @@ pub fn wire<F>(
     // controllers would hold each other alive for the life of the process.
     *ctl.coll_ctl.borrow_mut() = Some(Rc::downgrade(&coll_ctl));
 
+    // TRACES: FR-CAT-13
+    // The reload the settings page offers when a sidecar disagrees.
+    {
+        let weak = window.as_weak();
+        let ctl = ctl.clone();
+        window.on_settings_xmp_reload(move || {
+            if let Some(w) = weak.upgrade() {
+                start_xmp_reload(&w, &ctl);
+            }
+        });
+    }
+
     // TRACES: FR-UI-3 | FR-UI-4
     // Whether the rating strip waits to be hovered or stands open.
     //

ui/dr-ui/src/settings_ui.rs

@@ -154,6 +154,7 @@ pub fn render(window: &AppWindow, controller: &SettingsController) {
     window.set_settings_thumbnail_budget(budget::label(s.cache.thumbnail_budget_bytes).into());
     window.set_settings_thumbnail_unlimited(s.cache.thumbnail_budget_bytes.is_none());
     window.set_settings_keep_opened(s.cache.keep_opened_originals);
+    window.set_settings_write_xmp(s.library.write_xmp_sidecars);
 
     // --- develop -------------------------------------------------------
     //
@@ -501,6 +502,16 @@ pub fn wire<F, G>(
             render(&w, &ctl);
         });
     }
+    {
+        // TRACES: FR-CAT-13 | NFR-R4
+        let weak = window.as_weak();
+        let ctl = controller.clone();
+        window.on_settings_write_xmp_toggled(move |on| {
+            let Some(w) = weak.upgrade() else { return };
+            ctl.edit(|s| s.library.write_xmp_sidecars = on);
+            render(&w, &ctl);
+        });
+    }
 
     // --- faces ---------------------------------------------------------
     //

ui/dr-ui/src/xmp_sync.rs

@@ -0,0 +1,597 @@
+//! TRACES: FR-CAT-13 | FR-NC-9 | NFR-R4
+//! Standard XMP sidecars, read into the catalog and written back out of it.
+//!
+//! `dr-xmp` reads and writes the file. This is the other half the requirement
+//! asks for and the half `docs/outstanding.md` called "the larger": which
+//! images a given `.xmp` describes, what the catalog holds about them, how
+//! the two are reconciled, where a disagreement goes, and the write in the
+//! other direction. Nothing here parses XML and nothing in `dr-xmp` knows a
+//! catalog exists.
+//!
+//! # Two conventions for one file
+//!
+//! Lightroom writes `IMG_0001.xmp` beside `IMG_0001.CR3`; darktable writes
+//! `IMG_0001.CR3.xmp`. Both are answered by [`images_for`]: the path with
+//! `.xmp` taken off is tried as an image path first, exactly, and failing that
+//! as a stem shared with the images beside it — the rule DarkRoom's own
+//! sidecar already follows, under which a RAW and the JPEG the camera wrote
+//! beside it are one photograph (FR-CAT-11) and share the document.
+//!
+//! # Precedence, and where a disagreement goes
+//!
+//! A standard XMP carries no revision and no device, so nothing in it can say
+//! whether its rating is newer than the catalog's. The automatic pull
+//! therefore runs [`dr_xmp::reconcile`] with the catalog winning: keywords
+//! union, and every whole-valued field is taken only where the catalog holds
+//! none. A genuine disagreement — both sides hold a value, and different ones —
+//! is written to `xmp_conflicts` rather than resolved, and the requirement's
+//! "a metadata reload offered" is that table with a button in front of it.
+//! [`reload`] is the button: the same reconciliation with the sidecar winning,
+//! asked for by a person.
+//!
+//! # Detection
+//!
+//! "External modification of an XMP sidecar shall be detected." The scan
+//! already records the ETag of every sidecar it has taken in, and fetches
+//! only those whose ETag has moved. An `.xmp` edited in another application
+//! is precisely a file whose ETag has moved, so the detection is the pull's
+//! ordinary incrementality — nothing watches a directory, and nothing needs
+//! to. What is new is what happens after: the re-read reconciles again, and
+//! the second reading is where a conflict first appears.
+//!
+//! # Writing, and why it is off
+//!
+//! NFR-R4 makes source-adjacent writes opt-in. A library shared with another
+//! editor is one where a file DarkRoom wrote can be read by something else,
+//! and that is exactly what [`Xmp::rewrite`] is built for — it rewrites only
+//! the properties DarkRoom owns and copies everything else through byte for
+//! byte. But the option to write beside somebody's originals is theirs to
+//! switch on, and until they do the catalog and DarkRoom's own sidecar are the
+//! only things a judgement reaches.
+
+use dr_catalog::Catalog;
+use dr_types::{FlagState, ImageId};
+use dr_xmp::{Precedence, Rating, Xmp};
+use rusqlite::Connection;
+
+/// The two places an image's XMP sidecar may be, in the order they are
+/// tried when writing: the darktable spelling first, because it names the
+/// image unambiguously, then Lightroom's.
+///
+/// When reading, whichever the scan found is the one read; this is for the
+/// write, which has to choose. An existing file of either spelling is
+/// rewritten in place, and a photograph with neither gets Lightroom's, since
+/// it is the spelling more applications look for.
+pub fn candidate_paths(image_path: &str) -> [String; 2] {
+    let stem = match image_path.rsplit_once('.') {
+        Some((stem, ext)) if !ext.contains('/') => stem,
+        _ => image_path,
+    };
+    [format!("{image_path}.xmp"), format!("{stem}.xmp")]
+}
+
+/// Whether a listing entry is a standard XMP sidecar rather than DarkRoom's.
+pub fn is_xmp(path: &str) -> bool {
+    path.rsplit_once('.')
+        .is_some_and(|(_, ext)| ext.eq_ignore_ascii_case(dr_sync::scan::XMP_EXTENSION))
+}
+
+/// The images an `.xmp` at `path` describes: their ids and default versions.
+///
+/// Empty when the sidecar sits beside nothing this catalog knows, which is
+/// the ordinary case for a file that arrived before its photograph was
+/// scanned — the next pull reads it again, because no ETag is recorded for a
+/// sidecar that reached nothing.
+pub fn images_for(conn: &Connection, root_id: i64, path: &str) -> Vec<(ImageId, i64)> {
+    let Some(named) = path
+        .strip_suffix(".xmp")
+        .or_else(|| path.strip_suffix(".XMP"))
+    else {
+        return Vec::new();
+    };
+
+    let ids = |sql: &str, arg: &str| -> Vec<ImageId> {
+        let Ok(mut stmt) = conn.prepare(sql) else {
+            return Vec::new();
+        };
+        stmt.query_map(rusqlite::params![root_id, arg], |r| r.get::<_, i64>(0))
+            .map(|rows| rows.flatten().map(|i| ImageId(i as u64)).collect())
+            .unwrap_or_default()
+    };
+
+    // darktable's spelling: the name before `.xmp` is the image itself.
+    let mut images = ids(
+        "SELECT id FROM images WHERE root_id = ?1 AND source_ref = ?2",
+        named,
+    );
+    if images.is_empty() {
+        // Lightroom's: a stem shared with the photograph, and with the JPEG
+        // beside it. The escape is what makes `%` and `_` in a folder name
+        // literal, and the Rust-side check is what actually decides — a LIKE
+        // is a filter, not an answer.
+        let prefix = named
+            .replace('\\', "\\\\")
+            .replace('%', "\\%")
+            .replace('_', "\\_");
+        let candidates: Vec<(ImageId, String)> = {
+            let Ok(mut stmt) = conn.prepare(
+                "SELECT id, source_ref FROM images
+                  WHERE root_id = ?1 AND source_ref LIKE ?2 ESCAPE '\\'",
+            ) else {
+                return Vec::new();
+            };
+            stmt.query_map(rusqlite::params![root_id, format!("{prefix}.%")], |r| {
+                Ok((ImageId(r.get::<_, i64>(0)? as u64), r.get::<_, String>(1)?))
+            })
+            .map(|rows| rows.flatten().collect())
+            .unwrap_or_default()
+        };
+        images = candidates
+            .into_iter()
+            .filter(|(_, source)| candidate_paths(source)[1].eq_ignore_ascii_case(path))
+            .map(|(id, _)| id)
+            .collect();
+    }
+
+    images
+        .into_iter()
+        .filter_map(|image| {
+            let version = dr_catalog::rating::default_version_id(conn, image).ok()?;
+            Some((image, version))
+        })
+        .collect()
+}
+
+/// What the catalog holds about one image, as the sidecar would carry it.
+///
+/// Rating and flag are two axes here and one field there: a rejection is
+/// written as Adobe's `-1` and a rating as its stars, and a frame that is
+/// both rejected and starred loses the stars in the file — the loss the
+/// format has and `dr_xmp::Rating` records. Reading goes the other way in
+/// [`apply`].
+pub fn record_of(conn: &Connection, image: ImageId, version: i64) -> Xmp {
+    let mut xmp = Xmp::default();
+    let row = conn
+        .query_row(
+            "SELECT rating, flag, label FROM versions WHERE id = ?1",
+            [version],
+            |r| {
+                Ok((
+                    r.get::<_, i64>(0)?,
+                    r.get::<_, i64>(1)?,
+                    r.get::<_, Option<i64>>(2)?,
+                ))
+            },
+        )
+        .ok();
+    if let Some((rating, flag, label)) = row {
+        xmp.rating = if flag == flag_code(FlagState::Reject) {
+            Some(Rating::Rejected)
+        } else if rating > 0 {
+            Some(Rating::Stars(rating.clamp(0, 5) as u8))
+        } else {
+            None
+        };
+        xmp.set_colour(dr_catalog::rating::label_from_code(label));
+    }
+    xmp.keywords = dr_catalog::keywords::for_image(conn, image).unwrap_or_default();
+    xmp
+}
+
+/// Write a reconciled record onto one image.
+///
+/// Only what the record holds: a `None` rating is *unrated* and is not
+/// written over a star, for the reason `dr_pipeline::sidecar::merge_judgement`
+/// gives — a file that says nothing cannot erase an afternoon's culling. A
+/// rejection sets the flag and leaves the stars alone; stars set the rating
+/// and, if the frame was rejected, lift the rejection, since the file said
+/// it was worth a number. Keywords are added and never removed here, which
+/// is what a union means. `label` is set where the file names one of the
+/// five colours.
+///
+/// Returns whether any column moved.
+pub fn apply(
+    conn: &Connection,
+    image: ImageId,
+    version: i64,
+    record: &Xmp,
+) -> Result<bool, String> {
+    let mut changed = false;
+    match record.rating {
+        Some(Rating::Rejected) => {
+            changed |= conn
+                .execute(
+                    "UPDATE versions SET flag = ?2 WHERE id = ?1 AND flag <> ?2",
+                    rusqlite::params![version, flag_code(FlagState::Reject)],
+                )
+                .map_err(|e| e.to_string())?
+                > 0;
+        }
+        Some(Rating::Stars(n)) if n > 0 => {
+            changed |= conn
+                .execute(
+                    "UPDATE versions
+                        SET rating = ?2,
+                            flag = CASE WHEN flag = ?3 THEN 0 ELSE flag END
+                      WHERE id = ?1 AND (rating <> ?2 OR flag = ?3)",
+                    rusqlite::params![version, n.min(5) as i64, flag_code(FlagState::Reject)],
+                )
+                .map_err(|e| e.to_string())?
+                > 0;
+        }
+        _ => {}
+    }
+    if let Some(colour) = record.colour() {
+        changed |= conn
+            .execute(
+                "UPDATE versions SET label = ?2 WHERE id = ?1 AND label IS NOT ?2",
+                rusqlite::params![version, dr_catalog::rating::label_code(colour)],
+            )
+            .map_err(|e| e.to_string())?
+            > 0;
+    }
+    if !record.keywords.is_empty() {
+        let have = dr_catalog::keywords::for_image(conn, image).unwrap_or_default();
+        for word in &record.keywords {
+            if have.iter().any(|h| h.eq_ignore_ascii_case(word)) {
+                continue;
+            }
+            match dr_catalog::keywords::assign(conn, &[image], word) {
+                Ok(n) => changed |= n > 0,
+                // A word the vocabulary refuses — empty once trimmed, or a
+                // separator on its own — costs that word and not the file.
+                Err(e) => log::debug!("xmp keyword {word:?} on {}: {e}", image.0),
+            }
+        }
+    }
+    Ok(changed)
+}
+
+/// TRACES: FR-CAT-13
+/// Before rewriting an existing sidecar, keep what the catalog cannot hold.
+///
+/// `Xmp::rewrite` replaces the owned properties wholesale — that is what
+/// makes "no rating" mean the rating goes — and the catalog has columns for
+/// three of them: rating and flag, label, keywords. A record built from the
+/// catalog therefore says nothing about a title, a caption, a copyright
+/// line or a hierarchical subject, and writing it as it stands would delete
+/// all four from a sidecar Lightroom wrote, on every judgement. So those
+/// come through from the file, and so does a label that is not one of the
+/// five colours — the catalog kept no text for it, and the file's is the
+/// only copy.
+pub fn carry_through(record: &mut Xmp, existing: &Xmp) {
+    record.hierarchical_subjects = existing.hierarchical_subjects.clone();
+    record.title = existing.title.clone();
+    record.description = existing.description.clone();
+    record.creators = existing.creators.clone();
+    record.copyright = existing.copyright.clone();
+    record.credit = existing.credit.clone();
+    record.usage_terms = existing.usage_terms.clone();
+    if record.label.is_none() && existing.colour().is_none() {
+        record.label = existing.label.clone();
+    }
+}
+
+/// What one sidecar did when taken in.
+#[derive(Debug, Default, Clone, PartialEq, Eq)]
+pub struct TakenIn {
+    /// Images whose rows moved.
+    pub changed: usize,
+    /// Images the sidecar described at all. Zero means it sits beside nothing
+    /// this catalog has yet, and its ETag must not be recorded.
+    pub described: usize,
+    /// The fields both sides held and disagreed on, across every image.
+    pub conflicts: Vec<dr_xmp::Field>,
+}
+
+/// TRACES: FR-CAT-13 | FR-NC-9
+/// Take a standard sidecar into the catalog, with the catalog winning.
+///
+/// The automatic path. Every image the file describes is reconciled against
+/// what the catalog holds for it; the merged record is applied; and a
+/// disagreement is recorded in `xmp_conflicts` for a person to settle, never
+/// resolved here.
+pub fn take_in(
+    conn: &Connection,
+    root_id: i64,
+    path: &str,
+    text: &str,
+    now: i64,
+) -> Result<TakenIn, String> {
+    let sidecar = Xmp::parse(text).map_err(|e| e.to_string())?;
+    let mut out = TakenIn::default();
+    for (image, version) in images_for(conn, root_id, path) {
+        out.described += 1;
+        let mine = record_of(conn, image, version);
+        let reconciled = dr_xmp::reconcile(&mine, &sidecar, Precedence::Catalog);
+        if apply(conn, image, version, &reconciled.merged)? {
+            out.changed += 1;
+        }
+        for field in reconciled.conflicts {
+            if !out.conflicts.contains(&field) {
+                out.conflicts.push(field);
+            }
+        }
+    }
+    if out.conflicts.is_empty() {
+        clear_conflict(conn, root_id, path);
+    } else if out.described > 0 {
+        record_conflict(conn, root_id, path, &out.conflicts, now);
+    }
+    Ok(out)
+}
+
+/// TRACES: FR-CAT-13
+/// The offered reload: the same reconciliation with the sidecar winning.
+///
+/// Asked for by a person, per file, which is the consent the module
+/// documentation says this needs. Clears the conflict whatever the outcome —
+/// the person has now seen it, and if the file still disagrees the next pull
+/// that sees it change will say so again.
+pub fn reload(conn: &Connection, root_id: i64, path: &str, text: &str) -> Result<TakenIn, String> {
+    let sidecar = Xmp::parse(text).map_err(|e| e.to_string())?;
+    let mut out = TakenIn::default();
+    for (image, version) in images_for(conn, root_id, path) {
+        out.described += 1;
+        let mine = record_of(conn, image, version);
+        let reconciled = dr_xmp::reconcile(&mine, &sidecar, Precedence::Sidecar);
+        // The sidecar wins outright on the whole-valued fields, and that
+        // includes a rating it holds and the catalog's it replaces — which
+        // `apply` does. What `apply` will not do is *clear* a star the file
+        // does not mention, and a reload does not ask it to: the file said
+        // nothing, and nothing is not a judgement.
+        if apply(conn, image, version, &reconciled.merged)? {
+            out.changed += 1;
+        }
+    }
+    clear_conflict(conn, root_id, path);
+    Ok(out)
+}
+
+/// One outstanding disagreement, for the settings page.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct Conflict {
+    pub path: String,
+    pub fields: String,
+}
+
+/// Every sidecar in this library that disagrees with the catalog.
+pub fn conflicts(catalog: &Catalog, root_id: i64) -> Vec<Conflict> {
+    let Ok(mut stmt) = catalog
+        .connection()
+        .prepare("SELECT path, fields FROM xmp_conflicts WHERE root_id = ?1 ORDER BY path")
+    else {
+        return Vec::new();
+    };
+    stmt.query_map([root_id], |r| {
+        Ok(Conflict {
+            path: r.get(0)?,
+            fields: r.get(1)?,
+        })
+    })
+    .map(|rows| rows.flatten().collect())
+    .unwrap_or_default()
+}
+
+fn record_conflict(
+    conn: &Connection,
+    root_id: i64,
+    path: &str,
+    fields: &[dr_xmp::Field],
+    now: i64,
+) {
+    let fields = fields
+        .iter()
+        .map(|f| format!("{f:?}"))
+        .collect::<Vec<_>>()
+        .join(" ");
+    let done = conn.execute(
+        "INSERT INTO xmp_conflicts(root_id, path, fields, seen_at) VALUES (?1, ?2, ?3, ?4)
+         ON CONFLICT(root_id, path) DO UPDATE SET
+             fields = excluded.fields, seen_at = excluded.seen_at",
+        rusqlite::params![root_id, path, fields, now],
+    );
+    if let Err(e) = done {
+        log::debug!("recording xmp conflict for {path}: {e}");
+    }
+}
+
+fn clear_conflict(conn: &Connection, root_id: i64, path: &str) {
+    let _ = conn.execute(
+        "DELETE FROM xmp_conflicts WHERE root_id = ?1 AND path = ?2",
+        rusqlite::params![root_id, path],
+    );
+}
+
+fn flag_code(flag: FlagState) -> i64 {
+    match flag {
+        FlagState::Unflagged => 0,
+        FlagState::Pick => 1,
+        FlagState::Reject => 2,
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    fn catalog() -> Catalog {
+        let c = Catalog::in_memory().unwrap();
+        c.connection()
+            .execute_batch(
+                "INSERT INTO roots(id, kind, label) VALUES (1, 'remote', 'lib');
+                 INSERT INTO images(id, root_id, source_ref, added_at)
+                      VALUES (1, 1, 'Photos/IMG_0001.CR3', 0),
+                             (2, 1, 'Photos/IMG_0001.JPG', 0),
+                             (3, 1, 'Photos/IMG_0002.CR3', 0);",
+            )
+            .unwrap();
+        dr_catalog::rating::ensure_default_versions(c.connection()).unwrap();
+        c
+    }
+
+    const LIGHTROOM: &str = r#"<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
+<x:xmpmeta xmlns:x="adobe:ns:meta/">
+ <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
+  <rdf:Description rdf:about=""
+    xmlns:xmp="http://ns.adobe.com/xap/1.0/"
+    xmlns:dc="http://purl.org/dc/elements/1.1/"
+    xmp:Rating="4"
+    xmp:Label="Red">
+   <dc:subject><rdf:Bag><rdf:li>puffin</rdf:li><rdf:li>iceland</rdf:li></rdf:Bag></dc:subject>
+  </rdf:Description>
+ </rdf:RDF>
+</x:xmpmeta>
+<?xpacket end="w"?>"#;
+
+    /// Both spellings reach the photograph, and Lightroom's reaches the JPEG
+    /// beside it too.
+    #[test]
+    fn both_namings_find_their_images() {
+        let c = catalog();
+        let conn = c.connection();
+        let mut lr: Vec<u64> = images_for(conn, 1, "Photos/IMG_0001.xmp")
+            .into_iter()
+            .map(|(i, _)| i.0)
+            .collect();
+        lr.sort();
+        assert_eq!(lr, [1, 2], "the RAW and the JPEG are one photograph");
+
+        let dt: Vec<u64> = images_for(conn, 1, "Photos/IMG_0001.CR3.xmp")
+            .into_iter()
+            .map(|(i, _)| i.0)
+            .collect();
+        assert_eq!(dt, [1], "darktable's names the file exactly");
+
+        assert!(images_for(conn, 1, "Photos/IMG_9999.xmp").is_empty());
+    }
+
+    /// The ordinary pull: an empty catalog takes everything the file says.
+    #[test]
+    fn a_sidecar_fills_what_the_catalog_lacks() {
+        let c = catalog();
+        let conn = c.connection();
+        let taken = take_in(conn, 1, "Photos/IMG_0002.xmp", LIGHTROOM, 1).unwrap();
+        assert_eq!(taken.described, 1);
+        assert_eq!(taken.changed, 1);
+        assert!(taken.conflicts.is_empty());
+
+        let record = record_of(conn, ImageId(3), 3);
+        assert_eq!(record.rating, Some(Rating::Stars(4)));
+        assert_eq!(record.colour(), Some(dr_types::ColourLabel::Red));
+        assert_eq!(record.keywords, ["iceland", "puffin"]);
+        assert!(conflicts(&c, 1).is_empty());
+    }
+
+    /// A rating the catalog already holds is not overwritten by the file,
+    /// the disagreement is recorded, and the reload — a person asking —
+    /// takes the file's.
+    #[test]
+    fn a_disagreement_is_recorded_and_a_reload_settles_it() {
+        let c = catalog();
+        let conn = c.connection();
+        dr_catalog::rating::set_rating(conn, ImageId(3), 2).unwrap();
+
+        let taken = take_in(conn, 1, "Photos/IMG_0002.xmp", LIGHTROOM, 1).unwrap();
+        assert_eq!(taken.conflicts, [dr_xmp::Field::Rating]);
+        assert_eq!(
+            record_of(conn, ImageId(3), 3).rating,
+            Some(Rating::Stars(2)),
+            "the catalog's own rating stands"
+        );
+        assert_eq!(
+            record_of(conn, ImageId(3), 3).keywords,
+            ["iceland", "puffin"],
+            "while the keywords still union"
+        );
+        let open = conflicts(&c, 1);
+        assert_eq!(open.len(), 1);
+        assert_eq!(open[0].path, "Photos/IMG_0002.xmp");
+        assert_eq!(open[0].fields, "Rating");
+
+        reload(conn, 1, "Photos/IMG_0002.xmp", LIGHTROOM).unwrap();
+        assert_eq!(
+            record_of(conn, ImageId(3), 3).rating,
+            Some(Rating::Stars(4))
+        );
+        assert!(conflicts(&c, 1).is_empty(), "settled");
+    }
+
+    /// A rejection and a star count are two axes here and one field there.
+    #[test]
+    fn a_rejection_crosses_as_adobes_minus_one_and_back() {
+        let c = catalog();
+        let conn = c.connection();
+        dr_catalog::rating::set_flag(conn, ImageId(3), FlagState::Reject).unwrap();
+        assert_eq!(
+            record_of(conn, ImageId(3), 3).rating,
+            Some(Rating::Rejected)
+        );
+
+        let back = Xmp {
+            rating: Some(Rating::Stars(3)),
+            ..Default::default()
+        };
+        apply(conn, ImageId(3), 3, &back).unwrap();
+        let after = record_of(conn, ImageId(3), 3);
+        assert_eq!(
+            after.rating,
+            Some(Rating::Stars(3)),
+            "the stars lift the rejection"
+        );
+    }
+
+    /// A judgement written over a Lightroom sidecar must not cost the caption
+    /// Lightroom wrote, since the catalog never held it.
+    #[test]
+    fn a_rewrite_keeps_what_the_catalog_has_no_column_for() {
+        let theirs = Xmp {
+            title: Some("Puffins at dusk".into()),
+            copyright: Some("© Someone".into()),
+            hierarchical_subjects: vec!["Places|Iceland".into()],
+            label: Some("Second choice".into()),
+            rating: Some(Rating::Stars(1)),
+            ..Default::default()
+        };
+
+        let mut ours = Xmp {
+            rating: Some(Rating::Stars(4)),
+            keywords: vec!["puffin".into()],
+            ..Default::default()
+        };
+        carry_through(&mut ours, &theirs);
+
+        assert_eq!(ours.rating, Some(Rating::Stars(4)), "the judgement is ours");
+        assert_eq!(ours.title.as_deref(), Some("Puffins at dusk"));
+        assert_eq!(ours.copyright.as_deref(), Some("© Someone"));
+        assert_eq!(ours.hierarchical_subjects, ["Places|Iceland"]);
+        assert_eq!(
+            ours.label.as_deref(),
+            Some("Second choice"),
+            "a label that is not one of the five colours has no other copy"
+        );
+
+        let mut red = Xmp::default();
+        red.set_colour(Some(dr_types::ColourLabel::Red));
+        carry_through(&mut red, &theirs);
+        assert_eq!(
+            red.label.as_deref(),
+            Some("Red"),
+            "but a colour of ours wins"
+        );
+    }
+
+    /// The two spellings a write chooses between.
+    #[test]
+    fn the_write_tries_darktables_spelling_then_lightrooms() {
+        assert_eq!(
+            candidate_paths("Photos/IMG_0001.CR3"),
+            ["Photos/IMG_0001.CR3.xmp", "Photos/IMG_0001.xmp"]
+        );
+        assert_eq!(
+            candidate_paths("a.b/IMG"),
+            ["a.b/IMG.xmp", "a.b/IMG.xmp"],
+            "a dot in a folder is not an extension"
+        );
+    }
+}