Read the sidecars other editors write, and write them back on request
Benchmarks / CPU and I/O (per commit) (push) Successful in 10m59s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Successful in 1h33m36s
Build and test / Layer separation (push) Successful in 1m2s
Traceability / Requirement traces (push) Successful in 1m25s
🐳 Android image / Build and push (push) Successful in 9s
Build and test / android-image (push) Successful in 9s
Build and test / Android (aarch64) (push) Successful in 56m59s

FR-CAT-13 asked for standard XMP and `core/dr-xmp` answered the file: it
has read and written `dc:subject`, `xmp:Rating`, `xmp:Label` and the IPTC
core since 5fa4c07, under an ownership rule that leaves everything else in
the document untouched. What nothing did was call it. No scan found an
`.xmp` beside a raw, no catalog row was filled from one, no judgement
wrote one back, and the "external modification detected, reload offered"
clause had no mechanism. A library imported from Lightroom came in and
could not go back out.

The scan collects `.xmp` beside `.drsc` from the listings it was already
paying for, and the pull reads each one whose ETag has moved. Both
namings resolve: darktable's `IMG_0001.CR3.xmp` names its file exactly,
Lightroom's `IMG_0001.xmp` names the stem, and under the stem the JPEG
beside a RAW is the same photograph and takes the same document, as
DarkRoom's own sidecar already does. Each is reconciled with the catalog
winning — keywords union, a rating or label taken only where the catalog
has none — because a standard XMP carries nothing that could say whether
its value is newer. A genuine disagreement is not resolved; it is written
to a table, and the settings page offers the sidecars' values against it.
That button is the reload the requirement asks to be offered, and the
ETag that moved is the detection it asks for: an `.xmp` edited elsewhere
is exactly a file the pull's ordinary incrementality re-reads.

Writing goes the other way behind a setting that starts off, since NFR-R4
makes writes beside somebody's originals theirs to switch on. With it on,
a judgement or a keyword rewrites the sidecar of whichever spelling
exists, or creates Lightroom's. The record is read from the catalog
whole at that moment rather than carried from the gesture, so a rating
and a keyword a second apart are two writes of one file that agree. And
the file's own title, caption, copyright and hierarchy come through the
rewrite: the catalog has no columns for them, `rewrite` replaces the
owned set wholesale, and a record that said nothing about them would have
deleted them from a Lightroom sidecar on every star.

The rating's two axes cross the format's one field both ways: a
rejection is Adobe's `-1` and stars are stars, and stars arriving on a
rejected frame lift the rejection, since the file said it was worth a
number. An unrated file says nothing and clears nothing, on the rule the
`.drsc` merge keeps. `versions.label` finally has a reader and a writer,
with the code table moved out of the query so the two cannot drift.
This commit is contained in:
2026-09-12 01:08:11 +02:00
parent d3b6127db6
commit 896188a489
17 changed files with 1316 additions and 118 deletions
Generated
+1
View File
@@ -1669,6 +1669,7 @@ dependencies = [
"dr-sync-nextcloud",
"dr-thumbs",
"dr-types",
"dr-xmp",
"env_logger",
"jni 0.22.4",
"log",
+1 -7
View File
@@ -301,13 +301,7 @@ fn like_prefix(path: &str) -> String {
}
fn label_code(l: ColourLabel) -> i64 {
match l {
ColourLabel::Red => 1,
ColourLabel::Yellow => 2,
ColourLabel::Green => 3,
ColourLabel::Blue => 4,
ColourLabel::Purple => 5,
}
crate::rating::label_code(l)
}
fn flag_code(f: FlagState) -> i64 {
+30 -1
View File
@@ -30,7 +30,7 @@
use rusqlite::{Connection, OptionalExtension};
use dr_types::{FlagState, ImageId};
use dr_types::{ColourLabel, FlagState, ImageId};
use crate::error::CatalogError;
@@ -275,6 +275,35 @@ pub fn align_default_version_uuids(conn: &Connection) -> Result<usize, CatalogEr
/// version pass was interrupted between the image insert and the commit.
/// Failing a rating because of either would be the wrong answer — the user
/// pressed a key and expects a star.
/// TRACES: FR-CAT-13
/// How `versions.label` encodes a colour label, and back.
///
/// One place for both directions, so a label written by the XMP pull and a
/// label queried by the selector cannot drift apart: the query used to hold
/// its own copy of the forward mapping and nothing held the reverse.
pub fn label_code(l: ColourLabel) -> i64 {
match l {
ColourLabel::Red => 1,
ColourLabel::Yellow => 2,
ColourLabel::Green => 3,
ColourLabel::Blue => 4,
ColourLabel::Purple => 5,
}
}
/// The colour a `versions.label` value names, or `None` for NULL and for a
/// code this build does not know.
pub fn label_from_code(code: Option<i64>) -> Option<ColourLabel> {
Some(match code? {
1 => ColourLabel::Red,
2 => ColourLabel::Yellow,
3 => ColourLabel::Green,
4 => ColourLabel::Blue,
5 => ColourLabel::Purple,
_ => return None,
})
}
pub fn default_version_id(conn: &Connection, image: ImageId) -> Result<i64, CatalogError> {
let existing: Option<i64> = conn
.query_row(
+38 -1
View File
@@ -15,7 +15,7 @@ use rusqlite::Connection;
use crate::error::CatalogError;
/// Schema version this build writes and understands.
pub const SCHEMA_VERSION: i64 = 14;
pub const SCHEMA_VERSION: i64 = 15;
/// Apply migrations up to [`SCHEMA_VERSION`].
///
@@ -136,6 +136,13 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
tx.commit()?;
}
if from < 15 {
let tx = conn.unchecked_transaction()?;
tx.execute_batch(V15)?;
tx.pragma_update(None, "user_version", 15)?;
tx.commit()?;
}
Ok(from)
}
@@ -642,6 +649,36 @@ DELETE FROM face_index
AND f.model_id = face_index.model_id);
"#;
const V15: &str = r#"
-- TRACES: FR-CAT-13
-- Where a standard XMP sidecar and the catalog disagree.
--
-- An `.xmp` beside a photograph is read on the same pull as DarkRoom's own
-- sidecar, and reconciled field by field (`dr_xmp::reconcile`): keywords
-- union, and a rating, label or caption is taken only where the catalog holds
-- none. That rule is the safe one and it is not always the right one -- a
-- rating changed in Lightroom after it was changed here is a genuine
-- disagreement, and a standard XMP carries no revision to settle it by. So
-- the disagreement is written here instead of being resolved, and the
-- requirement's "a metadata reload offered" is a row in this table with a
-- button in front of it: the reload re-reads the file with the sidecar
-- winning, and deletes the row.
--
-- Keyed on the sidecar's path like `sidecars` is, and for the same reason: a
-- path is what the scan reports, what a fetch addresses, and what the ETag
-- that noticed the change belongs to. `fields` is the disagreeing fields as
-- `dr_xmp` names them, space-separated, for the line the settings page shows.
--
-- Rebuildable: the next pull that sees a changed ETag writes the row again.
CREATE TABLE IF NOT EXISTS xmp_conflicts (
root_id INTEGER NOT NULL REFERENCES roots(id) ON DELETE CASCADE,
path TEXT NOT NULL,
fields TEXT NOT NULL,
seen_at INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY(root_id, path)
);
"#;
const V9: &str = r#"
-- TRACES: FR-CULL-8
-- A record that face detection has *run* on an image, distinct from what it
+31 -5
View File
@@ -37,6 +37,17 @@ pub struct ScanProgress {
/// on the whole edit format to recognise four characters in a filename.
pub const SIDECAR_EXTENSION: &str = "drsc";
/// TRACES: FR-CAT-13
/// Extension of a standard XMP sidecar — Lightroom's `IMG_0001.xmp`,
/// darktable's `IMG_0001.CR3.xmp`, and every other editor's.
///
/// Collected alongside DarkRoom's own for the same reason and at the same
/// cost: it is in the listing already, and it is the file the ratings and
/// keywords of a library edited elsewhere are in. Which images a given
/// `.xmp` describes is the catalog's question, since the two naming
/// conventions resolve differently and only the catalog knows the images.
pub const XMP_EXTENSION: &str = "xmp";
/// The result of a scan.
#[derive(Debug, Clone, Default)]
pub struct ScanResult {
@@ -275,15 +286,18 @@ where
Ok(result)
}
/// Whether a filename is a DarkRoom sidecar.
/// Whether a filename is a sidecar — DarkRoom's own, or a standard XMP one.
///
/// Case-insensitive on the extension alone. A server that upper-cased the
/// suffix — or a file copied through a filesystem that did — still describes a
/// photograph, and failing to recognise it would silently lose the edit rather
/// than fail visibly.
fn is_sidecar(name: &str) -> bool {
name.rsplit_once('.')
.is_some_and(|(stem, ext)| !stem.is_empty() && ext.eq_ignore_ascii_case(SIDECAR_EXTENSION))
name.rsplit_once('.').is_some_and(|(stem, ext)| {
!stem.is_empty()
&& (ext.eq_ignore_ascii_case(SIDECAR_EXTENSION)
|| ext.eq_ignore_ascii_case(XMP_EXTENSION))
})
}
/// Whether pruning is worth attempting against this backend.
@@ -927,6 +941,10 @@ mod tests {
// Upper-cased by a filesystem somewhere along the way; still a
// sidecar, and losing it would lose the edit silently.
file("Photos/2025/b.DRSC"),
// TRACES: FR-CAT-13
// And the standard kind, in both of its spellings.
file("Photos/2025/a.xmp"),
file("Photos/2025/b.jpg.xmp"),
],
);
let before = *b.lists.borrow();
@@ -946,14 +964,22 @@ mod tests {
.iter()
.map(|e| e.path.as_str().to_string())
.collect::<Vec<_>>(),
vec!["Photos/2025/a.drsc", "Photos/2025/b.DRSC"]
vec![
"Photos/2025/a.drsc",
"Photos/2025/a.xmp",
"Photos/2025/b.DRSC",
"Photos/2025/b.jpg.xmp",
]
);
assert_eq!(*b.lists.borrow() - before, 3, "no extra requests");
// And they are not photographs: the count the user is shown must not
// double because a library has been edited.
assert_eq!(r.progress.images_found, 3);
assert!(r.images.iter().all(|e| !e.path.name().contains("drsc")));
assert!(r
.images
.iter()
.all(|e| !e.path.name().contains("drsc") && !e.path.name().contains("xmp")));
}
/// A file whose *name* is only an extension is not a sidecar for anything.
+13 -1
View File
@@ -89,6 +89,15 @@ pub struct LibrarySettings {
/// 64 bars on a phone held in the hand is finer than a finger can aim at;
/// 32 on a desktop monitor wastes most of a tall sidebar.
pub timeline_bars: u32,
/// TRACES: FR-CAT-13 | NFR-R4
/// Whether a judgement is also written to the standard XMP sidecar beside
/// the original — `IMG_0001.xmp`, or `IMG_0001.CR3.xmp` where one exists.
///
/// Off by default, because NFR-R4 says writes beside somebody's originals
/// are theirs to switch on. Reading is not gated: a sidecar another
/// editor wrote is taken in regardless, since reading changes nothing in
/// the folder.
pub write_xmp_sidecars: bool,
}
impl LibrarySettings {
@@ -106,7 +115,10 @@ impl Default for LibrarySettings {
// The coarser of the two. A bar has to be wide enough to hit with a
// finger before it has to be narrow enough to be precise, and the
// smallest screen is the one where getting this wrong hurts most.
Self { timeline_bars: 32 }
Self {
timeline_bars: 32,
write_xmp_sidecars: false,
}
}
}
+11 -11
View File
@@ -25,7 +25,7 @@ Sampling a neutral is the first move of the tonal pass — every colour judgemen
Anchored on the fingers' midpoint, and on the pointer, so the gesture reads as magnifying the picture rather than sliding it about. Double-tap is the way to an exact 1:1; this is the way to everything in between.
<sub>`ui/dr-ui/ui/app.slint:1965`</sub>
<sub>`ui/dr-ui/ui/app.slint:1974`</sub>
### Move a magnified photograph about
@@ -34,7 +34,7 @@ Anchored on the fingers' midpoint, and on the pointer, so the gesture reads as m
Only once there is something outside the viewport to reach, which is why the cursor becomes a hand exactly then. The view is clamped to the frame: panning past the edge would show undefined area beside the photograph, and that reads as a rendering fault rather than as the end of the picture.
<sub>`ui/dr-ui/ui/app.slint:2056`</sub>
<sub>`ui/dr-ui/ui/app.slint:2065`</sub>
### Paint a mask by hand
@@ -43,7 +43,7 @@ Only once there is something outside the viewport to reach, which is why the cur
A model's mask stops inside a shoulder and leaks into the hair, and no single edge control fixes two errors that go opposite ways. The whole stroke is one step in the history, so taking a mark back costs one press however long it took to make.
<sub>`ui/dr-ui/ui/app.slint:2143`</sub>
<sub>`ui/dr-ui/ui/app.slint:2152`</sub>
### Take back the last change
@@ -53,7 +53,7 @@ A model's mask stops inside a shoulder and leaks into the hair, and no single ed
A whole drag is one step, so undo takes back a decision rather than a frame of a gesture. The list is there because arriving six steps back costs what arriving from one does.
<sub>`ui/dr-ui/ui/app.slint:2364`</sub>
<sub>`ui/dr-ui/ui/app.slint:2373`</sub>
### Do it again after taking it back
@@ -61,7 +61,7 @@ A whole drag is one step, so undo takes back a decision rather than a frame of a
- **Pointer** — Click it, or press Redo in the History header
- **Keyboard** — Ctrl+Shift+Z
<sub>`ui/dr-ui/ui/app.slint:2377`</sub>
<sub>`ui/dr-ui/ui/app.slint:2386`</sub>
### Copy the settings from this photograph
@@ -71,7 +71,7 @@ A whole drag is one step, so undo takes back a decision rather than a frame of a
The panel is the copy that has to work: a tablet has no modifier key to hold and no menu bar to hang the action from. The shortcut is an accelerator for a control that is on screen either way.
<sub>`ui/dr-ui/ui/app.slint:2410`</sub>
<sub>`ui/dr-ui/ui/app.slint:2419`</sub>
### Paste the settings onto this photograph
@@ -81,7 +81,7 @@ The panel is the copy that has to work: a tablet has no modifier key to hold and
The button names what would be pasted — "3 adjustments", and whether the crop is coming with it — which the shortcut cannot say. Both paste the same scope.
<sub>`ui/dr-ui/ui/app.slint:2422`</sub>
<sub>`ui/dr-ui/ui/app.slint:2431`</sub>
### Change which group of adjustments is on screen
@@ -91,7 +91,7 @@ The button names what would be pasted — "3 adjustments", and whether the crop
The groups are whatever the operation set declares itself to be about, so there are as many as the pipeline has and no key can be assigned to one of them by name. Stepping is the binding that survives a node being added.
<sub>`ui/dr-ui/ui/app.slint:2450`</sub>
<sub>`ui/dr-ui/ui/app.slint:2459`</sub>
### Look at the photograph at 1:1
@@ -101,7 +101,7 @@ The groups are whatever the operation set declares itself to be about, so there
Noise reduction and capture sharpening are judgements about single pixels, and a fitted view averages several of the file's into each one on screen — so the frame looks softer than it is and the correction goes too far. The point and the magnification survive opening the next photograph, which is what makes checking the same eye across forty portraits forty keystrokes rather than forty pans.
<sub>`ui/dr-ui/ui/app.slint:2485`</sub>
<sub>`ui/dr-ui/ui/app.slint:2494`</sub>
### Move to the next or previous photograph
@@ -111,7 +111,7 @@ Noise reduction and capture sharpening are judgements about single pixels, and a
The edit on screen is saved on the way out, so stepping through a folder is as much a departure as going back to the grid and loses nothing.
<sub>`ui/dr-ui/ui/app.slint:2537`</sub>
<sub>`ui/dr-ui/ui/app.slint:2546`</sub>
### See the photograph before you edited it
@@ -121,7 +121,7 @@ The edit on screen is saved on the way out, so stepping through a folder is as m
Held rather than toggled, and no split screen: a split halves the working image on the tablet the column was sized for, and the comparison photographers describe making is a flick back and forth. It takes no history step, so checking whether a frame is overcooked costs nothing to undo afterwards.
<sub>`ui/dr-ui/ui/app.slint:2661`</sub>
<sub>`ui/dr-ui/ui/app.slint:2670`</sub>
### Put one control back to its default
+16 -14
View File
@@ -308,7 +308,7 @@ well optimised — ETag pruning under FR-NC-4 turns an unchanged 50k library int
gap is narrower than it reads. It is the *first* build against a large remote library that pays, and
that is the moment a new user meets.
**FR-CAT-13 — XMP interoperability, built but not yet wired.** `core/dr-xmp` now reads and writes
**FR-CAT-13 — XMP interoperability, wired on 2026-09-12.** `core/dr-xmp` reads and writes
standard XMP sidecars: `dc:subject` and `lr:hierarchicalSubject`, `xmp:Rating` and `xmp:Label`, and
the IPTC core fields, in both the attribute and the element form and whatever RDF container a file
happened to use. It states the ownership rule in one place — DarkRoom owns the properties in
@@ -316,20 +316,22 @@ happened to use. It states the ownership rule in one place — DarkRoom owns the
and enforces it by rewriting a packet event by event rather than serialising over it, so another
application's `crs:` settings, comments and processing instructions survive a write byte for byte.
**What remains is the wiring, and it is the larger half.** Nothing above the crate calls it: no scan
finds a `.xmp` beside a raw, no catalog row is populated from one, no edit writes one back, and the
external-modification detection the requirement also asks for does not exist. Two smaller gaps go
with it — GPS is not carried (`exif:GPSLatitude` is a format of its own, and `dr-decode` produces no
location for it to carry yet, which `dr-export`'s metadata module says about its own half), and the
filename convention is left to the caller, because Lightroom writes `IMG_0001.xmp` and darktable
writes `IMG_0001.CR3.xmp` and finding a file is not this crate's business.
**The wiring is `ui/dr-ui/src/xmp_sync.rs`.** The scan collects `.xmp` beside `.drsc` from the
listings it was already making; the pull reads each one whose ETag has moved and reconciles it
against the catalog with the catalog winning — keywords union, and a rating, label or caption taken
only where the catalog holds none. Both naming conventions resolve: `IMG_0001.CR3.xmp` names its
file, `IMG_0001.xmp` the stem, and the JPEG beside a RAW is the same photograph. A genuine
disagreement is written to `xmp_conflicts` and the settings page offers "Take the sidecars' values",
which is the reload the requirement asks for; the detection it asks for is the ETag that moved. The
write in the other direction is behind a setting that starts off (NFR-R4): a judgement or a keyword
then also rewrites the sidecar beside the original, keeping the file's own caption, copyright and
hierarchy, which the catalog has no columns for and would otherwise have deleted.
The precedence question is settled conservatively rather than fully: keywords union, following
`dr_catalog::merge`, and every other field is taken only where DarkRoom holds none, following
`Version::merge`'s judgement rule — because a standard XMP carries no revision and no device, so
FR-NC-9's ordering cannot be performed against it. What is *not* settled, and is written down in the
crate rather than guessed at, is when a reload may happen without asking; see the module
documentation's "The open question".
**What remains.** GPS is not carried — `exif:GPSLatitude` is a format of its own and `dr-decode`
produces no location for it to carry yet. Title, description and copyright are read and reconciled
but the catalog has nowhere to put them, so they pass through a rewrite rather than being editable.
An XMP write made offline is not queued: the catalog and the `.drsc` are authoritative, and the next
judgement online writes the file whole again.
`dr-preset-xmp` remains what it always was and is still not the counter-example it looks like: a
reader of Lightroom *presets* under FR-DEV-6, a different file for a different purpose.
+78 -78
View File
File diff suppressed because one or more lines are too long
+1
View File
@@ -29,6 +29,7 @@ tokio.workspace = true
reqwest.workspace = true
dr-plat.workspace = true
dr-sync.workspace = true
dr-xmp.workspace = true
dr-sync-folder.workspace = true
dr-sync-nextcloud.workspace = true
dr-export.workspace = true
+3
View File
@@ -59,6 +59,7 @@ mod settings_ui;
mod sidecar_cache;
mod spots_ui;
mod trash;
mod xmp_sync;
use std::cell::{Cell, RefCell};
use std::path::{Path, PathBuf};
@@ -1583,6 +1584,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
let stored = settings.snapshot();
library.set_cache_budget(stored.cache.original_budget_bytes);
library.set_keep_opened_originals(stored.cache.keep_opened_originals);
library.set_write_xmp_sidecars(stored.library.write_xmp_sidecars);
library.set_timeline_bars(stored.library.timeline_bars);
library.set_face_model_id(stored.faces.detector.model_id());
}
@@ -1714,6 +1716,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
// cache would sit over budget indefinitely.
lib.set_cache_budget(s.cache.original_budget_bytes);
lib.set_keep_opened_originals(s.cache.keep_opened_originals);
lib.set_write_xmp_sidecars(s.library.write_xmp_sidecars);
if let Some(w) = weak.upgrade() {
refresh_export_label(&w, &ctl);
}
+213
View File
@@ -1498,6 +1498,34 @@ async fn pull_sidecars(
};
let text = String::from_utf8_lossy(&bytes);
// TRACES: FR-CAT-13
// A standard XMP beside the photograph — Lightroom's, darktable's,
// anybody's — takes the other branch: reconciled field by field with
// the catalog winning, and a disagreement recorded for the reload
// the requirement asks to be offered. See `xmp_sync`.
if crate::xmp_sync::is_xmp(path) {
match crate::xmp_sync::take_in(conn, root_id, path, &text, now_secs()) {
Ok(taken) => {
applied += taken.changed;
if !taken.conflicts.is_empty() {
log::info!(
"xmp sidecar {path} disagrees with the catalog on {:?}; \
a reload is offered in Settings",
taken.conflicts
);
}
// Recorded only once it reached a photograph: a sidecar
// that arrived before its image is read again next time.
if taken.described > 0 {
record_sidecar_read(conn, root_id, path, &entry.validator);
}
}
Err(e) => log::warn!("xmp sidecar at {path} is unreadable ({e})"),
}
continue;
}
let mut sidecar = match dr_pipeline::Sidecar::parse(&text) {
Ok(s) => s,
// Unreadable is not empty. Recording the ETag would mean never
@@ -1534,6 +1562,191 @@ async fn pull_sidecars(
Ok(applied)
}
/// TRACES: FR-CAT-13 | NFR-R4
/// One image's ratings, label and keywords, on their way to the `.xmp`
/// beside it.
///
/// Distinct from [`SidecarWrite`], which amends DarkRoom's own document
/// through the cache and the outbox. This is best-effort in the other
/// direction: the catalog and the `.drsc` are authoritative, the `.xmp` is a
/// courtesy to whatever else reads the folder, and a write that cannot
/// happen now is written again — from the catalog, whole — the next time
/// anything about the photograph is judged. So nothing is queued.
#[derive(Debug, Clone)]
pub struct XmpWrite {
pub image_path: String,
pub record: dr_xmp::Xmp,
}
/// TRACES: FR-CAT-13 | NFR-R4
/// Write each record into the XMP sidecar beside its image.
///
/// An existing sidecar of either spelling is rewritten in place, which is the
/// whole point of `dr_xmp::rewrite`: only the properties DarkRoom owns move,
/// and another application's settings, comments and namespaces come through
/// byte for byte. A photograph with neither gets a new file under Lightroom's
/// name. Reported once at the end, as the judgement writes are.
pub fn spawn_xmp_writes(conn: Connection, writes: Vec<XmpWrite>) -> Receiver<XmpMessage> {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let mut written = 0usize;
let mut failed = 0usize;
let mut last_error = None;
let rt = match crate::net_runtime::build() {
Ok(rt) => rt,
Err(e) => {
let _ = tx.send(XmpMessage::Finished {
written: 0,
failed: writes.len(),
last_error: Some(e.to_string()),
});
return;
}
};
rt.block_on(async {
let backend = match crate::remote::connect(&conn) {
Ok(b) => b,
Err(e) => {
failed = writes.len();
last_error = Some(e.to_string());
return;
}
};
for w in &writes {
match write_one_xmp(&*backend, w).await {
Ok(()) => written += 1,
Err(e) => {
log::warn!("xmp sidecar for {}: {e}", w.image_path);
last_error = Some(e);
failed += 1;
}
}
}
});
let _ = tx.send(XmpMessage::Finished {
written,
failed,
last_error,
});
});
rx
}
/// The outcome of a batch of XMP writes.
#[derive(Debug)]
pub enum XmpMessage {
Finished {
written: usize,
failed: usize,
last_error: Option<String>,
},
}
/// Read-modify-write one image's XMP sidecar on the server.
async fn write_one_xmp(backend: &dyn RemoteBackend, w: &XmpWrite) -> Result<(), String> {
let [darktable, lightroom] = crate::xmp_sync::candidate_paths(&w.image_path);
// Whichever exists is the one rewritten; neither existing means the
// Lightroom spelling is created. An existing file this build cannot
// parse is left alone rather than replaced — it is somebody else's
// document, and a refusal is recoverable where an overwrite is not.
let mut target = lightroom.clone();
let mut existing: Option<String> = None;
for path in [&darktable, &lightroom] {
if let Ok(bytes) = backend
.get(&RemoteId::Path(RemotePath::new(path.clone())), None)
.await
{
if !bytes.is_empty() {
target = path.clone();
existing = Some(String::from_utf8_lossy(&bytes).into_owned());
break;
}
}
}
let text = match existing {
Some(text) => {
// The file's caption, copyright and hierarchy come through: the
// catalog has nowhere to keep them, and a rewrite that said
// nothing about them would remove them.
let theirs = dr_xmp::Xmp::parse(&text)
.map_err(|e| format!("{target} is not a sidecar this build can read: {e}"))?;
let mut record = w.record.clone();
crate::xmp_sync::carry_through(&mut record, &theirs);
record
.rewrite(&text)
.map_err(|e| format!("{target} is not a sidecar this build can rewrite: {e}"))?
}
None => w.record.to_text(),
};
backend
.put(&RemotePath::new(target), text.into_bytes(), None)
.await
.map(|_| ())
.map_err(|e| e.to_string())
}
/// TRACES: FR-CAT-13
/// The offered reload: re-read the sidecars a person chose to trust, with
/// the sidecar winning. One fetch per path, the catalog opened on this
/// thread as the scan opens it.
pub fn spawn_xmp_reload(
conn: Connection,
root: String,
catalog_path: PathBuf,
paths: Vec<String>,
) -> Receiver<XmpMessage> {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let mut written = 0usize;
let mut failed = 0usize;
let mut last_error = None;
let outcome: Result<(), String> = (|| {
let catalog = Catalog::open(&catalog_path).map_err(|e| e.to_string())?;
let root_id: i64 = catalog
.connection()
.query_row(
"SELECT id FROM roots WHERE label = ?1 AND kind = 'remote'",
[&root],
|r| r.get(0),
)
.map_err(|e| e.to_string())?;
let rt = crate::net_runtime::build().map_err(|e| e.to_string())?;
rt.block_on(async {
let backend = crate::remote::connect(&conn).map_err(|e| e.to_string())?;
for path in &paths {
let fetched = backend
.get(&RemoteId::Path(RemotePath::new(path.clone())), None)
.await
.map_err(|e| e.to_string())
.and_then(|bytes| {
let text = String::from_utf8_lossy(&bytes);
crate::xmp_sync::reload(catalog.connection(), root_id, path, &text)
});
match fetched {
Ok(taken) => written += taken.changed,
Err(e) => {
log::warn!("reloading {path}: {e}");
last_error = Some(e);
failed += 1;
}
}
}
Ok(())
})
})();
if let Err(e) = outcome {
failed = paths.len();
last_error = Some(e);
}
let _ = tx.send(XmpMessage::Finished {
written,
failed,
last_error,
});
});
rx
}
/// What this device has already read, so a pull fetches only what changed.
fn load_sidecar_etags(
catalog: &Catalog,
+215
View File
@@ -298,6 +298,9 @@ pub struct LibraryController {
/// Drains the sidecar writer. Held so a second judgement replaces the
/// timer rather than leaving two draining the same finished channel.
sidecar_timer: RefCell<Option<slint::Timer>>,
/// TRACES: FR-CAT-13
/// The same, for a batch of XMP writes or a reload.
xmp_timer: RefCell<Option<slint::Timer>>,
/// Which window load the model belongs to, bumped by [`load_window`].
///
/// A thumbnail worker addresses cells by *row index into the window that
@@ -381,6 +384,10 @@ pub struct LibraryController {
/// small budget still keeps a working set. A metered or small-disk device
/// wants the first.
keep_opened: std::cell::Cell<bool>,
/// TRACES: FR-CAT-13 | NFR-R4
/// Whether judgements and keywords also go to the `.xmp` beside the
/// original. Mirrors `LibrarySettings::write_xmp_sidecars`.
write_xmp: std::cell::Cell<bool>,
/// TRACES: FR-CAT-6
/// How many bars the capture-time axis is cut into, from the settings
/// page.
@@ -450,6 +457,7 @@ impl LibraryController {
current_bucket: RefCell::new(None),
filter: RefCell::new(library::RatingFilter::default()),
sidecar_timer: RefCell::new(None),
xmp_timer: RefCell::new(None),
generation: std::cell::Cell::new(0),
reachability: RefCell::new(dr_sync::Reachability::new()),
root_lost: RefCell::new(None),
@@ -466,6 +474,9 @@ impl LibraryController {
keep_opened: std::cell::Cell::new(
dr_types::CacheSettings::default().keep_opened_originals,
),
write_xmp: std::cell::Cell::new(
dr_types::LibrarySettings::default().write_xmp_sidecars,
),
timeline_bars: std::cell::Cell::new(dr_types::LibrarySettings::default().timeline_bars),
face_model_id: RefCell::new(dr_types::FaceDetector::default().model_id().to_string()),
})
@@ -505,6 +516,11 @@ impl LibraryController {
self.keep_opened.set(keep);
}
/// TRACES: FR-CAT-13 | NFR-R4
pub fn set_write_xmp_sidecars(&self, on: bool) {
self.write_xmp.set(on);
}
/// TRACES: FR-NC-6a
/// Set the ceiling on passively cached originals, and apply it now.
///
@@ -1266,6 +1282,11 @@ fn drain_scan(
log::info!("library folder is readable again");
}
refresh_offline(&w, ctl);
// TRACES: FR-CAT-13
// The pull may have found an `.xmp` that disagrees
// with the catalog; the settings page offers the
// reload, and this is what tells it how many.
refresh_xmp_conflicts(&w, ctl);
// An incremental rescan lists almost nothing, so
// reporting the listed count would read as "0 images"
@@ -3086,6 +3107,7 @@ fn apply_keyword(window: &AppWindow, ctl: &Rc<LibraryController>, word: &str, as
window.set_library_error(slint::SharedString::new());
window.set_library_status(keyword_summary(&word, n, images.len(), assigning).into());
refresh_keywords(window, ctl, &images);
start_xmp_writes(window, ctl, &images);
// A filtered grid may no longer hold what was just keyworded — taking
// "puffin" off an image while showing only puffins means it belongs
@@ -3194,6 +3216,7 @@ fn apply_judgement(
}
start_sidecar_writes(window, ctl, writes);
start_xmp_writes(window, ctl, images);
}
/// What the status line says about a judgement that just landed.
@@ -3385,6 +3408,186 @@ fn collect_sidecar_writes(
}
/// Push judgements out to sidecars on a worker, reporting once at the end.
/// TRACES: FR-CAT-13 | NFR-R4
/// Write these images' ratings, labels and keywords to the `.xmp` beside
/// each, where the user has switched that on.
///
/// The record is read from the catalog *now*, whole, rather than carried
/// from the gesture: a rating and a keyword typed a second apart are two
/// writes of the same file, and the second must not carry a copy of the
/// first taken before it landed.
pub(crate) fn start_xmp_writes(
window: &AppWindow,
ctl: &Rc<LibraryController>,
images: &[dr_types::ImageId],
) {
if !ctl.write_xmp.get() || images.is_empty() || ctl.is_offline() {
return;
}
let Some((conn, _)) = ctl.session.borrow().clone() else {
return;
};
let writes: Vec<library::XmpWrite> = {
let borrow = ctl.catalog.borrow();
let Some(catalog) = borrow.as_ref() else {
return;
};
let c = catalog.connection();
images
.iter()
.filter_map(|&image| {
let version = dr_catalog::rating::default_version_id(c, image).ok()?;
let image_path: String = c
.query_row(
"SELECT source_ref FROM images WHERE id = ?1",
[image.0 as i64],
|r| r.get(0),
)
.ok()?;
Some(library::XmpWrite {
image_path,
record: crate::xmp_sync::record_of(c, image, version),
})
})
.collect()
};
if writes.is_empty() {
return;
}
let count = writes.len();
let rx = library::spawn_xmp_writes(conn, writes);
let job = ctl.activity.begin(
crate::activity::Kind::Upload,
format!("Writing {count} XMP sidecar(s)"),
);
drain_xmp(window.as_weak(), ctl.clone(), rx, job, false);
}
/// TRACES: FR-CAT-13
/// The offered reload: take the sidecars' values for every photograph the
/// last pull found disagreeing with the catalog.
pub(crate) fn start_xmp_reload(window: &AppWindow, ctl: &Rc<LibraryController>) {
let Some((conn, _)) = ctl.session.borrow().clone() else {
return;
};
let paths: Vec<String> = {
let borrow = ctl.catalog.borrow();
let Some(catalog) = borrow.as_ref() else {
return;
};
let Some(root_id) = root_id_of(catalog, &conn.account.root) else {
return;
};
crate::xmp_sync::conflicts(catalog, root_id)
.into_iter()
.map(|c| c.path)
.collect()
};
if paths.is_empty() {
return;
}
let count = paths.len();
let rx = library::spawn_xmp_reload(
conn.clone(),
conn.account.root.clone(),
library::catalog_path(&conn.account),
paths,
);
let job = ctl.activity.begin(
crate::activity::Kind::Download,
format!("Reloading {count} XMP sidecar(s)"),
);
drain_xmp(window.as_weak(), ctl.clone(), rx, job, true);
}
/// Wait for a batch of XMP work to report, then say what it did.
fn drain_xmp(
weak: slint::Weak<AppWindow>,
ctl: Rc<LibraryController>,
rx: std::sync::mpsc::Receiver<library::XmpMessage>,
job: crate::activity::Activity,
reload: bool,
) {
let timer = slint::Timer::default();
let job = RefCell::new(Some(job));
let ctl_cb = ctl.clone();
timer.start(
slint::TimerMode::Repeated,
std::time::Duration::from_millis(200),
move || {
let ctl = &ctl_cb;
let message = match rx.try_recv() {
Ok(m) => m,
Err(std::sync::mpsc::TryRecvError::Empty) => return,
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
stop(&ctl.xmp_timer);
return;
}
};
stop(&ctl.xmp_timer);
let library::XmpMessage::Finished {
written,
failed,
last_error,
} = message;
let status = match (reload, failed, last_error) {
(false, 0, _) => format!("{written} XMP sidecar(s) written"),
(true, 0, _) => format!("{written} photograph(s) reloaded from XMP"),
(_, n, Some(e)) => format!("{n} XMP sidecar(s) failed: {e}"),
(_, n, None) => format!("{n} XMP sidecar(s) failed"),
};
if let Some(job) = job.borrow_mut().take() {
if failed > 0 {
job.fail(status.clone());
} else {
job.finish(status.clone());
}
}
if let Some(w) = weak.upgrade() {
w.set_library_status(status.into());
if reload {
// The grid draws what the reload changed, and the
// settings page stops offering what is settled.
let visible = ctl.visible_ids();
if let Some(catalog) = ctl.catalog.borrow().as_ref() {
sync_ratings(&w, catalog, &visible);
refresh_rating_counts(&w, catalog);
}
refresh_xmp_conflicts(&w, ctl);
}
}
},
);
*ctl.xmp_timer.borrow_mut() = Some(timer);
}
/// TRACES: FR-CAT-13
/// How many sidecars the last pull found disagreeing with the catalog, for
/// the settings page to offer the reload against.
pub(crate) fn refresh_xmp_conflicts(window: &AppWindow, ctl: &Rc<LibraryController>) {
let count = (|| {
let (conn, _) = ctl.session.borrow().clone()?;
let borrow = ctl.catalog.borrow();
let catalog = borrow.as_ref()?;
let root_id = root_id_of(catalog, &conn.account.root)?;
Some(crate::xmp_sync::conflicts(catalog, root_id).len())
})()
.unwrap_or(0);
window.set_settings_xmp_conflicts(count as i32);
}
fn root_id_of(catalog: &Catalog, root: &str) -> Option<i64> {
catalog
.connection()
.query_row(
"SELECT id FROM roots WHERE label = ?1 AND kind = 'remote'",
[root],
|r| r.get(0),
)
.ok()
}
pub(crate) fn start_sidecar_writes(
window: &AppWindow,
ctl: &Rc<LibraryController>,
@@ -5503,6 +5706,18 @@ pub fn wire<F>(
// controllers would hold each other alive for the life of the process.
*ctl.coll_ctl.borrow_mut() = Some(Rc::downgrade(&coll_ctl));
// TRACES: FR-CAT-13
// The reload the settings page offers when a sidecar disagrees.
{
let weak = window.as_weak();
let ctl = ctl.clone();
window.on_settings_xmp_reload(move || {
if let Some(w) = weak.upgrade() {
start_xmp_reload(&w, &ctl);
}
});
}
// TRACES: FR-UI-3 | FR-UI-4
// Whether the rating strip waits to be hovered or stands open.
//
+11
View File
@@ -154,6 +154,7 @@ pub fn render(window: &AppWindow, controller: &SettingsController) {
window.set_settings_thumbnail_budget(budget::label(s.cache.thumbnail_budget_bytes).into());
window.set_settings_thumbnail_unlimited(s.cache.thumbnail_budget_bytes.is_none());
window.set_settings_keep_opened(s.cache.keep_opened_originals);
window.set_settings_write_xmp(s.library.write_xmp_sidecars);
// --- develop -------------------------------------------------------
//
@@ -501,6 +502,16 @@ pub fn wire<F, G>(
render(&w, &ctl);
});
}
{
// TRACES: FR-CAT-13 | NFR-R4
let weak = window.as_weak();
let ctl = controller.clone();
window.on_settings_write_xmp_toggled(move |on| {
let Some(w) = weak.upgrade() else { return };
ctl.edit(|s| s.library.write_xmp_sidecars = on);
render(&w, &ctl);
});
}
// --- faces ---------------------------------------------------------
//
+597
View File
@@ -0,0 +1,597 @@
//! TRACES: FR-CAT-13 | FR-NC-9 | NFR-R4
//! Standard XMP sidecars, read into the catalog and written back out of it.
//!
//! `dr-xmp` reads and writes the file. This is the other half the requirement
//! asks for and the half `docs/outstanding.md` called "the larger": which
//! images a given `.xmp` describes, what the catalog holds about them, how
//! the two are reconciled, where a disagreement goes, and the write in the
//! other direction. Nothing here parses XML and nothing in `dr-xmp` knows a
//! catalog exists.
//!
//! # Two conventions for one file
//!
//! Lightroom writes `IMG_0001.xmp` beside `IMG_0001.CR3`; darktable writes
//! `IMG_0001.CR3.xmp`. Both are answered by [`images_for`]: the path with
//! `.xmp` taken off is tried as an image path first, exactly, and failing that
//! as a stem shared with the images beside it — the rule DarkRoom's own
//! sidecar already follows, under which a RAW and the JPEG the camera wrote
//! beside it are one photograph (FR-CAT-11) and share the document.
//!
//! # Precedence, and where a disagreement goes
//!
//! A standard XMP carries no revision and no device, so nothing in it can say
//! whether its rating is newer than the catalog's. The automatic pull
//! therefore runs [`dr_xmp::reconcile`] with the catalog winning: keywords
//! union, and every whole-valued field is taken only where the catalog holds
//! none. A genuine disagreement — both sides hold a value, and different ones —
//! is written to `xmp_conflicts` rather than resolved, and the requirement's
//! "a metadata reload offered" is that table with a button in front of it.
//! [`reload`] is the button: the same reconciliation with the sidecar winning,
//! asked for by a person.
//!
//! # Detection
//!
//! "External modification of an XMP sidecar shall be detected." The scan
//! already records the ETag of every sidecar it has taken in, and fetches
//! only those whose ETag has moved. An `.xmp` edited in another application
//! is precisely a file whose ETag has moved, so the detection is the pull's
//! ordinary incrementality — nothing watches a directory, and nothing needs
//! to. What is new is what happens after: the re-read reconciles again, and
//! the second reading is where a conflict first appears.
//!
//! # Writing, and why it is off
//!
//! NFR-R4 makes source-adjacent writes opt-in. A library shared with another
//! editor is one where a file DarkRoom wrote can be read by something else,
//! and that is exactly what [`Xmp::rewrite`] is built for — it rewrites only
//! the properties DarkRoom owns and copies everything else through byte for
//! byte. But the option to write beside somebody's originals is theirs to
//! switch on, and until they do the catalog and DarkRoom's own sidecar are the
//! only things a judgement reaches.
use dr_catalog::Catalog;
use dr_types::{FlagState, ImageId};
use dr_xmp::{Precedence, Rating, Xmp};
use rusqlite::Connection;
/// The two places an image's XMP sidecar may be, in the order they are
/// tried when writing: the darktable spelling first, because it names the
/// image unambiguously, then Lightroom's.
///
/// When reading, whichever the scan found is the one read; this is for the
/// write, which has to choose. An existing file of either spelling is
/// rewritten in place, and a photograph with neither gets Lightroom's, since
/// it is the spelling more applications look for.
pub fn candidate_paths(image_path: &str) -> [String; 2] {
let stem = match image_path.rsplit_once('.') {
Some((stem, ext)) if !ext.contains('/') => stem,
_ => image_path,
};
[format!("{image_path}.xmp"), format!("{stem}.xmp")]
}
/// Whether a listing entry is a standard XMP sidecar rather than DarkRoom's.
pub fn is_xmp(path: &str) -> bool {
path.rsplit_once('.')
.is_some_and(|(_, ext)| ext.eq_ignore_ascii_case(dr_sync::scan::XMP_EXTENSION))
}
/// The images an `.xmp` at `path` describes: their ids and default versions.
///
/// Empty when the sidecar sits beside nothing this catalog knows, which is
/// the ordinary case for a file that arrived before its photograph was
/// scanned — the next pull reads it again, because no ETag is recorded for a
/// sidecar that reached nothing.
pub fn images_for(conn: &Connection, root_id: i64, path: &str) -> Vec<(ImageId, i64)> {
let Some(named) = path
.strip_suffix(".xmp")
.or_else(|| path.strip_suffix(".XMP"))
else {
return Vec::new();
};
let ids = |sql: &str, arg: &str| -> Vec<ImageId> {
let Ok(mut stmt) = conn.prepare(sql) else {
return Vec::new();
};
stmt.query_map(rusqlite::params![root_id, arg], |r| r.get::<_, i64>(0))
.map(|rows| rows.flatten().map(|i| ImageId(i as u64)).collect())
.unwrap_or_default()
};
// darktable's spelling: the name before `.xmp` is the image itself.
let mut images = ids(
"SELECT id FROM images WHERE root_id = ?1 AND source_ref = ?2",
named,
);
if images.is_empty() {
// Lightroom's: a stem shared with the photograph, and with the JPEG
// beside it. The escape is what makes `%` and `_` in a folder name
// literal, and the Rust-side check is what actually decides — a LIKE
// is a filter, not an answer.
let prefix = named
.replace('\\', "\\\\")
.replace('%', "\\%")
.replace('_', "\\_");
let candidates: Vec<(ImageId, String)> = {
let Ok(mut stmt) = conn.prepare(
"SELECT id, source_ref FROM images
WHERE root_id = ?1 AND source_ref LIKE ?2 ESCAPE '\\'",
) else {
return Vec::new();
};
stmt.query_map(rusqlite::params![root_id, format!("{prefix}.%")], |r| {
Ok((ImageId(r.get::<_, i64>(0)? as u64), r.get::<_, String>(1)?))
})
.map(|rows| rows.flatten().collect())
.unwrap_or_default()
};
images = candidates
.into_iter()
.filter(|(_, source)| candidate_paths(source)[1].eq_ignore_ascii_case(path))
.map(|(id, _)| id)
.collect();
}
images
.into_iter()
.filter_map(|image| {
let version = dr_catalog::rating::default_version_id(conn, image).ok()?;
Some((image, version))
})
.collect()
}
/// What the catalog holds about one image, as the sidecar would carry it.
///
/// Rating and flag are two axes here and one field there: a rejection is
/// written as Adobe's `-1` and a rating as its stars, and a frame that is
/// both rejected and starred loses the stars in the file — the loss the
/// format has and `dr_xmp::Rating` records. Reading goes the other way in
/// [`apply`].
pub fn record_of(conn: &Connection, image: ImageId, version: i64) -> Xmp {
let mut xmp = Xmp::default();
let row = conn
.query_row(
"SELECT rating, flag, label FROM versions WHERE id = ?1",
[version],
|r| {
Ok((
r.get::<_, i64>(0)?,
r.get::<_, i64>(1)?,
r.get::<_, Option<i64>>(2)?,
))
},
)
.ok();
if let Some((rating, flag, label)) = row {
xmp.rating = if flag == flag_code(FlagState::Reject) {
Some(Rating::Rejected)
} else if rating > 0 {
Some(Rating::Stars(rating.clamp(0, 5) as u8))
} else {
None
};
xmp.set_colour(dr_catalog::rating::label_from_code(label));
}
xmp.keywords = dr_catalog::keywords::for_image(conn, image).unwrap_or_default();
xmp
}
/// Write a reconciled record onto one image.
///
/// Only what the record holds: a `None` rating is *unrated* and is not
/// written over a star, for the reason `dr_pipeline::sidecar::merge_judgement`
/// gives — a file that says nothing cannot erase an afternoon's culling. A
/// rejection sets the flag and leaves the stars alone; stars set the rating
/// and, if the frame was rejected, lift the rejection, since the file said
/// it was worth a number. Keywords are added and never removed here, which
/// is what a union means. `label` is set where the file names one of the
/// five colours.
///
/// Returns whether any column moved.
pub fn apply(
conn: &Connection,
image: ImageId,
version: i64,
record: &Xmp,
) -> Result<bool, String> {
let mut changed = false;
match record.rating {
Some(Rating::Rejected) => {
changed |= conn
.execute(
"UPDATE versions SET flag = ?2 WHERE id = ?1 AND flag <> ?2",
rusqlite::params![version, flag_code(FlagState::Reject)],
)
.map_err(|e| e.to_string())?
> 0;
}
Some(Rating::Stars(n)) if n > 0 => {
changed |= conn
.execute(
"UPDATE versions
SET rating = ?2,
flag = CASE WHEN flag = ?3 THEN 0 ELSE flag END
WHERE id = ?1 AND (rating <> ?2 OR flag = ?3)",
rusqlite::params![version, n.min(5) as i64, flag_code(FlagState::Reject)],
)
.map_err(|e| e.to_string())?
> 0;
}
_ => {}
}
if let Some(colour) = record.colour() {
changed |= conn
.execute(
"UPDATE versions SET label = ?2 WHERE id = ?1 AND label IS NOT ?2",
rusqlite::params![version, dr_catalog::rating::label_code(colour)],
)
.map_err(|e| e.to_string())?
> 0;
}
if !record.keywords.is_empty() {
let have = dr_catalog::keywords::for_image(conn, image).unwrap_or_default();
for word in &record.keywords {
if have.iter().any(|h| h.eq_ignore_ascii_case(word)) {
continue;
}
match dr_catalog::keywords::assign(conn, &[image], word) {
Ok(n) => changed |= n > 0,
// A word the vocabulary refuses — empty once trimmed, or a
// separator on its own — costs that word and not the file.
Err(e) => log::debug!("xmp keyword {word:?} on {}: {e}", image.0),
}
}
}
Ok(changed)
}
/// TRACES: FR-CAT-13
/// Before rewriting an existing sidecar, keep what the catalog cannot hold.
///
/// `Xmp::rewrite` replaces the owned properties wholesale — that is what
/// makes "no rating" mean the rating goes — and the catalog has columns for
/// three of them: rating and flag, label, keywords. A record built from the
/// catalog therefore says nothing about a title, a caption, a copyright
/// line or a hierarchical subject, and writing it as it stands would delete
/// all four from a sidecar Lightroom wrote, on every judgement. So those
/// come through from the file, and so does a label that is not one of the
/// five colours — the catalog kept no text for it, and the file's is the
/// only copy.
pub fn carry_through(record: &mut Xmp, existing: &Xmp) {
record.hierarchical_subjects = existing.hierarchical_subjects.clone();
record.title = existing.title.clone();
record.description = existing.description.clone();
record.creators = existing.creators.clone();
record.copyright = existing.copyright.clone();
record.credit = existing.credit.clone();
record.usage_terms = existing.usage_terms.clone();
if record.label.is_none() && existing.colour().is_none() {
record.label = existing.label.clone();
}
}
/// What one sidecar did when taken in.
#[derive(Debug, Default, Clone, PartialEq, Eq)]
pub struct TakenIn {
/// Images whose rows moved.
pub changed: usize,
/// Images the sidecar described at all. Zero means it sits beside nothing
/// this catalog has yet, and its ETag must not be recorded.
pub described: usize,
/// The fields both sides held and disagreed on, across every image.
pub conflicts: Vec<dr_xmp::Field>,
}
/// TRACES: FR-CAT-13 | FR-NC-9
/// Take a standard sidecar into the catalog, with the catalog winning.
///
/// The automatic path. Every image the file describes is reconciled against
/// what the catalog holds for it; the merged record is applied; and a
/// disagreement is recorded in `xmp_conflicts` for a person to settle, never
/// resolved here.
pub fn take_in(
conn: &Connection,
root_id: i64,
path: &str,
text: &str,
now: i64,
) -> Result<TakenIn, String> {
let sidecar = Xmp::parse(text).map_err(|e| e.to_string())?;
let mut out = TakenIn::default();
for (image, version) in images_for(conn, root_id, path) {
out.described += 1;
let mine = record_of(conn, image, version);
let reconciled = dr_xmp::reconcile(&mine, &sidecar, Precedence::Catalog);
if apply(conn, image, version, &reconciled.merged)? {
out.changed += 1;
}
for field in reconciled.conflicts {
if !out.conflicts.contains(&field) {
out.conflicts.push(field);
}
}
}
if out.conflicts.is_empty() {
clear_conflict(conn, root_id, path);
} else if out.described > 0 {
record_conflict(conn, root_id, path, &out.conflicts, now);
}
Ok(out)
}
/// TRACES: FR-CAT-13
/// The offered reload: the same reconciliation with the sidecar winning.
///
/// Asked for by a person, per file, which is the consent the module
/// documentation says this needs. Clears the conflict whatever the outcome —
/// the person has now seen it, and if the file still disagrees the next pull
/// that sees it change will say so again.
pub fn reload(conn: &Connection, root_id: i64, path: &str, text: &str) -> Result<TakenIn, String> {
let sidecar = Xmp::parse(text).map_err(|e| e.to_string())?;
let mut out = TakenIn::default();
for (image, version) in images_for(conn, root_id, path) {
out.described += 1;
let mine = record_of(conn, image, version);
let reconciled = dr_xmp::reconcile(&mine, &sidecar, Precedence::Sidecar);
// The sidecar wins outright on the whole-valued fields, and that
// includes a rating it holds and the catalog's it replaces — which
// `apply` does. What `apply` will not do is *clear* a star the file
// does not mention, and a reload does not ask it to: the file said
// nothing, and nothing is not a judgement.
if apply(conn, image, version, &reconciled.merged)? {
out.changed += 1;
}
}
clear_conflict(conn, root_id, path);
Ok(out)
}
/// One outstanding disagreement, for the settings page.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Conflict {
pub path: String,
pub fields: String,
}
/// Every sidecar in this library that disagrees with the catalog.
pub fn conflicts(catalog: &Catalog, root_id: i64) -> Vec<Conflict> {
let Ok(mut stmt) = catalog
.connection()
.prepare("SELECT path, fields FROM xmp_conflicts WHERE root_id = ?1 ORDER BY path")
else {
return Vec::new();
};
stmt.query_map([root_id], |r| {
Ok(Conflict {
path: r.get(0)?,
fields: r.get(1)?,
})
})
.map(|rows| rows.flatten().collect())
.unwrap_or_default()
}
fn record_conflict(
conn: &Connection,
root_id: i64,
path: &str,
fields: &[dr_xmp::Field],
now: i64,
) {
let fields = fields
.iter()
.map(|f| format!("{f:?}"))
.collect::<Vec<_>>()
.join(" ");
let done = conn.execute(
"INSERT INTO xmp_conflicts(root_id, path, fields, seen_at) VALUES (?1, ?2, ?3, ?4)
ON CONFLICT(root_id, path) DO UPDATE SET
fields = excluded.fields, seen_at = excluded.seen_at",
rusqlite::params![root_id, path, fields, now],
);
if let Err(e) = done {
log::debug!("recording xmp conflict for {path}: {e}");
}
}
fn clear_conflict(conn: &Connection, root_id: i64, path: &str) {
let _ = conn.execute(
"DELETE FROM xmp_conflicts WHERE root_id = ?1 AND path = ?2",
rusqlite::params![root_id, path],
);
}
fn flag_code(flag: FlagState) -> i64 {
match flag {
FlagState::Unflagged => 0,
FlagState::Pick => 1,
FlagState::Reject => 2,
}
}
#[cfg(test)]
mod tests {
use super::*;
fn catalog() -> Catalog {
let c = Catalog::in_memory().unwrap();
c.connection()
.execute_batch(
"INSERT INTO roots(id, kind, label) VALUES (1, 'remote', 'lib');
INSERT INTO images(id, root_id, source_ref, added_at)
VALUES (1, 1, 'Photos/IMG_0001.CR3', 0),
(2, 1, 'Photos/IMG_0001.JPG', 0),
(3, 1, 'Photos/IMG_0002.CR3', 0);",
)
.unwrap();
dr_catalog::rating::ensure_default_versions(c.connection()).unwrap();
c
}
const LIGHTROOM: &str = r#"<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/">
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<rdf:Description rdf:about=""
xmlns:xmp="http://ns.adobe.com/xap/1.0/"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmp:Rating="4"
xmp:Label="Red">
<dc:subject><rdf:Bag><rdf:li>puffin</rdf:li><rdf:li>iceland</rdf:li></rdf:Bag></dc:subject>
</rdf:Description>
</rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>"#;
/// Both spellings reach the photograph, and Lightroom's reaches the JPEG
/// beside it too.
#[test]
fn both_namings_find_their_images() {
let c = catalog();
let conn = c.connection();
let mut lr: Vec<u64> = images_for(conn, 1, "Photos/IMG_0001.xmp")
.into_iter()
.map(|(i, _)| i.0)
.collect();
lr.sort();
assert_eq!(lr, [1, 2], "the RAW and the JPEG are one photograph");
let dt: Vec<u64> = images_for(conn, 1, "Photos/IMG_0001.CR3.xmp")
.into_iter()
.map(|(i, _)| i.0)
.collect();
assert_eq!(dt, [1], "darktable's names the file exactly");
assert!(images_for(conn, 1, "Photos/IMG_9999.xmp").is_empty());
}
/// The ordinary pull: an empty catalog takes everything the file says.
#[test]
fn a_sidecar_fills_what_the_catalog_lacks() {
let c = catalog();
let conn = c.connection();
let taken = take_in(conn, 1, "Photos/IMG_0002.xmp", LIGHTROOM, 1).unwrap();
assert_eq!(taken.described, 1);
assert_eq!(taken.changed, 1);
assert!(taken.conflicts.is_empty());
let record = record_of(conn, ImageId(3), 3);
assert_eq!(record.rating, Some(Rating::Stars(4)));
assert_eq!(record.colour(), Some(dr_types::ColourLabel::Red));
assert_eq!(record.keywords, ["iceland", "puffin"]);
assert!(conflicts(&c, 1).is_empty());
}
/// A rating the catalog already holds is not overwritten by the file,
/// the disagreement is recorded, and the reload — a person asking —
/// takes the file's.
#[test]
fn a_disagreement_is_recorded_and_a_reload_settles_it() {
let c = catalog();
let conn = c.connection();
dr_catalog::rating::set_rating(conn, ImageId(3), 2).unwrap();
let taken = take_in(conn, 1, "Photos/IMG_0002.xmp", LIGHTROOM, 1).unwrap();
assert_eq!(taken.conflicts, [dr_xmp::Field::Rating]);
assert_eq!(
record_of(conn, ImageId(3), 3).rating,
Some(Rating::Stars(2)),
"the catalog's own rating stands"
);
assert_eq!(
record_of(conn, ImageId(3), 3).keywords,
["iceland", "puffin"],
"while the keywords still union"
);
let open = conflicts(&c, 1);
assert_eq!(open.len(), 1);
assert_eq!(open[0].path, "Photos/IMG_0002.xmp");
assert_eq!(open[0].fields, "Rating");
reload(conn, 1, "Photos/IMG_0002.xmp", LIGHTROOM).unwrap();
assert_eq!(
record_of(conn, ImageId(3), 3).rating,
Some(Rating::Stars(4))
);
assert!(conflicts(&c, 1).is_empty(), "settled");
}
/// A rejection and a star count are two axes here and one field there.
#[test]
fn a_rejection_crosses_as_adobes_minus_one_and_back() {
let c = catalog();
let conn = c.connection();
dr_catalog::rating::set_flag(conn, ImageId(3), FlagState::Reject).unwrap();
assert_eq!(
record_of(conn, ImageId(3), 3).rating,
Some(Rating::Rejected)
);
let back = Xmp {
rating: Some(Rating::Stars(3)),
..Default::default()
};
apply(conn, ImageId(3), 3, &back).unwrap();
let after = record_of(conn, ImageId(3), 3);
assert_eq!(
after.rating,
Some(Rating::Stars(3)),
"the stars lift the rejection"
);
}
/// A judgement written over a Lightroom sidecar must not cost the caption
/// Lightroom wrote, since the catalog never held it.
#[test]
fn a_rewrite_keeps_what_the_catalog_has_no_column_for() {
let theirs = Xmp {
title: Some("Puffins at dusk".into()),
copyright: Some("© Someone".into()),
hierarchical_subjects: vec!["Places|Iceland".into()],
label: Some("Second choice".into()),
rating: Some(Rating::Stars(1)),
..Default::default()
};
let mut ours = Xmp {
rating: Some(Rating::Stars(4)),
keywords: vec!["puffin".into()],
..Default::default()
};
carry_through(&mut ours, &theirs);
assert_eq!(ours.rating, Some(Rating::Stars(4)), "the judgement is ours");
assert_eq!(ours.title.as_deref(), Some("Puffins at dusk"));
assert_eq!(ours.copyright.as_deref(), Some("© Someone"));
assert_eq!(ours.hierarchical_subjects, ["Places|Iceland"]);
assert_eq!(
ours.label.as_deref(),
Some("Second choice"),
"a label that is not one of the five colours has no other copy"
);
let mut red = Xmp::default();
red.set_colour(Some(dr_types::ColourLabel::Red));
carry_through(&mut red, &theirs);
assert_eq!(
red.label.as_deref(),
Some("Red"),
"but a colour of ours wins"
);
}
/// The two spellings a write chooses between.
#[test]
fn the_write_tries_darktables_spelling_then_lightrooms() {
assert_eq!(
candidate_paths("Photos/IMG_0001.CR3"),
["Photos/IMG_0001.CR3.xmp", "Photos/IMG_0001.xmp"]
);
assert_eq!(
candidate_paths("a.b/IMG"),
["a.b/IMG.xmp", "a.b/IMG.xmp"],
"a dot in a folder is not an extension"
);
}
}
+9
View File
@@ -846,6 +846,11 @@ export component AppWindow inherits Window {
in property <[string]> settings-timeline-bar-labels;
in property <int> settings-timeline-bars-selected: 0;
callback settings-timeline-bars-picked(int);
/// TRACES: FR-CAT-13
in property <bool> settings-write-xmp: false;
callback settings-write-xmp-toggled(bool);
in property <int> settings-xmp-conflicts: 0;
callback settings-xmp-reload();
/// TRACES: FR-UI-1
/// The group-navigation preference, and what "Automatic" resolves to here.
@@ -1281,6 +1286,10 @@ in property <bool> panel-visible: true;
timeline-bar-labels: root.settings-timeline-bar-labels;
timeline-bars-selected: root.settings-timeline-bars-selected;
timeline-bars-picked(i) => { root.settings-timeline-bars-picked(i); }
write-xmp: root.settings-write-xmp;
write-xmp-toggled(on) => { root.settings-write-xmp-toggled(on); }
xmp-conflicts: root.settings-xmp-conflicts;
xmp-reload() => { root.settings-xmp-reload(); }
cache-usage: root.settings-cache-usage;
original-budget-changed(t) => { root.settings-original-budget-changed(t); }
+48
View File
@@ -181,6 +181,13 @@ export component SettingsPage inherits Rectangle {
in property <[string]> timeline-bar-labels;
in property <int> timeline-bars-selected: 0;
callback timeline-bars-picked(int);
/// TRACES: FR-CAT-13 | NFR-R4
/// Whether judgements also go to the `.xmp` beside the original, and how
/// many sidecars the last scan found disagreeing with the catalog.
in property <bool> write-xmp: false;
callback write-xmp-toggled(bool);
in property <int> xmp-conflicts: 0;
callback xmp-reload();
// --- export --------------------------------------------------------
in property <[string]> format-labels;
@@ -991,6 +998,47 @@ export component SettingsPage inherits Rectangle {
PanelHeading { text: "FILES AND METADATA"; }
// TRACES: FR-CAT-13 | NFR-R4
// Reading is not a choice — a sidecar another editor
// wrote is taken in regardless, since reading changes
// nothing in the folder. Writing beside somebody's
// originals is, and it starts off.
Check {
label: "Write ratings, labels and keywords to XMP sidecars";
hint: "Beside the originals, as Lightroom and darktable "
+ "do, so other applications see them. Only those "
+ "fields are written; everything else in an "
+ "existing sidecar is left exactly as it was.";
checked: root.write-xmp;
toggled(on) => { root.write-xmp-toggled(on); }
}
// The offered reload. Shown only while there is
// something to offer: a disagreement between an
// `.xmp` changed elsewhere and what this catalog
// holds, which the automatic pull records rather
// than resolves.
if root.xmp-conflicts > 0: Caption {
text: root.xmp-conflicts
+ (root.xmp-conflicts == 1
? " XMP sidecar disagrees"
: " XMP sidecars disagree")
+ " with this catalog about a rating, label or "
+ "caption. The catalog's values stand until you "
+ "say otherwise.";
wrap: word-wrap;
}
if root.xmp-conflicts > 0: Rectangle {
height: Theme.control-height;
Button {
x: 0;
text: "Take the sidecars' values";
clicked => { root.xmp-reload(); }
}
}
TextRow {
label: "Filename template";
hint: "{name} {seq} {date} {dimensions} {preset}";