Keep albums in the catalog: export folders and what went into them
An album is a named export destination. Its folder holds only the exported files; the catalog records, per file, the image it was rendered from, so an album can show the originals behind its JPEGs (FR-EXP-10). The tables are created on first use (CREATE TABLE IF NOT EXISTS), the way dedup_probes is, rather than by a schema migration: a new user_version makes every older build refuse this catalog's snapshot at sync, and the 0.16.0 tablet would stop merging collections, keywords and people for a feature it does not have. Albums merge as collections do: by uuid and revision, tombstones on delete, exports as a set union keyed on the server's file id (content hash for a folder library). A folder on the server lives on the album row and syncs; a folder on this device lives in album_folders, which the merge never reads and the upload snapshot drops, because a path or a SAF grant on one device means nothing on another. Exports are keyed on the file name, not the image: two crops of one photograph are two files and two rows, and an overwrite re-points the name at whatever wrote it last.
This commit is contained in:
@@ -0,0 +1,499 @@
|
|||||||
|
//! TRACES: FR-EXP-10 | FR-EXP-6 | FR-CAT-7
|
||||||
|
//! Albums: named export folders, and which photographs went into each.
|
||||||
|
//!
|
||||||
|
//! An album is where finished pictures go — a folder of JPEGs somebody else
|
||||||
|
//! looks at — as opposed to a collection, which is a set of originals the
|
||||||
|
//! photographer works on. The folder holds only the exported files. What the
|
||||||
|
//! catalog adds is the link back: each export is recorded against the image
|
||||||
|
//! it was rendered from, so opening an album in the library shows the RAWs
|
||||||
|
//! behind its JPEGs, and re-exporting after an edit is one selection away.
|
||||||
|
//!
|
||||||
|
//! # Where the folder is, and why that is two tables
|
||||||
|
//!
|
||||||
|
//! An album's folder is either on the library's server or on this device.
|
||||||
|
//!
|
||||||
|
//! A **server folder** is one path on the account, the same from every device
|
||||||
|
//! signed in to it, so it lives on the album row and syncs with it.
|
||||||
|
//!
|
||||||
|
//! A **local folder** — a filesystem path on a desktop, a Storage Access
|
||||||
|
//! Framework tree on Android — means nothing on any other device. It lives in
|
||||||
|
//! `album_folders`, which the merge never reads and the upload snapshot drops
|
||||||
|
//! ([`crate::sync::snapshot_for_upload`]). An album made on the desktop with a
|
||||||
|
//! local folder therefore reaches the tablet as an album with no folder there
|
||||||
|
//! yet, which is true, and which the tablet can fix by choosing one.
|
||||||
|
//!
|
||||||
|
//! # Created on first use, not by a migration
|
||||||
|
//!
|
||||||
|
//! A new schema version makes every older build refuse this catalog's
|
||||||
|
//! snapshot at sync (`crate::sync::remote_is_mergeable`), so the tablet would
|
||||||
|
//! stop merging collections, keywords and people until it was updated — for
|
||||||
|
//! a feature it does not have. The tables are created by [`ensure_tables`]
|
||||||
|
//! instead, the way `dedup_probes` is; an older build that meets them ignores
|
||||||
|
//! them, and its merge keeps working.
|
||||||
|
//!
|
||||||
|
//! # Sync
|
||||||
|
//!
|
||||||
|
//! Albums merge by uuid and revision with tombstones, and their exports as a
|
||||||
|
//! set union keyed on the image's server file id — the rules
|
||||||
|
//! [`crate::merge`] applies to collections, for the same reasons.
|
||||||
|
|
||||||
|
use rusqlite::{Connection, OptionalExtension};
|
||||||
|
|
||||||
|
use dr_types::ImageId;
|
||||||
|
|
||||||
|
use crate::error::CatalogError;
|
||||||
|
|
||||||
|
/// Identifies an album within one catalog. Local, like every integer id here;
|
||||||
|
/// the uuid is what crosses devices.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
|
||||||
|
pub struct AlbumId(pub u64);
|
||||||
|
|
||||||
|
/// Where an album's files go.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub enum Place {
|
||||||
|
/// A folder on the library's server, relative to the account root, with no
|
||||||
|
/// leading slash. The same on every device.
|
||||||
|
Server(String),
|
||||||
|
/// A folder on this device: a filesystem path, or on Android a SAF tree
|
||||||
|
/// URI. Never synced.
|
||||||
|
Local(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One album, as the sidebar and the export sheet show it.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct Album {
|
||||||
|
pub id: AlbumId,
|
||||||
|
pub uuid: String,
|
||||||
|
pub name: String,
|
||||||
|
/// Where exports go from this device, or `None` for an album whose folder
|
||||||
|
/// is local to another device and has not been chosen here.
|
||||||
|
pub place: Option<Place>,
|
||||||
|
/// Distinct photographs exported into it — what the grid shows when the
|
||||||
|
/// album is opened.
|
||||||
|
pub sources: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Create the album tables if this catalog does not have them yet.
|
||||||
|
///
|
||||||
|
/// Cheap when they exist: `IF NOT EXISTS` is answered from the schema, and
|
||||||
|
/// every function below calls this first so no caller has to remember to.
|
||||||
|
pub fn ensure_tables(conn: &Connection) -> Result<(), CatalogError> {
|
||||||
|
conn.execute_batch(
|
||||||
|
"CREATE TABLE IF NOT EXISTS albums (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
-- The merge identity; the integer id is local.
|
||||||
|
uuid TEXT NOT NULL UNIQUE,
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
-- A folder on the server, relative to the account root. NULL for
|
||||||
|
-- an album whose folder is local to some device.
|
||||||
|
server_path TEXT,
|
||||||
|
created INTEGER NOT NULL,
|
||||||
|
revision INTEGER NOT NULL DEFAULT 1,
|
||||||
|
modified INTEGER NOT NULL,
|
||||||
|
deleted INTEGER NOT NULL DEFAULT 0
|
||||||
|
);
|
||||||
|
-- One row per file written into an album. Keyed on the file, not the
|
||||||
|
-- image: a photograph exported twice — two crops, or once before an
|
||||||
|
-- edit and once after — is two files in the folder and two rows here.
|
||||||
|
CREATE TABLE IF NOT EXISTS album_exports (
|
||||||
|
album_id INTEGER NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
|
||||||
|
file_name TEXT NOT NULL,
|
||||||
|
image_id INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE,
|
||||||
|
exported_at INTEGER NOT NULL,
|
||||||
|
PRIMARY KEY (album_id, file_name)
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS album_exports_image ON album_exports(image_id);
|
||||||
|
-- This device's folder for an album. Never merged, never uploaded.
|
||||||
|
CREATE TABLE IF NOT EXISTS album_folders (
|
||||||
|
album_id INTEGER PRIMARY KEY REFERENCES albums(id) ON DELETE CASCADE,
|
||||||
|
folder TEXT NOT NULL
|
||||||
|
);",
|
||||||
|
)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Make an album.
|
||||||
|
///
|
||||||
|
/// The name is trimmed and must not be empty; two albums may share one, as
|
||||||
|
/// two collections may, because the uuid is the identity and refusing a
|
||||||
|
/// duplicate name here would refuse it on one device and not another.
|
||||||
|
pub fn create(conn: &Connection, name: &str, place: &Place) -> Result<AlbumId, CatalogError> {
|
||||||
|
ensure_tables(conn)?;
|
||||||
|
let name = name.trim();
|
||||||
|
if name.is_empty() {
|
||||||
|
return Err(CatalogError::EmptyName);
|
||||||
|
}
|
||||||
|
let now = now_secs();
|
||||||
|
let tx = conn.unchecked_transaction()?;
|
||||||
|
tx.execute(
|
||||||
|
"INSERT INTO albums(uuid, name, server_path, created, revision, modified)
|
||||||
|
VALUES (?1, ?2, ?3, ?4, 1, ?4)",
|
||||||
|
rusqlite::params![crate::collections::new_uuid(), name, server_path(place), now],
|
||||||
|
)?;
|
||||||
|
let id = AlbumId(tx.last_insert_rowid() as u64);
|
||||||
|
if let Place::Local(folder) = place {
|
||||||
|
tx.execute(
|
||||||
|
"INSERT INTO album_folders(album_id, folder) VALUES (?1, ?2)",
|
||||||
|
rusqlite::params![id.0 as i64, folder],
|
||||||
|
)?;
|
||||||
|
}
|
||||||
|
tx.commit()?;
|
||||||
|
Ok(id)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Rename an album. The folder keeps its name: the album is what the
|
||||||
|
/// photographer calls it, the folder is what is already out there.
|
||||||
|
pub fn rename(conn: &Connection, id: AlbumId, name: &str) -> Result<(), CatalogError> {
|
||||||
|
ensure_tables(conn)?;
|
||||||
|
let name = name.trim();
|
||||||
|
if name.is_empty() {
|
||||||
|
return Err(CatalogError::EmptyName);
|
||||||
|
}
|
||||||
|
let n = conn.execute(
|
||||||
|
"UPDATE albums SET name = ?2, revision = revision + 1, modified = ?3
|
||||||
|
WHERE id = ?1 AND deleted = 0",
|
||||||
|
rusqlite::params![id.0 as i64, name, now_secs()],
|
||||||
|
)?;
|
||||||
|
if n == 0 {
|
||||||
|
return Err(CatalogError::NoSuchAlbum(id.0));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Point an album at a different folder, from this device.
|
||||||
|
///
|
||||||
|
/// A server folder replaces the synced path, and bumps the revision so the
|
||||||
|
/// move reaches every device. A local folder is recorded for this device
|
||||||
|
/// only; it also clears a server path, because an album goes to one place and
|
||||||
|
/// the photographer has just said which.
|
||||||
|
pub fn set_place(conn: &Connection, id: AlbumId, place: &Place) -> Result<(), CatalogError> {
|
||||||
|
ensure_tables(conn)?;
|
||||||
|
let tx = conn.unchecked_transaction()?;
|
||||||
|
let n = tx.execute(
|
||||||
|
"UPDATE albums SET server_path = ?2, revision = revision + 1, modified = ?3
|
||||||
|
WHERE id = ?1 AND deleted = 0",
|
||||||
|
rusqlite::params![id.0 as i64, server_path(place), now_secs()],
|
||||||
|
)?;
|
||||||
|
if n == 0 {
|
||||||
|
return Err(CatalogError::NoSuchAlbum(id.0));
|
||||||
|
}
|
||||||
|
match place {
|
||||||
|
Place::Local(folder) => tx.execute(
|
||||||
|
"INSERT INTO album_folders(album_id, folder) VALUES (?1, ?2)
|
||||||
|
ON CONFLICT(album_id) DO UPDATE SET folder = excluded.folder",
|
||||||
|
rusqlite::params![id.0 as i64, folder],
|
||||||
|
)?,
|
||||||
|
Place::Server(_) => tx.execute(
|
||||||
|
"DELETE FROM album_folders WHERE album_id = ?1",
|
||||||
|
[id.0 as i64],
|
||||||
|
)?,
|
||||||
|
};
|
||||||
|
tx.commit()?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Delete an album, leaving a tombstone. The files in its folder are not
|
||||||
|
/// touched: they are finished work somebody may already have been sent a
|
||||||
|
/// link to, and the album was only ever this catalog's note of them.
|
||||||
|
pub fn delete(conn: &Connection, id: AlbumId) -> Result<(), CatalogError> {
|
||||||
|
ensure_tables(conn)?;
|
||||||
|
let tx = conn.unchecked_transaction()?;
|
||||||
|
let n = tx.execute(
|
||||||
|
"UPDATE albums SET deleted = 1, revision = revision + 1, modified = ?2
|
||||||
|
WHERE id = ?1 AND deleted = 0",
|
||||||
|
rusqlite::params![id.0 as i64, now_secs()],
|
||||||
|
)?;
|
||||||
|
if n == 0 {
|
||||||
|
return Err(CatalogError::NoSuchAlbum(id.0));
|
||||||
|
}
|
||||||
|
tx.execute("DELETE FROM album_exports WHERE album_id = ?1", [id.0 as i64])?;
|
||||||
|
tx.execute("DELETE FROM album_folders WHERE album_id = ?1", [id.0 as i64])?;
|
||||||
|
tx.commit()?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every live album, by name, with how many photographs each holds.
|
||||||
|
///
|
||||||
|
/// One statement: the counts are aggregated from `album_exports` first and
|
||||||
|
/// joined to the (few) albums, not counted per row.
|
||||||
|
pub fn list(conn: &Connection) -> Result<Vec<Album>, CatalogError> {
|
||||||
|
ensure_tables(conn)?;
|
||||||
|
let mut stmt = conn.prepare(
|
||||||
|
"SELECT a.id, a.uuid, a.name, a.server_path, f.folder, coalesce(e.n, 0)
|
||||||
|
FROM albums a
|
||||||
|
LEFT JOIN album_folders f ON f.album_id = a.id
|
||||||
|
LEFT JOIN (SELECT album_id, count(DISTINCT image_id) AS n
|
||||||
|
FROM album_exports GROUP BY album_id) e
|
||||||
|
ON e.album_id = a.id
|
||||||
|
WHERE a.deleted = 0
|
||||||
|
ORDER BY a.name COLLATE NOCASE, a.id",
|
||||||
|
)?;
|
||||||
|
let rows = stmt
|
||||||
|
.query_map([], album_from_row)?
|
||||||
|
.collect::<Result<Vec<_>, _>>()?;
|
||||||
|
Ok(rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One album, or `None` if it is gone.
|
||||||
|
pub fn get(conn: &Connection, id: AlbumId) -> Result<Option<Album>, CatalogError> {
|
||||||
|
ensure_tables(conn)?;
|
||||||
|
Ok(conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT a.id, a.uuid, a.name, a.server_path, f.folder,
|
||||||
|
(SELECT count(DISTINCT image_id) FROM album_exports WHERE album_id = a.id)
|
||||||
|
FROM albums a
|
||||||
|
LEFT JOIN album_folders f ON f.album_id = a.id
|
||||||
|
WHERE a.id = ?1 AND a.deleted = 0",
|
||||||
|
[id.0 as i64],
|
||||||
|
album_from_row,
|
||||||
|
)
|
||||||
|
.optional()?)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The album with this uuid, if this catalog holds it live.
|
||||||
|
pub fn id_for_uuid(conn: &Connection, uuid: &str) -> Result<Option<AlbumId>, CatalogError> {
|
||||||
|
ensure_tables(conn)?;
|
||||||
|
Ok(conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT id FROM albums WHERE uuid = ?1 AND deleted = 0",
|
||||||
|
[uuid],
|
||||||
|
|r| r.get::<_, i64>(0),
|
||||||
|
)
|
||||||
|
.optional()?
|
||||||
|
.map(|id| AlbumId(id as u64)))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Record the files one export wrote into an album, and which image each
|
||||||
|
/// came from. One transaction for the batch, however many files it placed.
|
||||||
|
///
|
||||||
|
/// A file name already recorded is re-pointed at the image that wrote it
|
||||||
|
/// last: an export that overwrote `IMG_0001.jpg` replaced the picture in the
|
||||||
|
/// folder, and the link must say what is there now.
|
||||||
|
pub fn record_exports(
|
||||||
|
conn: &Connection,
|
||||||
|
id: AlbumId,
|
||||||
|
files: &[(ImageId, String)],
|
||||||
|
) -> Result<(), CatalogError> {
|
||||||
|
ensure_tables(conn)?;
|
||||||
|
if files.is_empty() {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
let tx = conn.unchecked_transaction()?;
|
||||||
|
let now = now_secs();
|
||||||
|
{
|
||||||
|
let mut insert = tx.prepare(
|
||||||
|
"INSERT INTO album_exports(album_id, file_name, image_id, exported_at)
|
||||||
|
VALUES (?1, ?2, ?3, ?4)
|
||||||
|
ON CONFLICT(album_id, file_name) DO UPDATE SET
|
||||||
|
image_id = excluded.image_id, exported_at = excluded.exported_at",
|
||||||
|
)?;
|
||||||
|
for (image, name) in files {
|
||||||
|
insert.execute(rusqlite::params![id.0 as i64, name, image.0 as i64, now])?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
tx.commit()?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The photographs behind an album's files, most recently exported first —
|
||||||
|
/// what the grid shows when the album is opened.
|
||||||
|
pub fn sources(conn: &Connection, id: AlbumId) -> Result<Vec<ImageId>, CatalogError> {
|
||||||
|
ensure_tables(conn)?;
|
||||||
|
let mut stmt = conn.prepare(
|
||||||
|
"SELECT image_id FROM album_exports
|
||||||
|
WHERE album_id = ?1
|
||||||
|
GROUP BY image_id
|
||||||
|
ORDER BY max(exported_at) DESC, image_id",
|
||||||
|
)?;
|
||||||
|
let rows = stmt
|
||||||
|
.query_map([id.0 as i64], |r| r.get::<_, i64>(0))?
|
||||||
|
.map(|r| r.map(|i| ImageId(i as u64)))
|
||||||
|
.collect::<Result<Vec<_>, _>>()?;
|
||||||
|
Ok(rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The names of the files an image left in an album — the "which JPEG is
|
||||||
|
/// this" half of the link.
|
||||||
|
pub fn files_of(
|
||||||
|
conn: &Connection,
|
||||||
|
id: AlbumId,
|
||||||
|
image: ImageId,
|
||||||
|
) -> Result<Vec<String>, CatalogError> {
|
||||||
|
ensure_tables(conn)?;
|
||||||
|
let mut stmt = conn.prepare(
|
||||||
|
"SELECT file_name FROM album_exports
|
||||||
|
WHERE album_id = ?1 AND image_id = ?2
|
||||||
|
ORDER BY exported_at DESC, file_name",
|
||||||
|
)?;
|
||||||
|
let rows = stmt
|
||||||
|
.query_map(rusqlite::params![id.0 as i64, image.0 as i64], |r| r.get(0))?
|
||||||
|
.collect::<Result<Vec<_>, _>>()?;
|
||||||
|
Ok(rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn album_from_row(r: &rusqlite::Row<'_>) -> rusqlite::Result<Album> {
|
||||||
|
let server: Option<String> = r.get(3)?;
|
||||||
|
let local: Option<String> = r.get(4)?;
|
||||||
|
Ok(Album {
|
||||||
|
id: AlbumId(r.get::<_, i64>(0)? as u64),
|
||||||
|
uuid: r.get(1)?,
|
||||||
|
name: r.get(2)?,
|
||||||
|
// A server path wins: `set_place` clears the local folder when it
|
||||||
|
// sets one, so both being present means a merge brought a server
|
||||||
|
// path in over a local choice — and the newer revision decided that.
|
||||||
|
place: server.map(Place::Server).or(local.map(Place::Local)),
|
||||||
|
sources: r.get::<_, i64>(5)? as usize,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn server_path(place: &Place) -> Option<&str> {
|
||||||
|
match place {
|
||||||
|
Place::Server(p) => Some(p.trim_matches('/')),
|
||||||
|
Place::Local(_) => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn now_secs() -> i64 {
|
||||||
|
std::time::SystemTime::now()
|
||||||
|
.duration_since(std::time::UNIX_EPOCH)
|
||||||
|
.map(|d| d.as_secs() as i64)
|
||||||
|
.unwrap_or(0)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
/// A catalog, and the connection to it. The `Catalog` has to outlive the
|
||||||
|
/// connection it hands out, so tests hold both.
|
||||||
|
fn catalog() -> crate::Catalog {
|
||||||
|
let cat = crate::Catalog::in_memory().unwrap();
|
||||||
|
cat.connection()
|
||||||
|
.execute(
|
||||||
|
"INSERT INTO roots(id, kind, label) VALUES (1, 'local', 'lib')",
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
cat
|
||||||
|
}
|
||||||
|
|
||||||
|
fn image(conn: &Connection, path: &str) -> ImageId {
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO images(root_id, source_ref, added_at) VALUES (1, ?1, 0)",
|
||||||
|
[path],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
ImageId(conn.last_insert_rowid() as u64)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_album_lists_with_its_place_and_no_photographs() {
|
||||||
|
let cat = catalog();
|
||||||
|
let conn = cat.connection();
|
||||||
|
let web = create(conn, " Web ", &Place::Server("Shared/Web/".into())).unwrap();
|
||||||
|
let print = create(conn, "Print", &Place::Local("/mnt/print".into())).unwrap();
|
||||||
|
|
||||||
|
let all = list(conn).unwrap();
|
||||||
|
assert_eq!(all.len(), 2);
|
||||||
|
assert_eq!(all[0].id, print, "sorted by name");
|
||||||
|
assert_eq!(all[0].place, Some(Place::Local("/mnt/print".into())));
|
||||||
|
assert_eq!(all[1].id, web);
|
||||||
|
assert_eq!(all[1].name, "Web", "trimmed");
|
||||||
|
assert_eq!(all[1].place, Some(Place::Server("Shared/Web".into())));
|
||||||
|
assert_eq!(all[1].sources, 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_empty_name_is_refused() {
|
||||||
|
let cat = catalog();
|
||||||
|
let conn = cat.connection();
|
||||||
|
assert!(matches!(
|
||||||
|
create(conn, " ", &Place::Local("/x".into())),
|
||||||
|
Err(CatalogError::EmptyName)
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn exports_link_files_back_to_their_images() {
|
||||||
|
let cat = catalog();
|
||||||
|
let conn = cat.connection();
|
||||||
|
let album = create(conn, "Web", &Place::Local("/out".into())).unwrap();
|
||||||
|
let a = image(conn, "a.cr3");
|
||||||
|
let b = image(conn, "b.cr3");
|
||||||
|
|
||||||
|
record_exports(
|
||||||
|
conn,
|
||||||
|
album,
|
||||||
|
&[(a, "a.jpg".into()), (a, "a (1).jpg".into()), (b, "b.jpg".into())],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let got = get(conn, album).unwrap().unwrap();
|
||||||
|
assert_eq!(got.sources, 2, "two photographs, three files");
|
||||||
|
let mut s = sources(conn, album).unwrap();
|
||||||
|
s.sort();
|
||||||
|
assert_eq!(s, vec![a, b]);
|
||||||
|
assert_eq!(files_of(conn, album, a).unwrap().len(), 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_overwritten_file_points_at_what_wrote_it_last() {
|
||||||
|
let cat = catalog();
|
||||||
|
let conn = cat.connection();
|
||||||
|
let album = create(conn, "Web", &Place::Local("/out".into())).unwrap();
|
||||||
|
let a = image(conn, "a.cr3");
|
||||||
|
let b = image(conn, "b.cr3");
|
||||||
|
record_exports(conn, album, &[(a, "x.jpg".into())]).unwrap();
|
||||||
|
record_exports(conn, album, &[(b, "x.jpg".into())]).unwrap();
|
||||||
|
assert_eq!(sources(conn, album).unwrap(), vec![b]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn moving_to_the_server_forgets_the_local_folder() {
|
||||||
|
let cat = catalog();
|
||||||
|
let conn = cat.connection();
|
||||||
|
let album = create(conn, "Web", &Place::Local("/out".into())).unwrap();
|
||||||
|
set_place(conn, album, &Place::Server("Web".into())).unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
get(conn, album).unwrap().unwrap().place,
|
||||||
|
Some(Place::Server("Web".into()))
|
||||||
|
);
|
||||||
|
set_place(conn, album, &Place::Local("/again".into())).unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
get(conn, album).unwrap().unwrap().place,
|
||||||
|
Some(Place::Local("/again".into()))
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_deleted_album_is_gone_and_its_uuid_no_longer_resolves() {
|
||||||
|
let cat = catalog();
|
||||||
|
let conn = cat.connection();
|
||||||
|
let album = create(conn, "Web", &Place::Local("/out".into())).unwrap();
|
||||||
|
let uuid = get(conn, album).unwrap().unwrap().uuid;
|
||||||
|
let a = image(conn, "a.cr3");
|
||||||
|
record_exports(conn, album, &[(a, "a.jpg".into())]).unwrap();
|
||||||
|
|
||||||
|
delete(conn, album).unwrap();
|
||||||
|
assert!(list(conn).unwrap().is_empty());
|
||||||
|
assert_eq!(id_for_uuid(conn, &uuid).unwrap(), None);
|
||||||
|
assert!(matches!(
|
||||||
|
rename(conn, album, "Again"),
|
||||||
|
Err(CatalogError::NoSuchAlbum(_))
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_rename_bumps_the_revision_the_merge_compares() {
|
||||||
|
let cat = catalog();
|
||||||
|
let conn = cat.connection();
|
||||||
|
let album = create(conn, "Web", &Place::Local("/out".into())).unwrap();
|
||||||
|
rename(conn, album, "Website").unwrap();
|
||||||
|
let rev: i64 = conn
|
||||||
|
.query_row("SELECT revision FROM albums WHERE id = ?1", [album.0 as i64], |r| {
|
||||||
|
r.get(0)
|
||||||
|
})
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(rev, 2);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -65,6 +65,16 @@ pub enum CatalogError {
|
|||||||
#[error("no such collection: {0}")]
|
#[error("no such collection: {0}")]
|
||||||
NoSuchCollection(u64),
|
NoSuchCollection(u64),
|
||||||
|
|
||||||
|
/// An album the caller named is gone — deleted here, or by a merge while
|
||||||
|
/// its id sat in a UI model.
|
||||||
|
#[error("no such album: {0}")]
|
||||||
|
NoSuchAlbum(u64),
|
||||||
|
|
||||||
|
/// A name that is empty once trimmed. Refused rather than stored, because
|
||||||
|
/// a row with no name is one the sidebar cannot draw and nobody can pick.
|
||||||
|
#[error("a name is required")]
|
||||||
|
EmptyName,
|
||||||
|
|
||||||
/// A keyword the caller named is gone — deleted, or fused into another by a
|
/// A keyword the caller named is gone — deleted, or fused into another by a
|
||||||
/// merge while its id sat in a UI model.
|
/// merge while its id sat in a UI model.
|
||||||
///
|
///
|
||||||
|
|||||||
@@ -36,6 +36,7 @@ use std::path::Path;
|
|||||||
use dr_types::{Availability, ImageId};
|
use dr_types::{Availability, ImageId};
|
||||||
use rusqlite::Connection;
|
use rusqlite::Connection;
|
||||||
|
|
||||||
|
pub mod albums;
|
||||||
pub mod bursts;
|
pub mod bursts;
|
||||||
pub mod cache;
|
pub mod cache;
|
||||||
pub mod collections;
|
pub mod collections;
|
||||||
@@ -57,6 +58,7 @@ pub mod sync;
|
|||||||
pub mod trash;
|
pub mod trash;
|
||||||
pub mod walk;
|
pub mod walk;
|
||||||
|
|
||||||
|
pub use albums::{Album, AlbumId, Place};
|
||||||
pub use cache::{Budget, Cache, DEFAULT_BUDGET_BYTES};
|
pub use cache::{Budget, Cache, DEFAULT_BUDGET_BYTES};
|
||||||
pub use collections::{Collection, CollectionKind, TreeRow};
|
pub use collections::{Collection, CollectionKind, TreeRow};
|
||||||
pub use dedup::{seen_by_content, seen_by_metadata, set_content_hash};
|
pub use dedup::{seen_by_content, seen_by_metadata, set_content_hash};
|
||||||
|
|||||||
@@ -121,6 +121,15 @@ pub struct MergeReport {
|
|||||||
pub keywords_assigned: usize,
|
pub keywords_assigned: usize,
|
||||||
/// Images whose capture metadata was taken from the remote.
|
/// Images whose capture metadata was taken from the remote.
|
||||||
pub metadata_adopted: usize,
|
pub metadata_adopted: usize,
|
||||||
|
|
||||||
|
/// Albums the remote had and this device did not, or had renamed, moved
|
||||||
|
/// or deleted with the higher revision.
|
||||||
|
pub albums_taken: usize,
|
||||||
|
/// Albums where this device's revision was at least as high.
|
||||||
|
pub albums_kept_local: usize,
|
||||||
|
/// Exported files the remote had recorded into an album and this device
|
||||||
|
/// had not.
|
||||||
|
pub album_exports_added: usize,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl MergeReport {
|
impl MergeReport {
|
||||||
@@ -136,12 +145,17 @@ impl MergeReport {
|
|||||||
|| self.keywords_fused > 0
|
|| self.keywords_fused > 0
|
||||||
|| self.keywords_assigned > 0
|
|| self.keywords_assigned > 0
|
||||||
|| self.metadata_adopted > 0
|
|| self.metadata_adopted > 0
|
||||||
|
|| self.albums_taken > 0
|
||||||
|
|| self.album_exports_added > 0
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether the local catalog holds anything the remote did not, and so
|
/// Whether the local catalog holds anything the remote did not, and so
|
||||||
/// must be uploaded even if nothing was taken from the remote.
|
/// must be uploaded even if nothing was taken from the remote.
|
||||||
pub fn should_upload(&self) -> bool {
|
pub fn should_upload(&self) -> bool {
|
||||||
self.kept_local > 0 || self.keywords_kept_local > 0 || self.local_changed()
|
self.kept_local > 0
|
||||||
|
|| self.keywords_kept_local > 0
|
||||||
|
|| self.albums_kept_local > 0
|
||||||
|
|| self.local_changed()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -197,10 +211,134 @@ pub fn merge_all(conn: &Connection) -> Result<MergeReport, CatalogError> {
|
|||||||
merge_keywords_within(&tx, &mut report)?;
|
merge_keywords_within(&tx, &mut report)?;
|
||||||
merge_people_within(&tx, &mut report)?;
|
merge_people_within(&tx, &mut report)?;
|
||||||
merge_metadata_within(&tx, &mut report)?;
|
merge_metadata_within(&tx, &mut report)?;
|
||||||
|
merge_albums_within(&tx, &mut report)?;
|
||||||
tx.commit()?;
|
tx.commit()?;
|
||||||
Ok(report)
|
Ok(report)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Merge albums and what was exported into them, from an attached catalog.
|
||||||
|
pub fn merge_albums(conn: &Connection) -> Result<MergeReport, CatalogError> {
|
||||||
|
let tx = conn.unchecked_transaction()?;
|
||||||
|
let mut report = MergeReport::default();
|
||||||
|
merge_albums_within(&tx, &mut report)?;
|
||||||
|
tx.commit()?;
|
||||||
|
Ok(report)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The album half: rows by [`verdict`], exports as a set union.
|
||||||
|
///
|
||||||
|
/// `album_folders` is not read. It is this device's choice of a local folder
|
||||||
|
/// and means nothing on the device that sent the snapshot — which has in any
|
||||||
|
/// case dropped it before uploading (see [`crate::albums`]).
|
||||||
|
fn merge_albums_within(tx: &Connection, report: &mut MergeReport) -> Result<(), CatalogError> {
|
||||||
|
// A snapshot from a build before albums, or from a device that never
|
||||||
|
// made one, has no tables to read. Ours are made on demand so the
|
||||||
|
// statements below have somewhere to write.
|
||||||
|
if !remote_has(tx, "albums")? {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
crate::albums::ensure_tables(tx)?;
|
||||||
|
|
||||||
|
let rows: Vec<(String, String, Option<String>, i64, i64, i64, bool, MergeVerdict)> = {
|
||||||
|
let mut stmt = tx.prepare(
|
||||||
|
"SELECT r.uuid, r.name, r.server_path, r.created, r.revision, r.modified,
|
||||||
|
r.deleted, l.revision, l.modified
|
||||||
|
FROM remote_cat.albums r
|
||||||
|
LEFT JOIN main.albums l ON l.uuid = r.uuid",
|
||||||
|
)?;
|
||||||
|
let rows = stmt
|
||||||
|
.query_map([], |r| {
|
||||||
|
let revision: i64 = r.get(4)?;
|
||||||
|
let modified: i64 = r.get(5)?;
|
||||||
|
let deleted: bool = r.get::<_, i64>(6)? != 0;
|
||||||
|
let local_rev: Option<i64> = r.get(7)?;
|
||||||
|
let local_mod: Option<i64> = r.get(8)?;
|
||||||
|
Ok((
|
||||||
|
r.get(0)?,
|
||||||
|
r.get(1)?,
|
||||||
|
r.get(2)?,
|
||||||
|
r.get(3)?,
|
||||||
|
revision,
|
||||||
|
modified,
|
||||||
|
deleted,
|
||||||
|
verdict(local_rev.zip(local_mod), (revision, modified), deleted),
|
||||||
|
))
|
||||||
|
})?
|
||||||
|
.collect::<Result<Vec<_>, _>>()?;
|
||||||
|
rows
|
||||||
|
};
|
||||||
|
|
||||||
|
{
|
||||||
|
// One upsert covers insert, update and tombstone: the verdict has
|
||||||
|
// already decided the remote row wins, so its fields are the answer
|
||||||
|
// whichever of the three it is.
|
||||||
|
let mut take = tx.prepare(
|
||||||
|
"INSERT INTO main.albums
|
||||||
|
(uuid, name, server_path, created, revision, modified, deleted)
|
||||||
|
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)
|
||||||
|
ON CONFLICT(uuid) DO UPDATE SET
|
||||||
|
name = excluded.name, server_path = excluded.server_path,
|
||||||
|
revision = excluded.revision, modified = excluded.modified,
|
||||||
|
deleted = excluded.deleted",
|
||||||
|
)?;
|
||||||
|
let mut forget = tx.prepare(
|
||||||
|
"DELETE FROM main.album_exports
|
||||||
|
WHERE album_id = (SELECT id FROM main.albums WHERE uuid = ?1)",
|
||||||
|
)?;
|
||||||
|
let mut unfold = tx.prepare(
|
||||||
|
"DELETE FROM main.album_folders
|
||||||
|
WHERE album_id = (SELECT id FROM main.albums WHERE uuid = ?1)",
|
||||||
|
)?;
|
||||||
|
for (uuid, name, server_path, created, revision, modified, deleted, v) in rows {
|
||||||
|
if v == MergeVerdict::KeptLocal {
|
||||||
|
report.albums_kept_local += 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
take.execute(rusqlite::params![
|
||||||
|
uuid, name, server_path, created, revision, modified, deleted as i64
|
||||||
|
])?;
|
||||||
|
if deleted {
|
||||||
|
forget.execute([&uuid])?;
|
||||||
|
unfold.execute([&uuid])?;
|
||||||
|
} else if server_path.is_some() {
|
||||||
|
// Another device moved the album to the server with the newer
|
||||||
|
// revision; a local folder chosen here is no longer where it
|
||||||
|
// goes.
|
||||||
|
unfold.execute([&uuid])?;
|
||||||
|
}
|
||||||
|
report.albums_taken += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
report.album_exports_added =
|
||||||
|
tx.execute(ALBUM_EXPORTS_BY_FILE_ID, [])? + tx.execute(ALBUM_EXPORTS_BY_CONTENT_HASH, [])?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Exported files, matched to local images by the server's file id — the
|
||||||
|
/// identity [`MEMBERS_BY_FILE_ID`] explains. Tombstoned albums are excluded,
|
||||||
|
/// or a merge would refill an album it had just deleted.
|
||||||
|
const ALBUM_EXPORTS_BY_FILE_ID: &str = "
|
||||||
|
INSERT OR IGNORE INTO main.album_exports(album_id, file_name, image_id, exported_at)
|
||||||
|
SELECT la.id, re.file_name, li.id, re.exported_at
|
||||||
|
FROM remote_cat.album_exports re
|
||||||
|
JOIN remote_cat.albums ra ON ra.id = re.album_id
|
||||||
|
JOIN main.albums la ON la.uuid = ra.uuid AND la.deleted = 0
|
||||||
|
JOIN remote_cat.remote rr ON rr.image_id = re.image_id
|
||||||
|
JOIN main.remote lr ON lr.file_id = rr.file_id
|
||||||
|
JOIN main.images li ON li.id = lr.image_id";
|
||||||
|
|
||||||
|
/// The same union by content hash, for a library with no server behind it.
|
||||||
|
const ALBUM_EXPORTS_BY_CONTENT_HASH: &str = "
|
||||||
|
INSERT OR IGNORE INTO main.album_exports(album_id, file_name, image_id, exported_at)
|
||||||
|
SELECT la.id, re.file_name, li.id, re.exported_at
|
||||||
|
FROM remote_cat.album_exports re
|
||||||
|
JOIN remote_cat.albums ra ON ra.id = re.album_id
|
||||||
|
JOIN main.albums la ON la.uuid = ra.uuid AND la.deleted = 0
|
||||||
|
JOIN remote_cat.images ri ON ri.id = re.image_id
|
||||||
|
JOIN main.images li ON li.content_hash = ri.content_hash
|
||||||
|
WHERE ri.content_hash IS NOT NULL";
|
||||||
|
|
||||||
/// Adopt capture metadata from an attached catalog, on its own.
|
/// Adopt capture metadata from an attached catalog, on its own.
|
||||||
pub fn merge_metadata(conn: &Connection) -> Result<MergeReport, CatalogError> {
|
pub fn merge_metadata(conn: &Connection) -> Result<MergeReport, CatalogError> {
|
||||||
let tx = conn.unchecked_transaction()?;
|
let tx = conn.unchecked_transaction()?;
|
||||||
|
|||||||
@@ -103,6 +103,10 @@ pub fn snapshot_for_upload(conn: &Connection, dest: &Path) -> Result<(), Catalog
|
|||||||
|
|
||||||
let out = Connection::open(dest)?;
|
let out = Connection::open(dest)?;
|
||||||
build_snapshot(conn, &source, &out)?;
|
build_snapshot(conn, &source, &out)?;
|
||||||
|
// This device's local album folders are paths and SAF grants nobody
|
||||||
|
// else can use; the merge never reads them, and the snapshot is what
|
||||||
|
// a fresh device would otherwise adopt whole.
|
||||||
|
out.execute_batch("DROP TABLE IF EXISTS album_folders")?;
|
||||||
verify_snapshot(&out)?;
|
verify_snapshot(&out)?;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
@@ -443,6 +447,92 @@ mod tests {
|
|||||||
assert_eq!(n, 2);
|
assert_eq!(n, 2);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// One image, known to the server by `file_id`, in a catalog.
|
||||||
|
fn with_image(c: &Connection, file_id: i64) -> dr_types::ImageId {
|
||||||
|
c.execute(
|
||||||
|
"INSERT OR IGNORE INTO roots(id, kind, label) VALUES (1, 'remote', 'lib')",
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
c.execute(
|
||||||
|
"INSERT INTO images(root_id, source_ref, added_at) VALUES (1, ?1, 0)",
|
||||||
|
[format!("IMG_{file_id}.CR3")],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let id = c.last_insert_rowid();
|
||||||
|
c.execute(
|
||||||
|
"INSERT INTO remote(image_id, file_id) VALUES (?1, ?2)",
|
||||||
|
[id, file_id],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
dr_types::ImageId(id as u64)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_album_and_its_exports_reach_another_device_but_its_folder_does_not() {
|
||||||
|
use crate::albums::{self, Place};
|
||||||
|
let dir = tempdir();
|
||||||
|
let remote_path = dir.join("remote.sqlite");
|
||||||
|
let snap = dir.join("snap.sqlite");
|
||||||
|
{
|
||||||
|
// The desktop: two albums, one on the server and one on its own
|
||||||
|
// disk, each with an export of the same photograph.
|
||||||
|
let r = seeded(&dir.join("desktop.sqlite"));
|
||||||
|
let img = with_image(&r, 4242);
|
||||||
|
let web = albums::create(&r, "Web", &Place::Server("Shared/Web".into())).unwrap();
|
||||||
|
let print = albums::create(&r, "Print", &Place::Local("/mnt/print".into())).unwrap();
|
||||||
|
albums::record_exports(&r, web, &[(img, "IMG_4242.jpg".into())]).unwrap();
|
||||||
|
albums::record_exports(&r, print, &[(img, "IMG_4242.tif".into())]).unwrap();
|
||||||
|
snapshot_for_upload(&r, &snap).unwrap();
|
||||||
|
std::fs::rename(&snap, &remote_path).unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
// The tablet knows the same file under its own image id.
|
||||||
|
let local = seeded(&dir.join("tablet.sqlite"));
|
||||||
|
with_image(&local, 1);
|
||||||
|
let img = with_image(&local, 4242);
|
||||||
|
|
||||||
|
let report = merge_remote(&local, &remote_path).unwrap();
|
||||||
|
assert_eq!(report.albums_taken, 2);
|
||||||
|
assert_eq!(report.album_exports_added, 2);
|
||||||
|
assert!(report.local_changed());
|
||||||
|
|
||||||
|
let all = albums::list(&local).unwrap();
|
||||||
|
let print = all.iter().find(|a| a.name == "Print").unwrap();
|
||||||
|
let web = all.iter().find(|a| a.name == "Web").unwrap();
|
||||||
|
assert_eq!(print.place, None, "the desktop's disk is not the tablet's");
|
||||||
|
assert_eq!(web.place, Some(Place::Server("Shared/Web".into())));
|
||||||
|
assert_eq!(albums::sources(&local, web.id).unwrap(), vec![img]);
|
||||||
|
|
||||||
|
// Nothing changed on either side, so a second pass takes nothing.
|
||||||
|
let again = merge_remote(&local, &remote_path).unwrap();
|
||||||
|
assert_eq!(again.albums_taken, 0);
|
||||||
|
assert_eq!(again.album_exports_added, 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_device_that_never_made_an_album_still_uploads_this_ones() {
|
||||||
|
use crate::albums::{self, Place};
|
||||||
|
let dir = tempdir();
|
||||||
|
let remote_path = dir.join("remote.sqlite");
|
||||||
|
{
|
||||||
|
// A snapshot from a build that predates albums altogether.
|
||||||
|
let r = seeded(&remote_path);
|
||||||
|
checkpoint(&r).unwrap();
|
||||||
|
}
|
||||||
|
let local = seeded(&dir.join("local.sqlite"));
|
||||||
|
albums::create(&local, "Web", &Place::Server("Web".into())).unwrap();
|
||||||
|
|
||||||
|
let report = merge_remote(&local, &remote_path).unwrap();
|
||||||
|
assert_eq!(report.albums_taken, 0);
|
||||||
|
// Its albums table is absent, so nothing was compared — and the local
|
||||||
|
// album has still to reach the server.
|
||||||
|
assert!(
|
||||||
|
albums::list(&local).unwrap().len() == 1,
|
||||||
|
"the local album survives a merge with a catalog that has none"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_remote_can_be_merged_twice_without_attach_conflict() {
|
fn the_remote_can_be_merged_twice_without_attach_conflict() {
|
||||||
// Detach must happen even on the failure path, or the second attempt
|
// Detach must happen even on the failure path, or the second attempt
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user