Specify panorama merging: §3.11, D18, S15, and the design in panorama.md
A merge writes a new source file beside its sources (D18) rather than a multi-source Version, which answers the schema question §7 had been holding open for panorama, HDR merge and focus stacking together. The panorama is undeferred as FR-MRG-1 … 11; the other two stay in §7 with their data model decided. FR-MRG-10 and 11 fix where the work runs — every per-pixel stage on the GPU, the composite never held as one texture — because the output exceeds max_texture_dimension_2d before it exceeds memory. panorama.md carries the stage table, the chunked output driver, the model licences and the porting sources. S15 gates all of it. Coverage falls from 83.0% to 77.2%: thirteen requirements entered with no code, and outstanding.md §11 says so.
This commit is contained in:
+151
-2
@@ -1678,6 +1678,114 @@ application starts, opens a photograph, names the responsible plugin, and contin
|
||||
|
||||
---
|
||||
|
||||
### 3.11 Merging images
|
||||
|
||||
Several photographs become one. Panorama is the first merge and the only one specified; HDR merge
|
||||
and focus stacking share its data model (D18) and are still deferred in §7. The clauses below are
|
||||
written for the panorama and, where a clause is general to any merge, say so.
|
||||
|
||||
**FR-MRG-1 — Panorama from a selection.** Two or more selected images are aligned and blended
|
||||
into one composite, which is written as a new source file per D18. The tool is never automatic:
|
||||
it proposes an alignment, the photographer sees it and confirms, and nothing is written before
|
||||
that press.
|
||||
|
||||
The stated audience (D11) shoots panoramas and currently leaves the application to stitch them,
|
||||
which is the workflow break FR-DEV-8 was added to close for dust. It is also the first of the
|
||||
three §7 merges, and the one whose alignment problem is smallest — a rotation about one point,
|
||||
with no depth to recover — so it is where the shared machinery is built.
|
||||
|
||||
**FR-MRG-2 — What is stitched.** Each source enters the merge at develop-neutral scene-linear:
|
||||
after black and white levels, demosaic, camera profile and lens distortion correction, before any
|
||||
tone or colour adjustment, with one white balance — the first frame's — applied to all. The
|
||||
sources' own edits are not baked in. The composite is developed afterwards as if it were a new
|
||||
RAW.
|
||||
|
||||
This is the clause that decides what the output *is*. Stitching the rendered edits is what a JPEG
|
||||
stitcher does; the result cannot be re-developed, and any difference between the frames' edits
|
||||
becomes a seam. Stitching neutral pixels produces something that behaves like a photograph the
|
||||
camera could have taken, and every develop operation in §3.3 then applies to it once, not five
|
||||
times. Lens correction sits above the cut because a distorted frame does not align; white balance
|
||||
sits above it because the scalars must agree across frames or the overlaps do not match.
|
||||
|
||||
**FR-MRG-3 — The output file.** *(general to any merge)* Scene-linear, at least 16 bits per
|
||||
channel, in a wide gamut with the colour transform resolved or the profile carried, with capture
|
||||
metadata from the first source. Named from the first source with a stated suffix and placed
|
||||
beside it. Where the sources' folder is not writable — a remote-only tier, a read-only mount —
|
||||
it goes where an export goes (FR-EXP-6) and the interface says so before the merge starts.
|
||||
|
||||
The container is a spike result (S15), not a requirement: a linear DNG if rawler reads back what
|
||||
the application writes, else a float TIFF with a decode path of its own. The choice is invisible
|
||||
to everything above the decoder.
|
||||
|
||||
**FR-MRG-4 — Projection and framing.** Cylindrical, spherical or perspective, chosen from the
|
||||
field of view and overridable; the horizon levelled from the estimated rotations, overridable by a
|
||||
drag; auto-crop to the largest inscribed rectangle, overridable. No boundary fill: painting pixels
|
||||
that were never captured is the pixel editing §1.3 excludes.
|
||||
|
||||
**FR-MRG-5 — Honesty of failure.** *(general to any merge)* A frame that cannot be aligned is
|
||||
named, with why — too few matches, no overlap with any other frame, a residual above the stated
|
||||
bound — and the merge stops. Never a silent drop, never a best-effort composite with a frame
|
||||
missing.
|
||||
|
||||
The same rule as `spot-removal.md`'s and D17's: a tool that quietly alters or omits part of a
|
||||
photograph is the failure this application must not have, and here the omission would be an
|
||||
entire frame.
|
||||
|
||||
**FR-MRG-6 — Provenance.** *(general to any merge)* The composite's sidecar carries
|
||||
`derived_from`: the content hashes of its sources in order, and the merge parameters. The history
|
||||
records the merge as the first entry, and export metadata declares the composite as one. Sources
|
||||
trashed later leave the list dangling; the panel says so and nothing is blocked.
|
||||
|
||||
Provenance, not dependency. The composite renders from itself alone; `derived_from` exists so the
|
||||
photographer, and anyone they hand the file to, can see what it is made of. The audience is
|
||||
RAW-literate and a composite declares itself (D17). Nothing here specifies C2PA.
|
||||
|
||||
**FR-MRG-7 — Execution.** *(general to any merge)* A background job on the pattern FR-EXP-7
|
||||
established: its own thread, its own `GpuContext`, a row in the activity panel, cancellable with
|
||||
NFR-ARCH-3's bound. Alignment runs at proxy resolution and drives the preview; the full-resolution
|
||||
warp and blend run only on confirm.
|
||||
|
||||
**FR-MRG-8 — Model-optional.** Keypoint detection works without any model weights and better
|
||||
with them, the convention `dr-segment` set. Weights that ship are recorded in `models/LICENCE.md`
|
||||
before they land, under D8's compatibility test, and their absence degrades quality rather than
|
||||
disabling the feature.
|
||||
|
||||
**FR-MRG-9 — Platform.** Desktop first. Android runs the same code within NFR-RES-2 and
|
||||
FR-MRG-11, with a stated ceiling on frame count and source resolution, refused with a message,
|
||||
rather than an out-of-memory kill.
|
||||
|
||||
**FR-MRG-10 — Where the work runs.** *(general to any merge)* Every per-pixel stage of a merge —
|
||||
rendering the sources, the preview reprojection, the full-resolution warp, gain compensation, the
|
||||
seam and the blend — runs on the GPU as WGSL, under ARCH §6.4. The stages that are not per-pixel
|
||||
— keypoint detection at proxy resolution, descriptor matching, and the rotation solve over a few
|
||||
parameters per frame — run on the CPU, and the specification says so rather than leaving it to be
|
||||
"moved later".
|
||||
|
||||
The per-pixel stages are the whole cost, and the tablet is where the cost is paid: the output is
|
||||
larger than any single photograph the pipeline has rendered, and a CPU blend of it would take
|
||||
minutes there. The CPU stages are bounded by frame count, not by output size — detection is once
|
||||
per frame at 1024 px, on the same runtime faces and masks use — and moving a small model's
|
||||
convolutions to hand-written WGSL is real work for no visible gain. Seam finding is the one
|
||||
classic stage that resists the GPU; the seam algorithm is chosen for the GPU, not for the paper.
|
||||
|
||||
**FR-MRG-11 — Tiled in output space.** *(general to any merge)* No stage may hold the composite as
|
||||
one texture, on any platform. The warp, seam, blend and encode proceed in output-space chunks, each
|
||||
pulling only the source tiles that project into it, so the working set is one chunk plus its
|
||||
sources' tiles regardless of how large the composite is.
|
||||
|
||||
Two facts force this before memory does. `max_texture_dimension_2d` is 8192 on many mobile GPUs
|
||||
and 16384 on desktop, and a three-row panorama is routinely 20 000 px wide — the composite would
|
||||
not fit a texture even with the memory to spare. And five 24 MP frames at the working precision
|
||||
are ~1 GB together, which the tablet does not have. ARCH §6.2 applies to the composite as it
|
||||
applies to a source, and retrofitting it would be the rewrite it warns about.
|
||||
|
||||
**Non-goals, fixed now.** No translation solve or parallax correction — seam placement is the
|
||||
tool for a hand-held set, and a photograph with real parallax is not a panorama. No HDR panorama
|
||||
in one operation until HDR merge exists on its own. No live re-stitch: a different projection or
|
||||
crop after the fact is a new file, not an edit. No video.
|
||||
|
||||
---
|
||||
|
||||
## 4. Non-functional requirements
|
||||
|
||||
### 4.1 Performance targets
|
||||
@@ -1719,6 +1827,11 @@ the class where it was expressed.
|
||||
Everything the photographer produced or navigated to is a different matter, and none of it may be
|
||||
touched by a resize. That is the list in the criterion, and it is the testable half.
|
||||
|
||||
**NFR-MRG-1 — Merge latency.** For five 24 MP frames: the alignment preview (FR-MRG-7) within
|
||||
5 s on the reference desktop and 15 s on the reference tablet, of which keypoint detection is at
|
||||
most 1 s per frame on the tablet's CPU; the full merge written to disk within 60 s on the desktop.
|
||||
The tablet's full-merge figure is set by S15, not guessed here.
|
||||
|
||||
**Performance regressions fail the build.** §8's benchmark suite runs per-commit; a regression
|
||||
beyond a stated tolerance is a build failure, not a notification. Performance work rots otherwise.
|
||||
|
||||
@@ -1765,6 +1878,11 @@ means a second implementation of the operations. ARCH §6.4 stands as written, a
|
||||
fallback clause has been reworded to match. A second full pipeline was the alternative, and it was
|
||||
declined for the reason ARCH §6.4 gives: the GPU path is the product, not an optimisation of it.
|
||||
|
||||
**NFR-MRG-2 — Reproducible merges.** The same sources, the same settings and the same device
|
||||
produce a byte-identical composite. Across devices the comparison is tolerance-based, calibrated
|
||||
as S9 calibrates R1: the merge is float work end to end, and ARCH §6.13's bit-identity applies to
|
||||
integer state only.
|
||||
|
||||
### 4.3 Resource behaviour
|
||||
|
||||
**NFR-RES-1 — Bounded memory.** Memory use is bounded and configurable, independent of catalog
|
||||
@@ -2001,6 +2119,7 @@ Rationale, evidence, and the eliminated alternatives are recorded in
|
||||
| D10 | Interface strategy | One adaptive UI, tablet + desktop |
|
||||
| D11 | Product positioning | Culling-first differentiator; see below |
|
||||
| D12 | Scope versus pace | **DECIDED 2026-09-19** — settled by events; full scope stands, no v1 date |
|
||||
| D18 | Derived images | **DECIDED 2026-09-19** — a merge writes a new source file; no multi-source Version |
|
||||
|
||||
### D11 — product positioning
|
||||
|
||||
@@ -2195,6 +2314,9 @@ collides with four things this document says.
|
||||
priced before starting), trashing B must know A depends on it (FR-CAT-15), and `Version::merge`
|
||||
has never seen a cross-reference. Reopening ARCH §6.3 for this reopens it for the three deferred
|
||||
rows at once, which is the argument for doing it once and properly rather than for this alone.
|
||||
*D18 has since done it once, the other way:* a merge writes a new file and ARCH §6.3 stands.
|
||||
That leaves this case as the only one that would still need a cross-reference — the output is
|
||||
frame A, not a new file — so the cost above is now this feature's alone to justify.
|
||||
3. **Tone.** The source patch goes through A's chain, not B's — B demosaiced and run through A's
|
||||
parameters to the head of the detail chain, then sampled. A second small pipeline at proxy
|
||||
resolution; a second full demosaic at export. The heal hides lighting drift between frames; it
|
||||
@@ -2211,6 +2333,28 @@ Deferred under D12 until FR-CULL-8a exists and spot removal's disc has, in its o
|
||||
finished and used. The first cut, when it comes, is "clone from a neighbouring frame" as a spot
|
||||
source; the face-aware proposal is a layer over that.
|
||||
|
||||
### D18 — derived images · **DECIDED 2026-09-19**
|
||||
|
||||
**A merge produces a new source file, not a multi-source Version.** The composite is written
|
||||
beside its sources (FR-MRG-3), gets its own sidecar and content-hash identity, and is from then on
|
||||
an ordinary `Image`: developed, synced, exported and trashed like any other. Its sidecar carries
|
||||
`derived_from` (FR-MRG-6) as *provenance*, not as a *dependency* — trashing a source does not break
|
||||
the composite, and rendering it needs nothing but itself.
|
||||
|
||||
This is the ARCH §6.3 question §7 has been keeping open for panorama, HDR and focus stacking,
|
||||
answered once for all three. The alternative — a Version whose inputs are several other images,
|
||||
rendered live — was priced in D17: sidecar cross-references, `Version::merge` seeing a reference
|
||||
for the first time, FR-CAT-15 knowing that trashing B breaks A, FR-NC-6c requiring every source
|
||||
present and priced before A can render, and an export that decodes N files. Every one of those is
|
||||
a change to a subsystem that works today, and none of them buys the photographer anything a file
|
||||
does not. It is also what Lightroom does, and the audience (D11) knows it.
|
||||
|
||||
What it forecloses, so that it reads as a decision: re-merging with different settings is a new
|
||||
file, not an edit to the old one; and the composite occupies disk — a five-frame panorama is a
|
||||
100–200 MB file, which the photographer chose to make. D17 narrows accordingly to the one case
|
||||
where the output is still frame A, and inherits nothing from this decision but the provenance
|
||||
rule.
|
||||
|
||||
### D16 — plugin licensing · **OPEN, post-v1**
|
||||
|
||||
> Deferred with §3.10 on 2026-09-19. Still to be answered before the format is published as
|
||||
@@ -2248,8 +2392,9 @@ note where deferring now constrains the design later.
|
||||
| Deferred | Note |
|
||||
|---|---|
|
||||
| Tethered shooting | — |
|
||||
| Panorama and HDR merge | **Keep the schema open** — these produce images derived from multiple sources, which ARCH §6.3's single-source `Image` cannot express. |
|
||||
| Focus stacking | Same provenance consideration. |
|
||||
| ~~Panorama~~ | **Undeferred 2026-09-19** — §3.11 (FR-MRG-1 … FR-MRG-11), under D18, which answers the schema question this row was holding open: a merge is a new source file, and ARCH §6.3's single-source `Image` does not change. |
|
||||
| HDR merge | Deferred. D18 answers the data model; the merge itself — exposure alignment, ghost handling, the tone of the result — is not specified. FR-MRG-3, 5, 6, 7, 10 and 11 are written to be general to it. |
|
||||
| Focus stacking | Deferred, on the same terms as HDR merge. |
|
||||
| Cross-frame face repair ("best take") | A face from a neighbouring frame of the same burst, aligned by its landmarks and blended by FR-DEV-8's heal. Mechanically a spot whose source is another photograph; **the same multi-source schema question as the two rows above, arriving early** — D17. Deferred rather than refused, with its non-goals fixed now: never automatic, geometry not corrected, source frame declared in sidecar, history and export. |
|
||||
| Gaze and eye-contact estimation | Every open gaze model read on 2026-09-19 is trained on Gaze360, MPIIGaze or ETH-XGaze, all research-only, and Gaze360's licence restricts *models trained on it* by name — the InsightFace situation again (D13). Head pose from the five landmarks is the proxy (FR-CULL-8a). Revisit when weights with a clean data chain exist; iris offset within the eye crop is the licence-free fallback if the proxy proves too weak. |
|
||||
| Print layout | — |
|
||||
@@ -2324,6 +2469,8 @@ stacks.
|
||||
| **S13** | **Slint accessibility on Android:** verify TalkBack exposure of names, roles, and values | Whether NFR-A11Y-2 is achievable in the chosen toolkit | NFR-A11Y-2 |
|
||||
| **S14** | **Face pipeline in Rust, on a real personal library:** run a detector plus an embedder over ~2,000 images through a Rust ONNX runtime, at proxy resolution, on the reference desktop. Measure per-image latency, cluster purity against hand-labelled truth, and fit the FR-CULL-9 calibration to see whether it converges on a library-sized sample. **Resolve the model licence question before writing any of it** | Whether §3.9.1 is buildable without breaking the pure-Rust dependency policy, and whether the accuracy is worth the subsystem | D13, FR-CULL-8, FR-CULL-9 |
|
||||
|
||||
| **S15** | **Panorama pre-conditions, in order of what can kill it:** (1) write a linear DNG with the `tiff` crate and read it back through rawler — decides FR-MRG-3's container; (2) export XFeat to ONNX at a fixed 1024 px input and load it under tract with zero unsupported operators — the F6 check `segmentation.md` records, and the licence read first; (3) tap the working-space texture after lens correction and before tone, and confirm it carries what FR-MRG-2 asks for; (4) a tiled multi-band blend of a 100 MP output on the reference tablet, and XFeat's per-frame time on its CPU — the two halves of NFR-MRG-1 | Whether §3.11 is buildable on the pipeline as it stands, and what the tablet figure is | D18, FR-MRG-2, FR-MRG-3, FR-MRG-8, FR-MRG-11, NFR-MRG-1 |
|
||||
|
||||
### Why this order
|
||||
|
||||
**S1, S2, and S10 are the three that can invalidate the architecture.** S1 and S2 test ARCH §6.1 — the
|
||||
@@ -2354,4 +2501,6 @@ Android GPU vendors.
|
||||
- **Demosaic** — reconstructing full RGB from a colour-filter-array sensor capture.
|
||||
- **CFA** — colour filter array (Bayer, X-Trans).
|
||||
- **Sidecar** — a small file alongside the source holding edit metadata.
|
||||
- **Composite** — an image produced by a merge (§3.11) from several sources; a source file in its own right under D18.
|
||||
- **Merge** — an operation that produces a composite: panorama, HDR merge, focus stacking.
|
||||
- **Pixel pipeline** — the ordered chain of processing stages from sensor data to output.
|
||||
|
||||
Reference in New Issue
Block a user