panorama.md: MI-GAN as the border filler — MIT, six operators, 7.4 s a tile
Read and measured, not built. The bare 512 generator exports at a fixed shape and loads under tract with nothing unsupported; at f32 on the desktop CPU it takes 7.4 s per 512×512 tile, which puts a full-resolution fill of the fixture's border at ten minutes. The three routes that would make it viable are recorded, with the quarter-resolution fill the cheapest and Hexagon int8 the one the model was designed for.
This commit is contained in:
@@ -344,3 +344,47 @@ half.
|
||||
6. **`derived_from` names sources by file name**, not content hash: the
|
||||
catalog's `content_hash` is null for most images most of the time. The
|
||||
hash can join it when the catalog has one.
|
||||
|
||||
## 12. Filling the border instead of cropping it — MI-GAN, read and measured 2026-09-19
|
||||
|
||||
Raised after the first merges: the ragged border a cylinder leaves could be
|
||||
*filled* rather than cropped away. FR-MRG-4 says no boundary fill, on
|
||||
§1.3's "not a pixel editor"; this is the evidence for deciding whether to
|
||||
revise that, not a revision.
|
||||
|
||||
**The candidate: MI-GAN** (Sargsyan et al., ICCV 2023, Picsart AI Research).
|
||||
Image inpainting designed for mobile: ~6 M parameters, plain convolutions —
|
||||
no FFT, no attention — so it quantises to int8 and runs on a phone's DSP,
|
||||
with quality close to LaMa and CoModGAN.
|
||||
|
||||
**Licence: MIT, code and weights alike** (`LICENSE` and `LICENSE-WEIGHTS`
|
||||
in the repository, read the same day). The cleanest position of any model
|
||||
in the tree — GPL-compatible, store-compatible, no grant to read around.
|
||||
|
||||
**Export.** The HuggingFace ONNX files are the *pipeline* — uint8 image and
|
||||
mask in, crop-around-mask, resize and blend inside the graph, every
|
||||
dimension dynamic — and tract refuses them (F6 again). The bare generator
|
||||
exports cleanly from the `migan_512_places2.pt` state dict at a fixed
|
||||
`1×4×512×512` (`export_migan.py` in the spike directory; the input is
|
||||
`mask − 0.5` and the masked RGB in −1..1, the output RGB in −1..1, the
|
||||
caller composites). After slimming the graph is **six operator types**:
|
||||
`Add, Clip, Conv, LeakyRelu, Mul, Resize`. 28 MB.
|
||||
|
||||
**Under tract on the reference desktop: loads in 53 ms, runs in 7.4 s per
|
||||
512 × 512 tile, f32.** That is the number. The fixture's border is two
|
||||
ragged bands across 22 993 px — roughly ninety 512-px tiles at full
|
||||
resolution — so a CPU-f32 fill is ten minutes on the desktop and longer on
|
||||
the tablet. Three ways to make it viable, none built:
|
||||
|
||||
1. **Fill at a quarter of the resolution and upsample.** Sky and scree
|
||||
tolerate it; twenty-odd tiles, about three minutes on the desktop CPU. A
|
||||
background job with the outbox's patience, not an interactive one.
|
||||
2. **int8 on the tablet's Hexagon through QNN**, where the plain-conv design
|
||||
is the point and the whole graph should run in milliseconds. The setup
|
||||
exists from the eye-state work; MI-GAN is a candidate for the same path.
|
||||
3. **A WGSL runtime for those six operators.** A project of its own, and
|
||||
the only route that would make it interactive on the desktop.
|
||||
|
||||
Whichever, the fill is a *proposal* under FR-MRG-1's rule — shown, then
|
||||
confirmed — and it would sit beside the crop, not replace it: the crop is
|
||||
free and honest, the fill is invented pixels, and the photographer chooses.
|
||||
|
||||
+12
-12
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user