docs(VR-012): the quality knee, and what it retires
Sharpness is not a sufficient statistic for identity loss. Six cells at effectively identical measured sharpness span 15.3% to 91.0% TPI, ordered entirely by source size, because a scalar keyed on high-frequency energy cannot separate attenuated high frequencies from destroyed spatial sampling. AR-028's "kept separate, not collapsed into one scalar" now rests on a measurement rather than an argument -- and the reasoning it used to rest on, that the aligned crop is scale-normalised so a measure there cannot re-measure size, was wrong and is corrected in place. Variance of Laplacian -- the most widely used blur metric there is -- is anti-predictive at fixed degradation on all three blur families. The decile it calls sharpest is 2.6x less identifiable than the decile it calls blurriest, monotone across ten bins, because within a cell its residual variance is native contrast rather than detail, and hard shadows and JPEG ringing raise it while making a face harder to match. Gating on it would preferentially discard the more identifiable faces. Blur breaks confidence, not identity: rank-1 holds at 80.2% where TPI is 15.3%, and FPI never left 0.1% in any of the 108 cells. Degradation produces abstention, never a wrong name. That is also why sharpness fails as a compute gate -- even a visually destroyed face stays 46.9% identifiable, so a gate discards recoverable evidence at three times the cost of the free size filter. Discount, do not gate; the rule AR-028 already stated now has evidence for why it is right rather than merely cautious. Records the shape a discount must have (flat, then a cliff between sigma 2 and 3), that its cost scales with proximity to the decision boundary rather than with blur, and that the pose half of VR-012 has not been run. TRACES: VR-012, AR-028, AR-029, AR-030 | SR-002
This commit is contained in:
+77
-14
@@ -248,11 +248,29 @@ the same response.
|
|||||||
- **Size** — already AR-002, floor at 40×40 px in original resolution, measured
|
- **Size** — already AR-002, floor at 40×40 px in original resolution, measured
|
||||||
end to end by VR-013. It is the precedent for the other two: the
|
end to end by VR-013. It is the precedent for the other two: the
|
||||||
threshold was *located*, not chosen.
|
threshold was *located*, not chosen.
|
||||||
- **Sharpness** — motion blur and soft focus destroy the high-frequency detail
|
- **Sharpness** — motion blur and optical defocus destroy the high-frequency
|
||||||
the embedder keys on, and unlike size they leave the bounding box looking
|
detail the embedder keys on, and unlike size they leave the bounding box
|
||||||
perfectly healthy. Measured on the **112×112 aligned crop**, not the raw box:
|
looking perfectly healthy. Measured on the **112×112 aligned crop**, not the
|
||||||
the crop is already scale-normalised, so a measure taken there cannot silently
|
raw box.
|
||||||
re-measure face size and double-count it against AR-002.
|
|
||||||
|
An earlier version of this clause argued the crop is scale-normalised and so a
|
||||||
|
measure taken there "cannot re-measure face size and double-count it against
|
||||||
|
AR-002". **That reasoning is wrong and VR-012 measured it wrong.** The
|
||||||
|
normalisation is geometric, not informational: a 40 px face upscaled into the
|
||||||
|
canonical frame genuinely carries less high-frequency content than a 400 px
|
||||||
|
one downscaled into it, so every candidate measure *does* respond to source
|
||||||
|
size. What the crop yields is **effective resolution in canonical space** —
|
||||||
|
the union of "was small" and "was blurred", not blur alone.
|
||||||
|
|
||||||
|
The conclusion survives, for a better reason. VR-012 sorted its grid by
|
||||||
|
measured sharpness and found the six cells at effectively identical sharpness
|
||||||
|
(0.0003–0.0005) spanning **15.3% to 91.0% TPI**, ordered entirely by source
|
||||||
|
size. Sharpness is therefore not a sufficient statistic for identity loss: a
|
||||||
|
scalar keyed on high-frequency energy cannot separate *attenuated* high
|
||||||
|
frequencies from *destroyed* spatial sampling, because blur preserves
|
||||||
|
mid-frequency facial geometry exactly while downsampling destroys it. The two
|
||||||
|
axes are not redundant and neither substitutes for the other — which is what
|
||||||
|
"not collapsed into one scalar" above now rests on.
|
||||||
- **Visibility** — extreme pose or occlusion means the face presents fewer of the
|
- **Visibility** — extreme pose or occlusion means the face presents fewer of the
|
||||||
features the embedding assumes are present. The measure is the **residual of
|
features the embedding assumes are present. The measure is the **residual of
|
||||||
the AR-005 alignment fit**: the RMS landmark error, in canonical 112×112
|
the AR-005 alignment fit**: the RMS landmark error, in canonical 112×112
|
||||||
@@ -333,18 +351,63 @@ hand-chosen cutoff on an uncalibrated measure is the same unfalsifiable magic
|
|||||||
number AR-024 retired for similarity, and it would fail the same way: meaning
|
number AR-024 retired for similarity, and it would fail the same way: meaning
|
||||||
something different for every detector, every embedder and every film.
|
something different for every detector, every embedder and every film.
|
||||||
|
|
||||||
|
**A discount curve on sharpness must be flat, then steep.** VR-012 measured the
|
||||||
|
response as a cliff rather than a gradient: Gaussian sigma up to 1.5 costs under
|
||||||
|
1.5 points of TPI in every cell — at 16 px it is very slightly *positive*,
|
||||||
|
smoothing upscale artifacts — sigma 2 costs 1–3, and the 2→3 step costs 7–19. A
|
||||||
|
linear or sigmoid discount over the measure would penalise the whole flat region
|
||||||
|
where blur demonstrably costs nothing.
|
||||||
|
|
||||||
|
**Which blur is modelled is a first-order decision, not a detail.** VR-012 swept
|
||||||
|
three families at matched per-axis PSF spread, and at σ=3 px on a 112 px face
|
||||||
|
they cost 9%, 18% and **53%** error for Gaussian, motion and optical defocus
|
||||||
|
respectively. Defocus is the destructive one because its disc PSF has a jinc
|
||||||
|
transfer function with **exact zeros** — bands annihilated rather than
|
||||||
|
attenuated — where a Gaussian merely rolls off. It is also the case AR-002
|
||||||
|
cannot catch, since a defocused face is large and confidently detected. Any
|
||||||
|
future study that sweeps blur states its family and its justification; a
|
||||||
|
Gaussian-only sweep understated the effect by a factor of five and would have
|
||||||
|
retired this axis as not worth its cost.
|
||||||
|
|
||||||
|
**The cost of blur is proportional to proximity to the decision boundary, not to
|
||||||
|
blur itself.** Sigma 3 costs −22.5 points at 24 px, but only −7.9 at 112 px
|
||||||
|
(margin to spare) and −8.3 at 16 px (already below threshold). This is why the
|
||||||
|
axes must combine multiplicatively in `EvidenceDiscounter` rather than each
|
||||||
|
gating independently.
|
||||||
|
|
||||||
|
**Sharpness discounts; it must never gate.** VR-012 tried the gate directly, as
|
||||||
|
a compute saving: skipping the embed below a sharpness threshold costs 15.1% of
|
||||||
|
true identifications to save 20% of the work, against the size filter's 4.7% at
|
||||||
|
16.7% — three times the damage, from a measure that needs the warped crop plus a
|
||||||
|
DFT where size is a bbox dimension available for free. The reason is a ceiling
|
||||||
|
no measure can beat: **at 112 px with defocus radius 6 — visually destroyed —
|
||||||
|
46.9% of faces still identify correctly, and rank-1 is still 94.8%.** Apparent
|
||||||
|
blur does not determine the outcome. The size filter wins only because smallness
|
||||||
|
destroys identity more completely than blur does (16 px succeeds 23.5% of the
|
||||||
|
time), and that asymmetry is the measured justification for the rule above:
|
||||||
|
**failing sharpness discounts the observation, failing size may drop it.**
|
||||||
|
|
||||||
**Current:** visibility is measured and carried — `estimate_alignment()` in
|
**Current:** visibility is measured and carried — `estimate_alignment()` in
|
||||||
`src/face_utils.hpp` returns the residual alongside the transform, and
|
`src/face_utils.hpp` returns the residual alongside the transform, and
|
||||||
`FaceAlignerFunc` writes it to `DetectedFace::alignment_residual`. Size is
|
`FaceAlignerFunc` writes it to `DetectedFace::alignment_residual`. Sharpness is
|
||||||
`min_face_px` (40, decoded-frame space — AR-002 still open). Sharpness is
|
measured: `assess_sharpness()` in `src/quality.hpp` returns five AR-029
|
||||||
unmeasured. Nothing yet *consumes* any of it: no discount is applied, and
|
candidates over a fixed 64×64 window on the face interior, and VR-012 has ranked
|
||||||
`align_face()` still drops the degenerate-fit case without counting it.
|
them — `var_laplacian` and `tenengrad` are disqualified as discounts (see
|
||||||
|
AR-029), leaving `hf_energy_ratio` as the only correctly-signed survivor. Size
|
||||||
|
is `min_face_px` (40, decoded-frame space — AR-002 still open). All three are
|
||||||
|
exposed to studies through `sae_embed`. Nothing yet *consumes* any of it: no
|
||||||
|
discount is applied, and `align_face()` still drops the degenerate-fit case
|
||||||
|
without counting it.
|
||||||
|
|
||||||
**Gap:** AR-029 entirely. For AR-030, the measure exists but the discount does
|
**Gap:** the discount itself, on every axis. Neither sharpness nor the residual
|
||||||
not — it must reach `EvidenceDiscounter` as the reliability term. For AR-028, the
|
reaches `EvidenceDiscounter`, whose weight remains pure novelty — so a profile
|
||||||
residual does not yet reach the VR-001 dump, which is what VR-012 needs to run
|
or defocused view still moves a track's belief hardest when it deserves the
|
||||||
from fixtures; that is the next step, since it unblocks the study that sets
|
least trust. Neither reaches the VR-001 dump either, so VR-012 must still re-run
|
||||||
every remaining behaviour.
|
video rather than replay fixtures. VR-012's **pose half is not started**: the
|
||||||
|
AR-030 residual has no arm in the grid, so whether the 5-point proxy suffices or
|
||||||
|
a dedicated landmark model is needed remains open. And the sharpness result is
|
||||||
|
weak enough (best within-cell AUC 0.530) that whether AR-029 earns a discount at
|
||||||
|
all is still a judgement, not a measurement.
|
||||||
|
|
||||||
## AR-007, AR-008 — Tracking
|
## AR-007, AR-008 — Tracking
|
||||||
|
|
||||||
|
|||||||
@@ -53,6 +53,14 @@ every finding below.
|
|||||||
A training-set effect that did not reproduce on 5 held-out films once
|
A training-set effect that did not reproduce on 5 held-out films once
|
||||||
two methodology bugs in the comparison harness were found and fixed.
|
two methodology bugs in the comparison harness were found and fixed.
|
||||||
|
|
||||||
|
- :material-blur:{ .lg .middle } **[What does blur cost?](quality-knee.md)**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
Sharpness is not a sufficient statistic for identity loss, blur breaks
|
||||||
|
confidence rather than ranking, and variance-of-Laplacian is
|
||||||
|
anti-predictive at fixed resolution.
|
||||||
|
|
||||||
- :material-magnify-expand:{ .lg .middle } **[Deep dive: LVFace-B Glint360K](lvface-deep-dive.md)**
|
- :material-magnify-expand:{ .lg .middle } **[Deep dive: LVFace-B Glint360K](lvface-deep-dive.md)**
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -0,0 +1,308 @@
|
|||||||
|
# Quality knee: what does a blurred or small face cost, and can a measure predict it?
|
||||||
|
|
||||||
|
VR-012. Companion to the minimum-face-size studies VR-005 and VR-013 (see the
|
||||||
|
[requirement register](requirements.md)), which located the size floor at 40 px;
|
||||||
|
this asks the same question for **sharpness**, and asks whether any cheap
|
||||||
|
measure taken on the aligned crop can be acted on at inference.
|
||||||
|
|
||||||
|
Run by
|
||||||
|
[`scripts/validation/quality_knee.py`](https://REPOLINK/scripts/validation/quality_knee.py)
|
||||||
|
through the `sae_embed` bindings — detection, the ArcFace warp, the embedder,
|
||||||
|
the five candidate measures and the Platt calibration are all the shipped C++.
|
||||||
|
|
||||||
|
## Protocol
|
||||||
|
|
||||||
|
1670 gallery actors with 3 or more mugshots (of 2456 total), one image held out
|
||||||
|
per actor as a probe, the remaining 10326 embeddings staying in the gallery at
|
||||||
|
native resolution. Only the probe degrades — reference mugshots are clean and
|
||||||
|
the face coming out of the video is not.
|
||||||
|
|
||||||
|
Each probe passes through a **joint grid**: downscale to *S*×*S* and back to
|
||||||
|
112 (the sampling loss), then blur at level *L* in canonical pixels. Three blur
|
||||||
|
families, 36 cells each, 60120 probe-cell records per family:
|
||||||
|
|
||||||
|
| family | models | parameter |
|
||||||
|
|---|---|---|
|
||||||
|
| Gaussian | soft focus, a generic stand-in | sigma 0 … 3 |
|
||||||
|
| **Disc** | **real optical defocus** — the circle of confusion | radius 0 … 6 |
|
||||||
|
| Motion | camera pan or moving subject | length 0 … 21 px |
|
||||||
|
|
||||||
|
The three are not interchangeable, and sweeping only the first was the original
|
||||||
|
design error — one that would have produced a wrong answer, not merely an
|
||||||
|
incomplete one (Result 3). A defocused lens spreads a point into a **uniform
|
||||||
|
disc**, whose transfer function is a jinc — `2·J1(x)/x` — that crosses zero and
|
||||||
|
goes negative, annihilating whole frequency bands and returning the ones beyond
|
||||||
|
each zero phase-reversed. A Gaussian MTF is strictly positive and monotone and
|
||||||
|
does neither. More practically: defocus and motion are how a face ends up
|
||||||
|
**large and useless**, while Gaussian blur as swept here mostly co-occurs with
|
||||||
|
small faces. That difference decides whether sharpness carries anything the size
|
||||||
|
filter does not.
|
||||||
|
|
||||||
|
Families are compared at matched **per-axis PSF standard deviation** (σ for a
|
||||||
|
Gaussian, R/2 for a disc, L/√12 for a linear smear), never at equal raw
|
||||||
|
parameter, which would compare different amounts of damage.
|
||||||
|
|
||||||
|
Identification is the pipeline's own decision: per-actor best-of-N cosine →
|
||||||
|
Platt sigmoid → accept above `prob_threshold` 0.754. Never a raw cosine
|
||||||
|
(AR-024).
|
||||||
|
|
||||||
|
## Result 1 — sharpness is not a sufficient statistic
|
||||||
|
|
||||||
|
Sorting the 36 Gaussian cells by `hf_energy_ratio`, the six sigma-3 cells land
|
||||||
|
at effectively identical measured sharpness:
|
||||||
|
|
||||||
|
| size | sigma | hf_energy_ratio | TPI |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 16 | 3 | 0.0003 | **15.3%** |
|
||||||
|
| 24 | 3 | 0.0003 | 63.2% |
|
||||||
|
| 32 | 3 | 0.0003 | 79.4% |
|
||||||
|
| 48 | 3 | 0.0003 | 86.6% |
|
||||||
|
| 64 | 3 | 0.0004 | 88.4% |
|
||||||
|
| 112 | 3 | 0.0005 | **91.0%** |
|
||||||
|
|
||||||
|
Same measured sharpness, a **76-point spread in identification**. It inverts
|
||||||
|
too: 16 px unblurred measures 0.0033 and scores 23.5%, while 48 px at sigma 2
|
||||||
|
measures *lower* at 0.0021 and scores 96.6%.
|
||||||
|
|
||||||
|
A canonical-frame sharpness scalar cannot separate *attenuated* high
|
||||||
|
frequencies from *destroyed* spatial sampling. Blur suppresses the high band
|
||||||
|
while preserving mid-frequency facial geometry exactly; downsampling to 16 px
|
||||||
|
destroys that geometry outright. Both look alike to any measure keyed on
|
||||||
|
high-frequency energy.
|
||||||
|
|
||||||
|
This is the measured basis for AR-028's rule that the axes are **kept separate
|
||||||
|
and not collapsed into one scalar**, and it settles the double-counting
|
||||||
|
question: size and sharpness are not redundant, and neither substitutes for the
|
||||||
|
other.
|
||||||
|
|
||||||
|
## Result 2 — blur is a cliff, and it breaks confidence, not identity
|
||||||
|
|
||||||
|
TPI % by size (rows) against Gaussian sigma (columns):
|
||||||
|
|
||||||
|
| size | 0 | 0.5 | 1 | 1.5 | 2 | 3 |
|
||||||
|
|---|---|---|---|---|---|---|
|
||||||
|
| 16 | 23.5 | 24.0 | 25.0 | 24.6 | 23.9 | 15.3 |
|
||||||
|
| 24 | 85.7 | 85.1 | 85.6 | 85.9 | 82.6 | 63.2 |
|
||||||
|
| 32 | 95.9 | 95.9 | 96.0 | 95.5 | 93.7 | 79.4 |
|
||||||
|
| 48 | 98.7 | 98.7 | 98.4 | 98.1 | 96.6 | 86.6 |
|
||||||
|
| 64 | 98.6 | 98.6 | 98.8 | 98.4 | 97.5 | 88.4 |
|
||||||
|
| 112 | 98.9 | 98.9 | 98.8 | 98.6 | 98.0 | 91.0 |
|
||||||
|
|
||||||
|
Three regimes: **sigma ≤ 1.5 is free** (every cell moves under 1.5 points, sign
|
||||||
|
flipping at random — at 16 px it slightly *improves*, smoothing upscale
|
||||||
|
artifacts); sigma 2 costs 1–3 points; the 2→3 step costs 7–19. A smooth
|
||||||
|
discount curve is therefore the wrong shape — the response is flat, then falls
|
||||||
|
off a cliff.
|
||||||
|
|
||||||
|
**The cost peaks at the size knee, not at full resolution.** Sigma 3 costs
|
||||||
|
−22.5 points at 24 px but only −7.9 at 112 px and −8.3 at 16 px. Blur has no
|
||||||
|
intrinsic cost; it costs in proportion to how close the observation already sits
|
||||||
|
to the decision boundary. At 112 px there is margin to spare, at 16 px the probe
|
||||||
|
is already below threshold, and at 24 px it sits exactly on the knee.
|
||||||
|
|
||||||
|
**What blur destroys is confidence, not ranking.** Rank-1 barely moves: 99.3% →
|
||||||
|
99.2% at 112 px across the whole sigma range. The extreme case is 16 px at sigma
|
||||||
|
3, where rank-1 is **80.2%** while TPI is **15.3%** — 65 points of probes have
|
||||||
|
the correct actor ranked first and are rejected anyway for falling under the
|
||||||
|
probability threshold.
|
||||||
|
|
||||||
|
That is why **FPI never left 0.1% in any of the 108 cells across all three
|
||||||
|
families**. Degradation produces TBI, never a wrong name. The calibration
|
||||||
|
degrades gracefully, which is what SR-002 needs.
|
||||||
|
|
||||||
|
## Result 3 — the blur *family* matters more than the blur *amount*
|
||||||
|
|
||||||
|
Comparing families by their raw parameter is meaningless — sigma, radius and
|
||||||
|
length are different units. They are matched here by the **per-axis standard
|
||||||
|
deviation of the PSF**, which puts them on one scale:
|
||||||
|
|
||||||
|
| family | per-axis σ | level giving σ = 3 px |
|
||||||
|
|---|---|---|
|
||||||
|
| Gaussian σ | σ | 3 |
|
||||||
|
| Disc radius R | R/2 | 6 |
|
||||||
|
| Motion length L | L/√12 | 10.4 |
|
||||||
|
|
||||||
|
For reference the ArcFace template places the eyes 35.2 canonical px apart, so
|
||||||
|
σ = 3 px is 9% of the inter-ocular distance.
|
||||||
|
|
||||||
|
TPI at matched severity, interpolated within each family:
|
||||||
|
|
||||||
|
| size | σ=3 Gaussian | σ=3 Motion | σ=3 **Defocus** | defocus penalty |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| 16 | 15.3 | 15.1 | 11.0 | +4.3 |
|
||||||
|
| 24 | 63.2 | 61.1 | 41.4 | +21.9 |
|
||||||
|
| 32 | 79.4 | 76.1 | 50.4 | +29.0 |
|
||||||
|
| 48 | 86.6 | 80.9 | 52.6 | +34.0 |
|
||||||
|
| 64 | 88.4 | 81.7 | 51.0 | +37.4 |
|
||||||
|
| 112 | 91.0 | 82.1 | **46.9** | **+44.0** |
|
||||||
|
|
||||||
|
**Optical defocus is up to 44 points more destructive than a Gaussian of
|
||||||
|
identical spread**, and the ordering is defocus ≫ motion > Gaussian throughout.
|
||||||
|
At σ=1 the three families are indistinguishable, and at σ=2 they differ by under
|
||||||
|
5 points; the divergence appears only when both the blur is severe *and* the face
|
||||||
|
is large.
|
||||||
|
|
||||||
|
That pattern is physically consistent. At 16 px the resampling has already
|
||||||
|
removed the high frequencies, so the PSF's shape has nothing left to act on and
|
||||||
|
all three agree. At 112 px the full spectrum is present and shape decides: a
|
||||||
|
Gaussian MTF rolls off gently and always leaves *some* energy at every
|
||||||
|
frequency, so the embedder receives a merely attenuated signal, while a disc MTF
|
||||||
|
is a jinc that **hits exact zeros** — whole frequency bands annihilated rather
|
||||||
|
than attenuated, with the bands beyond each zero returning phase-reversed.
|
||||||
|
Motion sits between them because it ruins one axis and leaves the perpendicular
|
||||||
|
one untouched.
|
||||||
|
|
||||||
|
**The methodological consequence is the important one.** This study originally
|
||||||
|
swept Gaussian blur alone and concluded blur was a minor effect. On the family
|
||||||
|
that actually occurs in film, the same nominal severity costs **53% error
|
||||||
|
instead of 9%** at full resolution. A threshold set from the Gaussian arm would
|
||||||
|
have been wrong by a factor of five in error rate, and the axis would probably
|
||||||
|
have been dropped as not worth its cost.
|
||||||
|
|
||||||
|
**Defocus is also the case a size gate cannot catch.** Every one of those 112 px
|
||||||
|
faces is large and confidently detected, and sails through AR-002 untouched.
|
||||||
|
That, not the Gaussian result, is what justifies a sharpness axis existing at
|
||||||
|
all.
|
||||||
|
|
||||||
|
## Result 4 — variance of Laplacian is anti-predictive at fixed degradation
|
||||||
|
|
||||||
|
Pooled across all cells, every candidate scores AUC 0.76–0.80 for predicting
|
||||||
|
correct identification, with textbook `var_laplacian` top. That number is close
|
||||||
|
to worthless: it rewards a measure for detecting *how degraded the crop is*,
|
||||||
|
which all five do. The question a per-observation discount needs is whether, at
|
||||||
|
a **fixed** degradation, the measure predicts which faces fail:
|
||||||
|
|
||||||
|
| measure | Gaussian | Defocus | Motion |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `hf_energy_ratio` | **0.530** | **0.521** | **0.557** |
|
||||||
|
| `norm_var_laplacian` | 0.520 | 0.507 | 0.539 |
|
||||||
|
| `dir_min_tenengrad` | 0.524 | 0.512 | 0.506 |
|
||||||
|
| `tenengrad` | 0.433 | 0.437 | 0.457 |
|
||||||
|
| `var_laplacian` | 0.423 | 0.422 | 0.473 |
|
||||||
|
|
||||||
|
Best is 0.557 — barely above chance, and `hf_energy_ratio` wins on all three
|
||||||
|
families. `var_laplacian` is anti-predictive on all three too, so that finding
|
||||||
|
does not depend on the blur model.
|
||||||
|
|
||||||
|
**The two metrics measure different jobs, and the candidates split along that
|
||||||
|
line.** On the motion arm `dir_min_tenengrad` has the best *pooled* AUC by a
|
||||||
|
wide margin — **0.854** against 0.792 for the next — exactly as its synthetic
|
||||||
|
directional-blur ladder predicted, yet its within-cell AUC there is 0.506. It is
|
||||||
|
an excellent detector of *how badly smeared a crop is* and no guide at all to
|
||||||
|
*which face will be recognised*. Pooled AUC is the right metric for a
|
||||||
|
gross-degradation flag; within-cell AUC is the right one for a per-observation
|
||||||
|
discount; a measure can be strong at one and useless at the other.
|
||||||
|
|
||||||
|
Deciles within the 16 px Gaussian cell, where 1277 failures give the test real
|
||||||
|
power:
|
||||||
|
|
||||||
|
| `var_laplacian` decile | TPI |
|
||||||
|
|---|---|
|
||||||
|
| 0.00071–0.00192 (blurriest) | **37.1%** |
|
||||||
|
| 0.00242–0.00278 | 22.8% |
|
||||||
|
| 0.00397–0.00447 | 25.7% |
|
||||||
|
| 0.00625–0.01445 (sharpest) | **14.4%** |
|
||||||
|
|
||||||
|
The faces the measure calls sharpest are **2.6x less identifiable** than those
|
||||||
|
it calls blurriest, monotone across ten bins of 167. Within a cell every crop
|
||||||
|
received identical degradation, so the residual variance is *native contrast*,
|
||||||
|
not native detail — and hard shadows, high-contrast lighting, sharpening halos
|
||||||
|
and JPEG ringing all raise Laplacian variance while making a face harder to
|
||||||
|
match. The measure reads photographic style and encoding artifacts and calls
|
||||||
|
them sharpness.
|
||||||
|
|
||||||
|
`hf_energy_ratio` is the only candidate with a correctly-signed within-cell
|
||||||
|
trend (16.2% → 35.3% across the same deciles), being a pure ratio in which the
|
||||||
|
contrast factor cancels.
|
||||||
|
|
||||||
|
**Consequence:** a per-face quality *discount* keyed on variance of Laplacian —
|
||||||
|
the most widely used blur metric in production vision pipelines — would
|
||||||
|
systematically down-weight the *more* identifiable faces. It is worse than no
|
||||||
|
discount.
|
||||||
|
|
||||||
|
## Result 5 — as a compute gate, sharpness loses to the size filter
|
||||||
|
|
||||||
|
Skipping the embed for crops below a threshold, measured as compute saved
|
||||||
|
against true identifications lost:
|
||||||
|
|
||||||
|
| gate | skipped | true IDs lost | of skipped, doomed anyway |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `hf_energy_ratio` < 0.00023 | 10.0% | 7.6% | 37.9% |
|
||||||
|
| `hf_energy_ratio` < 0.00051 | 20.0% | 15.1% | 38.7% |
|
||||||
|
| **source size < 24 px** | **16.7%** | **4.7%** | **77.3%** |
|
||||||
|
|
||||||
|
At a comparable skip rate the size filter loses **4.7% against sharpness's
|
||||||
|
15.1%** — three times less damage — and it is free, being a bbox dimension
|
||||||
|
available before alignment or embedding, where sharpness needs the warped crop
|
||||||
|
plus a colour convert, three convolutions and a 64×64 DFT.
|
||||||
|
|
||||||
|
Restricting to large faces (≥64 px) on the **defocus** arm, where the size
|
||||||
|
filter is blind, improves the gate's precision 3.5x (37% of skipped crops doomed
|
||||||
|
versus 10.7% on the Gaussian arm) but not its trade: skip 10%, lose 7.0%.
|
||||||
|
|
||||||
|
A hard ceiling explains why. **At 112 px with defocus radius 6 — visually
|
||||||
|
destroyed — 46.9% of faces still identify correctly and rank-1 is still 94.8%.**
|
||||||
|
Blur does not determine the outcome, so any gate keyed on apparent blur is
|
||||||
|
predicting a coin flip. The size filter wins not because size is better
|
||||||
|
measured, but because *smallness destroys identity more completely than blur
|
||||||
|
does*: 16 px faces succeed only 23.5% of the time, so discarding them is cheap.
|
||||||
|
|
||||||
|
## What this means for the requirements
|
||||||
|
|
||||||
|
**Do not gate on sharpness; discount on it.** Heavily defocused faces remain
|
||||||
|
~47% identifiable, so a gate destroys recoverable evidence. This is the first
|
||||||
|
hard evidence that AR-028's "**discounts the observation, never deletes the
|
||||||
|
detection**" is right on the merits rather than merely cautious. Since ranking
|
||||||
|
survives where confidence does not, the per-track accumulation (AR-025) should
|
||||||
|
recover much of what a single-frame threshold rejects — which is also the
|
||||||
|
argument for the discount living in `EvidenceDiscounter` rather than in a filter.
|
||||||
|
|
||||||
|
**`var_laplacian` and `tenengrad` are disqualified as discounts** by Result 4,
|
||||||
|
on all three blur families. They remain usable as coarse *gross-degradation*
|
||||||
|
detectors, the role in which their pooled AUC is real — the same role the size
|
||||||
|
filter plays — but they must never weight a per-observation belief.
|
||||||
|
|
||||||
|
**`hf_energy_ratio` is the only surviving discount candidate**, best on all
|
||||||
|
three families, and its within-cell signal (0.52–0.56) is weak enough that
|
||||||
|
shipping a discount on it needs justification beyond this study.
|
||||||
|
|
||||||
|
**`dir_min_tenengrad` earns a different job.** Its pooled 0.854 on the motion arm
|
||||||
|
makes it the best available detector of gross directional smear — useful as a
|
||||||
|
per-frame "this shot is unusable" flag, which is a decision about a *frame*, not
|
||||||
|
a weighting of an *observation*. If AR-029 ships two measures for two roles, this
|
||||||
|
is the second one, and it must not be confused with the first.
|
||||||
|
|
||||||
|
**Model the blur family, not just its amount.** Result 3 makes the choice of
|
||||||
|
degradation model a first-order design decision rather than a detail: the same
|
||||||
|
matched severity costs 9% or 53% error depending on the PSF. Any future study
|
||||||
|
that sweeps blur must state which family it used and why.
|
||||||
|
|
||||||
|
**Any discount curve must be flat then steep**, not linear or sigmoid over the
|
||||||
|
measure. Blur costs nothing until it costs a great deal.
|
||||||
|
|
||||||
|
## Limitations
|
||||||
|
|
||||||
|
- **Cooperative population.** Gallery mugshots are frontal and well-lit;
|
||||||
|
within-cell failures are likely dominated by cross-view mismatch, which no
|
||||||
|
sharpness measure can predict. Read the ~chance within-cell AUCs as "sharpness
|
||||||
|
does not predict the dominant failure mode *here*", not as "sharpness is
|
||||||
|
meaningless".
|
||||||
|
- **Uniform grid, not a natural distribution.** Sizes and blur levels are
|
||||||
|
sampled evenly, so "skip 16.7%" is exactly the 16 px row. The gate comparisons
|
||||||
|
are like-for-like on identical records, but the absolute savings are not what
|
||||||
|
a film would show.
|
||||||
|
- **TensorRT fp16.** A different realisation of the embedder from the fp32 ONNX
|
||||||
|
reference — VR-005 measured ~0.85 cosine agreement with separation intact.
|
||||||
|
Gallery and probes share one session so the study is internally consistent,
|
||||||
|
but the absolute knee belongs to the fp16 space.
|
||||||
|
- **Blur is applied in the canonical frame**, after resampling, so its width is
|
||||||
|
independent of the cell's size. Real optics blur before sampling.
|
||||||
|
- **The top motion rung is an anchor, not an operating point.** Length 21 is a
|
||||||
|
per-axis σ of 6.1 — 17% of the inter-ocular distance, a streak rather than a
|
||||||
|
face — and it is swept to bound the curve, not because a frame like that is
|
||||||
|
worth reasoning about. Its 3.4% TPI at 112 px should not be quoted as a
|
||||||
|
headline. The same caution applies less severely to defocus radius 6 (σ = 3).
|
||||||
|
- **Per-axis σ equates spread, not perceptual damage.** It is the fairest single
|
||||||
|
scalar for comparing PSFs, but Result 3 is precisely the finding that equal
|
||||||
|
spread does *not* mean equal harm, so the matched-severity tables compare
|
||||||
|
like-for-like inputs, not like-for-like severity as a face would experience it.
|
||||||
@@ -56,7 +56,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
|
|||||||
| AR-026 | All similarity computed as GEMM, including annex and deferred pass | SR-001 | High | In Progress |
|
| AR-026 | All similarity computed as GEMM, including annex and deferred pass | SR-001 | High | In Progress |
|
||||||
| AR-027 | Throughput acceptable for **arbitrary** gallery size | SR-001 | High | Planned |
|
| AR-027 | Throughput acceptable for **arbitrary** gallery size | SR-001 | High | Planned |
|
||||||
| AR-028 | **Embedding input quality assessed and carried** — every face scored on size, sharpness and visibility before its embedding is used as identity evidence; the vector travels with the face and reaches the VR-001 dump | SR-002 | High | Planned |
|
| AR-028 | **Embedding input quality assessed and carried** — every face scored on size, sharpness and visibility before its embedding is used as identity evidence; the vector travels with the face and reaches the VR-001 dump | SR-002 | High | Planned |
|
||||||
| AR-029 | Sharpness measure on the **aligned crop** (scale-normalised, so it cannot re-measure size) | SR-002 | Medium | Planned |
|
| AR-029 | Sharpness measure on the **aligned crop**, consumed as a discount and **never as a gate** | SR-002 | Medium | **In Progress** — five candidates implemented (`src/quality.hpp`) and ranked by VR-012 over three blur families. `var_laplacian` and `tenengrad` are **disqualified as discounts**: within a fixed degradation they are anti-predictive on *all three* families (AUC 0.42–0.47; the decile the measure calls sharpest is 2.6× *less* identifiable), since their residual variance is native contrast, not detail. `hf_energy_ratio` is the only correctly-signed survivor, best on all three, and weak (0.52–0.56). `dir_min_tenengrad` is the best *gross-smear detector* (pooled AUC 0.854 on motion) but ~chance within-cell, so it serves a per-frame flag, not a per-observation weight. The parenthetical this row used to carry — "scale-normalised, so it cannot re-measure size" — was wrong: every candidate responds to source size, and the axes are separable for a different reason (see AR-028) |
|
||||||
| AR-030 | Visibility measure from the AR-001 5-point landmarks — extreme pose or occlusion **discounts the observation, never deletes the detection** | SR-002 | Medium | **In Progress** — measure is the AR-005 alignment residual (`estimate_alignment()`), carried on `DetectedFace`; roll/scale invariance and monotonicity under foreshortening asserted. Nothing consumes it as a discount yet |
|
| AR-030 | Visibility measure from the AR-001 5-point landmarks — extreme pose or occlusion **discounts the observation, never deletes the detection** | SR-002 | Medium | **In Progress** — measure is the AR-005 alignment residual (`estimate_alignment()`), carried on `DetectedFace`; roll/scale invariance and monotonicity under foreshortening asserted. Nothing consumes it as a discount yet |
|
||||||
|
|
||||||
## Deployment (DP)
|
## Deployment (DP)
|
||||||
@@ -114,7 +114,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
|
|||||||
| VR-009 | Verify accumulated posteriors are calibrated against held-out tracks | PR-002 | High | Planned |
|
| VR-009 | Verify accumulated posteriors are calibrated against held-out tracks | PR-002 | High | Planned |
|
||||||
| VR-010 | Dump provenance attributes — embedder model, detector settings, `dense_scale`, `scene_detect`, sample rate | PR-002 | **High** | Planned |
|
| VR-010 | Dump provenance attributes — embedder model, detector settings, `dense_scale`, `scene_detect`, sample rate | PR-002 | **High** | Planned |
|
||||||
| VR-011 | Rewrite the replay harness for the post-AR-012 output contract | PR-002 | High | Planned |
|
| VR-011 | Rewrite the replay harness for the post-AR-012 output contract | PR-002 | High | Planned |
|
||||||
| VR-012 | Quality-knee study — TPI/FPI vs sharpness and vs pose, as VR-005 did for size; also settles whether the 5-point pose proxy needs a dedicated landmark model | PR-002 | Medium | Planned |
|
| VR-012 | Quality-knee study — TPI/FPI vs sharpness and vs pose, as VR-005 did for size; also settles whether the 5-point pose proxy needs a dedicated landmark model | PR-002 | Medium | **In Progress** — sharpness half done ([`docs/quality-knee.md`](quality-knee.md)): 1670 actors, joint size×blur grid over three blur families (Gaussian, disc defocus, linear motion), 60120 probe-cell records each. Sharpness is **not a sufficient statistic** (equal measured sharpness spans 15.3–91.0% TPI, ordered by source size); **the blur family matters more than its amount** — at matched per-axis σ=3 on a 112 px face, Gaussian/motion/defocus cost 9/18/**53**% error, so a Gaussian-only sweep understates real lens blur fivefold; blur breaks **confidence, not ranking** (rank-1 80.2% where TPI is 15.3%), so FPI never left 0.1% in any of the 108 cells; a sharpness **gate** loses 3× more true presence than the free size filter at equal saving, because even destroyed faces stay 46.9% identifiable. **Pose half not started** — the AR-030 residual is exposed via `sae_embed.alignment_residual` but no pose arm has been run, so the dedicated-landmark-model question is still open |
|
||||||
| VR-014 | Audio-signature **offset recovery on real content** — a known trim recovered from film audio, not from the synthetic golden tone | PR-002 | Medium | **Done** — 40 random in-cap offsets, every one recovered to the nearest frame: **worst error 46 ms against a 500 ms budget**, and 46 ms is the floor rather than a result, since the offset is quantised to whole 92.88 ms frames. The `runtime/2` anchor confirmed through real head-trimmed files (a `delta` trim moves the window by `delta/2`). The one soft spot is **tier labelling, not accuracy**: the score falls with sub-frame misalignment (0.94–0.99 near a frame boundary, 0.69–0.73 at half a frame), so 27/40 correct alignments were demoted to `loose`. ±1 frame of slack in the *score* fixes it — measured, all 40 back to `audio` (min 0.906), false matches unmoved at 0.12–0.16, costing 81 ms of the budget |
|
| VR-014 | Audio-signature **offset recovery on real content** — a known trim recovered from film audio, not from the synthetic golden tone | PR-002 | Medium | **Done** — 40 random in-cap offsets, every one recovered to the nearest frame: **worst error 46 ms against a 500 ms budget**, and 46 ms is the floor rather than a result, since the offset is quantised to whole 92.88 ms frames. The `runtime/2` anchor confirmed through real head-trimmed files (a `delta` trim moves the window by `delta/2`). The one soft spot is **tier labelling, not accuracy**: the score falls with sub-frame misalignment (0.94–0.99 near a frame boundary, 0.69–0.73 at half a frame), so 27/40 correct alignments were demoted to `loose`. ±1 frame of slack in the *score* fixes it — measured, all 40 back to `audio` (min 0.906), false matches unmoved at 0.12–0.16, costing 81 ms of the budget |
|
||||||
| VR-013 | Cross-source identification probe — gallery from one recording, probes from another, swept over input resolution end to end | PR-002 | Medium | **In Progress** — holding 90% of the plateau needs ~50 px end to end against VR-005's ~22 px, the gap being detection and landmark error; **`min_face_px` 40, since 32 admits faces in the falling region** (AR-002). FPI 0.0% at every scale. Ceiling is cross-view, not resolution |
|
| VR-013 | Cross-source identification probe — gallery from one recording, probes from another, swept over input resolution end to end | PR-002 | Medium | **In Progress** — holding 90% of the plateau needs ~50 px end to end against VR-005's ~22 px, the gap being detection and landmark error; **`min_face_px` 40, since 32 admits faces in the falling region** (AR-002). FPI 0.0% at every scale. Ceiling is cross-view, not resolution |
|
||||||
|
|
||||||
|
|||||||
@@ -39,6 +39,7 @@ nav:
|
|||||||
- Best Model: best-model.md
|
- Best Model: best-model.md
|
||||||
- Gallery Scope (Full vs. Limited): gallery-scope.md
|
- Gallery Scope (Full vs. Limited): gallery-scope.md
|
||||||
- Pose Expansion: pose-expansion.md
|
- Pose Expansion: pose-expansion.md
|
||||||
|
- Quality Knee (Blur and Size): quality-knee.md
|
||||||
- LVFace Deep Dive: lvface-deep-dive.md
|
- LVFace Deep Dive: lvface-deep-dive.md
|
||||||
- Full Experiment Log: model-bakeoff.md
|
- Full Experiment Log: model-bakeoff.md
|
||||||
- Service Conversion (proposal): service-conversion.md
|
- Service Conversion (proposal): service-conversion.md
|
||||||
|
|||||||
Reference in New Issue
Block a user