docs(VR-012): the quality knee, and what it retires
Sharpness is not a sufficient statistic for identity loss. Six cells at effectively identical measured sharpness span 15.3% to 91.0% TPI, ordered entirely by source size, because a scalar keyed on high-frequency energy cannot separate attenuated high frequencies from destroyed spatial sampling. AR-028's "kept separate, not collapsed into one scalar" now rests on a measurement rather than an argument -- and the reasoning it used to rest on, that the aligned crop is scale-normalised so a measure there cannot re-measure size, was wrong and is corrected in place. Variance of Laplacian -- the most widely used blur metric there is -- is anti-predictive at fixed degradation on all three blur families. The decile it calls sharpest is 2.6x less identifiable than the decile it calls blurriest, monotone across ten bins, because within a cell its residual variance is native contrast rather than detail, and hard shadows and JPEG ringing raise it while making a face harder to match. Gating on it would preferentially discard the more identifiable faces. Blur breaks confidence, not identity: rank-1 holds at 80.2% where TPI is 15.3%, and FPI never left 0.1% in any of the 108 cells. Degradation produces abstention, never a wrong name. That is also why sharpness fails as a compute gate -- even a visually destroyed face stays 46.9% identifiable, so a gate discards recoverable evidence at three times the cost of the free size filter. Discount, do not gate; the rule AR-028 already stated now has evidence for why it is right rather than merely cautious. Records the shape a discount must have (flat, then a cliff between sigma 2 and 3), that its cost scales with proximity to the decision boundary rather than with blur, and that the pose half of VR-012 has not been run. TRACES: VR-012, AR-028, AR-029, AR-030 | SR-002
This commit is contained in:
@@ -56,7 +56,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
|
||||
| AR-026 | All similarity computed as GEMM, including annex and deferred pass | SR-001 | High | In Progress |
|
||||
| AR-027 | Throughput acceptable for **arbitrary** gallery size | SR-001 | High | Planned |
|
||||
| AR-028 | **Embedding input quality assessed and carried** — every face scored on size, sharpness and visibility before its embedding is used as identity evidence; the vector travels with the face and reaches the VR-001 dump | SR-002 | High | Planned |
|
||||
| AR-029 | Sharpness measure on the **aligned crop** (scale-normalised, so it cannot re-measure size) | SR-002 | Medium | Planned |
|
||||
| AR-029 | Sharpness measure on the **aligned crop**, consumed as a discount and **never as a gate** | SR-002 | Medium | **In Progress** — five candidates implemented (`src/quality.hpp`) and ranked by VR-012 over three blur families. `var_laplacian` and `tenengrad` are **disqualified as discounts**: within a fixed degradation they are anti-predictive on *all three* families (AUC 0.42–0.47; the decile the measure calls sharpest is 2.6× *less* identifiable), since their residual variance is native contrast, not detail. `hf_energy_ratio` is the only correctly-signed survivor, best on all three, and weak (0.52–0.56). `dir_min_tenengrad` is the best *gross-smear detector* (pooled AUC 0.854 on motion) but ~chance within-cell, so it serves a per-frame flag, not a per-observation weight. The parenthetical this row used to carry — "scale-normalised, so it cannot re-measure size" — was wrong: every candidate responds to source size, and the axes are separable for a different reason (see AR-028) |
|
||||
| AR-030 | Visibility measure from the AR-001 5-point landmarks — extreme pose or occlusion **discounts the observation, never deletes the detection** | SR-002 | Medium | **In Progress** — measure is the AR-005 alignment residual (`estimate_alignment()`), carried on `DetectedFace`; roll/scale invariance and monotonicity under foreshortening asserted. Nothing consumes it as a discount yet |
|
||||
|
||||
## Deployment (DP)
|
||||
@@ -114,7 +114,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
|
||||
| VR-009 | Verify accumulated posteriors are calibrated against held-out tracks | PR-002 | High | Planned |
|
||||
| VR-010 | Dump provenance attributes — embedder model, detector settings, `dense_scale`, `scene_detect`, sample rate | PR-002 | **High** | Planned |
|
||||
| VR-011 | Rewrite the replay harness for the post-AR-012 output contract | PR-002 | High | Planned |
|
||||
| VR-012 | Quality-knee study — TPI/FPI vs sharpness and vs pose, as VR-005 did for size; also settles whether the 5-point pose proxy needs a dedicated landmark model | PR-002 | Medium | Planned |
|
||||
| VR-012 | Quality-knee study — TPI/FPI vs sharpness and vs pose, as VR-005 did for size; also settles whether the 5-point pose proxy needs a dedicated landmark model | PR-002 | Medium | **In Progress** — sharpness half done ([`docs/quality-knee.md`](quality-knee.md)): 1670 actors, joint size×blur grid over three blur families (Gaussian, disc defocus, linear motion), 60120 probe-cell records each. Sharpness is **not a sufficient statistic** (equal measured sharpness spans 15.3–91.0% TPI, ordered by source size); **the blur family matters more than its amount** — at matched per-axis σ=3 on a 112 px face, Gaussian/motion/defocus cost 9/18/**53**% error, so a Gaussian-only sweep understates real lens blur fivefold; blur breaks **confidence, not ranking** (rank-1 80.2% where TPI is 15.3%), so FPI never left 0.1% in any of the 108 cells; a sharpness **gate** loses 3× more true presence than the free size filter at equal saving, because even destroyed faces stay 46.9% identifiable. **Pose half not started** — the AR-030 residual is exposed via `sae_embed.alignment_residual` but no pose arm has been run, so the dedicated-landmark-model question is still open |
|
||||
| VR-014 | Audio-signature **offset recovery on real content** — a known trim recovered from film audio, not from the synthetic golden tone | PR-002 | Medium | **Done** — 40 random in-cap offsets, every one recovered to the nearest frame: **worst error 46 ms against a 500 ms budget**, and 46 ms is the floor rather than a result, since the offset is quantised to whole 92.88 ms frames. The `runtime/2` anchor confirmed through real head-trimmed files (a `delta` trim moves the window by `delta/2`). The one soft spot is **tier labelling, not accuracy**: the score falls with sub-frame misalignment (0.94–0.99 near a frame boundary, 0.69–0.73 at half a frame), so 27/40 correct alignments were demoted to `loose`. ±1 frame of slack in the *score* fixes it — measured, all 40 back to `audio` (min 0.906), false matches unmoved at 0.12–0.16, costing 81 ms of the budget |
|
||||
| VR-013 | Cross-source identification probe — gallery from one recording, probes from another, swept over input resolution end to end | PR-002 | Medium | **In Progress** — holding 90% of the plateau needs ~50 px end to end against VR-005's ~22 px, the gap being detection and landmark error; **`min_face_px` 40, since 32 admits faces in the falling region** (AR-002). FPI 0.0% at every scale. Ceiling is cross-view, not resolution |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user