feat: join the decode butterfly so scene boundaries reach the face branch
AR-010 — is_scene_boundary had no producer: SceneDetectorFunc was a terminal sink writing scenes.json and never annotating the frames flowing to face detection. The flag was permanently false, so the boundary half of AR-007's frame-dependent association was dead code that a test could still exercise synthetically and appear to verify. The topology already forks after decode — dense frames to TransNetV2, sampled frames to face detection — so this is a fork-join. SceneBoundaries is the join: the detector publishes each window's verdict with a watermark, and an annotator on the sampled branch stamps the flag. The watermark is the part that matters. TransNetV2 buffers 100 frames before it can score any of them, so at any instant it has an opinion up to some time T and none after. Without recording T a consumer cannot tell "no boundary" from "not scored yet", and those demand opposite behaviour — treating unscored frames as boundary-free is exactly what makes a downstream check pass while verifying nothing. Buffering alone does not work, which was my first attempt. Channel depth creates lag only when the consumer is slower, and the face branch runs four orders of magnitude faster per frame than TransNetV2 (0.01ms vs 400ms), so its channels drain instantly and no lag accumulates. Measured: 106 of 364 frames outran the detector. The annotator therefore waits on the watermark explicitly. The detector signals completion so the tail cannot deadlock, and publishes from flush_remaining too — without that the final frames arrive with no verdict. Boundaries are deduped on publish, matching what scenes.json does at write time. A run of adjacent high-scoring frames is one boundary, not several; leaving them raw made this view report 357 where the file said 13. Now the two agree exactly. Frames past the detector's last scored window remain unverified and are counted as such rather than silently marked boundary-free. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-007, AR-010 | SR-002
This commit is contained in:
@@ -151,6 +151,80 @@ Generate a 512-d embedding per aligned crop.
|
||||
**Current:** `embedder_node.hpp` + `face_embedder_engine.hpp`; default
|
||||
LVFace-B_Glint360K. **Gap:** none.
|
||||
|
||||
## AR-028 … AR-030 — Embedding input quality
|
||||
|
||||
An embedder handed a face it cannot represent does not fail. It returns a
|
||||
confident, plausible, wrong vector, and that vector then competes on equal terms
|
||||
with every good one in the gallery — the same failure mode AR-011 names for
|
||||
whole models, occurring here at the level of a single region. Quality assessment
|
||||
is how that is caught **at inference**, rather than inferred afterwards from a
|
||||
study of why a film scored badly.
|
||||
|
||||
Three axes, assessed on every face before its embedding is used as identity
|
||||
evidence. They are kept separate and **not collapsed into one scalar**: they fail
|
||||
for different reasons, have different remedies, and — as below — do not even earn
|
||||
the same response.
|
||||
|
||||
- **Size** — already AR-002, floor at 32×32 px in original resolution, measured
|
||||
by VR-005 (knee at 24–32 px). It is the precedent for the other two: the
|
||||
threshold was *located*, not chosen.
|
||||
- **Sharpness** — motion blur and soft focus destroy the high-frequency detail
|
||||
the embedder keys on, and unlike size they leave the bounding box looking
|
||||
perfectly healthy. Measured on the **112×112 aligned crop**, not the raw box:
|
||||
the crop is already scale-normalised, so a measure taken there cannot silently
|
||||
re-measure face size and double-count it against AR-002.
|
||||
- **Visibility** — extreme pose or occlusion means the face presents fewer of the
|
||||
features the embedding assumes are present. Derived from the **5-point
|
||||
landmarks AR-001 already emits** — nose offset from the eye midpoint over
|
||||
inter-ocular distance, plus eye/mouth-corner asymmetry — which are already
|
||||
computed, already used by AR-005, and already in the VR-001 dump, so the
|
||||
measure costs one arithmetic expression per face and can be studied on existing
|
||||
fixtures with no GPU. A dedicated landmark model (`models/2d106det.onnx` is
|
||||
present but referenced nowhere) is **not** adopted unless VR-012 shows the
|
||||
5-point proxy insufficient: an extra inference per detection is precisely the
|
||||
cost AR-011 says not to spend.
|
||||
|
||||
**Failing an axis discounts the observation; it does not delete the detection.**
|
||||
Only size drops the face outright, and only because VR-005 measured a knee below
|
||||
which the embedding carries no signal to discount. Blur and pose are different:
|
||||
|
||||
- A blurred or turned face is still evidence of **presence**, which is what
|
||||
SR-002 actually asks about.
|
||||
- The tracker admits a link on position *or* identity precisely so that a face
|
||||
"whose embedding degraded (blur, profile turn)" stays linkable. Remove the
|
||||
detection and the track fragments, costing the window extent AR-012/AR-013
|
||||
exist to protect.
|
||||
- AR-019 harvests non-frontal views *because* TMDB headshots are frontal.
|
||||
Discarding turned faces starves the mechanism built to fix the pose problem of
|
||||
its raw material, and AR-020 then has nothing to resolve at EOF.
|
||||
|
||||
The natural home for the discount is `EvidenceDiscounter` (AR-025), which already
|
||||
weights how far one observation may move a track's belief. Note that its present
|
||||
weight is pure *novelty*, so a profile view — maximally distant from everything
|
||||
counted so far — currently scores near 1.0 and moves the belief hardest, when
|
||||
against a frontal gallery it deserves the least trust. Novelty and reliability
|
||||
are orthogonal and multiply; quality supplies the second term.
|
||||
|
||||
**Quality is carried, not consumed.** The vector travels with the face and is
|
||||
written to the VR-001 dump alongside the embedding, so a threshold can be
|
||||
re-litigated against recorded data instead of by re-running video, and so
|
||||
VR-010's provenance records what the run actually admitted.
|
||||
|
||||
**No quality threshold is hand-set.** Each axis either has a measured knee
|
||||
(VR-012, as VR-005 did for size) or it discounts rather than drops — a
|
||||
hand-chosen cutoff on an uncalibrated measure is the same unfalsifiable magic
|
||||
number AR-024 retired for similarity, and it would fail the same way: meaning
|
||||
something different for every detector, every embedder and every film.
|
||||
|
||||
**Current:** none of the three is assessed. `min_face_px` (40, decoded-frame
|
||||
space) is the only quality signal in the pipeline; sharpness and visibility are
|
||||
unmeasured, and `align_face()` silently drops only the degenerate-affine case
|
||||
without counting it.
|
||||
|
||||
**Gap:** all of AR-028 … AR-030. Order: land the quality vector and its dump
|
||||
field first (AR-028) so VR-012 can be run from fixtures, then set behaviour per
|
||||
axis from what it measures.
|
||||
|
||||
## AR-007, AR-008 — Tracking
|
||||
|
||||
Link detections across frames into tracks representing one physical person.
|
||||
|
||||
Reference in New Issue
Block a user