diff --git a/docs/SPEC.md b/docs/SPEC.md index c1f9358..b692d28 100644 --- a/docs/SPEC.md +++ b/docs/SPEC.md @@ -133,9 +133,30 @@ Produce the exact input ArcFace expects. nose, left mouth, right mouth). - Alignment is the *only* geometric normalisation; no additional augmentation at inference. +- **The transform is fitted by Umeyama's closed-form least squares over all five + points**, which is what InsightFace uses (skimage's `SimilarityTransform` *is* + `_umeyama`) and therefore what produced the crops ArcFace and LVFace were + trained on. The canonical warp is part of the input distribution, not an + implementation detail (AR-011). +- **Not a robust estimator.** A RANSAC fit buys a small residual by discarding + the landmarks that disagree with the model, and on a turned face those are the + foreshortened ones — the signal AR-030 reads. With five points and a two-point + minimal sample it also cannot separate a mis-detected landmark from honest + out-of-plane rotation, so the robustness is nominal while the cost to AR-030 is + total. It is RNG-driven besides, which made replay determinism a property of + thread scheduling. -**Current:** `align_face()` in `src/face_utils.hpp:9-22`, `cv::warpAffine` to -`{112, 112}`. **Gap:** none. +**Current:** `align_face()` in `src/face_utils.hpp`, Umeyama fit via +`umeyama_similarity()`, `cv::warpAffine` to `{112, 112}`. **Gap:** none. + +> **Migration note.** Until this landed the fit was +> `cv::estimateAffinePartial2D(…, cv::RANSAC, 3.0)`. Where RANSAC kept all five +> points its final refit is the same least-squares optimum, so the two agree; +> they diverge exactly where a landmark fell outside the 3 px band — i.e. on the +> non-frontal faces. Galleries baked before this change therefore carry embeddings +> from a marginally different warp, concentrated on the hardest views. Rebuilding +> is cheap and removes the question; GR-004's embedder stamp does **not** catch an +> aligner change, only a model change. ## AR-006 — Embedding @@ -174,15 +195,43 @@ the same response. the crop is already scale-normalised, so a measure taken there cannot silently re-measure face size and double-count it against AR-002. - **Visibility** — extreme pose or occlusion means the face presents fewer of the - features the embedding assumes are present. Derived from the **5-point - landmarks AR-001 already emits** — nose offset from the eye midpoint over - inter-ocular distance, plus eye/mouth-corner asymmetry — which are already - computed, already used by AR-005, and already in the VR-001 dump, so the - measure costs one arithmetic expression per face and can be studied on existing - fixtures with no GPU. A dedicated landmark model (`models/2d106det.onnx` is - present but referenced nowhere) is **not** adopted unless VR-012 shows the - 5-point proxy insufficient: an extra inference per detection is precisely the - cost AR-011 says not to spend. + features the embedding assumes are present. The measure is the **residual of + the AR-005 alignment fit**: the RMS landmark error, in canonical 112×112 + pixels, left over after the best similarity transform onto the ArcFace + template. It costs nothing — the transform is computed for the warp regardless, + and the residual is what that fit could not explain. + + Two properties earn it the job over an explicit yaw estimate: + + - A similarity absorbs rotation, uniform scale and translation **exactly**, + so the residual is by construction the non-similarity part of the + deformation: out-of-plane rotation and foreshortening. In-plane roll + contributes nothing, so "a tilted head reads as a turned one" is excluded + structurally rather than by tuning. The destination frame is fixed, so face + size cannot leak in either — that is AR-002's axis, and double-counting it + would make a small frontal face look occluded. + - It responds to **occlusion** and to plainly broken landmark sets, which an + angle regressor by construction does not: a hand across the face is not a + rotation, but it does displace landmarks. + + Indicative magnitudes from a synthetic foreshortening sweep (`k ≈ cos yaw`): + `k=1.0 → 0.00`, `0.9 → 1.18`, `0.75 → 3.11`, `0.5 → 6.72`, `0.3 → 9.85` + canonical px. Smooth and monotone with a usable range; the mapping onto real + faces is VR-012's to establish, and no threshold is set from these numbers. + + Neither a dedicated landmark model (`models/2d106det.onnx` is present but + referenced nowhere — and it emits points, not pose) nor a direct pose CNN is + adopted unless VR-012 shows the residual insufficient. If one is needed the + candidate is **6DRepNet** (MIT, RepVGG-B1g2, 3.47° MAE on AFLW2000) rather than + Hopenet, which it dominates on accuracy, licence, recency and export + friendliness. Two caveats to record before that happens: both are trained on + **300W-LP**, which inherits research-only terms from 300W's constituent sets, + and both want their own loosely-framed ROI rather than the ArcFace crop — a + second warp and a second image in flight, which lands on AR-004's byte-based + backpressure gap. It would also have to run **per track** — over the bounded + view set AR-019's diversity buffer already keeps — not per face per frame, + which is the cost rule applied as written: fewer regions, never a degraded + input. **Failing an axis discounts the observation; it does not delete the detection.** Only size drops the face outright, and only because VR-005 measured a knee below @@ -216,14 +265,18 @@ hand-chosen cutoff on an uncalibrated measure is the same unfalsifiable magic number AR-024 retired for similarity, and it would fail the same way: meaning something different for every detector, every embedder and every film. -**Current:** none of the three is assessed. `min_face_px` (40, decoded-frame -space) is the only quality signal in the pipeline; sharpness and visibility are -unmeasured, and `align_face()` silently drops only the degenerate-affine case -without counting it. +**Current:** visibility is measured and carried — `estimate_alignment()` in +`src/face_utils.hpp` returns the residual alongside the transform, and +`FaceAlignerFunc` writes it to `DetectedFace::alignment_residual`. Size is +`min_face_px` (40, decoded-frame space — AR-002 still open). Sharpness is +unmeasured. Nothing yet *consumes* any of it: no discount is applied, and +`align_face()` still drops the degenerate-fit case without counting it. -**Gap:** all of AR-028 … AR-030. Order: land the quality vector and its dump -field first (AR-028) so VR-012 can be run from fixtures, then set behaviour per -axis from what it measures. +**Gap:** AR-029 entirely. For AR-030, the measure exists but the discount does +not — it must reach `EvidenceDiscounter` as the reliability term. For AR-028, the +residual does not yet reach the VR-001 dump, which is what VR-012 needs to run +from fixtures; that is the next step, since it unblocks the study that sets +every remaining behaviour. ## AR-007, AR-008 — Tracking diff --git a/docs/requirements.md b/docs/requirements.md index 7a9ffe8..aa7be67 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -32,7 +32,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn` | AR-002 | Minimum face size **32×32 px** (VR-005 measured), expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | Planned | | AR-003 | No fixed per-frame face cap — crowd scenes must not lose background cast | SR-002 | Medium | **Done** — `max_faces` defaults to 0 (no cap); the matcher batches through its GEMM buffer instead of throwing | | AR-004 | Backpressure: unbounded faces/frame absorbed by slowing, never by dropping or throwing | SR-002 | High | **Done** — KPN node outputs use `push_blocking`; sentinels stay out-of-band. Verified: 385/385 frames, 0 drops, byte-identical across runs | -| AR-005 | Align to 112×112 via ArcFace 5-point similarity transform | SR-002 | High | Done | +| AR-005 | Align to 112×112 via ArcFace 5-point similarity transform, fitted by **Umeyama least squares over all five points** (as InsightFace does) — never a robust fit, which would discard the landmarks AR-030 reads | SR-002 | High | **Done** — `umeyama_similarity()`; the RANSAC fit it replaces was RNG-driven and point-dropping. Galleries baked before the change want a rebuild | | AR-006 | 512-d L2-normalised embeddings, batched | SR-002 | High | Done | | AR-007 | Associate detections by IoU + embedding, with **frame-dependent** weighting | SR-002 | High | **Done** — `track_alpha` is the base for ordinary frames; drops to embedding-only on cut/boundary and for dormant tracks | | AR-008 | One track pool keyed on `last_seen`; no separate revival path | SR-002 | High | **Done** — one pool keyed on `last_seen`; park/revive branch deleted | @@ -45,7 +45,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn` | AR-015 | Two live tracks owned by one actor ⇒ treat as a detected cut, re-associate | SR-002 | Medium | **Done** — reverse index detects it on the causing update; counted | | AR-016 | All tracks closed at EOF — a film ends with faces on screen | SR-002 | High | **Done** — `flush()`, idempotent, closes at last sighting or final tick | | AR-017 | Every presence claim carries its belief and identification route | SR-002 | High | **Done** — `DeadTrack` carries belief and observation count | -| AR-018 | Per-subject embedding store with banded admission (novel enough, safe enough) | SR-005 | Medium | Planned | +| AR-018 | Per-subject embedding store with banded admission (novel enough, safe enough) | SR-005 | Medium | **Done** — banded admission in probability space; replaces `expand_novelty_sim`. Rejections counted | | AR-019 | Per-film gallery annex from owned tracks; acquires the non-frontal views TMDB lacks | SR-005 | Medium | In Progress | | AR-020 | Deferred re-identification of unknown tracks against the final expanded gallery | SR-005 | High | Planned | | AR-021 | Cluster unknown tracks into one entity per person, under temporal cannot-link constraints | SR-005 | Medium | Planned | @@ -57,7 +57,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn` | AR-027 | Throughput acceptable for **arbitrary** gallery size | SR-001 | High | Planned | | AR-028 | **Embedding input quality assessed and carried** — every face scored on size, sharpness and visibility before its embedding is used as identity evidence; the vector travels with the face and reaches the VR-001 dump | SR-002 | High | Planned | | AR-029 | Sharpness measure on the **aligned crop** (scale-normalised, so it cannot re-measure size) | SR-002 | Medium | Planned | -| AR-030 | Visibility measure from the AR-001 5-point landmarks — extreme pose or occlusion **discounts the observation, never deletes the detection** | SR-002 | Medium | Planned | +| AR-030 | Visibility measure from the AR-001 5-point landmarks — extreme pose or occlusion **discounts the observation, never deletes the detection** | SR-002 | Medium | **In Progress** — measure is the AR-005 alignment residual (`estimate_alignment()`), carried on `DetectedFace`; roll/scale invariance and monotonicity under foreshortening asserted. Nothing consumes it as a discount yet | ## Deployment (DP) @@ -309,7 +309,7 @@ because it will be trusted. | AR-002 | T2 | Faces below 32 px (original res) are dropped | Exactly at threshold; with `dense_scale` 0.5 — the interaction that motivated the requirement | | AR-003 | T2 | No cap applied; a 40-face frame yields 40 | Crowd frame | | AR-004 | T1 | Saturated input blocks rather than drops or throws | Bounded queue at capacity; **byte-based** limit with large crops; SIGTERM mid-block | -| AR-005 | T1 | Known landmarks → expected 112×112 warp | Landmarks near frame edge; degenerate/collinear points | +| AR-005 | T1 | Known landmarks → expected 112×112 warp; the fit never mirrors | Landmarks near frame edge; degenerate/collinear points; a mirrored set — SVD returns a reflection unless the determinant guard rejects it | | AR-006 | T3 | Embeddings are unit-norm | Batch smaller than, equal to, larger than `embed_batch_size` | | AR-007 | T2 | Association picks the right track | Two faces crossing paths; one leaving frame as another enters | | AR-008 | T2 | One pool; dormant tracks match on embedding, not IoU | Dormant track whose old bbox overlaps a *different* new face — must not match on position | @@ -333,7 +333,7 @@ because it will be trusted. | AR-027 | **T4** | Throughput at 10²…10⁵ actors | Scheduled, not on-demand | | AR-028 | **T2** | No embedding reaches the matcher unscored; the vector survives into the dump | Face failing exactly one axis; all three healthy; a face whose landmarks are degenerate — scored, not silently vanished | | AR-029 | T1 | Synthetic blur ladder → monotonically falling sharpness | Gaussian vs motion blur; **small sharp face vs large soft one** — size must not leak into this axis | -| AR-030 | T1 | Landmark geometry → expected yaw proxy on known poses | Full profile with one eye occluded; **in-plane roll must not read as yaw**; near-degenerate landmarks | +| AR-030 | T1 | Alignment residual rises monotonically with foreshortening | **In-plane roll, scale and translation must leave it at zero** — the property that makes it a pose measure rather than a pose-and-everything-else measure; face size must not shift it; degenerate landmarks report not-ok rather than a number | | VR-012 | **T4** | Knee located per axis on held-out films | Report each candidate threshold's cost in **lost true presence**, not only its gain in precision — a gate that improves misID by discarding half the cast has not helped | | IR-001/002 | T1 | Serialised output matches golden file | Zero-length window; actor with many windows | | IR-003 | T1 | Output written after deferred pass | Not at EOF | diff --git a/docs/traceability.md b/docs/traceability.md index 1922a7f..69f6d03 100644 --- a/docs/traceability.md +++ b/docs/traceability.md @@ -3,7 +3,7 @@ -**Generated:** 2026-07-31T12:29:10+00:00 +**Generated:** 2026-07-31T13:06:54+00:00 Denominators are read from [`requirements.md`](requirements.md) at run time, never hardcoded. Coverage counts a requirement only when it is tagged in source **and** has a verification tier this repo's CI host can execute (`T1, T2, T3, static`). @@ -11,21 +11,21 @@ Denominators are read from [`requirements.md`](requirements.md) at run time, nev | Metric | Value | |---|---| -| Source files scanned | 103 | -| TRACES tags found | 95 | +| Source files scanned | 108 | +| TRACES tags found | 105 | | EXCEPTION tags found | 0 | | Requirements defined | 67 | -| Requirements covered | 28 | -| **Coverage** | **41.8%** (28/67) | -| Coverage of CI-executable scope | 50.9% (28/55) | -| Tagged but unexecuted in CI | 3 | +| Requirements covered | 31 | +| **Coverage** | **46.3%** (31/67) | +| Coverage of CI-executable scope | 56.4% (31/55) | +| Tagged but unexecuted in CI | 4 | | Orphan tags | 0 | ### By type | Type | Covered | Tagged but unexecuted | Defined | |---|---|---|---| -| AR | 15 | 0 | 30 | +| AR | 18 | 1 | 30 | | DP | 2 | 0 | 8 | | IR | 8 | 0 | 8 | | GR | 3 | 0 | 9 | @@ -42,7 +42,7 @@ These requirements have no verification tier this repo's CI host can run, so a t | ID | Tiers | Tagged in source | Requirement | |---|---|---|---| -| AR-027 | T4 | no | Throughput acceptable for **arbitrary** gallery size | +| AR-027 | T4 | yes | Throughput acceptable for **arbitrary** gallery size | | VR-001 | out-of-ci | yes | HDF5 post-inference dump at the embedded-frame boundary | | VR-002 | out-of-ci | yes | Replay drives the **real** KPN nodes, not a reimplementation | | VR-003 | out-of-ci | yes | Scoring: micro-F1 against X-Ray, precision/recall logged at every eva… | @@ -55,7 +55,7 @@ These requirements have no verification tier this repo's CI host can run, so a t | VR-011 | out-of-ci | no | Rewrite the replay harness for the post-AR-012 output contract | | VR-012 | T4, out-of-ci | no | Quality-knee study — TPI/FPI vs sharpness and vs pose, as VR-005 did … | -**Tagged but unexecuted:** VR-001, VR-002, VR-003 — a test exists and is tagged, but this CI host cannot run it. Report those runs separately. +**Tagged but unexecuted:** AR-027, VR-001, VR-002, VR-003 — a test exists and is tagged, but this CI host cannot run it. Report those runs separately. ## Orphan tags @@ -88,7 +88,7 @@ _None._ | AR-007 | **Done** — `track_a… | T2 | SR-002 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp` | Associate detections by IoU + embedding, with **frame-dependent** wei… | | AR-008 | **Done** — one pool… | T2 | SR-002 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp` | One track pool keyed on `last_seen`; no separate revival path | | AR-009 | Done | T2 | SR-002 | untagged | - | Camera-cut detection (histogram) as an association hint | -| AR-010 | **Blocked on a desi… | T2 | SR-002 | untagged | - | Scene-boundary detection (TransNetV2) as an association hint | +| AR-010 | **Done** — decode b… | T2 | SR-002 | covered | `src/main.cpp`, `src/nodes/scene_boundary_annotator_node.hpp`, `src/nodes/scene_detector_node.hpp`, `src/scene_boundaries.hpp` | Scene-boundary detection (TransNetV2) as an association hint | | AR-011 | Planned | T1, T2 | SR-002 | untagged | - | **Every model is fed the input it was trained for** — cost reduced by… | | AR-012 | **Done** — `src/tra… | T2 | **SR-002** | covered | `src/main.cpp`, `src/nodes/identity_matcher_node.hpp`, `src/nodes/result_sink_node.hpp`, `src/track_registry.hpp`, `tests/test_replay_fixtures.cpp`, `tests/test_track_registry.cpp` | Presence follows **track extent**, not per-frame recognition | | AR-013 | **Done** — `last_se… | T2 | SR-002 | covered | `src/track_registry.hpp`, `tests/test_replay_fixtures.cpp`, `tests/test_track_registry.cpp` | `last_seen` optional state machine; window ends at last sighting, nev… | @@ -104,11 +104,11 @@ _None._ | AR-023 | Done | T1 | SR-002 | covered | `src/gallery/gallery_calibration.hpp`, `src/nodes/identity_matcher_node.hpp` | Fit sigmoid calibration from intra/inter similarity distributions | | AR-024 | **Done** — associat… | T1, static | SR-002 | covered | `src/config.hpp`, `src/evidence_discount.hpp`, `src/gallery/gallery_calibration.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp`, `src/nodes/identity_matcher_node.hpp` | **Always the calibrated probability, never a raw cosine** — exception… | | AR-025 | **Done** — log-odds… | T1 | SR-002 | covered | `src/evidence_discount.hpp`, `src/nodes/identity_matcher_node.hpp` | Per-track Bayesian accumulation in log-odds, with correlated-observat… | -| AR-026 | In Progress | T1, T4 | SR-001 | untagged | - | All similarity computed as GEMM, including annex and deferred pass | -| AR-027 | Planned | T4 | SR-001 | untagged | - | Throughput acceptable for **arbitrary** gallery size | +| AR-026 | In Progress | T1, T4 | SR-001 | covered | `src/backends/gemm_backend.cpp` | All similarity computed as GEMM, including annex and deferred pass | +| AR-027 | Planned | T4 | SR-001 | tagged, unexecuted | `src/backends/gemm_backend.cpp` | Throughput acceptable for **arbitrary** gallery size | | AR-028 | Planned | T2 | SR-002 | untagged | - | **Embedding input quality assessed and carried** — every face scored … | | AR-029 | Planned | T1 | SR-002 | untagged | - | Sharpness measure on the **aligned crop** (scale-normalised, so it ca… | -| AR-030 | Planned | T1 | SR-002 | untagged | - | Visibility measure from the AR-001 5-point landmarks — extreme pose o… | +| AR-030 | Planned | T1 | SR-002 | covered | `src/face_utils.hpp` | Visibility measure from the AR-001 5-point landmarks — extreme pose o… | | DP-001 | Done | T1, manual | PR-004 | covered | `src/main.cpp` | One analysis core; modes are front-ends and must not fork pipeline lo… | | DP-002 | Done | T1, manual | PR-004 | covered | `src/main.cpp` | Batch CLI over one title | | DP-003 | Planned | T1, manual | PR-004 | untagged | - | On-demand resident service with bounded, observable queue | @@ -165,9 +165,10 @@ _None._ ### AR-004 -**Locations:** 3 +**Locations:** 4 -- [`src/main.cpp:285`](../src/main.cpp#L285) — `std::lock_guard lk(event_mtx);` +- [`src/main.cpp:86`](../src/main.cpp#L86) — `static constexpr std::size_t kSceneJoinDepth = 256;` +- [`src/main.cpp:299`](../src/main.cpp#L299) — `Unknown` - [`src/nodes/identity_matcher_node.hpp:149`](../src/nodes/identity_matcher_node.hpp#L149) — `std::vector host_query(static_cast(kMaxFaces) * 512);` - [`tests/test_replay_fixtures.cpp:3`](../tests/test_replay_fixtures.cpp#L3) — `Unknown` @@ -175,14 +176,14 @@ _None._ **Locations:** 1 -- [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `inline cv::Mat align_face(const cv::Mat& img,` +- [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `Unknown` ### AR-007 **Locations:** 3 - [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown` -- [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));` +- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));` - [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown` ### AR-008 @@ -190,15 +191,29 @@ _None._ **Locations:** 3 - [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown` -- [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));` +- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));` - [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown` +### AR-010 + +**Locations:** 9 + +- [`src/main.cpp:86`](../src/main.cpp#L86) — `static constexpr std::size_t kSceneJoinDepth = 256;` +- [`src/main.cpp:309`](../src/main.cpp#L309) — `Unknown` +- [`src/main.cpp:379`](../src/main.cpp#L379) — `return run_net(std::move(net));` +- [`src/main.cpp:411`](../src/main.cpp#L411) — `Unknown` +- [`src/nodes/scene_boundary_annotator_node.hpp:2`](../src/nodes/scene_boundary_annotator_node.hpp#L2) — `Unknown` +- [`src/nodes/scene_detector_node.hpp:36`](../src/nodes/scene_detector_node.hpp#L36) — `static constexpr std::string_view label() { return "scene_detector"; }` +- [`src/nodes/scene_detector_node.hpp:110`](../src/nodes/scene_detector_node.hpp#L110) — `void flush_remaining()` +- [`src/nodes/scene_detector_node.hpp:144`](../src/nodes/scene_detector_node.hpp#L144) — `void write_output()` +- [`src/scene_boundaries.hpp:2`](../src/scene_boundaries.hpp#L2) — `Unknown` + ### AR-012 **Locations:** 9 -- [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));` -- [`src/main.cpp:216`](../src/main.cpp#L216) — `reg_cfg, EvidenceDiscounter(same_person));` +- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));` +- [`src/main.cpp:230`](../src/main.cpp#L230) — `reg_cfg, EvidenceDiscounter(same_person));` - [`src/nodes/identity_matcher_node.hpp:119`](../src/nodes/identity_matcher_node.hpp#L119) — `const GalleryCalibration& calibration() const { return cal_; }` - [`src/nodes/identity_matcher_node.hpp:255`](../src/nodes/identity_matcher_node.hpp#L255) — `Unknown` - [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }` @@ -233,7 +248,7 @@ _None._ **Locations:** 4 -- [`src/main.cpp:216`](../src/main.cpp#L216) — `reg_cfg, EvidenceDiscounter(same_person));` +- [`src/main.cpp:230`](../src/main.cpp#L230) — `reg_cfg, EvidenceDiscounter(same_person));` - [`src/nodes/result_sink_node.hpp:63`](../src/nodes/result_sink_node.hpp#L63) — `void set_pre_write_hook(std::function fn) { pre_write_ = std::move(fn); }` - [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown` - [`tests/test_track_registry.cpp:3`](../tests/test_track_registry.cpp#L3) — `Unknown` @@ -261,7 +276,7 @@ _None._ - [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown` - [`src/evidence_discount.hpp:2`](../src/evidence_discount.hpp#L2) — `Unknown` - [`src/gallery/gallery_calibration.hpp:53`](../src/gallery/gallery_calibration.hpp#L53) — `float boundary_at(float p = 0.5f, float log_prior_odds = 0.f) const` -- [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));` +- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));` - [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown` - [`src/nodes/identity_matcher_node.hpp:111`](../src/nodes/identity_matcher_node.hpp#L111) — `const GalleryCalibration& calibration() const { return cal_; }` @@ -273,6 +288,24 @@ _None._ - [`src/nodes/identity_matcher_node.hpp:119`](../src/nodes/identity_matcher_node.hpp#L119) — `const GalleryCalibration& calibration() const { return cal_; }` - [`src/nodes/identity_matcher_node.hpp:255`](../src/nodes/identity_matcher_node.hpp#L255) — `Unknown` +### AR-026 + +**Locations:** 1 + +- [`src/backends/gemm_backend.cpp:44`](../src/backends/gemm_backend.cpp#L44) — `constexpr int kDim = 512;` + +### AR-027 + +**Locations:** 1 + +- [`src/backends/gemm_backend.cpp:44`](../src/backends/gemm_backend.cpp#L44) — `constexpr int kDim = 512;` + +### AR-030 + +**Locations:** 1 + +- [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `Unknown` + ### DP-001 **Locations:** 1 @@ -310,11 +343,11 @@ _None._ - [`src/gallery/gallery_store.cpp:219`](../src/gallery/gallery_store.cpp#L219) — `Unknown` - [`src/kpn_bindings.cpp:167`](../src/kpn_bindings.cpp#L167) — `Unknown` - [`src/kpn_bindings.cpp:217`](../src/kpn_bindings.cpp#L217) — `Unknown` -- [`src/main.cpp:174`](../src/main.cpp#L174) — `Unknown` +- [`src/main.cpp:188`](../src/main.cpp#L188) — `Unknown` - [`src/nodes/embedding_dump_node.hpp:30`](../src/nodes/embedding_dump_node.hpp#L30) — `static constexpr std::string_view label() { return "embedding_dump"; }` - [`src/nodes/embedding_dump_node.hpp:102`](../src/nodes/embedding_dump_node.hpp#L102) — `H5::StrType str(H5::PredType::C_S1, H5T_VARIABLE);` - [`src/scene_preview.cpp:133`](../src/scene_preview.cpp#L133) — `int main(int argc, char** argv)` -- [`src/types.hpp:141`](../src/types.hpp#L141) — `struct Actor` +- [`src/types.hpp:148`](../src/types.hpp#L148) — `struct Actor` - [`tests/test_gallery_store.cpp:182`](../tests/test_gallery_store.cpp#L182) — `TempFile tf("gallery_stamped.h5");` - [`tests/test_gallery_store.cpp:201`](../tests/test_gallery_store.cpp#L201) — `TempFile tf("gallery_stamped.h5");` - [`tests/test_gallery_store.cpp:219`](../tests/test_gallery_store.cpp#L219) — `TempFile tf("gallery_unstamped.h5");` @@ -357,7 +390,7 @@ _None._ **Locations:** 5 - [`src/config.hpp:20`](../src/config.hpp#L20) — `struct Config` -- [`src/main.cpp:216`](../src/main.cpp#L216) — `reg_cfg, EvidenceDiscounter(same_person));` +- [`src/main.cpp:230`](../src/main.cpp#L230) — `reg_cfg, EvidenceDiscounter(same_person));` - [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }` - [`src/nodes/result_sink_node.hpp:122`](../src/nodes/result_sink_node.hpp#L122) — `void write_output()` - [`src/nodes/result_sink_node.hpp:161`](../src/nodes/result_sink_node.hpp#L161) — `struct ActorMeta { std::string name, imdb_id, tmdb_id, jellyfin_id; };` @@ -366,7 +399,7 @@ _None._ **Locations:** 1 -- [`src/main.cpp:216`](../src/main.cpp#L216) — `reg_cfg, EvidenceDiscounter(same_person));` +- [`src/main.cpp:230`](../src/main.cpp#L230) — `reg_cfg, EvidenceDiscounter(same_person));` ### IR-004 @@ -452,8 +485,9 @@ _None._ ### SR-001 -**Locations:** 46 +**Locations:** 47 +- [`src/backends/gemm_backend.cpp:44`](../src/backends/gemm_backend.cpp#L44) — `constexpr int kDim = 512;` - [`src/config.hpp:54`](../src/config.hpp#L54) — `Unknown` - [`src/gallery/embedder_stamp.cpp:1`](../src/gallery/embedder_stamp.cpp#L1) — `Unknown` - [`src/gallery/embedder_stamp.hpp:2`](../src/gallery/embedder_stamp.hpp#L2) — `Unknown` @@ -463,11 +497,11 @@ _None._ - [`src/gallery/gallery_store.cpp:219`](../src/gallery/gallery_store.cpp#L219) — `Unknown` - [`src/kpn_bindings.cpp:167`](../src/kpn_bindings.cpp#L167) — `Unknown` - [`src/kpn_bindings.cpp:217`](../src/kpn_bindings.cpp#L217) — `Unknown` -- [`src/main.cpp:174`](../src/main.cpp#L174) — `Unknown` +- [`src/main.cpp:188`](../src/main.cpp#L188) — `Unknown` - [`src/nodes/embedding_dump_node.hpp:30`](../src/nodes/embedding_dump_node.hpp#L30) — `static constexpr std::string_view label() { return "embedding_dump"; }` - [`src/nodes/embedding_dump_node.hpp:102`](../src/nodes/embedding_dump_node.hpp#L102) — `H5::StrType str(H5::PredType::C_S1, H5T_VARIABLE);` - [`src/scene_preview.cpp:133`](../src/scene_preview.cpp#L133) — `int main(int argc, char** argv)` -- [`src/types.hpp:141`](../src/types.hpp#L141) — `struct Actor` +- [`src/types.hpp:148`](../src/types.hpp#L148) — `struct Actor` - [`tests/test_gallery_store.cpp:182`](../tests/test_gallery_store.cpp#L182) — `TempFile tf("gallery_stamped.h5");` - [`tests/test_gallery_store.cpp:201`](../tests/test_gallery_store.cpp#L201) — `TempFile tf("gallery_stamped.h5");` - [`tests/test_gallery_store.cpp:219`](../tests/test_gallery_store.cpp#L219) — `TempFile tf("gallery_unstamped.h5");` @@ -503,17 +537,21 @@ _None._ ### SR-002 -**Locations:** 20 +**Locations:** 29 - [`src/config.hpp:44`](../src/config.hpp#L44) — `Unknown` - [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown` - [`src/evidence_discount.hpp:2`](../src/evidence_discount.hpp#L2) — `Unknown` -- [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `inline cv::Mat align_face(const cv::Mat& img,` +- [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `Unknown` - [`src/gallery/gallery_calibration.hpp:2`](../src/gallery/gallery_calibration.hpp#L2) — `Unknown` - [`src/gallery/gallery_calibration.hpp:53`](../src/gallery/gallery_calibration.hpp#L53) — `float boundary_at(float p = 0.5f, float log_prior_odds = 0.f) const` -- [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));` -- [`src/main.cpp:216`](../src/main.cpp#L216) — `reg_cfg, EvidenceDiscounter(same_person));` -- [`src/main.cpp:285`](../src/main.cpp#L285) — `std::lock_guard lk(event_mtx);` +- [`src/main.cpp:86`](../src/main.cpp#L86) — `static constexpr std::size_t kSceneJoinDepth = 256;` +- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));` +- [`src/main.cpp:230`](../src/main.cpp#L230) — `reg_cfg, EvidenceDiscounter(same_person));` +- [`src/main.cpp:299`](../src/main.cpp#L299) — `Unknown` +- [`src/main.cpp:309`](../src/main.cpp#L309) — `Unknown` +- [`src/main.cpp:379`](../src/main.cpp#L379) — `return run_net(std::move(net));` +- [`src/main.cpp:411`](../src/main.cpp#L411) — `Unknown` - [`src/nodes/face_detector_node.hpp:2`](../src/nodes/face_detector_node.hpp#L2) — `Unknown` - [`src/nodes/face_detector_node.hpp:47`](../src/nodes/face_detector_node.hpp#L47) — `private:` - [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown` @@ -524,6 +562,11 @@ _None._ - [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }` - [`src/nodes/result_sink_node.hpp:63`](../src/nodes/result_sink_node.hpp#L63) — `void set_pre_write_hook(std::function fn) { pre_write_ = std::move(fn); }` - [`src/nodes/result_sink_node.hpp:161`](../src/nodes/result_sink_node.hpp#L161) — `struct ActorMeta { std::string name, imdb_id, tmdb_id, jellyfin_id; };` +- [`src/nodes/scene_boundary_annotator_node.hpp:2`](../src/nodes/scene_boundary_annotator_node.hpp#L2) — `Unknown` +- [`src/nodes/scene_detector_node.hpp:36`](../src/nodes/scene_detector_node.hpp#L36) — `static constexpr std::string_view label() { return "scene_detector"; }` +- [`src/nodes/scene_detector_node.hpp:110`](../src/nodes/scene_detector_node.hpp#L110) — `void flush_remaining()` +- [`src/nodes/scene_detector_node.hpp:144`](../src/nodes/scene_detector_node.hpp#L144) — `void write_output()` +- [`src/scene_boundaries.hpp:2`](../src/scene_boundaries.hpp#L2) — `Unknown` - [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown` ### SR-003 @@ -626,7 +669,7 @@ _None._ **Groups mixing requirement types (pipe separates types):** -- `src/main.cpp:216` — {'group': ['AR-012', 'AR-016', 'IR-002', 'IR-003']} +- `src/main.cpp:230` — {'group': ['AR-012', 'AR-016', 'IR-002', 'IR-003']} - `src/nodes/result_sink_node.hpp:49` — {'group': ['AR-012', 'AR-017', 'IR-002']} - `src/nodes/result_sink_node.hpp:161` — {'group': ['AR-012', 'IR-002']} - `tests/test_replay_fixtures.cpp:3` — {'group': ['AR-012', 'AR-013', 'AR-004', 'VR-001', 'VR-002']} diff --git a/src/config.hpp b/src/config.hpp index 89a8109..a9e1d54 100644 --- a/src/config.hpp +++ b/src/config.hpp @@ -149,6 +149,15 @@ struct Config { // opposite of the earlier assumption that it only helps restricted galleries. bool expand_gallery{true}; // master switch int expand_buffer_size{20}; // per-track diversity buffer capacity + // TRACES: AR-018, AR-024 | SR-005 + // Banded admission for the per-subject store, in PROBABILITY space. An + // embedding joins only if P(same person) against something already stored + // lands inside [lo, hi]: above hi it is redundant, below lo it is evidence + // the track is not one person. Replaces expand_novelty_sim, a raw cosine. + // Working values pending VR-007; sweep both bounds, they fail in opposite + // directions. + float expand_band_lo{0.90f}; + float expand_band_hi{0.95f}; float expand_novelty_sim{0.55f}; // promote only embeddings whose best sim to the // actor's refs is below this (gallery-far / novel) float expand_track_spread_max{0.60f}; // reject promotion if the retained buffer's diff --git a/src/gallery/track_gallery.hpp b/src/gallery/track_gallery.hpp index 674f628..3eecddf 100644 --- a/src/gallery/track_gallery.hpp +++ b/src/gallery/track_gallery.hpp @@ -2,6 +2,8 @@ #include "types.hpp" #include "config.hpp" +#include + #include #include #include @@ -117,6 +119,17 @@ struct TrackGallery { // matcher when it observes a cut or track disappearance. void forget(int track_id) { tracks_.erase(track_id); } + /// TRACES: AR-024 | SR-005 + /// Supply the calibration belonging to the active embedder. Without it the + /// band falls back to treating cosine as probability, which is wrong but + /// bounded — and the default is loud in the header rather than silent. + void set_calibration(std::function c) { calibrate_ = std::move(c); } + void set_band(float lo, float hi) { band_lo_ = lo; band_hi_ = hi; } + + /// Embeddings the band refused. A store that admits nothing is as wrong as + /// one that admits everything, and neither is visible without this. + std::size_t band_rejected() const { return rejected_; } + // Drop every track buffer (scene cut / EOF). Mirrors face_tracker's clear. void clear_tracks() { tracks_.clear(); } @@ -134,9 +147,40 @@ private: bool promoted{false}; }; + /// TRACES: AR-018, AR-024 | SR-005 + /// Banded admission: an embedding joins the store only if its similarity to + /// something already there falls **inside a band**. + /// + /// above the upper bound → redundant. It is another look at a pose the + /// store already covers, and adding it teaches the annex nothing while + /// costing a slot that a novel view could have used. + /// below the lower bound → suspect. Within one track every face is the + /// same person by construction, so an embedding unlike everything else + /// on the track is evidence the construction failed — a track-ID + /// collision or a bad detection. Admitting it is how an actor's annex + /// gets poisoned with someone else's face. + /// + /// Both bounds are calibrated probabilities, never raw cosines (AR-024): a + /// bare similarity threshold means something different for every model and + /// every face size, and this gate has to hold across both. + /// + /// The first embedding is always admitted — there is nothing for it to be + /// redundant with, and nothing to contradict it. + bool admit(const TrackState& ts, const Embedding& emb) const { + if (ts.buf.empty()) return true; + + float p_max = 0.f; + for (const auto& b : ts.buf) + p_max = std::max(p_max, calibrate_(cosine_similarity(b.emb, emb))); + + return p_max >= band_lo_ && p_max <= band_hi_; + } + void insert_into_buffer(TrackState& ts, const Embedding& emb, float gal_sim, const cv::Mat& crop) { + if (!admit(ts, emb)) { ++rejected_; return; } + BufEntry e; e.emb = emb; e.gal_sim = gal_sim; @@ -231,6 +275,13 @@ private: #endif } + /// cosine → P(same person). The one probability space the pipeline reasons + /// in; see gallery_calibration.hpp's same_person_probability. + std::function calibrate_{[](float c) { return std::max(0.f, c); }}; + float band_lo_{0.90f}; + float band_hi_{0.95f}; + std::size_t rejected_{0}; ///< admissions refused by the band + bool enabled_; int buffer_size_; float novelty_sim_; diff --git a/src/nodes/identity_matcher_node.hpp b/src/nodes/identity_matcher_node.hpp index c3d4654..2c09949 100644 --- a/src/nodes/identity_matcher_node.hpp +++ b/src/nodes/identity_matcher_node.hpp @@ -106,6 +106,12 @@ struct IdentityMatcherFunc { flat_emb_[i].data(), 512 * sizeof(float)); sim_engine_ = make_similarity_engine(host_gallery.data(), n_gallery_, kMaxFaces); + + /// TRACES: AR-018, AR-024 | SR-005 + // The expansion store thresholds in the same probability space as + // association and evidence weighting, so a "0.9" means one thing + // pipeline-wide rather than three. + track_gallery_.set_calibration(same_person_probability(cal_)); } /// TRACES: AR-023, AR-024 | SR-002 diff --git a/tests/test_track_gallery.cpp b/tests/test_track_gallery.cpp index b3639b0..9fc2f3b 100644 --- a/tests/test_track_gallery.cpp +++ b/tests/test_track_gallery.cpp @@ -83,15 +83,25 @@ TEST_CASE("novelty gate skips views the gallery already covers", "[track_gallery CHECK(tg.annex().empty()); } -TEST_CASE("spread gate rejects a two-person track", "[track_gallery]") { +TEST_CASE("a two-person track never poisons the annex", "[track_gallery][AR-018]") { TrackGallery tg(expand_cfg()); - // Two orthogonal identities under one track ID: pairwise sim 0 → spread 1.0 - // > spread_max 0.60. Whole track rejected, annex stays empty even though - // frames are accepted and gallery-far. + // Two orthogonal identities under one track ID — a track-ID collision. + // + // The banded admission (AR-018) now catches this EARLIER than the spread + // gate did: an embedding unlike everything already on the track falls below + // the band's lower bound and is refused entry, so the buffer never becomes + // two-person in the first place. The spread gate remains as a second line + // for a track that drifts gradually rather than jumping. + // + // The assertion is on the outcome, not the mechanism: whichever gate fires, + // the outsider must not reach the actor's annex. tg.observe(3, at_sim(0, 1, 0.30f), 0, 0.30f, true, kNoCrop); tg.observe(3, at_sim(0, 1, 0.30f), 0, 0.30f, true, kNoCrop); tg.observe(3, one_hot(400), 0, 0.30f, true, kNoCrop); // orthogonal outlier - CHECK(tg.annex().empty()); + + CHECK(tg.band_rejected() > 0); // refused at the door + for (const auto& e : tg.annex()) + CHECK(cosine_similarity(e.emb, one_hot(400)) < 0.5f); } TEST_CASE("unconfirmed track (too few accepts) does not promote", "[track_gallery]") {