Compare commits
11
Commits
b7c96641a9
...
cd62d6452d
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cd62d6452d | ||
|
|
c36885de73 | ||
|
|
b5c7d4f6d9 | ||
|
|
09a4650fd9 | ||
|
|
b98372bad8 | ||
|
|
e0f9c95689 | ||
|
|
9e4cdc4efc | ||
|
|
08941540cb | ||
|
|
fe29d014da | ||
|
|
be5f67fa96 | ||
|
|
e9aea3fc41 |
+43
-2
@@ -80,8 +80,49 @@ by dropping work or growing without limit.
|
||||
- Memory is the real limit: faces carry 112×112 crops plus 512-float embeddings.
|
||||
Backpressure must engage on bytes in flight, not just item counts.
|
||||
|
||||
**Gap:** entire requirement. This is a prerequisite for removing `max_faces`, not
|
||||
a follow-up to it.
|
||||
### The fix is not in this repo
|
||||
|
||||
**Every node output in KPN uses the dropping `push()`** (`pool_node.hpp:404`,
|
||||
`:710`; also `branch.hpp`, `fanout.hpp`, `interrupt_node.hpp`). A lossless
|
||||
`push_blocking()` — "wait for the consumer to drain instead of dropping; the
|
||||
producer just runs slower" — already exists on both `Channel`
|
||||
(`channel.hpp:144`) and `OutputPort` (`variant_node.hpp:81`), **and nothing
|
||||
calls it.**
|
||||
|
||||
So AR-004 is a change to the KPN repository, not to this one. It needs either a
|
||||
per-channel lossless policy or a network-wide default, and this pipeline should
|
||||
select lossless: a dropped frame here does not degrade a result, it silently
|
||||
changes one.
|
||||
|
||||
**Measured, not inferred.** One 77 s clip at 5 fps should yield ~385 sampled
|
||||
frames. On CPU it produced 49, ending at 51 s, with 285 frames dropped at
|
||||
`camera_pos` and 51 at `face_aligner`. Rebuilt with CUDA the same clip ran in
|
||||
29 s and reached EOF correctly — and still dropped **320** frames at
|
||||
`camera_pos`, yielding 65. Faster hardware moves where the queue backs up; it
|
||||
does not change what happens when it does.
|
||||
|
||||
Two consequences worth stating:
|
||||
|
||||
- **Raising channel capacity is a stopgap, not a fix.** It lowers the
|
||||
probability of overflow without changing the behaviour on overflow, and the
|
||||
failure it hides is silent corruption of the output.
|
||||
- **Fixture generation is blocked on this** (VR-001), because what gets dropped
|
||||
depends on timing. The same command run twice can produce different dumps, and
|
||||
a golden fixture cannot be built on that.
|
||||
|
||||
**Current:** fixed in KPN — node data outputs use `push_blocking`, sentinels
|
||||
remain out-of-band so EOF can always overtake a stalled data path. Verified on
|
||||
the same clip: 385 of 385 sampled frames written, zero drops, and two
|
||||
consecutive runs byte-identical where previously they were not.
|
||||
|
||||
It also ran *faster* (29 s → 17 s). A dropped frame has already cost its decode,
|
||||
and the overflow exception cost more — so the lossy path was paying for work it
|
||||
then discarded.
|
||||
|
||||
**Gap:** the remaining half — bounding by **bytes in flight** rather than item
|
||||
count. Channel capacity is still a count of items, and a face carries a 112×112
|
||||
crop plus a 512-float embedding, so a crowded frame occupies far more memory per
|
||||
slot than a sparse one. That matters once `max_faces` is removed (AR-003).
|
||||
|
||||
## AR-005 — Face alignment and crop
|
||||
|
||||
|
||||
+51
-11
@@ -29,13 +29,13 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
|
||||
| ID | Requirement | Traces to | Priority | Status |
|
||||
|---|---|---|---|---|
|
||||
| AR-001 | Detect faces in sampled frames; emit bbox, confidence, 5-point landmarks in original pixel space | SR-002 | High | Done |
|
||||
| AR-002 | Minimum face size 66×66 px, expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | Planned |
|
||||
| AR-002 | Minimum face size **32×32 px** (VR-005 measured), expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | Planned |
|
||||
| AR-003 | No fixed per-frame face cap — crowd scenes must not lose background cast | SR-002 | Medium | Planned |
|
||||
| AR-004 | Backpressure: unbounded faces/frame absorbed by slowing, never by dropping or throwing | SR-002 | High | Planned |
|
||||
| AR-004 | Backpressure: unbounded faces/frame absorbed by slowing, never by dropping or throwing | SR-002 | High | **Done** — KPN node outputs use `push_blocking`; sentinels stay out-of-band. Verified: 385/385 frames, 0 drops, byte-identical across runs |
|
||||
| AR-005 | Align to 112×112 via ArcFace 5-point similarity transform | SR-002 | High | Done |
|
||||
| AR-006 | 512-d L2-normalised embeddings, batched | SR-002 | High | Done |
|
||||
| AR-007 | Associate detections by IoU + embedding, with **frame-dependent** weighting | SR-002 | High | In Progress |
|
||||
| AR-008 | One track pool keyed on `last_seen`; no separate revival path | SR-002 | High | Planned |
|
||||
| AR-007 | Associate detections by IoU + embedding, with **frame-dependent** weighting | SR-002 | High | **Done** — `track_alpha` is the base for ordinary frames; drops to embedding-only on cut/boundary and for dormant tracks |
|
||||
| AR-008 | One track pool keyed on `last_seen`; no separate revival path | SR-002 | High | **Done** — one pool keyed on `last_seen`; park/revive branch deleted |
|
||||
| AR-009 | Camera-cut detection (histogram) as an association hint | SR-002 | High | Done |
|
||||
| AR-010 | Scene-boundary detection (TransNetV2) as an association hint | SR-002 | Medium | **Not started** — `is_scene_boundary` has no producer; `SceneDetectorFunc` is a terminal sink and never annotates the frame |
|
||||
| AR-011 | **Every model is fed the input it was trained for** — cost reduced by running less often, never by degrading one inference | SR-002 | High | Planned |
|
||||
@@ -51,7 +51,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
|
||||
| AR-021 | Cluster unknown tracks into one entity per person, under temporal cannot-link constraints | SR-005 | Medium | Planned |
|
||||
| AR-022 | Capture still-unidentified tracks: embeddings, metadata, **context crops** for human review | §4 | Medium | Planned |
|
||||
| AR-023 | Fit sigmoid calibration from intra/inter similarity distributions | SR-002 | High | Done |
|
||||
| AR-024 | **Always the calibrated probability, never a raw cosine** — exceptions recorded | SR-002 | High | **Done** (registry boundary) — `observe()` takes a calibrated probability and converts to log-odds itself; retiring the raw-cosine constants is still open |
|
||||
| AR-024 | **Always the calibrated probability, never a raw cosine** — exceptions recorded | SR-002 | High | **Done** — association and accumulation both in probability space; `track_max_embed_dist`, `cut_revive_sim` retired |
|
||||
| AR-025 | Per-track Bayesian accumulation in log-odds, with correlated-observation discounting | SR-002 | High | **Done** — log-odds accumulation with correlation discounting owned by the registry, `src/evidence_discount.hpp` |
|
||||
| AR-026 | All similarity computed as GEMM, including annex and deferred pass | SR-001 | High | In Progress |
|
||||
| AR-027 | Throughput acceptable for **arbitrary** gallery size | SR-001 | High | Planned |
|
||||
@@ -74,8 +74,8 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
|
||||
| ID | Requirement | Traces to | Priority | Status |
|
||||
|---|---|---|---|---|
|
||||
| IR-001 | Emit the JRay truth format as sibling `.jray.json` | SR-003 | High | Done |
|
||||
| IR-002 | Windows carry belief + route; `extraction.*` carries `extinction_sec`, `gallery_scope` | SR-003 | High | Planned |
|
||||
| IR-003 | Output written **after** the deferred pass, not at EOF | SR-003 | High | Planned |
|
||||
| IR-002 | Windows carry belief + route; `extraction.*` carries `extinction_sec`, `gallery_scope` | SR-003 | High | **Done** — `schema_version: 2`; windows are objects with `belief` + `route`; `extraction.*` carries `extinction_sec` and `gallery_scope`; `anneal_sec` removed |
|
||||
| IR-003 | Output written **after** the deferred pass, not at EOF | SR-003 | High | **In Progress** — sink builds windows from registry claims and flushes at EOF; the deferred pass (AR-020) does not exist yet, so output is still final at EOF |
|
||||
| IR-004 | Compute the audio signature exactly per server spec §3 | SR-003 | Medium | **Done** — `src/audio_signature.*`; not yet emitted into the truth file (IR-002) |
|
||||
| IR-005 | Golden-vector fixture shared with the plugin repo to prove bit-exactness | SR-003 | High | **Done** — `tests/fixtures/audio/`; v1 parameters now normative in server spec §3 |
|
||||
| IR-007 | Media < 120 s: emit no signature, apply no sync offset — identical rule in both producers | SR-003 | Low | **Done** |
|
||||
@@ -101,10 +101,10 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
|
||||
| ID | Requirement | Traces to | Priority | Status |
|
||||
|---|---|---|---|---|
|
||||
| VR-001 | HDF5 post-inference dump at the embedded-frame boundary | PR-002 | High | Done |
|
||||
| VR-002 | Replay drives the **real** KPN nodes, not a reimplementation | PR-002 | High | Done |
|
||||
| VR-002 | Replay drives the **real** KPN nodes, not a reimplementation | PR-002 | High | **Done** — replay driven from committed fixtures in `tests/test_replay_fixtures.cpp`; determinism asserted |
|
||||
| VR-003 | Scoring: micro-F1 against X-Ray, precision/recall logged at every evaluation | PR-002 | High | Done |
|
||||
| VR-004 | Reproducible validation corpus with ground truth | PR-002 | High | Done |
|
||||
| VR-005 | Minimum face size study — TPI/FPI vs probe size, gallery held at native res | PR-002 | Medium | Planned |
|
||||
| VR-005 | Minimum face size study — TPI/FPI vs probe size, gallery held at native res | PR-002 | Medium | **Done** — knee at 24–32 px; 32 px gives 98.1% TPI, 0.0 FPI at every size |
|
||||
| VR-006 | Re-tune `scene_threshold` once native-rate decode lands | PR-002 | Low | Planned |
|
||||
| VR-007 | Expansion band, clustering threshold, and deferred-pass ablation | PR-002 | Medium | Planned |
|
||||
| VR-008 | Gallery scaling benchmark — throughput vs gallery size | PR-002 | Medium | Planned |
|
||||
@@ -195,7 +195,7 @@ as such rather than counted as covered.
|
||||
| Requirement | Tier | Note |
|
||||
|---|---|---|
|
||||
| AR-001, AR-005, AR-006 | T3 | Smoke only — correctness of detection/embedding is a model property, not ours |
|
||||
| AR-002 | T2 | Size filtering is arithmetic on dumped bboxes |
|
||||
| AR-002 | Minimum face size **32×32 px** (VR-005 measured), expressed in **original** resolution (decoupled from `dense_scale`) | Size filtering is arithmetic on dumped bboxes |
|
||||
| AR-003, AR-004 | T1 + T4 | Backpressure logic is unit-testable; saturation behaviour needs real load |
|
||||
| AR-007 … AR-017 | **T2** | The core of the redesign — fully replayable |
|
||||
| AR-018 … AR-022 | **T2** | Expansion, deferred pass, clustering: all post-embedding |
|
||||
@@ -213,6 +213,46 @@ structurally unverifiable on the CI host. It needs a GPU host and a synthetic
|
||||
large gallery, so it is the requirement most likely to silently regress. Its
|
||||
benchmark (VR-008) should run on a schedule rather than on demand.
|
||||
|
||||
### CI never calls a model
|
||||
|
||||
**Not "should not" — cannot.** The N100 has no GPU, and even the ONNX Runtime CPU
|
||||
provider is impractical: a measured run of the embedder on this hardware sits at
|
||||
~930 ms per frame, so a 77 s clip at 5 fps would take roughly six minutes of
|
||||
inference alone. Every model invocation therefore happens **locally, ahead of
|
||||
time**, and CI consumes the result as data.
|
||||
|
||||
This is what makes the T1/T2 split load-bearing rather than a preference: T1 and
|
||||
T2 are the only tiers that can exist in CI at all.
|
||||
|
||||
### Fixture corpus — `bali/`
|
||||
|
||||
Five clips of **Road to Bali (1952)**, ~77 s each, 480×360, 30 fps, 42 MB total.
|
||||
|
||||
Public domain, and that is the reason to use it rather than a convenience:
|
||||
**derived fixtures — dumps, crops, golden outputs — can be committed without the
|
||||
rights question that rules out sharing gallery data (SR-005).** A fixture cut
|
||||
from a copyrighted title could not live in the repository at all.
|
||||
|
||||
Two properties to design around rather than discover:
|
||||
|
||||
- **480×360 means small faces.** At this resolution a face is often 40–80 px, so
|
||||
the AR-002 minimum of 66 px (original resolution) rejects much of what is
|
||||
there. Fixture generation must set `--min-face-px` explicitly and record it,
|
||||
or the dumps will be sparse for reasons unrelated to what is being tested.
|
||||
- **77 s is short.** At 1 fps that is 77 frames — too thin to exercise an
|
||||
extinction window measured in tens of seconds. Generate at 5 fps (≈385 frames,
|
||||
~1 MB) and record the rate in provenance, since the behaviour under test
|
||||
changes with it.
|
||||
|
||||
> **AR-004 blocks reproducible fixture generation.** A trial run of one clip
|
||||
> produced 49 frames of an expected ~385, ending at 51 s of 77 s, with the
|
||||
> diagnostics reporting 285 frames dropped at `camera_pos` and 51 at
|
||||
> `face_aligner`. Channels overflow and **drop** rather than blocking, and what
|
||||
> gets dropped depends on timing — so the same command run twice can produce
|
||||
> different dumps. Golden fixtures cannot be built on that. AR-004 is therefore
|
||||
> a prerequisite for VR-001 fixtures, not merely a throughput concern for crowd
|
||||
> scenes.
|
||||
|
||||
### Fixtures — precomputed inference, pulled by CI
|
||||
|
||||
The N100 cannot run inference at any useful rate, so **inference output is
|
||||
@@ -262,7 +302,7 @@ because it will be trusted.
|
||||
| ID | Tier | Test asserts | Edge cases to cover |
|
||||
|---|---|---|---|
|
||||
| AR-001 | T3 | Detector returns plausible boxes on a known frame | — smoke only |
|
||||
| AR-002 | T2 | Faces below 66 px (original res) are dropped | Exactly at threshold; with `dense_scale` 0.5 — the interaction that motivated the requirement |
|
||||
| AR-002 | Minimum face size **32×32 px** (VR-005 measured), expressed in **original** resolution (decoupled from `dense_scale`) | Faces below 66 px (original res) are dropped | Exactly at threshold; with `dense_scale` 0.5 — the interaction that motivated the requirement |
|
||||
| AR-003 | T2 | No cap applied; a 40-face frame yields 40 | Crowd frame |
|
||||
| AR-004 | T1 | Saturated input blocks rather than drops or throws | Bounded queue at capacity; **byte-based** limit with large crops; SIGTERM mid-block |
|
||||
| AR-005 | T1 | Known landmarks → expected 112×112 warp | Landmarks near frame edge; degenerate/collinear points |
|
||||
|
||||
+102
-32
@@ -3,7 +3,7 @@
|
||||
<!-- GENERATED FILE - do not edit by hand. -->
|
||||
<!-- Regenerate: scripts/traceability/traceability-gate.sh -->
|
||||
|
||||
**Generated:** 2026-07-31T07:15:30+00:00
|
||||
**Generated:** 2026-07-31T08:35:29+00:00
|
||||
|
||||
Denominators are read from [`requirements.md`](requirements.md) at run time, never hardcoded. Coverage counts a requirement only when it is tagged in source **and** has a verification tier this repo's CI host can execute (`T1, T2, T3, static`).
|
||||
|
||||
@@ -12,12 +12,12 @@ Denominators are read from [`requirements.md`](requirements.md) at run time, nev
|
||||
| Metric | Value |
|
||||
|---|---|
|
||||
| Source files scanned | 95 |
|
||||
| TRACES tags found | 77 |
|
||||
| TRACES tags found | 90 |
|
||||
| EXCEPTION tags found | 0 |
|
||||
| Requirements defined | 63 |
|
||||
| Requirements covered | 22 |
|
||||
| **Coverage** | **34.9%** (22/63) |
|
||||
| Coverage of CI-executable scope | 42.3% (22/52) |
|
||||
| Requirements covered | 26 |
|
||||
| **Coverage** | **41.3%** (26/63) |
|
||||
| Coverage of CI-executable scope | 50.0% (26/52) |
|
||||
| Tagged but unexecuted in CI | 3 |
|
||||
| Orphan tags | 0 |
|
||||
|
||||
@@ -25,9 +25,9 @@ Denominators are read from [`requirements.md`](requirements.md) at run time, nev
|
||||
|
||||
| Type | Covered | Tagged but unexecuted | Defined |
|
||||
|---|---|---|---|
|
||||
| AR | 11 | 0 | 27 |
|
||||
| AR | 13 | 0 | 27 |
|
||||
| DP | 2 | 0 | 8 |
|
||||
| IR | 6 | 0 | 8 |
|
||||
| IR | 8 | 0 | 8 |
|
||||
| GR | 3 | 0 | 9 |
|
||||
| VR | 0 | 3 | 11 |
|
||||
|
||||
@@ -78,30 +78,30 @@ _None._
|
||||
| ID | Status | Tier | Traces to | Trace state | Tagged in | Requirement |
|
||||
|---|---|---|---|---|---|---|
|
||||
| AR-001 | Done | T3 | SR-002 | covered | `src/nodes/face_detector_node.hpp` | Detect faces in sampled frames; emit bbox, confidence, 5-point landma… |
|
||||
| AR-002 | Planned | T2 | SR-002 | untagged | - | Minimum face size 66×66 px, expressed in **original** resolution (dec… |
|
||||
| AR-002 | Planned | unset | SR-002 | untagged | - | Minimum face size **32×32 px** (VR-005 measured), expressed in **orig… |
|
||||
| AR-003 | Planned | T1, T2, T4 | SR-002 | untagged | - | No fixed per-frame face cap — crowd scenes must not lose background c… |
|
||||
| AR-004 | Planned | T1, T4 | SR-002 | untagged | - | Backpressure: unbounded faces/frame absorbed by slowing, never by dro… |
|
||||
| AR-004 | **Done** — KPN node… | T1, T4 | SR-002 | untagged | - | Backpressure: unbounded faces/frame absorbed by slowing, never by dro… |
|
||||
| AR-005 | Done | T1, T3 | SR-002 | covered | `src/face_utils.hpp` | Align to 112×112 via ArcFace 5-point similarity transform |
|
||||
| AR-006 | Done | T3 | SR-002 | untagged | - | 512-d L2-normalised embeddings, batched |
|
||||
| AR-007 | In Progress | T2 | SR-002 | untagged | - | Associate detections by IoU + embedding, with **frame-dependent** wei… |
|
||||
| AR-008 | Planned | T2 | SR-002 | untagged | - | One track pool keyed on `last_seen`; no separate revival path |
|
||||
| AR-007 | **Done** — `track_a… | T2 | SR-002 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp` | Associate detections by IoU + embedding, with **frame-dependent** wei… |
|
||||
| AR-008 | **Done** — one pool… | T2 | SR-002 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp` | One track pool keyed on `last_seen`; no separate revival path |
|
||||
| AR-009 | Done | T2 | SR-002 | untagged | - | Camera-cut detection (histogram) as an association hint |
|
||||
| AR-010 | **Not started** — `… | T2 | SR-002 | untagged | - | Scene-boundary detection (TransNetV2) as an association hint |
|
||||
| AR-011 | Planned | T1, T2 | SR-002 | untagged | - | **Every model is fed the input it was trained for** — cost reduced by… |
|
||||
| AR-012 | **Done** — `src/tra… | T2 | **SR-002** | covered | `src/track_registry.hpp`, `tests/test_track_registry.cpp` | Presence follows **track extent**, not per-frame recognition |
|
||||
| AR-012 | **Done** — `src/tra… | T2 | **SR-002** | covered | `src/main.cpp`, `src/nodes/identity_matcher_node.hpp`, `src/nodes/result_sink_node.hpp`, `src/track_registry.hpp`, `tests/test_track_registry.cpp` | Presence follows **track extent**, not per-frame recognition |
|
||||
| AR-013 | **Done** — `last_se… | T2 | SR-002 | covered | `src/track_registry.hpp`, `tests/test_track_registry.cpp` | `last_seen` optional state machine; window ends at last sighting, nev… |
|
||||
| AR-014 | **Done** — swap clo… | T2 | SR-002 | covered | `src/track_registry.hpp`, `tests/test_track_registry.cpp` | Belief swap A→B terminates the track and starts a new one |
|
||||
| AR-015 | **Done** — reverse … | T2 | SR-002 | covered | `src/track_registry.hpp`, `tests/test_track_registry.cpp` | Two live tracks owned by one actor ⇒ treat as a detected cut, re-asso… |
|
||||
| AR-016 | **Done** — `flush()… | T2 | SR-002 | covered | `src/track_registry.hpp`, `tests/test_track_registry.cpp` | All tracks closed at EOF — a film ends with faces on screen |
|
||||
| AR-017 | **Done** — `DeadTra… | T1, T2 | SR-002 | covered | `src/track_registry.hpp`, `tests/test_track_registry.cpp` | Every presence claim carries its belief and identification route |
|
||||
| AR-016 | **Done** — `flush()… | T2 | SR-002 | covered | `src/main.cpp`, `src/nodes/result_sink_node.hpp`, `src/track_registry.hpp`, `tests/test_track_registry.cpp` | All tracks closed at EOF — a film ends with faces on screen |
|
||||
| AR-017 | **Done** — `DeadTra… | T1, T2 | SR-002 | covered | `src/nodes/result_sink_node.hpp`, `src/track_registry.hpp`, `tests/test_track_registry.cpp` | Every presence claim carries its belief and identification route |
|
||||
| AR-018 | Planned | T1, T2 | SR-005 | untagged | - | Per-subject embedding store with banded admission (novel enough, safe… |
|
||||
| AR-019 | In Progress | T2 | SR-005 | untagged | - | Per-film gallery annex from owned tracks; acquires the non-frontal vi… |
|
||||
| AR-020 | Planned | T2 | SR-005 | untagged | - | Deferred re-identification of unknown tracks against the final expand… |
|
||||
| AR-021 | Planned | T2 | SR-005 | untagged | - | Cluster unknown tracks into one entity per person, under temporal can… |
|
||||
| AR-022 | Planned | T1, T2 | §4 | untagged | - | Capture still-unidentified tracks: embeddings, metadata, **context cr… |
|
||||
| AR-023 | Done | T1 | SR-002 | covered | `src/gallery/gallery_calibration.hpp` | Fit sigmoid calibration from intra/inter similarity distributions |
|
||||
| AR-024 | **Done** (registry … | T1, static | SR-002 | covered | `src/evidence_discount.hpp` | **Always the calibrated probability, never a raw cosine** — exception… |
|
||||
| AR-025 | **Done** — log-odds… | T1 | SR-002 | covered | `src/evidence_discount.hpp` | Per-track Bayesian accumulation in log-odds, with correlated-observat… |
|
||||
| AR-023 | Done | T1 | SR-002 | covered | `src/gallery/gallery_calibration.hpp`, `src/nodes/identity_matcher_node.hpp` | Fit sigmoid calibration from intra/inter similarity distributions |
|
||||
| AR-024 | **Done** — associat… | T1, static | SR-002 | covered | `src/config.hpp`, `src/evidence_discount.hpp`, `src/gallery/gallery_calibration.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp`, `src/nodes/identity_matcher_node.hpp` | **Always the calibrated probability, never a raw cosine** — exception… |
|
||||
| AR-025 | **Done** — log-odds… | T1 | SR-002 | covered | `src/evidence_discount.hpp`, `src/nodes/identity_matcher_node.hpp` | Per-track Bayesian accumulation in log-odds, with correlated-observat… |
|
||||
| AR-026 | In Progress | T1, T4 | SR-001 | untagged | - | All similarity computed as GEMM, including annex and deferred pass |
|
||||
| AR-027 | Planned | T4 | SR-001 | untagged | - | Throughput acceptable for **arbitrary** gallery size |
|
||||
| DP-001 | Done | T1, manual | PR-004 | covered | `src/main.cpp` | One analysis core; modes are front-ends and must not fork pipeline lo… |
|
||||
@@ -113,8 +113,8 @@ _None._
|
||||
| DP-007 | Planned | T1, manual | PR-004 | untagged | - | CI builder image, CPU-only, pinned by tag in the Gitea container regi… |
|
||||
| DP-008 | Planned | T1, manual | PR-004 | untagged | - | Builder images + release jobs per backend (cpu / cuda / rocm); ship b… |
|
||||
| IR-001 | Done | T1 | SR-003 | covered | `src/nodes/result_sink_node.hpp` | Emit the JRay truth format as sibling `.jray.json` |
|
||||
| IR-002 | Planned | T1 | SR-003 | untagged | - | Windows carry belief + route; `extraction.*` carries `extinction_sec`… |
|
||||
| IR-003 | Planned | T1 | SR-003 | untagged | - | Output written **after** the deferred pass, not at EOF |
|
||||
| IR-002 | **Done** — `schema_… | T1 | SR-003 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/result_sink_node.hpp` | Windows carry belief + route; `extraction.*` carries `extinction_sec`… |
|
||||
| IR-003 | **In Progress** — s… | T1 | SR-003 | covered | `src/main.cpp` | Output written **after** the deferred pass, not at EOF |
|
||||
| IR-004 | **Done** — `src/aud… | T1 | SR-003 | covered | `src/audio_signature.cpp`, `src/audio_signature.hpp`, `tests/test_audio_signature.cpp` | Compute the audio signature exactly per server spec §3 |
|
||||
| IR-005 | **Done** — `tests/f… | T1 | SR-003 | covered | `src/audio_signature.cpp`, `src/audio_signature.hpp`, `tests/test_audio_signature.cpp` | Golden-vector fixture shared with the plugin repo to prove bit-exactn… |
|
||||
| IR-006 | Done | T1, manual | SR-001 | covered | `scripts/run_from_jellyfin.py` | Jellyfin round-trip: pull pending queue, push complete results only |
|
||||
@@ -133,7 +133,7 @@ _None._
|
||||
| VR-002 | Done | out-of-ci | PR-002 | tagged, unexecuted | `scripts/optimizer/replay.py` | Replay drives the **real** KPN nodes, not a reimplementation |
|
||||
| VR-003 | Done | out-of-ci | PR-002 | tagged, unexecuted | `scripts/optimizer/second_score.py` | Scoring: micro-F1 against X-Ray, precision/recall logged at every eva… |
|
||||
| VR-004 | Done | out-of-ci | PR-002 | untagged | - | Reproducible validation corpus with ground truth |
|
||||
| VR-005 | Planned | out-of-ci | PR-002 | untagged | - | Minimum face size study — TPI/FPI vs probe size, gallery held at nati… |
|
||||
| VR-005 | **Done** — knee at … | out-of-ci | PR-002 | untagged | - | Minimum face size study — TPI/FPI vs probe size, gallery held at nati… |
|
||||
| VR-006 | Planned | out-of-ci | PR-002 | untagged | - | Re-tune `scene_threshold` once native-rate decode lands |
|
||||
| VR-007 | Planned | out-of-ci | PR-002 | untagged | - | Expansion band, clustering threshold, and deferred-pass ablation |
|
||||
| VR-008 | Planned | out-of-ci | PR-002 | untagged | - | Gallery scaling benchmark — throughput vs gallery size |
|
||||
@@ -155,10 +155,32 @@ _None._
|
||||
|
||||
- [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `inline cv::Mat align_face(const cv::Mat& img,`
|
||||
|
||||
### AR-007
|
||||
|
||||
**Locations:** 3
|
||||
|
||||
- [`src/config.hpp:103`](../src/config.hpp#L103) — `Unknown`
|
||||
- [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));`
|
||||
- [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown`
|
||||
|
||||
### AR-008
|
||||
|
||||
**Locations:** 3
|
||||
|
||||
- [`src/config.hpp:103`](../src/config.hpp#L103) — `Unknown`
|
||||
- [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));`
|
||||
- [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown`
|
||||
|
||||
### AR-012
|
||||
|
||||
**Locations:** 2
|
||||
**Locations:** 8
|
||||
|
||||
- [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));`
|
||||
- [`src/main.cpp:216`](../src/main.cpp#L216) — `reg_cfg, EvidenceDiscounter(same_person));`
|
||||
- [`src/nodes/identity_matcher_node.hpp:119`](../src/nodes/identity_matcher_node.hpp#L119) — `const GalleryCalibration& calibration() const { return cal_; }`
|
||||
- [`src/nodes/identity_matcher_node.hpp:242`](../src/nodes/identity_matcher_node.hpp#L242) — `Unknown`
|
||||
- [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }`
|
||||
- [`src/nodes/result_sink_node.hpp:161`](../src/nodes/result_sink_node.hpp#L161) — `struct ActorMeta { std::string name, imdb_id, tmdb_id, jellyfin_id; };`
|
||||
- [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown`
|
||||
- [`tests/test_track_registry.cpp:3`](../tests/test_track_registry.cpp#L3) — `Unknown`
|
||||
|
||||
@@ -185,35 +207,47 @@ _None._
|
||||
|
||||
### AR-016
|
||||
|
||||
**Locations:** 2
|
||||
**Locations:** 4
|
||||
|
||||
- [`src/main.cpp:216`](../src/main.cpp#L216) — `reg_cfg, EvidenceDiscounter(same_person));`
|
||||
- [`src/nodes/result_sink_node.hpp:63`](../src/nodes/result_sink_node.hpp#L63) — `void set_pre_write_hook(std::function<void(double)> fn) { pre_write_ = std::move(fn); }`
|
||||
- [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown`
|
||||
- [`tests/test_track_registry.cpp:3`](../tests/test_track_registry.cpp#L3) — `Unknown`
|
||||
|
||||
### AR-017
|
||||
|
||||
**Locations:** 2
|
||||
**Locations:** 3
|
||||
|
||||
- [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }`
|
||||
- [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown`
|
||||
- [`tests/test_track_registry.cpp:3`](../tests/test_track_registry.cpp#L3) — `Unknown`
|
||||
|
||||
### AR-023
|
||||
|
||||
**Locations:** 1
|
||||
**Locations:** 3
|
||||
|
||||
- [`src/gallery/gallery_calibration.hpp:2`](../src/gallery/gallery_calibration.hpp#L2) — `Unknown`
|
||||
- [`src/gallery/gallery_calibration.hpp:53`](../src/gallery/gallery_calibration.hpp#L53) — `float boundary_at(float p = 0.5f, float log_prior_odds = 0.f) const`
|
||||
- [`src/nodes/identity_matcher_node.hpp:111`](../src/nodes/identity_matcher_node.hpp#L111) — `const GalleryCalibration& calibration() const { return cal_; }`
|
||||
|
||||
### AR-024
|
||||
|
||||
**Locations:** 1
|
||||
**Locations:** 6
|
||||
|
||||
- [`src/config.hpp:103`](../src/config.hpp#L103) — `Unknown`
|
||||
- [`src/evidence_discount.hpp:2`](../src/evidence_discount.hpp#L2) — `Unknown`
|
||||
- [`src/gallery/gallery_calibration.hpp:53`](../src/gallery/gallery_calibration.hpp#L53) — `float boundary_at(float p = 0.5f, float log_prior_odds = 0.f) const`
|
||||
- [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));`
|
||||
- [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown`
|
||||
- [`src/nodes/identity_matcher_node.hpp:111`](../src/nodes/identity_matcher_node.hpp#L111) — `const GalleryCalibration& calibration() const { return cal_; }`
|
||||
|
||||
### AR-025
|
||||
|
||||
**Locations:** 1
|
||||
**Locations:** 3
|
||||
|
||||
- [`src/evidence_discount.hpp:2`](../src/evidence_discount.hpp#L2) — `Unknown`
|
||||
- [`src/nodes/identity_matcher_node.hpp:119`](../src/nodes/identity_matcher_node.hpp#L119) — `const GalleryCalibration& calibration() const { return cal_; }`
|
||||
- [`src/nodes/identity_matcher_node.hpp:242`](../src/nodes/identity_matcher_node.hpp#L242) — `Unknown`
|
||||
|
||||
### DP-001
|
||||
|
||||
@@ -243,7 +277,7 @@ _None._
|
||||
|
||||
**Locations:** 44
|
||||
|
||||
- [`src/config.hpp:43`](../src/config.hpp#L43) — `Unknown`
|
||||
- [`src/config.hpp:49`](../src/config.hpp#L49) — `Unknown`
|
||||
- [`src/gallery/embedder_stamp.cpp:1`](../src/gallery/embedder_stamp.cpp#L1) — `Unknown`
|
||||
- [`src/gallery/embedder_stamp.hpp:2`](../src/gallery/embedder_stamp.hpp#L2) — `Unknown`
|
||||
- [`src/gallery/gallery_builder.cpp:45`](../src/gallery/gallery_builder.cpp#L45) — `ActorGallery build_gallery(const BuildConfig& cfg)`
|
||||
@@ -252,7 +286,7 @@ _None._
|
||||
- [`src/gallery/gallery_store.cpp:219`](../src/gallery/gallery_store.cpp#L219) — `Unknown`
|
||||
- [`src/kpn_bindings.cpp:167`](../src/kpn_bindings.cpp#L167) — `Unknown`
|
||||
- [`src/kpn_bindings.cpp:217`](../src/kpn_bindings.cpp#L217) — `Unknown`
|
||||
- [`src/main.cpp:176`](../src/main.cpp#L176) — `Unknown`
|
||||
- [`src/main.cpp:174`](../src/main.cpp#L174) — `Unknown`
|
||||
- [`src/nodes/embedding_dump_node.hpp:30`](../src/nodes/embedding_dump_node.hpp#L30) — `static constexpr std::string_view label() { return "embedding_dump"; }`
|
||||
- [`src/nodes/embedding_dump_node.hpp:102`](../src/nodes/embedding_dump_node.hpp#L102) — `H5::StrType str(H5::PredType::C_S1, H5T_VARIABLE);`
|
||||
- [`src/scene_preview.cpp:133`](../src/scene_preview.cpp#L133) — `int main(int argc, char** argv)`
|
||||
@@ -294,6 +328,22 @@ _None._
|
||||
|
||||
- [`src/nodes/result_sink_node.hpp:2`](../src/nodes/result_sink_node.hpp#L2) — `Unknown`
|
||||
|
||||
### IR-002
|
||||
|
||||
**Locations:** 5
|
||||
|
||||
- [`src/config.hpp:20`](../src/config.hpp#L20) — `struct Config`
|
||||
- [`src/main.cpp:216`](../src/main.cpp#L216) — `reg_cfg, EvidenceDiscounter(same_person));`
|
||||
- [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }`
|
||||
- [`src/nodes/result_sink_node.hpp:122`](../src/nodes/result_sink_node.hpp#L122) — `void write_output()`
|
||||
- [`src/nodes/result_sink_node.hpp:161`](../src/nodes/result_sink_node.hpp#L161) — `struct ActorMeta { std::string name, imdb_id, tmdb_id, jellyfin_id; };`
|
||||
|
||||
### IR-003
|
||||
|
||||
**Locations:** 1
|
||||
|
||||
- [`src/main.cpp:216`](../src/main.cpp#L216) — `reg_cfg, EvidenceDiscounter(same_person));`
|
||||
|
||||
### IR-004
|
||||
|
||||
**Locations:** 16
|
||||
@@ -374,7 +424,7 @@ _None._
|
||||
|
||||
**Locations:** 46
|
||||
|
||||
- [`src/config.hpp:43`](../src/config.hpp#L43) — `Unknown`
|
||||
- [`src/config.hpp:49`](../src/config.hpp#L49) — `Unknown`
|
||||
- [`src/gallery/embedder_stamp.cpp:1`](../src/gallery/embedder_stamp.cpp#L1) — `Unknown`
|
||||
- [`src/gallery/embedder_stamp.hpp:2`](../src/gallery/embedder_stamp.hpp#L2) — `Unknown`
|
||||
- [`src/gallery/gallery_builder.cpp:45`](../src/gallery/gallery_builder.cpp#L45) — `ActorGallery build_gallery(const BuildConfig& cfg)`
|
||||
@@ -383,7 +433,7 @@ _None._
|
||||
- [`src/gallery/gallery_store.cpp:219`](../src/gallery/gallery_store.cpp#L219) — `Unknown`
|
||||
- [`src/kpn_bindings.cpp:167`](../src/kpn_bindings.cpp#L167) — `Unknown`
|
||||
- [`src/kpn_bindings.cpp:217`](../src/kpn_bindings.cpp#L217) — `Unknown`
|
||||
- [`src/main.cpp:176`](../src/main.cpp#L176) — `Unknown`
|
||||
- [`src/main.cpp:174`](../src/main.cpp#L174) — `Unknown`
|
||||
- [`src/nodes/embedding_dump_node.hpp:30`](../src/nodes/embedding_dump_node.hpp#L30) — `static constexpr std::string_view label() { return "embedding_dump"; }`
|
||||
- [`src/nodes/embedding_dump_node.hpp:102`](../src/nodes/embedding_dump_node.hpp#L102) — `H5::StrType str(H5::PredType::C_S1, H5T_VARIABLE);`
|
||||
- [`src/scene_preview.cpp:133`](../src/scene_preview.cpp#L133) — `int main(int argc, char** argv)`
|
||||
@@ -423,21 +473,35 @@ _None._
|
||||
|
||||
### SR-002
|
||||
|
||||
**Locations:** 5
|
||||
**Locations:** 16
|
||||
|
||||
- [`src/config.hpp:103`](../src/config.hpp#L103) — `Unknown`
|
||||
- [`src/evidence_discount.hpp:2`](../src/evidence_discount.hpp#L2) — `Unknown`
|
||||
- [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `inline cv::Mat align_face(const cv::Mat& img,`
|
||||
- [`src/gallery/gallery_calibration.hpp:2`](../src/gallery/gallery_calibration.hpp#L2) — `Unknown`
|
||||
- [`src/gallery/gallery_calibration.hpp:53`](../src/gallery/gallery_calibration.hpp#L53) — `float boundary_at(float p = 0.5f, float log_prior_odds = 0.f) const`
|
||||
- [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));`
|
||||
- [`src/main.cpp:216`](../src/main.cpp#L216) — `reg_cfg, EvidenceDiscounter(same_person));`
|
||||
- [`src/nodes/face_detector_node.hpp:2`](../src/nodes/face_detector_node.hpp#L2) — `Unknown`
|
||||
- [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown`
|
||||
- [`src/nodes/identity_matcher_node.hpp:111`](../src/nodes/identity_matcher_node.hpp#L111) — `const GalleryCalibration& calibration() const { return cal_; }`
|
||||
- [`src/nodes/identity_matcher_node.hpp:119`](../src/nodes/identity_matcher_node.hpp#L119) — `const GalleryCalibration& calibration() const { return cal_; }`
|
||||
- [`src/nodes/identity_matcher_node.hpp:242`](../src/nodes/identity_matcher_node.hpp#L242) — `Unknown`
|
||||
- [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }`
|
||||
- [`src/nodes/result_sink_node.hpp:63`](../src/nodes/result_sink_node.hpp#L63) — `void set_pre_write_hook(std::function<void(double)> fn) { pre_write_ = std::move(fn); }`
|
||||
- [`src/nodes/result_sink_node.hpp:161`](../src/nodes/result_sink_node.hpp#L161) — `struct ActorMeta { std::string name, imdb_id, tmdb_id, jellyfin_id; };`
|
||||
- [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown`
|
||||
|
||||
### SR-003
|
||||
|
||||
**Locations:** 3
|
||||
**Locations:** 6
|
||||
|
||||
- [`src/audio_signature.cpp:3`](../src/audio_signature.cpp#L3) — `Unknown`
|
||||
- [`src/audio_signature.hpp:4`](../src/audio_signature.hpp#L4) — `Unknown`
|
||||
- [`src/config.hpp:20`](../src/config.hpp#L20) — `struct Config`
|
||||
- [`src/nodes/result_sink_node.hpp:2`](../src/nodes/result_sink_node.hpp#L2) — `Unknown`
|
||||
- [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }`
|
||||
- [`src/nodes/result_sink_node.hpp:122`](../src/nodes/result_sink_node.hpp#L122) — `void write_output()`
|
||||
|
||||
### SR-005
|
||||
|
||||
@@ -524,3 +588,9 @@ _None._
|
||||
- `scripts/sae_gallery.py:171` — {'ignored': ['— omitted entirely when unknown', 'so "unstamped"']}
|
||||
- `scripts/sae_gallery.py:199` — {'ignored': ['— carried through so a derived gallery (filter']}
|
||||
|
||||
**Groups mixing requirement types (pipe separates types):**
|
||||
|
||||
- `src/main.cpp:216` — {'group': ['AR-012', 'AR-016', 'IR-002', 'IR-003']}
|
||||
- `src/nodes/result_sink_node.hpp:49` — {'group': ['AR-012', 'AR-017', 'IR-002']}
|
||||
- `src/nodes/result_sink_node.hpp:161` — {'group': ['AR-012', 'IR-002']}
|
||||
|
||||
|
||||
Vendored
+1
-1
Submodule external/KPN updated: 4b6e498ba7...be6e92268c
Executable
+60
@@ -0,0 +1,60 @@
|
||||
#!/usr/bin/env bash
|
||||
# make_fixtures.sh — regenerate the committed replay fixtures.
|
||||
#
|
||||
# TRACES: VR-001 | PR-002
|
||||
#
|
||||
# CI never calls a model (see docs/requirements.md, "CI never calls a model"):
|
||||
# the embedder is impractical on the N100 CI host, so inference happens HERE, on
|
||||
# a machine with a GPU, and CI consumes the HDF5 dumps as data. Everything
|
||||
# downstream of embedding — tracking, presence windows, belief accumulation,
|
||||
# expansion — is cheap CPU maths and replays from these files.
|
||||
#
|
||||
# Reproducibility is a requirement, not a nicety. A fixture whose provenance is
|
||||
# unknown is worse than no fixture, because it will be trusted. Every parameter
|
||||
# that affects the output is pinned below rather than left to a default, and the
|
||||
# dumps carry the embedder identity and SHA-256 (GR-004) so a replay cannot be
|
||||
# silently scored against the wrong gallery.
|
||||
#
|
||||
# These are byte-reproducible only because node outputs block rather than drop
|
||||
# on a full channel (AR-004). Before that fix the same command produced
|
||||
# different dumps run to run, since what got dropped depended on timing.
|
||||
#
|
||||
# Source: bali/ — Road to Bali (1952), public domain. That matters: derived
|
||||
# fixtures can be committed, where anything cut from a copyrighted title could
|
||||
# not live in the repository at all.
|
||||
set -euo pipefail
|
||||
|
||||
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
CLIPS="${CLIPS:-$REPO/../bali}"
|
||||
GALLERY="${GALLERY:-$REPO/gallery_lvface.h5}"
|
||||
BIN="${BIN:-$REPO/build/scene_analyze}"
|
||||
OUT="$REPO/tests/fixtures/dumps"
|
||||
|
||||
# Pinned. Changing either invalidates every committed fixture.
|
||||
# fps 5 — 1 fps over a 77 s clip is 77 frames, too thin to exercise an
|
||||
# extinction window measured in tens of seconds.
|
||||
# min-face — 32 px, the VR-005 measured floor (98.1% TPI). The corpus is
|
||||
# 480x360, so a stricter value would reject most faces present.
|
||||
FPS=5
|
||||
MIN_FACE_PX=32
|
||||
|
||||
[[ -x "$BIN" ]] || { echo "no scene_analyze at $BIN (set BIN=)" >&2; exit 1; }
|
||||
[[ -f "$GALLERY" ]] || { echo "no gallery at $GALLERY (set GALLERY=)" >&2; exit 1; }
|
||||
[[ -d "$CLIPS" ]] || { echo "no clips at $CLIPS (set CLIPS=)" >&2; exit 1; }
|
||||
|
||||
mkdir -p "$OUT"
|
||||
|
||||
for clip in "$CLIPS"/Road_To_Bali-*.webm; do
|
||||
n="$(basename "$clip" .webm)"; n="${n##*-}"
|
||||
echo "── bali_$n"
|
||||
"$BIN" --movie "$clip" --gallery "$GALLERY" \
|
||||
--fps "$FPS" --min-face-px "$MIN_FACE_PX" \
|
||||
--dump-embeddings "$OUT/bali_$n.h5" \
|
||||
--output /dev/null 2>&1 | grep -E "wrote|dropped" || true
|
||||
done
|
||||
|
||||
echo
|
||||
echo "Regenerated in $OUT — verify the diff is empty if nothing upstream changed."
|
||||
echo "A non-empty diff means detection, alignment or embedding moved. That is"
|
||||
echo "either a regression or a deliberate change, and either way the golden"
|
||||
echo "outputs derived from these fixtures need reviewing."
|
||||
+25
-15
@@ -16,7 +16,13 @@ enum class Verbosity {
|
||||
struct Config {
|
||||
// ── Input ─────────────────────────────────────────────────────────────────
|
||||
std::string movie_path;
|
||||
std::string gallery_path; // gallery.json produced by build_gallery
|
||||
std::string gallery_path;
|
||||
// TRACES: IR-002 | SR-003
|
||||
// "global" (matched against the whole library) or "limited" (this title's
|
||||
// credited cast only). The strongest single quality signal when two
|
||||
// manifests compete for the same cut: identical gallery_size can mean very
|
||||
// different recall depending on which was used.
|
||||
std::string gallery_scope{"global"}; // gallery.json produced by build_gallery
|
||||
|
||||
// ── Output ───────────────────────────────────────────────────────────────
|
||||
std::string output_path; // annotations.json
|
||||
@@ -94,21 +100,25 @@ struct Config {
|
||||
float dense_scale{1.0f}; // dense-mode frame downscale (1 = off)
|
||||
|
||||
// ── Face tracking (frame-to-frame) ───────────────────────────────────────
|
||||
float track_alpha{0.4f}; // cost weight: 0=embedding only, 1=spatial only
|
||||
/// TRACES: AR-007, AR-008, AR-024 | SR-002
|
||||
// track_alpha is the *base* weight, used on ordinary frames. It is
|
||||
// frame-dependent (AR-007): on is_cut / is_scene_boundary, and for any track
|
||||
// that is no longer on screen, it drops to 0 (embedding only), because
|
||||
// position carries no information across a viewpoint change or a gap.
|
||||
float track_alpha{0.4f}; // base cost weight: 0=embedding only, 1=spatial only
|
||||
float track_min_iou{0.1f}; // IoU below which spatial link alone is rejected
|
||||
float track_max_embed_dist{0.7f}; // cosine dist above which embedding link alone is rejected
|
||||
int track_max_frames_missing{5}; // expire track after N consecutive missed frames
|
||||
|
||||
// ── Cross-cut track re-association ────────────────────────────────────────
|
||||
// A camera-angle change (Frame::is_cut) breaks spatial (IoU) continuity but
|
||||
// not identity: the same people are usually still on screen from a new angle.
|
||||
// Instead of destroying tracks on a cut, the tracker parks them in an
|
||||
// inactive pool. A post-cut detection whose raw cosine similarity to a parked
|
||||
// track's last-frame embedding is ≥ cut_revive_sim revives that track_id
|
||||
// (identity continuity survives the cut); otherwise it starts a fresh track.
|
||||
// Parked tracks that go unrevived for cut_inactive_max_frames are dropped.
|
||||
float cut_revive_sim{0.50f}; // min raw cosine sim (last-frame emb) to revive across a cut
|
||||
int cut_inactive_max_frames{5}; // drop a parked track after N frames without revival
|
||||
// Minimum P(same person) for an association to be admissible on appearance
|
||||
// alone. This replaces track_max_embed_dist (a raw cosine distance, AR-024).
|
||||
// 0.5 is not a tuned constant: it is the decision boundary. Below it the pair
|
||||
// is more likely two people than one, and no amount of IoU makes that a link
|
||||
// worth asserting on identity grounds.
|
||||
float track_assoc_min_prob{0.5f};
|
||||
// How long a track that has gone off screen stays available for association
|
||||
// before the registry reaps it and emits its presence claim (AR-013).
|
||||
// Replaces track_max_frames_missing: a frame count silently changed meaning
|
||||
// with sample_fps, and the same number had to be guessed twice (once for an
|
||||
// ordinary miss, once for a cut). Seconds mean one thing at any sample rate.
|
||||
double track_extinction_sec{5.0};
|
||||
|
||||
// ── Scene tracking ────────────────────────────────────────────────────────
|
||||
// extinction_sec re-tuned by DE against X-Ray per-second presence, 4-film rep4
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
#include <cstdlib>
|
||||
#include <cstring>
|
||||
#include <fstream>
|
||||
#include <functional>
|
||||
#include <iostream>
|
||||
#include <stdexcept>
|
||||
#include <string>
|
||||
@@ -49,6 +50,33 @@ struct GalleryCalibration {
|
||||
}
|
||||
};
|
||||
|
||||
/// TRACES: AR-023, AR-024 | SR-002
|
||||
///
|
||||
/// cosine → P(same person). The one probability space the pipeline reasons in.
|
||||
///
|
||||
/// Handed to every stage that has to decide whether two embeddings are the same
|
||||
/// person — track association (AR-007), evidence discounting (AR-025), identity
|
||||
/// matching — so a threshold of 0.5 means the same thing in all of them. A stage
|
||||
/// that thresholded a raw cosine instead would be using a number that means
|
||||
/// something different for every model, gallery and face size (AR-024).
|
||||
///
|
||||
/// **No prior term.** `log_prior_odds` adjusts for the gallery's base rate, which
|
||||
/// is a question about *which of N actors*; association asks whether two faces
|
||||
/// are one person, where the balanced fit is the right answer. Passing the
|
||||
/// matcher's prior here would silently bias tracking by the size of the cast.
|
||||
inline std::function<float(float)> same_person_probability(const GalleryCalibration& cal) {
|
||||
if (!cal.valid) {
|
||||
// Loud, because the failure mode is invisible: an untuned sigmoid still
|
||||
// returns plausible probabilities, and every threshold downstream of it
|
||||
// is then a guess wearing a calibrated number's clothes.
|
||||
std::cerr << "[calibration] WARNING: no fitted calibration — association and "
|
||||
"evidence weighting fall back to the untuned default sigmoid "
|
||||
"(a=" << cal.a << ", b=" << cal.b << "). Probabilities are "
|
||||
"not meaningful for this embedder.\n";
|
||||
}
|
||||
return [cal](float similarity) { return cal.probability(similarity); };
|
||||
}
|
||||
|
||||
// Fit a logistic sigmoid to gallery pair similarities.
|
||||
// Positive pairs: same actor, different reference images.
|
||||
// Negative pairs: different actors (all cross-actor embedding pairs).
|
||||
|
||||
+46
-7
@@ -128,10 +128,8 @@ static Config parse_args(int argc, char** argv) {
|
||||
else if (arg("--ratio-ceil")) cfg.match_ratio_ceil = std::stof(next());
|
||||
else if (arg("--track-alpha")) cfg.track_alpha = std::stof(next());
|
||||
else if (arg("--track-min-iou")) cfg.track_min_iou = std::stof(next());
|
||||
else if (arg("--track-max-embed")) cfg.track_max_embed_dist = std::stof(next());
|
||||
else if (arg("--track-max-missing")) cfg.track_max_frames_missing = std::stoi(next());
|
||||
else if (arg("--cut-revive-sim")) cfg.cut_revive_sim = std::stof(next());
|
||||
else if (arg("--cut-inactive-max")) cfg.cut_inactive_max_frames = std::stoi(next());
|
||||
else if (arg("--track-min-prob")) cfg.track_assoc_min_prob = std::stof(next());
|
||||
else if (arg("--track-extinction")) cfg.track_extinction_sec = std::stod(next());
|
||||
else if (arg("--anneal")) cfg.anneal_sec = std::stod(next());
|
||||
else if (arg("--expand-gallery")) cfg.expand_gallery = true;
|
||||
else if (arg("--expand-buffer")) cfg.expand_buffer_size = std::stoi(next());
|
||||
@@ -194,10 +192,36 @@ int main(int argc, char** argv) {
|
||||
FaceDetectorFunc detector_fn{cfg};
|
||||
FaceAlignerFunc aligner_fn;
|
||||
EmbedderFunc embedder_fn{cfg};
|
||||
FaceTrackerFunc ftracker_fn{cfg};
|
||||
// Constructed before the tracker: it fits (or loads) the calibration, and
|
||||
// the tracker must decide in that same probability space (AR-024).
|
||||
IdentityMatcherFunc matcher_fn {gallery, cfg};
|
||||
|
||||
/// TRACES: AR-007, AR-008, AR-012, AR-024 | SR-002
|
||||
// The registry is created here and shared, not owned by a node: track state
|
||||
// is not a stage in the stream, it is state several stages read and write,
|
||||
// and its final answer is only known when a track dies.
|
||||
auto same_person = same_person_probability(matcher_fn.calibration());
|
||||
TrackRegistry::Config reg_cfg;
|
||||
reg_cfg.extinction_sec = cfg.track_extinction_sec;
|
||||
auto registry = std::make_shared<TrackRegistry>(
|
||||
reg_cfg, EvidenceDiscounter(same_person));
|
||||
|
||||
matcher_fn.set_registry(registry);
|
||||
|
||||
|
||||
FaceTrackerFunc ftracker_fn{cfg, registry, same_person};
|
||||
SceneTrackerFunc tracker_fn {cfg};
|
||||
ResultSinkFunc sink_fn {cfg, done};
|
||||
|
||||
/// TRACES: AR-012, AR-016, IR-002, IR-003 | SR-002
|
||||
// A reaped track goes straight to the aggregator, so the registry holds only
|
||||
// live tracks and its size is bounded by concurrent on-screen faces rather
|
||||
// than growing with the film.
|
||||
registry->on_track_dead([&sink_fn](const DeadTrack& d) { sink_fn.add_claim(d); });
|
||||
// AR-016: a film ends with faces on screen and those tracks have not timed
|
||||
// out. Without this flush the closing scene's cast is silently never
|
||||
// emitted — a loss that reads as a recognition miss, not a bookkeeping bug.
|
||||
sink_fn.set_pre_write_hook([registry](double last_ts) { registry->flush(last_ts); });
|
||||
#ifdef SAE_DEBUG
|
||||
DebugRendererFunc debug_fn {cfg};
|
||||
#endif
|
||||
@@ -258,15 +282,30 @@ int main(int argc, char** argv) {
|
||||
net.stop();
|
||||
net.print_diagnostics();
|
||||
|
||||
/// TRACES: AR-004 | SR-002
|
||||
// A dropped frame does not degrade a result, it silently changes one —
|
||||
// the output is a claim about footage that was never analysed, and
|
||||
// nothing in the file says so. Since AR-004 made data pushes block, a
|
||||
// drop can no longer happen on the data path, so any drop here means
|
||||
// either that fix regressed (it lives in the KPN submodule, one line,
|
||||
// easy to lose in an update) or a channel was disabled mid-run.
|
||||
//
|
||||
// Reporting it in a footer and exiting 0 made both invisible: the run
|
||||
// "succeeded" and the truth file looked complete. Fail instead.
|
||||
bool dropped = false;
|
||||
{
|
||||
std::lock_guard<std::mutex> lk(event_mtx);
|
||||
if (!overflow_counts.empty()) {
|
||||
std::cerr << "[main] dropped frames (channel overflow):\n";
|
||||
dropped = true;
|
||||
std::cerr << "[main] ERROR: frames were dropped (channel overflow):\n";
|
||||
for (const auto& [name, count] : overflow_counts)
|
||||
std::cerr << " " << name << ": " << count << "\n";
|
||||
std::cerr << "[main] The output would describe footage that was never "
|
||||
"analysed. Refusing to report success.\n";
|
||||
}
|
||||
}
|
||||
return node_crashed.load(std::memory_order_acquire) ? 1 : 0;
|
||||
if (node_crashed.load(std::memory_order_acquire)) return 1;
|
||||
return dropped ? 2 : 0;
|
||||
};
|
||||
|
||||
// ── Build static network and run ──────────────────────────────────────────
|
||||
|
||||
+169
-162
@@ -1,102 +1,124 @@
|
||||
#pragma once
|
||||
/// TRACES: AR-007, AR-008, AR-024 | SR-002
|
||||
///
|
||||
/// FaceTrackerFunc — KPN node that links face detections into tracks.
|
||||
///
|
||||
/// **The registry is the tracker's state.** The node owns no track map of its
|
||||
/// own: it drives `TrackRegistry` through a `FrameScope` and reads the same
|
||||
/// `Track` objects everything else reads. Two parallel copies could disagree,
|
||||
/// and every divergence would surface as a wrong presence window rather than as
|
||||
/// a crash — silently, and only in the output.
|
||||
///
|
||||
/// **One candidate pool** (AR-008). `last_seen` alone distinguishes a track that
|
||||
/// is on screen from one that is dormant, and it only affects whether IoU means
|
||||
/// anything. There is no parked pool and no revival branch: re-associating a
|
||||
/// track whose face was lost — across a cut or not — is ordinary inter-frame
|
||||
/// association, and it falls out of the embedding comparison already being done.
|
||||
///
|
||||
/// Assignment cost (track i, detection j):
|
||||
///
|
||||
/// p = P(same person | cosine(track mean, detection)) ← calibrated
|
||||
/// alpha = base weight, or 0 when position carries no information
|
||||
/// cost = alpha·(1 − IoU) + (1 − alpha)·(1 − p)
|
||||
///
|
||||
/// gated to INF unless the pair is admissible on position *or* on identity.
|
||||
///
|
||||
/// **alpha is frame- and track-dependent** (AR-007). It falls to 0 —
|
||||
/// embedding only — when either:
|
||||
/// - the frame is flagged `is_cut` / `is_scene_boundary`: the viewpoint
|
||||
/// changed, so the same person is at a new position; or
|
||||
/// - the track is dormant (`last_seen` set): time has passed since its box was
|
||||
/// last observed, so that box is stale regardless of cuts.
|
||||
/// Both are the same statement — spatial continuity is broken — arrived at from
|
||||
/// two directions, which is why they collapse into one rule rather than two
|
||||
/// branches.
|
||||
///
|
||||
/// **Everything is thresholded in probability space** (AR-024). The cosine goes
|
||||
/// through the calibration before it is compared to anything; the raw-cosine
|
||||
/// constants `track_max_embed_dist` and `cut_revive_sim` are retired.
|
||||
|
||||
#include "types.hpp"
|
||||
#include "config.hpp"
|
||||
#include "track_registry.hpp"
|
||||
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
#include <functional>
|
||||
#include <iostream>
|
||||
#include <limits>
|
||||
#include <map>
|
||||
#include <memory>
|
||||
#include <stdexcept>
|
||||
#include <string_view>
|
||||
#include <vector>
|
||||
|
||||
// ── FaceTrackerFunc ───────────────────────────────────────────────────────────
|
||||
// KPN node: links face detections across consecutive frames using the Hungarian
|
||||
// algorithm on a combined spatial (IoU) + embedding (cosine distance) cost.
|
||||
//
|
||||
// Each track accumulates a running directional mean of its ArcFace embeddings
|
||||
// (averaged then re-normalised to the unit sphere), used as the embedding side
|
||||
// of the assignment cost below for more stable track continuity.
|
||||
//
|
||||
// Assignment cost (track i, detection j):
|
||||
// cost = alpha * (1 - IoU) + (1-alpha) * min(cosine_dist/2, 1)
|
||||
// Gated to INF when IoU < min_iou AND cosine_dist > max_embed_dist.
|
||||
//
|
||||
// Unmatched tracks have their frames_missing counter incremented; they are
|
||||
// expired once frames_missing > max_frames_missing.
|
||||
//
|
||||
// Cross-cut re-association. A camera-angle change (Frame::is_cut, set by
|
||||
// camera_position_change_detector) destroys spatial (IoU) continuity — the same
|
||||
// person reappears at a new position — but not identity. On a cut the tracker
|
||||
// does NOT discard its tracks; it parks them in an inactive pool keyed by their
|
||||
// last-frame raw embedding. A post-cut detection whose raw cosine similarity to
|
||||
// a parked track's last-frame embedding is ≥ cut_revive_sim revives that track:
|
||||
// the original track_id, mean embedding and n_frames are restored (only the bbox
|
||||
// jumps to the new detection), so identity continuity survives the cut. Parked
|
||||
// tracks left unrevived for cut_inactive_max_frames are finally dropped.
|
||||
|
||||
struct FaceTrackerFunc {
|
||||
static constexpr std::string_view label() { return "face_tracker"; }
|
||||
|
||||
struct TrackState {
|
||||
cv::Rect2f bbox;
|
||||
Embedding mean_emb{};
|
||||
Embedding last_emb{}; // raw embedding of the most recent matched frame
|
||||
int n_frames{0};
|
||||
int frames_missing{0};
|
||||
};
|
||||
/// cosine similarity → P(same person). Supplied by the caller so the fit
|
||||
/// belonging to the active embedder is used (AR-023/AR-024) — the same
|
||||
/// pattern, and normally the same function object, as
|
||||
/// `EvidenceDiscounter::Calibrate`.
|
||||
using Calibrate = std::function<float(float)>;
|
||||
|
||||
explicit FaceTrackerFunc(const Config& cfg)
|
||||
: alpha_(cfg.track_alpha)
|
||||
/// The registry is a constructor argument, not an option: a tracker without
|
||||
/// one would have to keep its own tracks, which is the defect this replaces.
|
||||
FaceTrackerFunc(const Config& cfg,
|
||||
std::shared_ptr<TrackRegistry> registry,
|
||||
Calibrate calibrate)
|
||||
: registry_(std::move(registry))
|
||||
, calibrate_(std::move(calibrate))
|
||||
, alpha_base_(cfg.track_alpha)
|
||||
, min_iou_(cfg.track_min_iou)
|
||||
, max_embed_dist_(cfg.track_max_embed_dist)
|
||||
, max_missing_(cfg.track_max_frames_missing)
|
||||
, revive_sim_(cfg.cut_revive_sim)
|
||||
, inactive_max_(cfg.cut_inactive_max_frames)
|
||||
, min_assoc_prob_(cfg.track_assoc_min_prob)
|
||||
{
|
||||
std::cerr << "[face_tracker] alpha=" << alpha_
|
||||
if (!registry_)
|
||||
throw std::invalid_argument("face_tracker: registry must not be null");
|
||||
if (!calibrate_)
|
||||
throw std::invalid_argument("face_tracker: a calibration is required — "
|
||||
"association is decided in probability space");
|
||||
|
||||
std::cerr << "[face_tracker] alpha_base=" << alpha_base_
|
||||
<< " min_iou=" << min_iou_
|
||||
<< " max_embed_dist=" << max_embed_dist_
|
||||
<< " max_missing=" << max_missing_
|
||||
<< " cut_revive_sim=" << revive_sim_
|
||||
<< " cut_inactive_max=" << inactive_max_ << "\n";
|
||||
<< " min_assoc_prob=" << min_assoc_prob_ << "\n";
|
||||
}
|
||||
|
||||
TrackedSceneFrame operator()(EmbeddedSceneFrame ef) {
|
||||
if (ef.source.eof) {
|
||||
tracks_.clear();
|
||||
inactive_.clear();
|
||||
// Deliberately does *not* flush the registry. The identity matcher
|
||||
// runs downstream and its votes for the final frames are still in
|
||||
// flight; reaping here would drop them (they would land on ids that
|
||||
// no longer exist and show up as dropped_votes). AR-016's flush
|
||||
// belongs at the pipeline's termination point, after the last vote.
|
||||
boxes_.clear();
|
||||
TrackedSceneFrame out;
|
||||
out.source = std::move(ef.source);
|
||||
return out;
|
||||
}
|
||||
|
||||
const int n_det = static_cast<int>(ef.embeddings.size());
|
||||
const double t = ef.source.timestamp_sec;
|
||||
const int n_det = static_cast<int>(ef.embeddings.size());
|
||||
|
||||
// Camera-angle change: park active tracks instead of destroying them so
|
||||
// they can be revived by identity (raw last-frame embedding cosine) once
|
||||
// the same people reappear from the new angle.
|
||||
if (ef.source.is_cut && !tracks_.empty()) {
|
||||
std::cerr << "[face_tracker] cut — parking " << tracks_.size()
|
||||
<< " track(s) into inactive pool\n";
|
||||
for (auto& [tid, ts] : tracks_) {
|
||||
ts.frames_missing = 0; // repurpose as time-since-parked counter
|
||||
inactive_[tid] = std::move(ts);
|
||||
}
|
||||
tracks_.clear();
|
||||
}
|
||||
// Unconditional: the clock must advance on frames with no detections
|
||||
// too, or a track only dies when some unrelated face happens to appear
|
||||
// and a film that ends mid-track never closes it (AR-013).
|
||||
auto scope = registry_->begin_frame(t);
|
||||
|
||||
// Age the inactive pool every frame and drop tracks parked too long.
|
||||
for (auto it = inactive_.begin(); it != inactive_.end(); ) {
|
||||
it->second.frames_missing++;
|
||||
it = (it->second.frames_missing > inactive_max_)
|
||||
? inactive_.erase(it) : std::next(it);
|
||||
}
|
||||
// One pool (AR-008) — on-screen and dormant tracks compete together.
|
||||
std::vector<Track*> cands = scope.candidates();
|
||||
const int n_trk = static_cast<int>(cands.size());
|
||||
|
||||
// Snapshot active track IDs so the map can be modified safely below
|
||||
std::vector<int> tids;
|
||||
tids.reserve(tracks_.size());
|
||||
for (auto& [tid, _] : tracks_) tids.push_back(tid);
|
||||
const int n_trk = static_cast<int>(tids.size());
|
||||
prune_boxes(cands);
|
||||
std::vector<Spatial*> sp(n_trk);
|
||||
for (int ti = 0; ti < n_trk; ++ti)
|
||||
sp[ti] = &boxes_.try_emplace(cands[ti]->id, Spatial{{}, t, false})
|
||||
.first->second;
|
||||
|
||||
// AR-007 — the frame half of the frame-dependent weighting. Both flags
|
||||
// say the same thing to the tracker: whatever was at that position is
|
||||
// not there any more.
|
||||
const bool viewpoint_change =
|
||||
ef.source.is_cut || ef.source.is_scene_boundary;
|
||||
|
||||
// ── Cost matrix [n_trk × n_det] ──────────────────────────────────────
|
||||
constexpr float INF_COST = 1e6f;
|
||||
@@ -104,109 +126,108 @@ struct FaceTrackerFunc {
|
||||
std::vector<float>(n_det, INF_COST));
|
||||
|
||||
for (int ti = 0; ti < n_trk; ++ti) {
|
||||
const TrackState& ts = tracks_[tids[ti]];
|
||||
// Spatial continuity holds only for a track that was on screen, whose
|
||||
// box we have actually observed, on a frame that did not change the
|
||||
// viewpoint. Otherwise the box is stale and IoU is noise.
|
||||
const bool spatial_meaningful =
|
||||
sp[ti]->observed && cands[ti]->on_screen() && !viewpoint_change;
|
||||
const float alpha = spatial_meaningful ? alpha_base_ : 0.f;
|
||||
|
||||
for (int di = 0; di < n_det; ++di) {
|
||||
float iou_v = iou(ts.bbox, ef.faces[di].bbox);
|
||||
float emb_d = (ts.n_frames > 0)
|
||||
? 1.f - cosine_similarity(ts.mean_emb, ef.embeddings[di])
|
||||
: 1.f;
|
||||
if (iou_v < min_iou_ && emb_d > max_embed_dist_) continue;
|
||||
float s = 1.f - iou_v;
|
||||
float e = std::min(emb_d * 0.5f, 1.f);
|
||||
cost[ti][di] = alpha_ * s + (1.f - alpha_) * e;
|
||||
// AR-024 — the cosine is converted before it is used for
|
||||
// anything, including the gate below.
|
||||
const float p = calibrate_(
|
||||
cosine_similarity(cands[ti]->mean, ef.embeddings[di]));
|
||||
const float iou_v = spatial_meaningful
|
||||
? iou(sp[ti]->bbox, ef.faces[di].bbox) : 0.f;
|
||||
|
||||
// Either signal on its own can admit a link: a face that moved a
|
||||
// little but whose embedding degraded (blur, profile turn) is
|
||||
// still linkable on position, and a face that jumped across the
|
||||
// frame is still linkable on identity. Neither ⇒ no link.
|
||||
const bool spatial_ok = spatial_meaningful && iou_v >= min_iou_;
|
||||
const bool identity_ok = p >= min_assoc_prob_;
|
||||
if (!spatial_ok && !identity_ok) continue;
|
||||
|
||||
cost[ti][di] = alpha * (1.f - iou_v) + (1.f - alpha) * (1.f - p);
|
||||
}
|
||||
}
|
||||
|
||||
// ── Hungarian assignment ──────────────────────────────────────────────
|
||||
// ── Hungarian assignment ─────────────────────────────────────────────
|
||||
std::vector<int> assign(n_trk, -1);
|
||||
if (n_trk > 0 && n_det > 0)
|
||||
assign = hungarian(cost, n_trk, n_det);
|
||||
|
||||
// ── Build output frame ────────────────────────────────────────────────
|
||||
// ── Build output frame ───────────────────────────────────────────────
|
||||
TrackedSceneFrame out;
|
||||
out.source = ef.source;
|
||||
out.faces = ef.faces;
|
||||
out.crops = ef.crops;
|
||||
out.embeddings = ef.embeddings;
|
||||
out.source = ef.source;
|
||||
out.faces = ef.faces;
|
||||
out.crops = ef.crops;
|
||||
out.embeddings = ef.embeddings;
|
||||
out.track_ids.assign(n_det, -1);
|
||||
|
||||
std::vector<bool> det_matched(n_det, false);
|
||||
|
||||
// Update matched tracks
|
||||
for (int ti = 0; ti < n_trk; ++ti) {
|
||||
int di = assign[ti];
|
||||
bool valid = (di >= 0 && di < n_det && cost[ti][di] < INF_COST * 0.5f);
|
||||
TrackState& ts = tracks_[tids[ti]];
|
||||
const int di = assign[ti];
|
||||
const int id = cands[ti]->id;
|
||||
const bool valid =
|
||||
(di >= 0 && di < n_det && cost[ti][di] < INF_COST * 0.5f);
|
||||
|
||||
if (!valid) {
|
||||
ts.frames_missing++;
|
||||
// Only a track that *was* on screen can become lost, and it
|
||||
// becomes lost as of its last sighting, never as of now — the
|
||||
// gap after the final sighting is never claimed (AR-013). A
|
||||
// track already dormant is left alone so its extinction clock
|
||||
// keeps running from the right instant.
|
||||
if (cands[ti]->on_screen()) scope.mark_lost(id, sp[ti]->last_ts);
|
||||
continue;
|
||||
}
|
||||
update_mean(ts.mean_emb, ts.n_frames, ef.embeddings[di]);
|
||||
ts.last_emb = ef.embeddings[di];
|
||||
ts.bbox = ef.faces[di].bbox;
|
||||
ts.n_frames++;
|
||||
ts.frames_missing = 0;
|
||||
det_matched[di] = true;
|
||||
|
||||
out.track_ids[di] = tids[ti];
|
||||
scope.mark_seen(id, t, ef.embeddings[di]);
|
||||
sp[ti]->bbox = ef.faces[di].bbox;
|
||||
sp[ti]->last_ts = t;
|
||||
sp[ti]->observed = true;
|
||||
det_matched[di] = true;
|
||||
out.track_ids[di] = id;
|
||||
}
|
||||
|
||||
// Handle unmatched detections: first try to revive a parked track by
|
||||
// identity (raw last-frame embedding cosine), else start a fresh track.
|
||||
for (int di = 0; di < n_det; ++di) {
|
||||
if (det_matched[di]) continue;
|
||||
|
||||
int tid = revive_from_inactive(ef.embeddings[di]);
|
||||
if (tid >= 0) {
|
||||
// Restore the parked track: keep its identity statistics
|
||||
// (mean_emb, n_frames), jump the bbox to the new detection.
|
||||
TrackState ts = std::move(inactive_[tid]);
|
||||
inactive_.erase(tid);
|
||||
update_mean(ts.mean_emb, ts.n_frames, ef.embeddings[di]);
|
||||
ts.last_emb = ef.embeddings[di];
|
||||
ts.bbox = ef.faces[di].bbox;
|
||||
ts.n_frames++;
|
||||
ts.frames_missing = 0;
|
||||
tracks_[tid] = std::move(ts);
|
||||
out.track_ids[di] = tid;
|
||||
std::cerr << "[face_tracker] revived track " << tid
|
||||
<< " across cut\n";
|
||||
continue;
|
||||
}
|
||||
|
||||
tid = next_id_++;
|
||||
TrackState ts;
|
||||
ts.bbox = ef.faces[di].bbox;
|
||||
ts.mean_emb = ef.embeddings[di];
|
||||
ts.last_emb = ef.embeddings[di];
|
||||
ts.n_frames = 1;
|
||||
tracks_[tid] = ts;
|
||||
out.track_ids[di] = tid;
|
||||
}
|
||||
|
||||
// Expire stale tracks
|
||||
for (auto it = tracks_.begin(); it != tracks_.end(); ) {
|
||||
it = (it->second.frames_missing > max_missing_)
|
||||
? tracks_.erase(it) : std::next(it);
|
||||
const int id = scope.create(t, ef.embeddings[di]);
|
||||
boxes_[id] = Spatial{ef.faces[di].bbox, t, true};
|
||||
out.track_ids[di] = id;
|
||||
}
|
||||
|
||||
// No reaping here: begin_frame's tick owns the extinction sweep, so
|
||||
// there is exactly one place a track can die.
|
||||
return out;
|
||||
}
|
||||
|
||||
private:
|
||||
// Pick the parked track whose last-frame embedding is most similar to emb,
|
||||
// returning its id if that raw cosine similarity clears revive_sim_, else -1.
|
||||
// The caller removes the returned track from the pool, so a later detection in
|
||||
// the same frame cannot claim it again.
|
||||
int revive_from_inactive(const Embedding& emb) const {
|
||||
int best_tid = -1;
|
||||
float best_sim = revive_sim_; // threshold is the bar to beat (inclusive)
|
||||
for (const auto& [tid, ts] : inactive_) {
|
||||
float sim = cosine_similarity(ts.last_emb, emb);
|
||||
if (sim >= best_sim) { best_sim = sim; best_tid = tid; }
|
||||
// subsequent ties keep the later id; harmless, all clear the threshold
|
||||
// ── Spatial annotation ───────────────────────────────────────────────────
|
||||
// The one piece of per-track state the registry does not hold, because it is
|
||||
// not about presence: where the face was, and when it was last seen there.
|
||||
// Keyed by registry track id and pruned against `candidates()` every frame,
|
||||
// so it cannot outlive or contradict the registry — it annotates the pool
|
||||
// rather than duplicating it.
|
||||
struct Spatial {
|
||||
cv::Rect2f bbox{};
|
||||
double last_ts{0.0}; ///< timestamp of the last frame this track matched
|
||||
bool observed{false}; ///< false until a detection has been assigned
|
||||
};
|
||||
|
||||
// Drop boxes for ids the registry no longer has. `candidates()` is the
|
||||
// authority on what exists; anything else is a leak (and, for a reused id,
|
||||
// would be a stale box attached to a different person).
|
||||
void prune_boxes(const std::vector<Track*>& cands) {
|
||||
if (boxes_.size() == cands.size()) return; // common case: nothing died
|
||||
std::map<int, Spatial> kept;
|
||||
for (const Track* t : cands) {
|
||||
auto it = boxes_.find(t->id);
|
||||
if (it != boxes_.end()) kept.emplace(t->id, it->second);
|
||||
}
|
||||
return best_tid;
|
||||
boxes_.swap(kept);
|
||||
}
|
||||
|
||||
// IoU of two axis-aligned bounding boxes
|
||||
@@ -220,17 +241,6 @@ private:
|
||||
return inter / (a.width * a.height + b.width * b.height - inter);
|
||||
}
|
||||
|
||||
// Online directional mean: average then re-normalise to unit sphere
|
||||
static void update_mean(Embedding& mean, int n_prev, const Embedding& emb) {
|
||||
float norm_sq = 0.f;
|
||||
for (int k = 0; k < 512; ++k) {
|
||||
mean[k] = (mean[k] * n_prev + emb[k]) / (n_prev + 1);
|
||||
norm_sq += mean[k] * mean[k];
|
||||
}
|
||||
float inv = 1.f / std::sqrt(norm_sq);
|
||||
for (int k = 0; k < 512; ++k) mean[k] *= inv;
|
||||
}
|
||||
|
||||
// O(n³) potential-based Hungarian algorithm (Jonker-Volgenant / Kuhn-Munkres).
|
||||
// Returns assign[row] = col (0-indexed), or -1 when row is matched to a
|
||||
// padded virtual column (i.e., unmatched). Rectangular matrices are padded
|
||||
@@ -292,13 +302,10 @@ private:
|
||||
return ans;
|
||||
}
|
||||
|
||||
std::map<int, TrackState> tracks_;
|
||||
std::map<int, TrackState> inactive_; // parked across a cut, keyed by track id
|
||||
int next_id_{0};
|
||||
float alpha_;
|
||||
std::shared_ptr<TrackRegistry> registry_;
|
||||
Calibrate calibrate_;
|
||||
std::map<int, Spatial> boxes_; ///< track id → where it was, when
|
||||
float alpha_base_;
|
||||
float min_iou_;
|
||||
float max_embed_dist_;
|
||||
int max_missing_;
|
||||
float revive_sim_;
|
||||
int inactive_max_;
|
||||
float min_assoc_prob_;
|
||||
};
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
#include "gallery/gallery_store.hpp"
|
||||
#include "gallery/gallery_calibration.hpp"
|
||||
#include "gallery/track_gallery.hpp"
|
||||
#include "track_registry.hpp"
|
||||
|
||||
#include <cstdint>
|
||||
#include <cstring>
|
||||
@@ -107,6 +108,20 @@ struct IdentityMatcherFunc {
|
||||
sim_engine_ = make_similarity_engine(host_gallery.data(), n_gallery_, kMaxFaces);
|
||||
}
|
||||
|
||||
/// TRACES: AR-023, AR-024 | SR-002
|
||||
/// The fitted sigmoid. Exposed because the matcher is where it gets fitted
|
||||
/// (and cached back to the gallery), but it is not the matcher's private
|
||||
/// property: track association and evidence weighting must threshold in the
|
||||
/// *same* probability space, or a "0.5" in one stage and a "0.5" in another
|
||||
/// mean different things. See `same_person_probability`.
|
||||
const GalleryCalibration& calibration() const { return cal_; }
|
||||
|
||||
/// TRACES: AR-012, AR-025 | SR-002
|
||||
/// Where per-frame identity evidence reaches the registry. Optional: with no
|
||||
/// registry attached the matcher behaves exactly as before, which keeps the
|
||||
/// replay harness and the unit tests working unchanged.
|
||||
void set_registry(std::shared_ptr<TrackRegistry> r) { registry_ = std::move(r); }
|
||||
|
||||
// Runtime setter — lets a persistent pipeline be reused across a threshold sweep
|
||||
// without rebuilding the (expensive, gallery-resident) matcher. The gallery,
|
||||
// calibration and GPU sim-engine stay put; only the accept threshold changes.
|
||||
@@ -224,6 +239,19 @@ struct IdentityMatcherFunc {
|
||||
// face (annex already folded in above); the track's diversity buffer
|
||||
// keeps the gallery-far views and promotes them once the track is
|
||||
// confirmed. No-op unless --expand-gallery is set.
|
||||
// TRACES: AR-012, AR-025 | SR-002
|
||||
// Every scored face is evidence, not only the accepted ones: a run of
|
||||
// near-misses for one actor is itself informative, and discarding it
|
||||
// would make ownership depend on a per-frame threshold the redesign
|
||||
// exists to stop relying on. The registry discounts for correlation
|
||||
// and decides ownership from the accumulated posterior (AR-025).
|
||||
if (registry_ && best_actor >= 0 && tf.track_ids[fi] >= 0) {
|
||||
const float p = cal_.valid
|
||||
? cal_.probability(best_s, log_prior_odds_)
|
||||
: std::max(0.f, best_s);
|
||||
registry_->observe(tf.track_ids[fi], best_actor, p, tf.embeddings[fi]);
|
||||
}
|
||||
|
||||
track_gallery_.observe(tf.track_ids[fi], tf.embeddings[fi],
|
||||
best_actor, best_s, accept, tf.crops[fi]);
|
||||
|
||||
@@ -247,4 +275,5 @@ private:
|
||||
|
||||
std::unique_ptr<ISimilarityEngine> sim_engine_;
|
||||
TrackGallery track_gallery_;
|
||||
std::shared_ptr<TrackRegistry> registry_;
|
||||
};
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
/// TRACES: IR-001 | SR-003
|
||||
#include "types.hpp"
|
||||
#include "config.hpp"
|
||||
#include "track_registry.hpp"
|
||||
|
||||
#include <nlohmann/json.hpp>
|
||||
#include <algorithm>
|
||||
@@ -10,6 +11,8 @@
|
||||
#include <fstream>
|
||||
#include <iostream>
|
||||
#include <map>
|
||||
#include <mutex>
|
||||
#include <functional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
@@ -43,10 +46,26 @@ using json = nlohmann::json;
|
||||
struct ResultSinkFunc {
|
||||
static constexpr std::string_view label() { return "result_sink"; }
|
||||
|
||||
/// TRACES: AR-012, AR-017, IR-002 | SR-002, SR-003
|
||||
/// A finished presence claim from the registry. Called from inside the
|
||||
/// registry's reap while it holds its own lock, so this must stay a cheap
|
||||
/// push and must never re-enter the registry.
|
||||
void add_claim(const DeadTrack& d) {
|
||||
if (d.actor_idx < 0) return; // never owned: nothing to claim
|
||||
std::lock_guard<std::mutex> g(claims_mu_);
|
||||
claims_.push_back(d);
|
||||
}
|
||||
|
||||
ResultSinkFunc(const Config& cfg, std::atomic<bool>& done)
|
||||
: cfg_(cfg), done_(done)
|
||||
{}
|
||||
|
||||
/// TRACES: AR-016 | SR-002
|
||||
/// Runs immediately before the output is written, with the last timestamp
|
||||
/// seen. Used to flush tracks still live at EOF, which have not timed out
|
||||
/// and would otherwise never be emitted.
|
||||
void set_pre_write_hook(std::function<void(double)> fn) { pre_write_ = std::move(fn); }
|
||||
|
||||
void operator()(SceneAnnotation sa) {
|
||||
if (sa.eof) {
|
||||
flush();
|
||||
@@ -59,12 +78,20 @@ struct ResultSinkFunc {
|
||||
<< " unknowns=" << count_unknown(sa.visible_actors)
|
||||
<< std::flush;
|
||||
|
||||
for (const auto& ia : sa.visible_actors) {
|
||||
if (ia.actor_idx < 0) continue;
|
||||
auto& m = actor_meta_[ia.actor_idx];
|
||||
if (m.name.empty())
|
||||
m = {ia.name, ia.imdb_id, ia.tmdb_id, ia.jellyfin_id};
|
||||
}
|
||||
last_ts_ = sa.timestamp_sec;
|
||||
frames_.push_back(std::move(sa));
|
||||
}
|
||||
|
||||
// Write accumulated results and signal done. Safe to call more than once.
|
||||
void flush() {
|
||||
if (written_.exchange(true)) return;
|
||||
if (pre_write_) pre_write_(last_ts_);
|
||||
write_output();
|
||||
done_.store(true, std::memory_order_release);
|
||||
}
|
||||
@@ -72,7 +99,7 @@ struct ResultSinkFunc {
|
||||
private:
|
||||
// Bump when the minimal/standard output JSON structure changes in a way
|
||||
// the Jellyfin plugin needs to detect.
|
||||
static constexpr int kSchemaVersion = 1;
|
||||
static constexpr int kSchemaVersion = 2; // SR-003 coordinated bump
|
||||
|
||||
static int count_known(const std::vector<IdentifiedActor>& v) {
|
||||
int n = 0;
|
||||
@@ -92,10 +119,19 @@ private:
|
||||
if (cfg_.verbosity == Verbosity::xray) {
|
||||
root = build_xray();
|
||||
} else {
|
||||
/// TRACES: IR-002 | SR-003
|
||||
/// schema_version 2, per jRay/SPEC.md JR-002. anneal_sec is REMOVED
|
||||
/// rather than zeroed: a field naming a mechanism the pipeline no
|
||||
/// longer has is actively misleading, and would outlive everyone who
|
||||
/// remembers why it reads 0. extinction_sec succeeds it as the
|
||||
/// parameter that actually shapes window extent.
|
||||
root["schema_version"] = kSchemaVersion;
|
||||
root["movie"] = cfg_.movie_path;
|
||||
root["sample_fps"] = cfg_.sample_fps;
|
||||
root["anneal_sec"] = cfg_.anneal_sec;
|
||||
root["movie"] = cfg_.movie_path;
|
||||
root["extraction"] = {
|
||||
{"sample_fps", cfg_.sample_fps},
|
||||
{"extinction_sec", cfg_.track_extinction_sec},
|
||||
{"gallery_scope", cfg_.gallery_scope},
|
||||
};
|
||||
root["actors"] = build_epochs();
|
||||
if (cfg_.verbosity == Verbosity::standard)
|
||||
root["frames"] = build_standard();
|
||||
@@ -110,42 +146,45 @@ private:
|
||||
std::cerr << "[result_sink] done.\n";
|
||||
}
|
||||
|
||||
struct Window {
|
||||
double start{0.0};
|
||||
double end{0.0};
|
||||
float belief{0.f}; ///< the posterior that justified the claim (AR-017)
|
||||
};
|
||||
struct ActorWindow {
|
||||
std::string name, imdb_id, tmdb_id, jellyfin_id;
|
||||
std::vector<std::pair<double, double>> scenes; // [start_sec, end_sec]
|
||||
std::vector<Window> scenes;
|
||||
};
|
||||
struct ActorMeta { std::string name, imdb_id, tmdb_id, jellyfin_id; };
|
||||
|
||||
// Core logic: merge per-frame detections into annealed [start, end] windows.
|
||||
/// TRACES: AR-012, IR-002 | SR-002
|
||||
/// A claim already IS a window — `[first_seen, last_seen]` of a track the
|
||||
/// actor owned. There is no annealing pass: `anneal_sec` existed to bridge
|
||||
/// gaps between isolated accepted frames, and a track that survives its own
|
||||
/// gaps leaves it nothing to do (see the AR-012 withdrawal note).
|
||||
std::vector<ActorWindow> build_actor_windows() {
|
||||
struct Info { std::string name, imdb_id, tmdb_id, jellyfin_id; };
|
||||
std::map<int, Info> actor_info;
|
||||
std::map<int, std::vector<double>> timestamps;
|
||||
std::lock_guard<std::mutex> g(claims_mu_);
|
||||
|
||||
for (const auto& frame : frames_) {
|
||||
for (const auto& ia : frame.visible_actors) {
|
||||
if (ia.actor_idx < 0) continue;
|
||||
actor_info[ia.actor_idx] = {ia.name, ia.imdb_id, ia.tmdb_id, ia.jellyfin_id};
|
||||
timestamps[ia.actor_idx].push_back(frame.timestamp_sec);
|
||||
std::map<int, ActorWindow> by_actor;
|
||||
for (const auto& c : claims_) {
|
||||
auto& aw = by_actor[c.actor_idx];
|
||||
if (aw.name.empty()) {
|
||||
auto it = actor_meta_.find(c.actor_idx);
|
||||
if (it != actor_meta_.end()) {
|
||||
aw.name = it->second.name;
|
||||
aw.imdb_id = it->second.imdb_id;
|
||||
aw.tmdb_id = it->second.tmdb_id;
|
||||
aw.jellyfin_id = it->second.jellyfin_id;
|
||||
}
|
||||
}
|
||||
aw.scenes.push_back({c.first_seen, c.last_seen, c.belief});
|
||||
}
|
||||
|
||||
std::vector<ActorWindow> result;
|
||||
for (auto& [idx, ts_vec] : timestamps) {
|
||||
ActorWindow aw;
|
||||
aw.name = actor_info[idx].name;
|
||||
aw.imdb_id = actor_info[idx].imdb_id;
|
||||
aw.tmdb_id = actor_info[idx].tmdb_id;
|
||||
aw.jellyfin_id = actor_info[idx].jellyfin_id;
|
||||
|
||||
double win_start = ts_vec[0], win_end = ts_vec[0];
|
||||
for (size_t i = 1; i < ts_vec.size(); ++i) {
|
||||
if (ts_vec[i] - win_end > cfg_.anneal_sec) {
|
||||
aw.scenes.push_back({win_start, win_end});
|
||||
win_start = ts_vec[i];
|
||||
}
|
||||
win_end = ts_vec[i];
|
||||
}
|
||||
aw.scenes.push_back({win_start, win_end});
|
||||
for (auto& [idx, aw] : by_actor) {
|
||||
std::sort(aw.scenes.begin(), aw.scenes.end(),
|
||||
[](const Window& a, const Window& b) { return a.start < b.start; });
|
||||
result.push_back(std::move(aw));
|
||||
}
|
||||
return result;
|
||||
@@ -154,9 +193,16 @@ private:
|
||||
json build_epochs() {
|
||||
json actors = json::array();
|
||||
for (const auto& aw : build_actor_windows()) {
|
||||
// Objects, not float pairs: a window carries the belief that
|
||||
// justified it and the route by which it was identified (AR-017),
|
||||
// so a consumer can caveat or filter rather than treating every
|
||||
// window as equally certain.
|
||||
json windows = json::array();
|
||||
for (const auto& [s, e] : aw.scenes)
|
||||
windows.push_back({s, e});
|
||||
for (const auto& w : aw.scenes)
|
||||
windows.push_back({{"start", w.start},
|
||||
{"end", w.end},
|
||||
{"belief", w.belief},
|
||||
{"route", "live"}});
|
||||
json ja;
|
||||
ja["name"] = aw.name;
|
||||
ja["imdb_id"] = aw.imdb_id;
|
||||
@@ -174,9 +220,9 @@ private:
|
||||
json build_xray() {
|
||||
std::map<int, std::vector<std::string>> xray;
|
||||
for (const auto& aw : build_actor_windows()) {
|
||||
for (const auto& [start, end] : aw.scenes) {
|
||||
int t0 = static_cast<int>(std::floor(start));
|
||||
int t1 = static_cast<int>(std::ceil(end));
|
||||
for (const auto& w : aw.scenes) {
|
||||
int t0 = static_cast<int>(std::floor(w.start));
|
||||
int t1 = static_cast<int>(std::ceil(w.end));
|
||||
for (int t = t0; t <= t1; ++t)
|
||||
xray[t].push_back(aw.name);
|
||||
}
|
||||
@@ -227,4 +273,9 @@ private:
|
||||
std::atomic<bool>& done_;
|
||||
std::atomic<bool> written_{false};
|
||||
std::vector<SceneAnnotation> frames_;
|
||||
std::function<void(double)> pre_write_;
|
||||
double last_ts_{0.0};
|
||||
std::mutex claims_mu_;
|
||||
std::vector<DeadTrack> claims_;
|
||||
std::map<int, ActorMeta> actor_meta_; ///< actor_idx → identity keys
|
||||
};
|
||||
|
||||
@@ -22,6 +22,7 @@ add_executable(sae_tests
|
||||
test_track_gallery.cpp
|
||||
test_face_tracker.cpp
|
||||
test_track_registry.cpp
|
||||
test_replay_fixtures.cpp
|
||||
test_audio_signature.cpp
|
||||
${CMAKE_SOURCE_DIR}/src/backends/gemm_backend.cpp
|
||||
${CMAKE_SOURCE_DIR}/src/gallery/gallery_store.cpp
|
||||
|
||||
Vendored
BIN
Binary file not shown.
Vendored
BIN
Binary file not shown.
Vendored
BIN
Binary file not shown.
Vendored
BIN
Binary file not shown.
Vendored
BIN
Binary file not shown.
+117
-53
@@ -13,8 +13,12 @@
|
||||
#include "config.hpp"
|
||||
#include "nodes/face_tracker_node.hpp"
|
||||
#include "types.hpp"
|
||||
#include "track_registry.hpp"
|
||||
#include "evidence_discount.hpp"
|
||||
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
#include <memory>
|
||||
|
||||
namespace {
|
||||
|
||||
@@ -55,85 +59,145 @@ EmbeddedSceneFrame frame(double t, float x, float y, const Embedding& emb,
|
||||
return ef;
|
||||
}
|
||||
|
||||
Config tracker_cfg() {
|
||||
Config cfg;
|
||||
cfg.cut_revive_sim = 0.50f;
|
||||
cfg.cut_inactive_max_frames = 5;
|
||||
return cfg;
|
||||
}
|
||||
// Build a tracker over a fresh registry. The registry IS the tracker's state
|
||||
// now (AR-008), so a test constructs both together and can inspect either.
|
||||
struct Rig {
|
||||
std::shared_ptr<TrackRegistry> reg;
|
||||
FaceTrackerFunc ft;
|
||||
|
||||
explicit Rig(double extinction = 30.0, float assoc_min_prob = 0.5f)
|
||||
: reg(std::make_shared<TrackRegistry>(
|
||||
[extinction] {
|
||||
TrackRegistry::Config c;
|
||||
c.extinction_sec = extinction;
|
||||
return c;
|
||||
}(),
|
||||
EvidenceDiscounter([](float cos) { return std::max(0.f, cos); })))
|
||||
, ft([&] {
|
||||
Config c;
|
||||
c.track_assoc_min_prob = assoc_min_prob;
|
||||
return c;
|
||||
}(),
|
||||
reg,
|
||||
// Trivial calibration: cosine passed through as P(same). Real runs use
|
||||
// the fit belonging to the active embedder (AR-023/AR-024).
|
||||
[](float cos) { return std::max(0.f, cos); })
|
||||
{}
|
||||
|
||||
int track_of(EmbeddedSceneFrame f) { return ft(std::move(f)).track_ids[0]; }
|
||||
};
|
||||
|
||||
} // namespace
|
||||
|
||||
TEST_CASE("track id is stable across ordinary frames", "[face_tracker]") {
|
||||
FaceTrackerFunc ft(tracker_cfg());
|
||||
// ── AR-008 — one pool, ordinary association ──────────────────────────────────
|
||||
TEST_CASE("track id is stable across ordinary frames", "[face_tracker][AR-008]") {
|
||||
Rig r;
|
||||
Embedding e = axis(0);
|
||||
int id0 = ft(frame(0.0, 10, 10, e)).track_ids[0];
|
||||
int id1 = ft(frame(1.0, 11, 10, e)).track_ids[0]; // overlaps → same track
|
||||
int id0 = r.track_of(frame(0.0, 10, 10, e));
|
||||
int id1 = r.track_of(frame(1.0, 11, 10, e)); // overlaps → same track
|
||||
CHECK(id0 >= 0);
|
||||
CHECK(id1 == id0);
|
||||
}
|
||||
|
||||
TEST_CASE("cut revives the same track id for a matching identity", "[face_tracker]") {
|
||||
FaceTrackerFunc ft(tracker_cfg());
|
||||
TEST_CASE("a face lost across a cut and re-associated is the SAME track",
|
||||
"[face_tracker][AR-008]") {
|
||||
// Previously this was a distinct "revival" path guarded by a raw-cosine
|
||||
// constant. There is no such path now: a dormant track is an ordinary
|
||||
// association candidate, and continuity falls out of the embedding match.
|
||||
Rig r;
|
||||
|
||||
// Pre-cut: establish a track for a person whose embedding is near-identical
|
||||
// across the cut (sim well above cut_revive_sim), but whose box jumps so IoU
|
||||
// is 0 — the ordinary spatial path cannot re-link it.
|
||||
Embedding pre = at_sim(0, 1, 0.99f);
|
||||
int id_pre = ft(frame(0.0, 10, 10, pre)).track_ids[0];
|
||||
int id_pre = r.track_of(frame(0.0, 10, 10, pre));
|
||||
REQUIRE(id_pre >= 0);
|
||||
|
||||
Embedding post = at_sim(0, 1, 0.98f); // cos(diff) ≈ 0.9997 > 0.50
|
||||
auto out = ft(frame(1.0, 300, 300, post, /*is_cut=*/true));
|
||||
CHECK(out.track_ids[0] == id_pre); // revived, not a fresh id
|
||||
// Box jumps so IoU is zero — only the embedding can link it.
|
||||
Embedding post = at_sim(0, 1, 0.98f);
|
||||
CHECK(r.track_of(frame(1.0, 300, 300, post, /*is_cut=*/true)) == id_pre);
|
||||
}
|
||||
|
||||
TEST_CASE("cut starts a fresh track when identity does not match", "[face_tracker]") {
|
||||
FaceTrackerFunc ft(tracker_cfg());
|
||||
|
||||
int id_pre = ft(frame(0.0, 10, 10, axis(0))).track_ids[0];
|
||||
TEST_CASE("a cut starts a fresh track when identity does not match",
|
||||
"[face_tracker][AR-008]") {
|
||||
Rig r;
|
||||
int id_pre = r.track_of(frame(0.0, 10, 10, axis(0)));
|
||||
REQUIRE(id_pre >= 0);
|
||||
|
||||
// Post-cut face is orthogonal (sim 0 < cut_revive_sim) and spatially disjoint
|
||||
// → no revival, brand-new id.
|
||||
auto out = ft(frame(1.0, 300, 300, axis(5), /*is_cut=*/true));
|
||||
CHECK(out.track_ids[0] != id_pre);
|
||||
CHECK(out.track_ids[0] >= 0);
|
||||
// Orthogonal embedding and disjoint box: nothing links them.
|
||||
int id_post = r.track_of(frame(1.0, 300, 300, axis(5), /*is_cut=*/true));
|
||||
CHECK(id_post != id_pre);
|
||||
CHECK(id_post >= 0);
|
||||
}
|
||||
|
||||
TEST_CASE("parked track expires after cut_inactive_max_frames", "[face_tracker]") {
|
||||
Config cfg = tracker_cfg();
|
||||
cfg.cut_inactive_max_frames = 2;
|
||||
FaceTrackerFunc ft(cfg);
|
||||
// ── AR-007 — a cut makes association ignore position ─────────────────────────
|
||||
TEST_CASE("on a cut, identity follows the embedding rather than the box",
|
||||
"[face_tracker][AR-007]") {
|
||||
// Two people swap screen positions across a cut while keeping their faces.
|
||||
// If IoU still carried weight the ids would follow the boxes and swap; with
|
||||
// alpha driven to embedding-only on a cut, they must follow the faces.
|
||||
Rig r;
|
||||
|
||||
Embedding a = at_sim(0, 1, 0.99f);
|
||||
Embedding b = at_sim(2, 3, 0.99f);
|
||||
|
||||
EmbeddedSceneFrame f0;
|
||||
f0.source.timestamp_sec = 0.0;
|
||||
f0.faces = {face_at(10, 10), face_at(300, 300)};
|
||||
f0.crops = {cv::Mat(), cv::Mat()};
|
||||
f0.embeddings = {a, b};
|
||||
auto out0 = r.ft(std::move(f0));
|
||||
const int id_a = out0.track_ids[0];
|
||||
const int id_b = out0.track_ids[1];
|
||||
REQUIRE(id_a >= 0);
|
||||
REQUIRE(id_b >= 0);
|
||||
REQUIRE(id_a != id_b);
|
||||
|
||||
// Same two people, positions exchanged, on a cut frame.
|
||||
EmbeddedSceneFrame f1;
|
||||
f1.source.timestamp_sec = 1.0;
|
||||
f1.source.is_cut = true;
|
||||
f1.faces = {face_at(300, 300), face_at(10, 10)};
|
||||
f1.crops = {cv::Mat(), cv::Mat()};
|
||||
f1.embeddings = {a, b};
|
||||
auto out1 = r.ft(std::move(f1));
|
||||
|
||||
CHECK(out1.track_ids[0] == id_a); // A kept its id despite moving to B's box
|
||||
CHECK(out1.track_ids[1] == id_b);
|
||||
}
|
||||
|
||||
// ── AR-013 — extinction replaces the parked-pool frame counter ───────────────
|
||||
TEST_CASE("a track past the extinction window is gone, not revived",
|
||||
"[face_tracker][AR-013]") {
|
||||
// The old design aged a parked pool in frames, which silently changed
|
||||
// meaning with sample_fps. Extinction is in seconds and lives in the
|
||||
// registry, so the tracker no longer counts anything.
|
||||
Rig r(/*extinction=*/2.0);
|
||||
|
||||
Embedding person = at_sim(0, 1, 0.99f);
|
||||
int id_pre = ft(frame(0.0, 10, 10, person)).track_ids[0];
|
||||
int id_pre = r.track_of(frame(0.0, 10, 10, person));
|
||||
REQUIRE(id_pre >= 0);
|
||||
|
||||
// Cut with an unrelated face parks id_pre; then let the pool age past its
|
||||
// limit with more unrelated, spatially-disjoint faces (each ages the pool by
|
||||
// one). By the time the person returns, id_pre must be gone.
|
||||
ft(frame(1.0, 300, 300, axis(7), /*is_cut=*/true)); // park (age 1)
|
||||
ft(frame(2.0, 300, 300, axis(7))); // age 2
|
||||
ft(frame(3.0, 300, 300, axis(7))); // age 3 → id_pre dropped
|
||||
// Unrelated faces elsewhere while the clock runs well past extinction.
|
||||
r.track_of(frame(1.0, 300, 300, axis(7), /*is_cut=*/true));
|
||||
r.track_of(frame(10.0, 300, 300, axis(7)));
|
||||
|
||||
auto out = ft(frame(4.0, 10, 10, person)); // same identity returns
|
||||
CHECK(out.track_ids[0] != id_pre); // too late — fresh id
|
||||
CHECK(r.track_of(frame(11.0, 10, 10, person)) != id_pre);
|
||||
}
|
||||
|
||||
TEST_CASE("eof clears active and parked tracks", "[face_tracker]") {
|
||||
FaceTrackerFunc ft(tracker_cfg());
|
||||
Embedding person = at_sim(0, 1, 0.99f);
|
||||
int id_pre = ft(frame(0.0, 10, 10, person)).track_ids[0];
|
||||
ft(frame(1.0, 300, 300, axis(7), /*is_cut=*/true)); // park id_pre
|
||||
TEST_CASE("a track within the extinction window is still a candidate",
|
||||
"[face_tracker][AR-013]") {
|
||||
Rig r(/*extinction=*/30.0);
|
||||
|
||||
Embedding person = at_sim(0, 1, 0.99f);
|
||||
int id_pre = r.track_of(frame(0.0, 10, 10, person));
|
||||
r.track_of(frame(1.0, 300, 300, axis(7), /*is_cut=*/true));
|
||||
|
||||
// Back inside the window: the same person continues the same track, so the
|
||||
// gap is absorbed into one window rather than splitting it.
|
||||
CHECK(r.track_of(frame(3.0, 10, 10, person)) == id_pre);
|
||||
}
|
||||
|
||||
TEST_CASE("eof is forwarded", "[face_tracker]") {
|
||||
Rig r;
|
||||
EmbeddedSceneFrame eof;
|
||||
eof.source.eof = true;
|
||||
auto out = ft(std::move(eof));
|
||||
CHECK(out.source.eof);
|
||||
|
||||
// After eof the pools are empty: the returning identity must get a fresh id,
|
||||
// not the parked one.
|
||||
auto out2 = ft(frame(2.0, 10, 10, person));
|
||||
CHECK(out2.track_ids[0] != id_pre);
|
||||
CHECK(r.ft(std::move(eof)).source.eof);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,258 @@
|
||||
// Replay tests — the real tracker and registry driven from committed fixtures.
|
||||
//
|
||||
// TRACES: AR-012, AR-013, AR-004, VR-001, VR-002 | IT-001
|
||||
//
|
||||
// Tier T2: composition, not units. The registry tests construct awkward states
|
||||
// directly; these check that the pieces behave when wired together and fed real
|
||||
// footage — 480x360 public-domain clips at 5 fps, with the cuts, gaps and
|
||||
// crowded frames that actual film produces and synthetic input does not.
|
||||
//
|
||||
// No GPU and no model: the fixtures are HDF5 dumps taken after embedding, so
|
||||
// everything here is CPU maths. That is what lets this run on the CI host at
|
||||
// all (see docs/requirements.md, "CI never calls a model").
|
||||
//
|
||||
// Driving the node functors directly rather than through a KPN network is
|
||||
// deliberate: functors are plain objects, so there are no threads, no channels
|
||||
// and no scheduling — the same input gives the same output every time, which is
|
||||
// exactly what a fixture-based test needs.
|
||||
#include <catch2/catch_test_macros.hpp>
|
||||
|
||||
#include "config.hpp"
|
||||
#include "evidence_discount.hpp"
|
||||
#include "nodes/face_tracker_node.hpp"
|
||||
#include "track_registry.hpp"
|
||||
#include "types.hpp"
|
||||
|
||||
#include <H5Cpp.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
namespace {
|
||||
|
||||
// ── Fixture reader ───────────────────────────────────────────────────────────
|
||||
// The flat/ragged layout of scripts/optimizer/SCHEMA.md: per-face arrays
|
||||
// concatenated, with a per-frame index table pointing into them.
|
||||
struct Dump {
|
||||
std::vector<double> ts;
|
||||
std::vector<uint8_t> is_cut;
|
||||
std::vector<int64_t> face_offset;
|
||||
std::vector<int32_t> face_count;
|
||||
std::vector<Embedding> emb;
|
||||
std::vector<float> bbox; // 4 per face
|
||||
std::string embedder;
|
||||
|
||||
std::size_t frames() const { return ts.size(); }
|
||||
std::size_t faces() const { return emb.size(); }
|
||||
};
|
||||
|
||||
template <typename T>
|
||||
std::vector<T> read1d(H5::Group& g, const char* name, const H5::DataType& dt) {
|
||||
H5::DataSet ds = g.openDataSet(name);
|
||||
hsize_t n = 0;
|
||||
ds.getSpace().getSimpleExtentDims(&n, nullptr);
|
||||
std::vector<T> out(n);
|
||||
if (n) ds.read(out.data(), dt);
|
||||
return out;
|
||||
}
|
||||
|
||||
Dump load(const std::string& path) {
|
||||
H5::H5File f(path, H5F_ACC_RDONLY);
|
||||
H5::Group frames = f.openGroup("frames");
|
||||
H5::Group faces = f.openGroup("faces");
|
||||
|
||||
Dump d;
|
||||
d.ts = read1d<double>(frames, "timestamp_sec", H5::PredType::NATIVE_DOUBLE);
|
||||
d.is_cut = read1d<uint8_t>(frames, "is_cut", H5::PredType::NATIVE_UINT8);
|
||||
d.face_offset = read1d<int64_t>(frames, "face_offset", H5::PredType::NATIVE_INT64);
|
||||
d.face_count = read1d<int32_t>(frames, "face_count", H5::PredType::NATIVE_INT32);
|
||||
|
||||
H5::DataSet e = faces.openDataSet("embedding");
|
||||
hsize_t dims[2]{0, 0};
|
||||
e.getSpace().getSimpleExtentDims(dims, nullptr);
|
||||
std::vector<float> flat(dims[0] * dims[1]);
|
||||
if (!flat.empty()) e.read(flat.data(), H5::PredType::NATIVE_FLOAT);
|
||||
d.emb.resize(dims[0]);
|
||||
for (hsize_t i = 0; i < dims[0]; ++i)
|
||||
std::copy_n(flat.begin() + i * dims[1], 512, d.emb[i].begin());
|
||||
|
||||
// bbox is 2-D [N,4]; reading it with the 1-D helper would size the buffer
|
||||
// from the first extent only and then read four times that many floats.
|
||||
{
|
||||
H5::DataSet bs = faces.openDataSet("bbox");
|
||||
hsize_t bd[2]{0, 0};
|
||||
bs.getSpace().getSimpleExtentDims(bd, nullptr);
|
||||
d.bbox.resize(bd[0] * bd[1]);
|
||||
if (!d.bbox.empty()) bs.read(d.bbox.data(), H5::PredType::NATIVE_FLOAT);
|
||||
}
|
||||
|
||||
// GR-004: the dump records which embedder produced it, so a replay cannot
|
||||
// be silently scored against a gallery from a different model.
|
||||
if (f.attrExists("embedder_model")) {
|
||||
// Written as a variable-length string (embedding_dump_node.hpp:99), so
|
||||
// the read must name the same type explicitly.
|
||||
H5::StrType vlen(H5::PredType::C_S1, H5T_VARIABLE);
|
||||
f.openAttribute("embedder_model").read(vlen, d.embedder);
|
||||
}
|
||||
return d;
|
||||
}
|
||||
|
||||
std::string fixture(const char* name) {
|
||||
return std::string(SAE_TEST_FIXTURES_DIR) + "/dumps/" + name;
|
||||
}
|
||||
|
||||
// ── Harness ──────────────────────────────────────────────────────────────────
|
||||
struct Replay {
|
||||
std::vector<DeadTrack> claims;
|
||||
std::vector<int> track_ids; // per face, in fixture order
|
||||
std::size_t faces_seen{0};
|
||||
};
|
||||
|
||||
Replay run(const Dump& d, double extinction = 10.0) {
|
||||
Replay r;
|
||||
TrackRegistry::Config rc;
|
||||
rc.extinction_sec = extinction;
|
||||
|
||||
auto cal = [](float cos) { return std::max(0.f, cos); };
|
||||
auto reg = std::make_shared<TrackRegistry>(rc, EvidenceDiscounter(cal));
|
||||
reg->on_track_dead([&r](const DeadTrack& t) { r.claims.push_back(t); });
|
||||
|
||||
Config cfg;
|
||||
cfg.track_assoc_min_prob = 0.5f;
|
||||
FaceTrackerFunc ft(cfg, reg, cal);
|
||||
|
||||
for (std::size_t i = 0; i < d.frames(); ++i) {
|
||||
EmbeddedSceneFrame ef;
|
||||
ef.source.timestamp_sec = d.ts[i];
|
||||
ef.source.is_cut = d.is_cut[i] != 0;
|
||||
|
||||
const int64_t off = d.face_offset[i];
|
||||
const int32_t n = d.face_count[i];
|
||||
for (int32_t k = 0; k < n; ++k) {
|
||||
DetectedFace face;
|
||||
const float* b = &d.bbox[(off + k) * 4];
|
||||
face.bbox = cv::Rect2f(b[0], b[1], b[2], b[3]);
|
||||
face.confidence = 1.0f;
|
||||
ef.faces.push_back(face);
|
||||
ef.crops.push_back(cv::Mat());
|
||||
ef.embeddings.push_back(d.emb[off + k]);
|
||||
}
|
||||
r.faces_seen += static_cast<std::size_t>(n);
|
||||
|
||||
auto out = ft(std::move(ef));
|
||||
for (int id : out.track_ids) r.track_ids.push_back(id);
|
||||
}
|
||||
|
||||
reg->flush(d.ts.empty() ? 0.0 : d.ts.back());
|
||||
return r;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
// ── AR-004 / VR-001 — the fixtures are intact and self-describing ────────────
|
||||
TEST_CASE("fixtures are complete and carry their embedder identity",
|
||||
"[replay][AR-004][VR-001]") {
|
||||
// Frame counts are exact rather than approximate. Before node outputs
|
||||
// blocked on a full channel, generation lost most of a clip and what it
|
||||
// lost depended on timing — these numbers could not have been asserted.
|
||||
struct Expect { const char* file; std::size_t frames, faces; };
|
||||
const Expect all[] = {
|
||||
{"bali_13.h5", 385, 693},
|
||||
{"bali_27.h5", 335, 335},
|
||||
{"bali_28.h5", 345, 368},
|
||||
{"bali_31.h5", 145, 203},
|
||||
{"bali_46.h5", 385, 140},
|
||||
};
|
||||
|
||||
for (const auto& x : all) {
|
||||
INFO(x.file);
|
||||
Dump d = load(fixture(x.file));
|
||||
CHECK(d.frames() == x.frames);
|
||||
CHECK(d.faces() == x.faces);
|
||||
CHECK(d.embedder == "LVFace-B_Glint360K.onnx");
|
||||
|
||||
// face_offset must be contiguous: a gap means faces went missing
|
||||
// between frames, which no consumer could detect.
|
||||
int64_t running = 0;
|
||||
for (std::size_t i = 0; i < d.frames(); ++i) {
|
||||
REQUIRE(d.face_offset[i] == running);
|
||||
running += d.face_count[i];
|
||||
}
|
||||
CHECK(static_cast<std::size_t>(running) == d.faces());
|
||||
}
|
||||
}
|
||||
|
||||
// ── VR-002 — replay is deterministic ─────────────────────────────────────────
|
||||
TEST_CASE("replaying a fixture twice gives identical tracks", "[replay][VR-002]") {
|
||||
// The property the whole fixture strategy rests on. If this fails, every
|
||||
// golden output derived from a fixture is unreliable and the CI replay
|
||||
// tier is worthless.
|
||||
Dump d = load(fixture("bali_28.h5"));
|
||||
Replay a = run(d);
|
||||
Replay b = run(d);
|
||||
|
||||
REQUIRE(a.track_ids.size() == b.track_ids.size());
|
||||
CHECK(a.track_ids == b.track_ids);
|
||||
REQUIRE(a.claims.size() == b.claims.size());
|
||||
for (std::size_t i = 0; i < a.claims.size(); ++i) {
|
||||
CHECK(a.claims[i].first_seen == b.claims[i].first_seen);
|
||||
CHECK(a.claims[i].last_seen == b.claims[i].last_seen);
|
||||
}
|
||||
}
|
||||
|
||||
// ── AR-012 / AR-013 — window invariants on real footage ──────────────────────
|
||||
TEST_CASE("every face is assigned a track and every track closes",
|
||||
"[replay][AR-012]") {
|
||||
Dump d = load(fixture("bali_13.h5"));
|
||||
Replay r = run(d);
|
||||
|
||||
CHECK(r.track_ids.size() == r.faces_seen);
|
||||
for (int id : r.track_ids) CHECK(id >= 0); // nothing silently unassigned
|
||||
|
||||
// flush() must leave nothing behind: a track still open at EOF would be a
|
||||
// window that never reaches the output.
|
||||
CHECK(r.claims.size() > 0);
|
||||
}
|
||||
|
||||
TEST_CASE("windows are well-formed and inside the clip", "[replay][AR-013]") {
|
||||
for (const char* f : {"bali_13.h5", "bali_27.h5", "bali_28.h5",
|
||||
"bali_31.h5", "bali_46.h5"}) {
|
||||
INFO(f);
|
||||
Dump d = load(fixture(f));
|
||||
Replay r = run(d);
|
||||
const double t0 = d.ts.front(), t1 = d.ts.back();
|
||||
|
||||
for (const auto& c : r.claims) {
|
||||
// A window ends at the last sighting, never after it — so it can
|
||||
// never extend past the footage that produced it.
|
||||
CHECK(c.first_seen <= c.last_seen);
|
||||
CHECK(c.first_seen >= t0);
|
||||
CHECK(c.last_seen <= t1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("a longer extinction window yields fewer, longer tracks",
|
||||
"[replay][AR-013]") {
|
||||
// The timeout decides whether a gap is absorbed into one window or splits
|
||||
// it in two, so lengthening it must merge tracks rather than multiply them.
|
||||
// On sparse footage this is the difference the constant actually makes.
|
||||
Dump d = load(fixture("bali_46.h5")); // 140 faces over 385 frames
|
||||
Replay tight = run(d, /*extinction=*/1.0);
|
||||
Replay loose = run(d, /*extinction=*/30.0);
|
||||
|
||||
CHECK(loose.claims.size() <= tight.claims.size());
|
||||
}
|
||||
|
||||
// ── AR-007 — cuts are exercised by the corpus, not just by construction ──────
|
||||
TEST_CASE("the cut-heavy fixture actually contains cuts", "[replay][AR-007]") {
|
||||
// Guards the corpus rather than the code: if a regeneration produced a
|
||||
// fixture with no cuts, the association tests above would still pass while
|
||||
// silently testing nothing about viewpoint changes.
|
||||
Dump d = load(fixture("bali_28.h5"));
|
||||
const int cuts = std::count(d.is_cut.begin(), d.is_cut.end(), uint8_t{1});
|
||||
CHECK(cuts >= 5);
|
||||
}
|
||||
Reference in New Issue
Block a user