feat: backpressure — the pipeline slows instead of losing frames

Picks up the KPN fix: node data outputs block on a full channel rather than
dropping. Sentinels stay out-of-band, so EOF can always overtake a stalled data
path and the hold-and-wait deadlock that comment warns about is not reachable.

Verified on a 77s clip at 5 fps, which should yield 385 sampled frames:

  before  65 written, 320 dropped, 29s, two runs differ
  after   385 written, 0 dropped, 17s, two runs byte-identical

The determinism is the part that matters. Golden fixtures were impossible while
what got dropped depended on timing; VR-001 fixture generation is unblocked by
this, and so is the CI replay strategy that depends on it.

Faster rather than slower, which is worth recording because the intuition runs
the other way: a dropped frame has already cost its decode, and the overflow
exception cost more still.

AR-004 is not fully closed. Channel capacity remains a count of items, while a
face carries a 112x112 crop and a 512-float embedding — so a crowded frame
occupies far more memory per slot than a sparse one. Bounding by bytes in
flight is the remaining half, and it matters once max_faces is removed (AR-003).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-004, VR-001 | SR-002
This commit is contained in:
2026-07-31 10:35:29 +02:00
co-authored by Claude Opus 5
parent b98372bad8
commit 09a4650fd9
3 changed files with 15 additions and 4 deletions
+13 -2
View File
@@ -110,8 +110,19 @@ Two consequences worth stating:
depends on timing. The same command run twice can produce different dumps, and depends on timing. The same command run twice can produce different dumps, and
a golden fixture cannot be built on that. a golden fixture cannot be built on that.
**Gap:** entire requirement, and it is **cross-repo** — the drop-versus-block **Current:** fixed in KPN — node data outputs use `push_blocking`, sentinels
decision lives in KPN. Removing `max_faces` (AR-003) remains gated on it. remain out-of-band so EOF can always overtake a stalled data path. Verified on
the same clip: 385 of 385 sampled frames written, zero drops, and two
consecutive runs byte-identical where previously they were not.
It also ran *faster* (29 s → 17 s). A dropped frame has already cost its decode,
and the overflow exception cost more — so the lossy path was paying for work it
then discarded.
**Gap:** the remaining half — bounding by **bytes in flight** rather than item
count. Channel capacity is still a count of items, and a face carries a 112×112
crop plus a 512-float embedding, so a crowded frame occupies far more memory per
slot than a sparse one. That matters once `max_faces` is removed (AR-003).
## AR-005 — Face alignment and crop ## AR-005 — Face alignment and crop
+1 -1
View File
@@ -31,7 +31,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
| AR-001 | Detect faces in sampled frames; emit bbox, confidence, 5-point landmarks in original pixel space | SR-002 | High | Done | | AR-001 | Detect faces in sampled frames; emit bbox, confidence, 5-point landmarks in original pixel space | SR-002 | High | Done |
| AR-002 | Minimum face size **32×32 px** (VR-005 measured), expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | Planned | | AR-002 | Minimum face size **32×32 px** (VR-005 measured), expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | Planned |
| AR-003 | No fixed per-frame face cap — crowd scenes must not lose background cast | SR-002 | Medium | Planned | | AR-003 | No fixed per-frame face cap — crowd scenes must not lose background cast | SR-002 | Medium | Planned |
| AR-004 | Backpressure: unbounded faces/frame absorbed by slowing, never by dropping or throwing | SR-002 | High | **Blocked — cross-repo.** Every KPN node output uses the dropping `push()`; `push_blocking()` exists but is never called. Needs a KPN change | | AR-004 | Backpressure: unbounded faces/frame absorbed by slowing, never by dropping or throwing | SR-002 | High | **Done** KPN node outputs use `push_blocking`; sentinels stay out-of-band. Verified: 385/385 frames, 0 drops, byte-identical across runs |
| AR-005 | Align to 112×112 via ArcFace 5-point similarity transform | SR-002 | High | Done | | AR-005 | Align to 112×112 via ArcFace 5-point similarity transform | SR-002 | High | Done |
| AR-006 | 512-d L2-normalised embeddings, batched | SR-002 | High | Done | | AR-006 | 512-d L2-normalised embeddings, batched | SR-002 | High | Done |
| AR-007 | Associate detections by IoU + embedding, with **frame-dependent** weighting | SR-002 | High | **Done**`track_alpha` is the base for ordinary frames; drops to embedding-only on cut/boundary and for dormant tracks | | AR-007 | Associate detections by IoU + embedding, with **frame-dependent** weighting | SR-002 | High | **Done**`track_alpha` is the base for ordinary frames; drops to embedding-only on cut/boundary and for dormant tracks |
+1 -1