test: replay the real tracker and registry from committed fixtures

Tier T2 — composition rather than units. The registry tests construct awkward states directly; these feed the pieces real 480x360 footage with the cuts, gaps and crowded frames that synthetic input does not produce. Six cases: - fixture integrity: exact frame and face counts, contiguous face_offset, and the embedder identity each dump carries (GR-004). The counts are asserted exactly rather than approximately, which was impossible before AR-004 — what a lossy run dropped depended on timing. - determinism: replaying a fixture twice gives identical track ids and windows. This is the property the whole fixture strategy rests on; without it every golden output derived from a fixture is unreliable and the CI replay tier is worthless. - every face is assigned a track, and flush leaves nothing open — a track still live at EOF is a window that never reaches the output. - windows are well-formed and inside the clip. A window ends at the last sighting, so it can never extend past the footage that produced it. - a longer extinction window yields fewer, longer tracks. On the sparse fixture (140 faces over 385 frames) that is the difference the constant actually makes: absorbing a gap versus splitting a window. - the cut-heavy fixture still contains cuts. This guards the corpus, not the code: a regeneration that produced cut-free fixtures would leave the association tests passing while silently testing nothing. Driving the functors directly rather than through a KPN network is deliberate — no threads, no channels, no scheduling, so the same input gives the same output. Suite: 86 cases, 6106 assertions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-004, AR-012, AR-013, VR-001, VR-002 | SR-002
2026-07-31 11:23:17 +02:00
parent c12838b9fd
commit 61e487fbee
4 changed files with 264 additions and 5 deletions
@@ -0,0 +1,258 @@
+// Replay tests — the real tracker and registry driven from committed fixtures.
+//
+// TRACES: AR-012, AR-013, AR-004, VR-001, VR-002 | IT-001
+//
+// Tier T2: composition, not units. The registry tests construct awkward states
+// directly; these check that the pieces behave when wired together and fed real
+// footage — 480x360 public-domain clips at 5 fps, with the cuts, gaps and
+// crowded frames that actual film produces and synthetic input does not.
+//
+// No GPU and no model: the fixtures are HDF5 dumps taken after embedding, so
+// everything here is CPU maths. That is what lets this run on the CI host at
+// all (see docs/requirements.md, "CI never calls a model").
+//
+// Driving the node functors directly rather than through a KPN network is
+// deliberate: functors are plain objects, so there are no threads, no channels
+// and no scheduling — the same input gives the same output every time, which is
+// exactly what a fixture-based test needs.
+#include <catch2/catch_test_macros.hpp>
+
+#include "config.hpp"
+#include "evidence_discount.hpp"
+#include "nodes/face_tracker_node.hpp"
+#include "track_registry.hpp"
+#include "types.hpp"
+
+#include <H5Cpp.h>
+
+#include <algorithm>
+#include <cmath>
+#include <memory>
+#include <string>
+#include <vector>
+
+namespace {
+
+// ── Fixture reader ───────────────────────────────────────────────────────────
+// The flat/ragged layout of scripts/optimizer/SCHEMA.md: per-face arrays
+// concatenated, with a per-frame index table pointing into them.
+struct Dump {
+    std::vector<double>    ts;
+    std::vector<uint8_t>   is_cut;
+    std::vector<int64_t>   face_offset;
+    std::vector<int32_t>   face_count;
+    std::vector<Embedding> emb;
+    std::vector<float>     bbox;      // 4 per face
+    std::string            embedder;
+
+    std::size_t frames() const { return ts.size(); }
+    std::size_t faces()  const { return emb.size(); }
+};
+
+template <typename T>
+std::vector<T> read1d(H5::Group& g, const char* name, const H5::DataType& dt) {
+    H5::DataSet ds = g.openDataSet(name);
+    hsize_t n = 0;
+    ds.getSpace().getSimpleExtentDims(&n, nullptr);
+    std::vector<T> out(n);
+    if (n) ds.read(out.data(), dt);
+    return out;
+}
+
+Dump load(const std::string& path) {
+    H5::H5File f(path, H5F_ACC_RDONLY);
+    H5::Group frames = f.openGroup("frames");
+    H5::Group faces  = f.openGroup("faces");
+
+    Dump d;
+    d.ts          = read1d<double>(frames, "timestamp_sec", H5::PredType::NATIVE_DOUBLE);
+    d.is_cut      = read1d<uint8_t>(frames, "is_cut",       H5::PredType::NATIVE_UINT8);
+    d.face_offset = read1d<int64_t>(frames, "face_offset",  H5::PredType::NATIVE_INT64);
+    d.face_count  = read1d<int32_t>(frames, "face_count",   H5::PredType::NATIVE_INT32);
+
+    H5::DataSet e = faces.openDataSet("embedding");
+    hsize_t dims[2]{0, 0};
+    e.getSpace().getSimpleExtentDims(dims, nullptr);
+    std::vector<float> flat(dims[0] * dims[1]);
+    if (!flat.empty()) e.read(flat.data(), H5::PredType::NATIVE_FLOAT);
+    d.emb.resize(dims[0]);
+    for (hsize_t i = 0; i < dims[0]; ++i)
+        std::copy_n(flat.begin() + i * dims[1], 512, d.emb[i].begin());
+
+    // bbox is 2-D [N,4]; reading it with the 1-D helper would size the buffer
+    // from the first extent only and then read four times that many floats.
+    {
+        H5::DataSet bs = faces.openDataSet("bbox");
+        hsize_t bd[2]{0, 0};
+        bs.getSpace().getSimpleExtentDims(bd, nullptr);
+        d.bbox.resize(bd[0] * bd[1]);
+        if (!d.bbox.empty()) bs.read(d.bbox.data(), H5::PredType::NATIVE_FLOAT);
+    }
+
+    // GR-004: the dump records which embedder produced it, so a replay cannot
+    // be silently scored against a gallery from a different model.
+    if (f.attrExists("embedder_model")) {
+        // Written as a variable-length string (embedding_dump_node.hpp:99), so
+        // the read must name the same type explicitly.
+        H5::StrType vlen(H5::PredType::C_S1, H5T_VARIABLE);
+        f.openAttribute("embedder_model").read(vlen, d.embedder);
+    }
+    return d;
+}
+
+std::string fixture(const char* name) {
+    return std::string(SAE_TEST_FIXTURES_DIR) + "/dumps/" + name;
+}
+
+// ── Harness ──────────────────────────────────────────────────────────────────
+struct Replay {
+    std::vector<DeadTrack> claims;
+    std::vector<int>       track_ids;   // per face, in fixture order
+    std::size_t            faces_seen{0};
+};
+
+Replay run(const Dump& d, double extinction = 10.0) {
+    Replay r;
+    TrackRegistry::Config rc;
+    rc.extinction_sec = extinction;
+
+    auto cal = [](float cos) { return std::max(0.f, cos); };
+    auto reg = std::make_shared<TrackRegistry>(rc, EvidenceDiscounter(cal));
+    reg->on_track_dead([&r](const DeadTrack& t) { r.claims.push_back(t); });
+
+    Config cfg;
+    cfg.track_assoc_min_prob = 0.5f;
+    FaceTrackerFunc ft(cfg, reg, cal);
+
+    for (std::size_t i = 0; i < d.frames(); ++i) {
+        EmbeddedSceneFrame ef;
+        ef.source.timestamp_sec = d.ts[i];
+        ef.source.is_cut        = d.is_cut[i] != 0;
+
+        const int64_t off = d.face_offset[i];
+        const int32_t n   = d.face_count[i];
+        for (int32_t k = 0; k < n; ++k) {
+            DetectedFace face;
+            const float* b = &d.bbox[(off + k) * 4];
+            face.bbox = cv::Rect2f(b[0], b[1], b[2], b[3]);
+            face.confidence = 1.0f;
+            ef.faces.push_back(face);
+            ef.crops.push_back(cv::Mat());
+            ef.embeddings.push_back(d.emb[off + k]);
+        }
+        r.faces_seen += static_cast<std::size_t>(n);
+
+        auto out = ft(std::move(ef));
+        for (int id : out.track_ids) r.track_ids.push_back(id);
+    }
+
+    reg->flush(d.ts.empty() ? 0.0 : d.ts.back());
+    return r;
+}
+
+}  // namespace
+
+// ── AR-004 / VR-001 — the fixtures are intact and self-describing ────────────
+TEST_CASE("fixtures are complete and carry their embedder identity",
+          "[replay][AR-004][VR-001]") {
+    // Frame counts are exact rather than approximate. Before node outputs
+    // blocked on a full channel, generation lost most of a clip and what it
+    // lost depended on timing — these numbers could not have been asserted.
+    struct Expect { const char* file; std::size_t frames, faces; };
+    const Expect all[] = {
+        {"bali_13.h5", 385, 693},
+        {"bali_27.h5", 335, 335},
+        {"bali_28.h5", 345, 368},
+        {"bali_31.h5", 145, 203},
+        {"bali_46.h5", 385, 140},
+    };
+
+    for (const auto& x : all) {
+        INFO(x.file);
+        Dump d = load(fixture(x.file));
+        CHECK(d.frames() == x.frames);
+        CHECK(d.faces()  == x.faces);
+        CHECK(d.embedder == "LVFace-B_Glint360K.onnx");
+
+        // face_offset must be contiguous: a gap means faces went missing
+        // between frames, which no consumer could detect.
+        int64_t running = 0;
+        for (std::size_t i = 0; i < d.frames(); ++i) {
+            REQUIRE(d.face_offset[i] == running);
+            running += d.face_count[i];
+        }
+        CHECK(static_cast<std::size_t>(running) == d.faces());
+    }
+}
+
+// ── VR-002 — replay is deterministic ─────────────────────────────────────────
+TEST_CASE("replaying a fixture twice gives identical tracks", "[replay][VR-002]") {
+    // The property the whole fixture strategy rests on. If this fails, every
+    // golden output derived from a fixture is unreliable and the CI replay
+    // tier is worthless.
+    Dump d = load(fixture("bali_28.h5"));
+    Replay a = run(d);
+    Replay b = run(d);
+
+    REQUIRE(a.track_ids.size() == b.track_ids.size());
+    CHECK(a.track_ids == b.track_ids);
+    REQUIRE(a.claims.size() == b.claims.size());
+    for (std::size_t i = 0; i < a.claims.size(); ++i) {
+        CHECK(a.claims[i].first_seen == b.claims[i].first_seen);
+        CHECK(a.claims[i].last_seen  == b.claims[i].last_seen);
+    }
+}
+
+// ── AR-012 / AR-013 — window invariants on real footage ──────────────────────
+TEST_CASE("every face is assigned a track and every track closes",
+          "[replay][AR-012]") {
+    Dump d = load(fixture("bali_13.h5"));
+    Replay r = run(d);
+
+    CHECK(r.track_ids.size() == r.faces_seen);
+    for (int id : r.track_ids) CHECK(id >= 0);       // nothing silently unassigned
+
+    // flush() must leave nothing behind: a track still open at EOF would be a
+    // window that never reaches the output.
+    CHECK(r.claims.size() > 0);
+}
+
+TEST_CASE("windows are well-formed and inside the clip", "[replay][AR-013]") {
+    for (const char* f : {"bali_13.h5", "bali_27.h5", "bali_28.h5",
+                          "bali_31.h5", "bali_46.h5"}) {
+        INFO(f);
+        Dump d = load(fixture(f));
+        Replay r = run(d);
+        const double t0 = d.ts.front(), t1 = d.ts.back();
+
+        for (const auto& c : r.claims) {
+            // A window ends at the last sighting, never after it — so it can
+            // never extend past the footage that produced it.
+            CHECK(c.first_seen <= c.last_seen);
+            CHECK(c.first_seen >= t0);
+            CHECK(c.last_seen  <= t1);
+        }
+    }
+}
+
+TEST_CASE("a longer extinction window yields fewer, longer tracks",
+          "[replay][AR-013]") {
+    // The timeout decides whether a gap is absorbed into one window or splits
+    // it in two, so lengthening it must merge tracks rather than multiply them.
+    // On sparse footage this is the difference the constant actually makes.
+    Dump d = load(fixture("bali_46.h5"));   // 140 faces over 385 frames
+    Replay tight = run(d, /*extinction=*/1.0);
+    Replay loose = run(d, /*extinction=*/30.0);
+
+    CHECK(loose.claims.size() <= tight.claims.size());
+}
+
+// ── AR-007 — cuts are exercised by the corpus, not just by construction ──────
+TEST_CASE("the cut-heavy fixture actually contains cuts", "[replay][AR-007]") {
+    // Guards the corpus rather than the code: if a regeneration produced a
+    // fixture with no cuts, the association tests above would still pass while
+    // silently testing nothing about viewpoint changes.
+    Dump d = load(fixture("bali_28.h5"));
+    const int cuts = std::count(d.is_cut.begin(), d.is_cut.end(), uint8_t{1});
+    CHECK(cuts >= 5);
+}