dtourolle/scene-actor-extraction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: presence windows come from registry claims (schema_version 2)

Browse Source 
The sink no longer reconstructs presence from per-frame detections. A reaped
track already IS a window — [first_seen, last_seen] of a track an actor owned —
so it is pushed straight to the aggregator when it dies and written out as-is.

AR-012 completed end to end. The annealing pass is deleted, not disabled:
anneal_sec existed only to bridge gaps between isolated accepted frames, and a
track that survives its own gaps leaves it nothing to do. The field is REMOVED
from the output rather than zeroed — a field naming a mechanism the pipeline no
longer has is actively misleading to anyone reading a manifest, and would
outlive everyone who remembers why it reads 0.

IR-002 — schema_version 2, matching jRay/SPEC.md JR-002. Windows become objects
carrying `belief` and `route` rather than bare float pairs, so a consumer can
caveat or filter instead of treating every window as equally certain. The new
`extraction` block carries `extinction_sec` (the successor to anneal_sec, and
what a consumer actually needs to interpret a window) and `gallery_scope` —
global vs limited being the strongest single quality signal when two manifests
compete for one cut, since identical gallery_size can mean very different
recall.

AR-016 wired: a pre-write hook flushes the registry with the last timestamp
seen, so tracks still live at EOF are emitted. A film ends with faces on screen
and those tracks have not timed out; without this the closing scene's cast is
silently dropped, which reads as a recognition miss rather than a bookkeeping
bug.

IR-003 stays In Progress deliberately: the sink now writes after the flush, but
the deferred re-identification pass (AR-020) does not exist yet, so output is
still final at EOF rather than after it.

This is a BREAKING format change and part of the coordinated SR-003 bump — it
must ship together with the jRay reader and the server's acceptance of the new
shape, not ahead of them.

Suite: 80 cases, 3250 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-012, AR-016, IR-002, IR-003 | SR-002, SR-003
This commit is contained in:
dtourolle
2026-07-31 10:10:51 +02:00
co-authored by Claude Opus 5
parent fe29d014da
commit 08941540cb
5 changed files with 151 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/nodes/result_sink_node.hpp
+85 -34
View File
@@ -2,6 +2,7 @@
/// TRACES: IR-001 | SR-003
#include "types.hpp"
#include "config.hpp"
#include "track_registry.hpp"
#include <nlohmann/json.hpp>
#include <algorithm>
@@ -10,6 +11,8 @@
#include <fstream>
#include <iostream>
#include <map>
#include <mutex>
#include <functional>
#include <string>
#include <vector>
@@ -43,10 +46,26 @@ using json = nlohmann::json;
struct ResultSinkFunc {
static constexpr std::string_view label() { return "result_sink"; }
/// TRACES: AR-012, AR-017, IR-002 | SR-002, SR-003
/// A finished presence claim from the registry. Called from inside the
/// registry's reap while it holds its own lock, so this must stay a cheap
/// push and must never re-enter the registry.
void add_claim(const DeadTrack& d) {
if (d.actor_idx < 0) return; // never owned: nothing to claim
std::lock_guard<std::mutex> g(claims_mu_);
claims_.push_back(d);
}
ResultSinkFunc(const Config& cfg, std::atomic<bool>& done)
: cfg_(cfg), done_(done)
{}
/// TRACES: AR-016 | SR-002
/// Runs immediately before the output is written, with the last timestamp
/// seen. Used to flush tracks still live at EOF, which have not timed out
/// and would otherwise never be emitted.
void set_pre_write_hook(std::function<void(double)> fn) { pre_write_ = std::move(fn); }
void operator()(SceneAnnotation sa) {
if (sa.eof) {
flush();
@@ -59,12 +78,20 @@ struct ResultSinkFunc {
<< " unknowns=" << count_unknown(sa.visible_actors)
<< std::flush;
for (const auto& ia : sa.visible_actors) {
if (ia.actor_idx < 0) continue;
auto& m = actor_meta_[ia.actor_idx];
if (m.name.empty())
m = {ia.name, ia.imdb_id, ia.tmdb_id, ia.jellyfin_id};
}
last_ts_ = sa.timestamp_sec;
frames_.push_back(std::move(sa));
}
// Write accumulated results and signal done. Safe to call more than once.
void flush() {
if (written_.exchange(true)) return;
if (pre_write_) pre_write_(last_ts_);
write_output();
done_.store(true, std::memory_order_release);
}
@@ -72,7 +99,7 @@ struct ResultSinkFunc {
private:
// Bump when the minimal/standard output JSON structure changes in a way
// the Jellyfin plugin needs to detect.
static constexpr int kSchemaVersion = 1;
static constexpr int kSchemaVersion = 2; // SR-003 coordinated bump
static int count_known(const std::vector<IdentifiedActor>& v) {
int n = 0;
@@ -92,10 +119,19 @@ private:
if (cfg_.verbosity == Verbosity::xray) {
root = build_xray();
} else {
/// TRACES: IR-002 | SR-003
/// schema_version 2, per jRay/SPEC.md JR-002. anneal_sec is REMOVED
/// rather than zeroed: a field naming a mechanism the pipeline no
/// longer has is actively misleading, and would outlive everyone who
/// remembers why it reads 0. extinction_sec succeeds it as the
/// parameter that actually shapes window extent.
root["schema_version"] = kSchemaVersion;
root["movie"] = cfg_.movie_path;
root["sample_fps"] = cfg_.sample_fps;
root["anneal_sec"] = cfg_.anneal_sec;
root["movie"] = cfg_.movie_path;
root["extraction"] = {
{"sample_fps", cfg_.sample_fps},
{"extinction_sec", cfg_.track_extinction_sec},
{"gallery_scope", cfg_.gallery_scope},
};
root["actors"] = build_epochs();
if (cfg_.verbosity == Verbosity::standard)
root["frames"] = build_standard();
@@ -110,42 +146,45 @@ private:
std::cerr << "[result_sink] done.\n";
}
struct Window {
double start{0.0};
double end{0.0};
float belief{0.f}; ///< the posterior that justified the claim (AR-017)
};
struct ActorWindow {
std::string name, imdb_id, tmdb_id, jellyfin_id;
std::vector<std::pair<double, double>> scenes; // [start_sec, end_sec]
std::vector<Window> scenes;
};
struct ActorMeta { std::string name, imdb_id, tmdb_id, jellyfin_id; };
// Core logic: merge per-frame detections into annealed [start, end] windows.
/// TRACES: AR-012, IR-002 | SR-002
/// A claim already IS a window — `[first_seen, last_seen]` of a track the
/// actor owned. There is no annealing pass: `anneal_sec` existed to bridge
/// gaps between isolated accepted frames, and a track that survives its own
/// gaps leaves it nothing to do (see the AR-012 withdrawal note).
std::vector<ActorWindow> build_actor_windows() {
struct Info { std::string name, imdb_id, tmdb_id, jellyfin_id; };
std::map<int, Info> actor_info;
std::map<int, std::vector<double>> timestamps;
std::lock_guard<std::mutex> g(claims_mu_);
for (const auto& frame : frames_) {
for (const auto& ia : frame.visible_actors) {
if (ia.actor_idx < 0) continue;
actor_info[ia.actor_idx] = {ia.name, ia.imdb_id, ia.tmdb_id, ia.jellyfin_id};
timestamps[ia.actor_idx].push_back(frame.timestamp_sec);
std::map<int, ActorWindow> by_actor;
for (const auto& c : claims_) {
auto& aw = by_actor[c.actor_idx];
if (aw.name.empty()) {
auto it = actor_meta_.find(c.actor_idx);
if (it != actor_meta_.end()) {
aw.name = it->second.name;
aw.imdb_id = it->second.imdb_id;
aw.tmdb_id = it->second.tmdb_id;
aw.jellyfin_id = it->second.jellyfin_id;
}
}
aw.scenes.push_back({c.first_seen, c.last_seen, c.belief});
}
std::vector<ActorWindow> result;
for (auto& [idx, ts_vec] : timestamps) {
ActorWindow aw;
aw.name = actor_info[idx].name;
aw.imdb_id = actor_info[idx].imdb_id;
aw.tmdb_id = actor_info[idx].tmdb_id;
aw.jellyfin_id = actor_info[idx].jellyfin_id;
double win_start = ts_vec[0], win_end = ts_vec[0];
for (size_t i = 1; i < ts_vec.size(); ++i) {
if (ts_vec[i] - win_end > cfg_.anneal_sec) {
aw.scenes.push_back({win_start, win_end});
win_start = ts_vec[i];
}
win_end = ts_vec[i];
}
aw.scenes.push_back({win_start, win_end});
for (auto& [idx, aw] : by_actor) {
std::sort(aw.scenes.begin(), aw.scenes.end(),
[](const Window& a, const Window& b) { return a.start < b.start; });
result.push_back(std::move(aw));
}
return result;
@@ -154,9 +193,16 @@ private:
json build_epochs() {
json actors = json::array();
for (const auto& aw : build_actor_windows()) {
// Objects, not float pairs: a window carries the belief that
// justified it and the route by which it was identified (AR-017),
// so a consumer can caveat or filter rather than treating every
// window as equally certain.
json windows = json::array();
for (const auto& [s, e] : aw.scenes)
windows.push_back({s, e});
for (const auto& w : aw.scenes)
windows.push_back({{"start", w.start},
{"end", w.end},
{"belief", w.belief},
{"route", "live"}});
json ja;
ja["name"] = aw.name;
ja["imdb_id"] = aw.imdb_id;
@@ -174,9 +220,9 @@ private:
json build_xray() {
std::map<int, std::vector<std::string>> xray;
for (const auto& aw : build_actor_windows()) {
for (const auto& [start, end] : aw.scenes) {
int t0 = static_cast<int>(std::floor(start));
int t1 = static_cast<int>(std::ceil(end));
for (const auto& w : aw.scenes) {
int t0 = static_cast<int>(std::floor(w.start));
int t1 = static_cast<int>(std::ceil(w.end));
for (int t = t0; t <= t1; ++t)
xray[t].push_back(aw.name);
}
@@ -227,4 +273,9 @@ private:
std::atomic<bool>& done_;
std::atomic<bool> written_{false};
std::vector<SceneAnnotation> frames_;
std::function<void(double)> pre_write_;
double last_ts_{0.0};
std::mutex claims_mu_;
std::vector<DeadTrack> claims_;
std::map<int, ActorMeta> actor_meta_; ///< actor_idx → identity keys
};