dtourolle/scene-actor-extraction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(engine): HDF5-native galleries with embedded calibration; TensorRT backends; scene detection

Browse Source 
Gallery format switches from JSON to HDF5 exclusively (JSON read-only kept for
back-compat): save_gallery always writes HDF5, and the fitted Platt-sigmoid
calibration (a, b, valid, hash) is now embedded directly in the gallery file
instead of a sidecar .calib_cache.json — identity_matcher reads it from the
loaded gallery and writes back only when the embeddings actually changed
(hash mismatch), skipping the O(n^2) refit otherwise.

Also includes: TensorRT inference backend support (ort_backend.cpp,
trt_backend.cpp), gemm_backend improvements, TransNetV2-based scene-boundary
detection wired through frame_source/face_tracker/main, and CMake build
target updates for the new sources.

Bumps the KPN submodule to feature/persistent-pipeline-reuse (push_blocking
backpressure, node_ptr/node_stats introspection, ObjectVariantNodeWrapper for
stateful functors) — needed by the optimizer's sae_kpn Python bindings.
This commit is contained in:
dtourolle
2026-07-19 19:04:03 +02:00
parent aca6147d69
commit 41a277bc19
19 changed files with 1151 additions and 216 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/scene_preview.cpp
+30 -4
View File
@@ -3,8 +3,11 @@
//
// KPN topology:
//
// [frame_source] ──► [face_detector] ──► [face_aligner] ──► [embedder]
// [frame_source] ──► [camera_pos] ──► [face_detector] ──► [face_aligner] ──► [embedder]
// ──► [identity_matcher] ──► FanoutNode<MatchedSceneFrame,2>
//
// camera_pos (histogram cut detector) stamps Frame::cut_score / is_cut, which
// ride through to the preview HUD's cut-score meter.
// ├──► [scene_tracker] ──► [result_sink] (background thread)
// └──► [preview_node] (main thread)
//
@@ -20,6 +23,7 @@
#include "types.hpp"
#include "gallery/gallery_store.hpp"
#include "nodes/frame_source_node.hpp"
#include "nodes/camera_position_change_detector_node.hpp"
#include "nodes/face_detector_node.hpp"
#include "nodes/face_aligner_node.hpp"
#include "nodes/embedder_node.hpp"
@@ -87,6 +91,25 @@ static Config parse_args(int argc, char** argv) {
else if (arg("--no-trt-fp16")) cfg.trt.fp16 = false;
else if (arg("--trt-int8")) cfg.trt.int8 = true;
else if (arg("--embed-batch")) cfg.embed_batch_size = std::stoi(next());
// Per-film gallery expansion — preview supports it (same cfg fields).
else if (arg("--expand-gallery")) cfg.expand_gallery = true;
else if (arg("--expand-buffer")) cfg.expand_buffer_size = std::stoi(next());
else if (arg("--expand-novelty-sim")) cfg.expand_novelty_sim = std::stof(next());
else if (arg("--expand-spread-max")) cfg.expand_track_spread_max = std::stof(next());
else if (arg("--expand-min-anchor")) cfg.expand_min_anchor_frames = std::stoi(next());
// Scene detection is scene_analyze-only (needs the dense TransNetV2 branch).
// Accept the flags so a shared command line runs, but note they're inert
// here — the preview shows the histogram cut-score meter instead.
else if (arg("--scene-detect")) {
std::cerr << "[preview] note: --scene-detect is inert in preview "
"(TransNetV2 needs the dense scene_analyze pipeline); "
"showing the histogram cut-score meter instead\n";
}
else if (arg("--scene-detector") || arg("--scene-detector-engine") ||
arg("--scene-threshold") || arg("--scene-stride") ||
arg("--scene-decode-fps") || arg("--dense-scale")) {
next(); // consume the value; inert in preview
}
else { std::cerr << "[warn] unknown flag: " << argv[i] << "\n"; }
}
if (cfg.movie_path.empty()) throw std::runtime_error("--movie is required");
@@ -112,7 +135,8 @@ int main(int argc, char** argv) {
// ── Functors ──────────────────────────────────────────────────────────────
std::atomic<bool> done{false};
FrameSourceFunc source_fn {cfg};
FrameSourceFunc source_fn {cfg};
CameraPositionChangeDetectorFunc campos_fn {cfg};
FaceDetectorFunc detector_fn{cfg};
FaceAlignerFunc aligner_fn;
EmbedderFunc embedder_fn{cfg};
@@ -122,7 +146,8 @@ int main(int argc, char** argv) {
ResultSinkFunc sink_fn {cfg, done};
// ── KPN ObjectNodes ───────────────────────────────────────────────────────
kpn::ObjectNode<FrameSourceFunc, kpn::in<>, kpn::out<"frame">, "frame_source", 0> source (source_fn, 32);
kpn::ObjectNode<FrameSourceFunc, kpn::in<>, kpn::out<"raw">, "frame_source", 0> source (source_fn, 32);
kpn::ObjectNode<CameraPositionChangeDetectorFunc, kpn::in<"raw">, kpn::out<"frame">, "camera_pos", 0> campos (campos_fn, 32);
kpn::ObjectNode<FaceDetectorFunc, kpn::in<"frame">, kpn::out<"scene">, "face_detector", 0> detector (detector_fn, 64);
kpn::ObjectNode<FaceAlignerFunc, kpn::in<"scene">, kpn::out<"aligned">, "face_aligner", 0> aligner (aligner_fn, 64);
kpn::ObjectNode<EmbedderFunc, kpn::in<"aligned">, kpn::out<"embedded">, "embedder", 0> embedder (embedder_fn, 32);
@@ -136,7 +161,8 @@ int main(int argc, char** argv) {
// matcher → FanoutNode<MatchedSceneFrame,2> → [scene_tracker, preview] (auto-inserted)
auto net = kpn::make_network(
kpn::edge(source.output<"frame">(), detector.input<"frame">()),
kpn::edge(source.output<"raw">(), campos.input<"raw">()),
kpn::edge(campos.output<"frame">(), detector.input<"frame">()),
kpn::edge(detector.output<"scene">(), aligner.input<"scene">()),
kpn::edge(aligner.output<"aligned">(), embedder.input<"aligned">()),
kpn::edge(embedder.output<"embedded">(), ftracker.input<"embedded">()),