dtourolle
d98dc2855a
Bali was chosen because the TRECVID DVU set ships character mugshots, but its reference crops are unusable at scale: median detected face 27 px against a 69 px maximum, so every reference was upscaled 4x or more past what the embedder was trained for (AR-011). A 66 px floor left 2 of 69 references; no threshold exists that both keeps the faces in distribution and leaves enough of them to calibrate. SuperHero is 69 px median and 241 px max. Its gallery builds at a 66 px floor with 14 references over 5 characters, and calibrates on its own (a=15.2867 b=-4.98633, 100% train accuracy) instead of borrowing constants. Measured on the fused 17-minute film, one stream rather than per-scene clips so presence windows cross real scene boundaries as SR-002 intends: precision 1.00, recall 0.65, F1 0.79 — 13 true positives, 0 false positives, 7 misses. Every out-of-gallery character was declined rather than forced onto a nearest match. The misses are the short scenes (14 s, 38 s, 27 s), consistent with per-track accumulation needing sightings. - build_gallery gains --min-face-px, filtering the *detected face* rather than the crop. The DVU images are scene crops, not mugshots, so crop dimensions say nothing about face scale. A poisoned reference is permanent in a way a bad frame is not: it corrupts every future match against that identity. - scripts/fetch_dvu.sh fetches mugshots, scene graphs and segmentation for any DVU film. NIST names the same film three different ways, so KG_DIR and KG_FILE are overridable rather than derived. This exists as a script because the first copy of this data was assembled ad hoc in /tmp and was lost with it, taking the working gallery along. - Replay fixtures move to the artifact registry: push/pull_artifacts.sh gain a replay-fixtures target, and tests/fixtures/dumps/.gitignore keeps them out of git. superhero.h5 is ~9 MB and regenerating it needs the film, the models and a GPU — none of which CI has. The gallery ships with the dumps, since a dump only replays against the gallery it was produced with. - AR-012 and AR-013 coverage is ported onto the new fixture rather than dropped with the Bali cases: 12369 assertions, up from 7991, since the film is an order of magnitude larger than the clips. Suite: 15679 assertions, 101 test cases. TRACES: AR-011, AR-012, AR-013 | VR-001, VR-005 | SR-002
116 lines
4.5 KiB
C++
116 lines
4.5 KiB
C++
// build_gallery — build an actor embedding gallery from a directory of images
|
|
//
|
|
// Gallery directory layout:
|
|
// gallery_root/
|
|
// nm0000093_Brad_Pitt/
|
|
// img1.jpg
|
|
// img2.jpg
|
|
// nm0000129_Cate_Blanchett/
|
|
// ...
|
|
//
|
|
// Usage:
|
|
// build_gallery --root <gallery_root> --output <gallery.json> [options]
|
|
//
|
|
// Options:
|
|
// --detector <path> SCRFD detector model (default: models/scrfd_500m_bnkps.onnx)
|
|
// --arcface <path> ArcFace model (default: models/arcface_w600k_r50.onnx)
|
|
// --conf <f> face detection confidence threshold (default: 0.5)
|
|
// --nms <f> NMS IoU threshold (default: 0.4)
|
|
|
|
#include "gallery/gallery_builder.hpp"
|
|
#include "gallery/gallery_report.hpp"
|
|
#include "gallery/gallery_calibration.hpp"
|
|
#include "gallery/gallery_store.hpp"
|
|
#include "config.hpp"
|
|
|
|
#include <cstring>
|
|
#include <iostream>
|
|
#include <stdexcept>
|
|
#include <string>
|
|
|
|
int main(int argc, char** argv) {
|
|
std::string root_path, output_path;
|
|
std::string detector_model = kDefaultDetectorModel;
|
|
std::string arcface_model = kDefaultArcfaceModel;
|
|
float conf = 0.5f, nms_thr = 0.4f;
|
|
int max_side = 500;
|
|
float min_face_px = 0.f;
|
|
|
|
for (int i = 1; i < argc; ++i) {
|
|
auto arg = [&](const char* f) { return std::strcmp(argv[i], f) == 0; };
|
|
auto next = [&]() -> std::string {
|
|
if (++i >= argc)
|
|
throw std::runtime_error(std::string("missing arg after ") + argv[i-1]);
|
|
return argv[i];
|
|
};
|
|
try {
|
|
if (arg("--root")) root_path = next();
|
|
else if (arg("--output")) output_path = next();
|
|
else if (arg("--detector")) detector_model = next();
|
|
else if (arg("--arcface")) arcface_model = next();
|
|
else if (arg("--conf")) conf = std::stof(next());
|
|
else if (arg("--nms")) nms_thr = std::stof(next());
|
|
else if (arg("--max-side")) max_side = std::stoi(next());
|
|
else if (arg("--min-face-px")) min_face_px = std::stof(next());
|
|
else { std::cerr << "[warn] unknown flag: " << argv[i] << "\n"; }
|
|
} catch (const std::exception& e) {
|
|
std::cerr << "Error: " << e.what() << "\n";
|
|
return 1;
|
|
}
|
|
}
|
|
|
|
if (root_path.empty() || output_path.empty()) {
|
|
std::cerr << "Usage: build_gallery --root <dir> --output <gallery.json> "
|
|
"[--detector <path>] [--arcface <path>] [--max-side <N>]\n"
|
|
" [--min-face-px <px>]\n";
|
|
return 1;
|
|
}
|
|
|
|
BuildConfig cfg;
|
|
cfg.gallery_root = root_path;
|
|
cfg.detector_model = detector_model;
|
|
cfg.arcface_model = arcface_model;
|
|
cfg.detector_conf = conf;
|
|
cfg.detector_nms = nms_thr;
|
|
cfg.max_side = max_side;
|
|
cfg.min_face_px = min_face_px;
|
|
|
|
try {
|
|
ActorGallery gallery = build_gallery(cfg);
|
|
if (gallery.actors.empty()) {
|
|
std::cerr << "No actors built — check your gallery directory.\n";
|
|
return 1;
|
|
}
|
|
save_gallery(output_path, gallery);
|
|
std::cerr << "Gallery saved to: " << output_path << "\n";
|
|
|
|
/// TRACES: GR-003 | SR-001
|
|
// Fit the calibration here and persist what it learned. The matcher
|
|
// fits the same sigmoid at analysis time, but that is the wrong place
|
|
// to audit a gallery from: by then the answer is per-run and nobody is
|
|
// looking. Build time is when the gallery's quality is decided, and a
|
|
// gallery can be quietly bad — heavily overlapping intra/inter
|
|
// distributions, actors with no usable image — while looking fine.
|
|
std::vector<Embedding> flat;
|
|
std::vector<int> flat_actor;
|
|
for (int ai = 0; ai < static_cast<int>(gallery.actors.size()); ++ai)
|
|
for (const auto& e : gallery.actors[ai].embeddings) {
|
|
flat.push_back(e);
|
|
flat_actor.push_back(ai);
|
|
}
|
|
|
|
GalleryCalibrationStats stats;
|
|
GalleryCalibration cal = calibrate_gallery(flat, flat_actor, &stats);
|
|
|
|
const GalleryReport report =
|
|
build_gallery_report(gallery, cal, stats, nullptr, output_path);
|
|
const std::string report_path = gallery_report_path(output_path);
|
|
save_gallery_report(report_path, report);
|
|
std::cerr << "Gallery report saved to: " << report_path << "\n";
|
|
} catch (const std::exception& e) {
|
|
std::cerr << "Fatal: " << e.what() << "\n";
|
|
return 1;
|
|
}
|
|
return 0;
|
|
}
|