feat(quality): score every face on sharpness and alignment before it is evidence

Every embedding now carries the quality of the input it came from. Both
axes fall out of the AR-005 warp for free: crop_sharpness() is the
normalised Laplacian variance over the aligned 112x112, so contrast and
size cannot leak into it, and the alignment residual is the part of the
landmark deformation a similarity transform cannot explain, so in-plane
roll reads as zero and foreshortening does not.

Carried, not consumed. Nothing discounts or thresholds on either number
yet -- that is AR-030 and VR-012, and the knee has to be located against
recorded data before a gate is chosen. What this change buys is that the
data exists to locate it with.

No face is admitted unscored: the -1 sentinel is preserved rather than
clamped, and a degenerate landmark fit is counted rather than silently
dropped.

Takes the VR-001 dump to schema_version 2. The bump is not for readers,
which check for the datasets by name and replay a v1 dump unchanged; it
is so a consumer can tell "never scored" from "scored zero", which is
not recoverable from the arrays afterwards.

TRACES: AR-028, AR-029, AR-030 | VR-001 | SR-002
This commit is contained in:
2026-08-05 14:37:30 +02:00
parent c1155cb607
commit 777c98cb33
10 changed files with 726 additions and 17 deletions
+26 -3
View File
@@ -1,5 +1,5 @@
#pragma once
/// TRACES: VR-001, VR-010 | PR-002
/// TRACES: AR-028 | VR-001, VR-010 | PR-002
#include "types.hpp"
#include "config.hpp"
#include "gallery/embedder_stamp.hpp"
@@ -178,6 +178,16 @@ struct EmbeddingDumpFunc {
lmk_.push_back(f.landmarks[k].y);
}
conf_.push_back(f.confidence);
/// TRACES: AR-028 | SR-002
// The quality vector, carried rather than consumed: written beside
// the embedding it describes so VR-012 can locate its knees against
// recorded data instead of by re-running video. Size is the third
// axis and is already here as bbox + the bbox_upscale attribute.
// Both are -1 only if a face reached the dump unscored, which the
// aligner does not allow — the sentinel is preserved rather than
// clamped so that a future path which did would be visible.
sharp_.push_back(f.sharpness);
resid_.push_back(f.alignment_residual);
const auto& e = ef.embeddings[i];
emb_.insert(emb_.end(), e.begin(), e.end());
}
@@ -194,11 +204,20 @@ struct EmbeddingDumpFunc {
}
private:
// Root attributes are additive: schema_version stays 1 across VR-010, because
// Root attributes are additive: schema_version stayed 1 across VR-010, because
// every reader takes attributes by name with a default (replay.py) or an
// existence check (read_dump_provenance), so an old dump loses nothing and a
// new dump breaks nothing. A bump is for a change to the *datasets*.
static constexpr int kSchemaVersion = 1;
//
// v2 is that change: AR-028 adds faces/sharpness and faces/alignment_residual.
// The bump is not about readers — those check for the datasets by name, and a
// v1 dump still replays. It is so a *consumer of the quality vector* can tell
// "this film's faces were never scored" from "this film's faces scored zero",
// which is the same distinction scene_detect exists to make and is likewise
// not recoverable from the arrays. A v1 dump reports the vector as unknown;
// re-dump to acquire it, since nobody can assert after the fact how sharp a
// face was.
static constexpr int kSchemaVersion = 2;
static constexpr int kEmbedDim = 512;
static std::string basename_of(const std::string& path) {
@@ -274,6 +293,9 @@ private:
write_vec(faces, "bbox", bbox_, H5::PredType::NATIVE_FLOAT, 4);
write_vec(faces, "landmarks", lmk_, H5::PredType::NATIVE_FLOAT, 10);
write_vec(faces, "confidence", conf_, H5::PredType::NATIVE_FLOAT);
/// TRACES: AR-028 | SR-002
write_vec(faces, "sharpness", sharp_, H5::PredType::NATIVE_FLOAT);
write_vec(faces, "alignment_residual", resid_, H5::PredType::NATIVE_FLOAT);
std::cerr << "[embedding_dump] wrote " << ts_.size() << " frames, "
<< conf_.size() << " faces → " << path_ << "\n";
@@ -292,4 +314,5 @@ private:
std::vector<int64_t> face_off_;
std::vector<int32_t> face_cnt_;
std::vector<float> emb_, bbox_, lmk_, conf_;
std::vector<float> sharp_, resid_; // AR-028 quality vector, parallel to conf_
};