feat(quality): five AR-029 sharpness candidates on the aligned crop
assess_sharpness() scores a 112x112 crop on variance-of-Laplacian, a contrast-normalised variant, Tenengrad, a spectral high-frequency ratio and dir_min_tenengrad, over a fixed 64x64 window on the face interior. The window excludes the corners because studio headshots are routinely shot at a wide aperture, and background bokeh measured over the whole crop would drag the score down on the sharpest images in the set. Five rather than one because AR-029's threshold has to be located, not chosen: VR-012 ranks them by how well each predicts real identity loss. The T1 ladders drove two corrections during development. The spectral ratio applied its Hann window before removing the mean, so the DC term smeared into the low-frequency bins and the "ratio" tracked absolute brightness (a 20/255 brightening moved it 23%). And no measure taken from the literature survived directional blur: normalising by total energy divides out the loss being measured, so both ratio measures are U-shaped in motion-blur length and score a 21 px smear about as sharp as a 3 px one. dir_min_tenengrad exists to fix that -- a low-frequency contrast denominator that blur leaves alone, and the worse of the two Sobel axes rather than their sum. The tests pin the disqualifying behaviours as well as the desirable ones, so a change that makes var_laplacian contrast-free is a deliberate act rather than an accident. They also record that every candidate falls under downscale-upscale as well as under blur: the aligned crop is scale-normalised geometrically, not informationally. Exposed through sae_embed alongside the AR-030 alignment residual, so a study scores through shipped code rather than a numpy copy -- the same argument that already applies to the calibration. TRACES: AR-028, AR-029 | SR-002
This commit is contained in:
@@ -19,6 +19,7 @@ add_executable(sae_tests
|
||||
test_calibration.cpp
|
||||
test_gallery_store.cpp
|
||||
test_face_utils.cpp
|
||||
test_quality.cpp
|
||||
test_track_gallery.cpp
|
||||
test_face_tracker.cpp
|
||||
test_track_registry.cpp
|
||||
|
||||
@@ -0,0 +1,302 @@
|
||||
// TRACES: AR-029 | SR-002
|
||||
//
|
||||
// T1 for the AR-029 sharpness candidates: the properties that have to hold
|
||||
// before a study is allowed to pick between them. GPU-free, model-free.
|
||||
//
|
||||
// The register's acceptance criterion is "synthetic blur ladder ->
|
||||
// monotonically falling sharpness; Gaussian vs motion blur; small sharp face vs
|
||||
// large soft one — size must not leak into this axis". The last clause needs
|
||||
// care, and the tests below split it in two:
|
||||
//
|
||||
// - What must NOT leak is *geometric* scale. The measure is taken in the
|
||||
// canonical frame, so changing how big the face was in the source while
|
||||
// preserving its detail must not move the score. That is structural: the
|
||||
// window is fixed at 64x64 canonical px.
|
||||
// - What DOES legitimately move the score is lost *detail*. A face that was
|
||||
// 40 px before being warped up to 112 really does carry less
|
||||
// high-frequency content than one that was 400 px, and a measure blind to
|
||||
// that would be blind to the thing it exists to catch.
|
||||
//
|
||||
// So "size must not leak" cannot mean "invariant to the source face size", and
|
||||
// the ladder test below asserts the opposite on purpose. What it buys is that
|
||||
// the overlap with AR-002 is a recorded property with a test naming it, rather
|
||||
// than a surprise VR-012 discovers when the two axes turn out to be correlated.
|
||||
#include <catch2/catch_test_macros.hpp>
|
||||
#include <catch2/matchers/catch_matchers_floating_point.hpp>
|
||||
|
||||
#include "quality.hpp"
|
||||
#include "types.hpp" // kArcFaceRef, for the window-placement test
|
||||
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
using Catch::Matchers::WithinAbs;
|
||||
using Catch::Matchers::WithinRel;
|
||||
|
||||
namespace {
|
||||
|
||||
// A deterministic 112x112 stand-in for a face crop.
|
||||
//
|
||||
// **Broadband, not a sum of a few sinusoids.** An earlier version of this
|
||||
// fixture used three discrete spatial frequencies, and the resampling ladder
|
||||
// below was non-monotone for hf_energy_ratio because of it: a period-7
|
||||
// component downsampled to 32 px lands exactly at Nyquist and aliases, so the
|
||||
// ratio rose at one rung instead of falling. That is a property of a
|
||||
// three-tone test pattern meeting a resampler, not of the measure or of any
|
||||
// face — a real crop has energy spread across the band, where such a
|
||||
// resonance averages out. Deterministic value noise, smoothed to give the
|
||||
// roughly 1/f falloff of a photograph, exercises the whole band at once.
|
||||
//
|
||||
// Mid-grey base with bounded amplitude, so scaling the contrast in the tests
|
||||
// below does not clip.
|
||||
cv::Mat synthetic_crop() {
|
||||
// Fixed LCG rather than cv::randu: the suite must not depend on OpenCV's
|
||||
// RNG state, which other tests share.
|
||||
uint32_t seed = 0x5eed1234u;
|
||||
auto next = [&seed] {
|
||||
seed = seed * 1664525u + 1013904223u;
|
||||
return (seed >> 16) & 0xffffu;
|
||||
};
|
||||
|
||||
cv::Mat noise(112, 112, CV_32F);
|
||||
for (int y = 0; y < 112; ++y)
|
||||
for (int x = 0; x < 112; ++x)
|
||||
noise.at<float>(y, x) = float(next()) / 65535.f - 0.5f;
|
||||
|
||||
// Mild smoothing: white noise is flat to Nyquist, which no lens produces
|
||||
// and which would make the sharpest rung of every ladder unrealistic.
|
||||
cv::Mat smooth;
|
||||
cv::GaussianBlur(noise, smooth, cv::Size(0, 0), 0.8);
|
||||
cv::normalize(smooth, smooth, -1.0, 1.0, cv::NORM_MINMAX);
|
||||
|
||||
cv::Mat img(112, 112, CV_8UC3);
|
||||
for (int y = 0; y < 112; ++y) {
|
||||
for (int x = 0; x < 112; ++x) {
|
||||
double v = 128.0 + 70.0 * smooth.at<float>(y, x);
|
||||
const auto b = static_cast<uchar>(std::clamp(v, 0.0, 255.0));
|
||||
img.at<cv::Vec3b>(y, x) = {b, b, b};
|
||||
}
|
||||
}
|
||||
return img;
|
||||
}
|
||||
|
||||
cv::Mat gaussian(const cv::Mat& src, double sigma) {
|
||||
cv::Mat out;
|
||||
cv::GaussianBlur(src, out, cv::Size(0, 0), sigma, sigma);
|
||||
return out;
|
||||
}
|
||||
|
||||
// Horizontal box blur — the camera-pan case, and the one an isotropic measure
|
||||
// could in principle miss.
|
||||
cv::Mat motion(const cv::Mat& src, int len) {
|
||||
cv::Mat kernel = cv::Mat::zeros(1, len, CV_32F);
|
||||
kernel.setTo(1.0f / len);
|
||||
cv::Mat out;
|
||||
cv::filter2D(src, out, -1, kernel);
|
||||
return out;
|
||||
}
|
||||
|
||||
// Throw away detail a face detected at size x size never had, then warp back up
|
||||
// to the 112x112 the embedder is fed — the VR-005 degradation.
|
||||
cv::Mat rescale(const cv::Mat& src, int size) {
|
||||
if (size == 112) return src.clone();
|
||||
cv::Mat small, out;
|
||||
cv::resize(src, small, {size, size}, 0, 0, cv::INTER_AREA);
|
||||
cv::resize(small, out, {112, 112}, 0, 0, cv::INTER_LINEAR);
|
||||
return out;
|
||||
}
|
||||
|
||||
std::vector<float> field(const std::vector<SharpnessScores>& s,
|
||||
float SharpnessScores::* m) {
|
||||
std::vector<float> v;
|
||||
v.reserve(s.size());
|
||||
for (const auto& x : s) v.push_back(x.*m);
|
||||
return v;
|
||||
}
|
||||
|
||||
void check_strictly_falling(const std::vector<float>& v, const char* what) {
|
||||
INFO(what);
|
||||
for (size_t i = 1; i < v.size(); ++i) {
|
||||
INFO("step " << i << ": " << v[i - 1] << " -> " << v[i]);
|
||||
CHECK(v[i] < v[i - 1]);
|
||||
}
|
||||
}
|
||||
|
||||
const std::vector<std::pair<const char*, float SharpnessScores::*>> kMeasures{
|
||||
{"var_laplacian", &SharpnessScores::var_laplacian},
|
||||
{"norm_var_laplacian", &SharpnessScores::norm_var_laplacian},
|
||||
{"tenengrad", &SharpnessScores::tenengrad},
|
||||
{"hf_energy_ratio", &SharpnessScores::hf_energy_ratio},
|
||||
{"dir_min_tenengrad", &SharpnessScores::dir_min_tenengrad},
|
||||
};
|
||||
|
||||
} // namespace
|
||||
|
||||
TEST_CASE("every candidate falls monotonically along a Gaussian blur ladder",
|
||||
"[quality][AR-029]") {
|
||||
const cv::Mat base = synthetic_crop();
|
||||
std::vector<SharpnessScores> ladder;
|
||||
for (double sigma : {0.0, 0.5, 1.0, 1.5, 2.0, 3.0})
|
||||
ladder.push_back(assess_sharpness(sigma == 0.0 ? base : gaussian(base, sigma)));
|
||||
|
||||
for (const auto& [name, m] : kMeasures) {
|
||||
REQUIRE(ladder.front().ok);
|
||||
check_strictly_falling(field(ladder, m), name);
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("only the absolute and directional measures survive motion blur",
|
||||
"[quality][AR-029]") {
|
||||
// Motion blur is the commonest way a film frame is unusable, and it is
|
||||
// where the candidates separate. A horizontal smear destroys horizontal
|
||||
// detail and leaves vertical detail untouched, so what a measure does here
|
||||
// depends on whether it can be fooled by the surviving axis.
|
||||
const cv::Mat base = synthetic_crop();
|
||||
std::vector<SharpnessScores> ladder{assess_sharpness(base)};
|
||||
for (int len : {3, 5, 9, 15, 21})
|
||||
ladder.push_back(assess_sharpness(motion(base, len)));
|
||||
|
||||
// Total gradient/Laplacian energy keeps falling: nothing replaces what the
|
||||
// smear removed.
|
||||
check_strictly_falling(field(ladder, &SharpnessScores::var_laplacian),
|
||||
"var_laplacian");
|
||||
check_strictly_falling(field(ladder, &SharpnessScores::tenengrad),
|
||||
"tenengrad");
|
||||
// The fix for the two below: low-frequency denominator, and the worse of
|
||||
// the two axes rather than their sum.
|
||||
check_strictly_falling(field(ladder, &SharpnessScores::dir_min_tenengrad),
|
||||
"dir_min_tenengrad");
|
||||
|
||||
// The disqualifying behaviour, pinned rather than hidden. Both measures
|
||||
// normalise by a quantity that contains the detail they are measuring, so
|
||||
// once the horizontal band is gone the quotient climbs back toward its
|
||||
// unblurred value: each is U-shaped in blur length, and a single score
|
||||
// maps to two very different amounts of blur. A 21 px smear scores about
|
||||
// as sharp as a 3 px one.
|
||||
for (const auto& [name, m] : {
|
||||
std::pair{"norm_var_laplacian", &SharpnessScores::norm_var_laplacian},
|
||||
std::pair{"hf_energy_ratio", &SharpnessScores::hf_energy_ratio}}) {
|
||||
const std::vector<float> v = field(ladder, m);
|
||||
INFO(name);
|
||||
const auto trough = std::min_element(v.begin(), v.end());
|
||||
CHECK(trough != v.begin()); // it does fall at first …
|
||||
CHECK(trough != v.end() - 1); // … then turns back up
|
||||
CHECK(v.back() > 0.8f * v[1]); // recovering most of one rung
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("sharpness falls under downscale-upscale as well as under blur",
|
||||
"[quality][AR-029]") {
|
||||
// The overlap with AR-002, asserted rather than assumed. Losing resolution
|
||||
// and losing focus are the same loss of high-frequency content, so every
|
||||
// candidate reads a small upscaled face as less sharp. VR-012's joint
|
||||
// size x sigma grid decides whether that makes a sharpness discount a
|
||||
// double-count against the size gate, or whether the two axes carry
|
||||
// separable information.
|
||||
const cv::Mat base = synthetic_crop();
|
||||
std::vector<SharpnessScores> ladder;
|
||||
for (int size : {112, 64, 48, 32, 24, 16})
|
||||
ladder.push_back(assess_sharpness(rescale(base, size)));
|
||||
|
||||
for (const auto& [name, m] : kMeasures)
|
||||
check_strictly_falling(field(ladder, m), name);
|
||||
}
|
||||
|
||||
TEST_CASE("the ratio measures are contrast-free and the raw ones are not",
|
||||
"[quality][AR-029]") {
|
||||
// The confound that decides the bake-off. A gallery drawn from thousands of
|
||||
// cameras, lighting setups and JPEG pipelines varies enormously in
|
||||
// contrast, and a measure that reads a low-contrast sharp face as blurred
|
||||
// would discount it for the photographer's choices rather than for anything
|
||||
// the embedder cares about.
|
||||
const cv::Mat base = synthetic_crop();
|
||||
|
||||
// Halve the contrast about mid-grey, leaving spatial structure untouched.
|
||||
cv::Mat low;
|
||||
base.convertTo(low, CV_8UC3, 0.5, 64.0);
|
||||
|
||||
const auto s_hi = assess_sharpness(base);
|
||||
const auto s_lo = assess_sharpness(low);
|
||||
REQUIRE(s_hi.ok);
|
||||
REQUIRE(s_lo.ok);
|
||||
|
||||
// Invariant by construction: both are ratios in which the contrast factor
|
||||
// cancels.
|
||||
CHECK_THAT(s_lo.norm_var_laplacian,
|
||||
WithinRel(s_hi.norm_var_laplacian, 0.02f));
|
||||
CHECK_THAT(s_lo.hf_energy_ratio, WithinRel(s_hi.hf_energy_ratio, 0.02f));
|
||||
|
||||
// Not invariant: both scale with the square of the contrast factor, so
|
||||
// halving the contrast quarters them. This is the disqualifying behaviour,
|
||||
// pinned so that a change making them contrast-free is a deliberate one.
|
||||
CHECK_THAT(s_lo.var_laplacian, WithinRel(0.25f * s_hi.var_laplacian, 0.05f));
|
||||
CHECK_THAT(s_lo.tenengrad, WithinRel(0.25f * s_hi.tenengrad, 0.05f));
|
||||
}
|
||||
|
||||
TEST_CASE("brightness alone moves nothing", "[quality][AR-029]") {
|
||||
const cv::Mat base = synthetic_crop();
|
||||
cv::Mat bright;
|
||||
base.convertTo(bright, CV_8UC3, 1.0, 20.0);
|
||||
|
||||
const auto a = assess_sharpness(base);
|
||||
const auto b = assess_sharpness(bright);
|
||||
for (const auto& [name, m] : kMeasures) {
|
||||
INFO(name);
|
||||
CHECK_THAT(b.*m, WithinRel(a.*m, 0.02f));
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("a flat crop is scored not-ok rather than given a number",
|
||||
"[quality][AR-029]") {
|
||||
// A face whose sharpness cannot be computed is a fact to record, not an
|
||||
// absence — the same rule AR-030 follows for degenerate landmarks.
|
||||
const cv::Mat flat(112, 112, CV_8UC3, cv::Scalar(128, 128, 128));
|
||||
const auto s = assess_sharpness(flat);
|
||||
CHECK_FALSE(s.ok);
|
||||
for (const auto& [name, m] : kMeasures) {
|
||||
INFO(name);
|
||||
CHECK_THAT(s.*m, WithinAbs(0.0f, 1e-6f));
|
||||
CHECK_FALSE(std::isnan(s.*m));
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("a crop smaller than the measurement window is scored not-ok",
|
||||
"[quality][AR-029]") {
|
||||
const cv::Mat small(64, 64, CV_8UC3, cv::Scalar(40, 90, 160));
|
||||
CHECK_FALSE(assess_sharpness(small).ok);
|
||||
CHECK_FALSE(assess_sharpness(cv::Mat()).ok);
|
||||
}
|
||||
|
||||
TEST_CASE("the measurement window covers the face interior of the crop",
|
||||
"[quality][AR-029]") {
|
||||
// The landmarks the ArcFace template pins must all fall inside the window,
|
||||
// or the measure is scoring background and hair rather than the face.
|
||||
const cv::Rect w = sharpness_window();
|
||||
CHECK(w.x >= 0);
|
||||
CHECK(w.y >= 0);
|
||||
CHECK(w.x + w.width <= 112);
|
||||
CHECK(w.y + w.height <= 112);
|
||||
for (int i = 0; i < 5; ++i) {
|
||||
INFO("landmark " << i);
|
||||
CHECK(w.contains(cv::Point(static_cast<int>(kArcFaceRef[i][0]),
|
||||
static_cast<int>(kArcFaceRef[i][1]))));
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("a single-channel crop scores the same as its BGR equivalent",
|
||||
"[quality][AR-029]") {
|
||||
// The dump replays crops; nothing should depend on whether they arrived as
|
||||
// three identical channels or one.
|
||||
const cv::Mat base = synthetic_crop();
|
||||
cv::Mat gray;
|
||||
cv::cvtColor(base, gray, cv::COLOR_BGR2GRAY);
|
||||
|
||||
const auto a = assess_sharpness(base);
|
||||
const auto b = assess_sharpness(gray);
|
||||
for (const auto& [name, m] : kMeasures) {
|
||||
INFO(name);
|
||||
CHECK_THAT(b.*m, WithinRel(a.*m, 1e-3f));
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user