docs: deploy from 0bd2747
This commit is contained in:
+121
-118
@@ -80,7 +80,7 @@
|
||||
<div data-md-component="skip">
|
||||
|
||||
|
||||
<a href="#whole-gallery-vs-limited-cast-restricted-gallery" class="md-skip">
|
||||
<a href="#whole-gallery-vs-cast-restricted-gallery" class="md-skip">
|
||||
Skip to content
|
||||
</a>
|
||||
|
||||
@@ -242,6 +242,25 @@
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-tabs__item">
|
||||
<a href="../methodology/" class="md-tabs__link">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
How We Score Against X-Ray
|
||||
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -272,26 +291,7 @@
|
||||
|
||||
|
||||
|
||||
Model Bake-off & Re-tune (full log)
|
||||
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-tabs__item">
|
||||
<a href="../optimizer-experiments/" class="md-tabs__link">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Optimizer Experiments (prior round)
|
||||
Full Experiment Log
|
||||
|
||||
</a>
|
||||
</li>
|
||||
@@ -393,6 +393,33 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../methodology/" class="md-nav__link">
|
||||
|
||||
|
||||
|
||||
<span class="md-ellipsis">
|
||||
|
||||
|
||||
How We Score Against X-Ray
|
||||
|
||||
|
||||
|
||||
</span>
|
||||
|
||||
|
||||
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -414,10 +441,10 @@
|
||||
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_2" checked>
|
||||
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_3" checked>
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2" id="__nav_2_label" tabindex="">
|
||||
<label class="md-nav__link" for="__nav_3" id="__nav_3_label" tabindex="">
|
||||
|
||||
|
||||
|
||||
@@ -435,8 +462,8 @@
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_2_label" aria-expanded="true">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_3_label" aria-expanded="true">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
|
||||
|
||||
@@ -541,10 +568,10 @@
|
||||
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#the-result" class="md-nav__link">
|
||||
<a href="#result" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
The result
|
||||
Result
|
||||
|
||||
</span>
|
||||
</a>
|
||||
@@ -552,10 +579,10 @@
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#why-this-isnt-the-shipped-default" class="md-nav__link">
|
||||
<a href="#why-this-is-not-the-shipped-default" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
Why this isn't the shipped default
|
||||
Why this is not the shipped default
|
||||
|
||||
</span>
|
||||
</a>
|
||||
@@ -648,34 +675,7 @@
|
||||
<span class="md-ellipsis">
|
||||
|
||||
|
||||
Model Bake-off & Re-tune (full log)
|
||||
|
||||
|
||||
|
||||
</span>
|
||||
|
||||
|
||||
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../optimizer-experiments/" class="md-nav__link">
|
||||
|
||||
|
||||
|
||||
<span class="md-ellipsis">
|
||||
|
||||
|
||||
Optimizer Experiments (prior round)
|
||||
Full Experiment Log
|
||||
|
||||
|
||||
|
||||
@@ -742,10 +742,10 @@
|
||||
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#the-result" class="md-nav__link">
|
||||
<a href="#result" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
The result
|
||||
Result
|
||||
|
||||
</span>
|
||||
</a>
|
||||
@@ -753,10 +753,10 @@
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#why-this-isnt-the-shipped-default" class="md-nav__link">
|
||||
<a href="#why-this-is-not-the-shipped-default" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
Why this isn't the shipped default
|
||||
Why this is not the shipped default
|
||||
|
||||
</span>
|
||||
</a>
|
||||
@@ -780,14 +780,15 @@
|
||||
|
||||
|
||||
|
||||
<h1 id="whole-gallery-vs-limited-cast-restricted-gallery">Whole gallery vs. limited (cast-restricted) gallery<a class="headerlink" href="#whole-gallery-vs-limited-cast-restricted-gallery" title="Permanent link">¶</a></h1>
|
||||
<p>Two ways to run the matcher: <strong>full</strong> scores every detected face against the
|
||||
entire library gallery (2418 actors across the 9-film benchmark set); <strong>restricted</strong>
|
||||
pre-filters each film's gallery down to just its Jellyfin-credited cast (typically
|
||||
~15 top-billed actors) before the matcher ever runs.</p>
|
||||
<h2 id="the-result">The result<a class="headerlink" href="#the-result" title="Permanent link">¶</a></h2>
|
||||
<p>Averaged across all 4 models and both expansion settings, on the 4 bake-off training
|
||||
films:</p>
|
||||
<h1 id="whole-gallery-vs-cast-restricted-gallery">Whole gallery vs. cast-restricted gallery<a class="headerlink" href="#whole-gallery-vs-cast-restricted-gallery" title="Permanent link">¶</a></h1>
|
||||
<p>Two ways to run the matcher. Full mode scores every detected face against
|
||||
the entire 2418-actor gallery. Restricted mode pre-filters each film's
|
||||
gallery down to just its Jellyfin-credited cast (typically around 15
|
||||
top-billed actors) before the matcher runs.</p>
|
||||
<h2 id="result">Result<a class="headerlink" href="#result" title="Permanent link">¶</a></h2>
|
||||
<p>Averaged across the 3 compared models (r50 excluded, see
|
||||
<a href="../model-bakeoff/">the full experiment log</a>) and both expansion settings, on
|
||||
the 4 training films:</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
@@ -795,68 +796,70 @@ films:</p>
|
||||
<th>F1</th>
|
||||
<th>P</th>
|
||||
<th>R</th>
|
||||
<th>total misID (8 evals)</th>
|
||||
<th>total misID</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>full</td>
|
||||
<td>71.2%</td>
|
||||
<td>91.1%</td>
|
||||
<td>59.0%</td>
|
||||
<td>1073</td>
|
||||
<td>71.1%</td>
|
||||
<td>89.6%</td>
|
||||
<td>59.6%</td>
|
||||
<td>1121</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>restricted</strong></td>
|
||||
<td><strong>74.5%</strong></td>
|
||||
<td>92.2%</td>
|
||||
<td><strong>62.9%</strong></td>
|
||||
<td><strong>329</strong></td>
|
||||
<td>restricted</td>
|
||||
<td>75.9%</td>
|
||||
<td>90.4%</td>
|
||||
<td>65.6%</td>
|
||||
<td>299</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p>This is not a precision/recall trade — restriction wins on every axis at once:
|
||||
<strong>+3.3pp F1, +3.9pp recall, and less than a third the total misIDs.</strong> Fewer
|
||||
<p>Restriction improves every metric at once, not a precision/recall trade:
|
||||
+4.8pp F1, +6.0pp recall, roughly a quarter the total misIDs. Fewer
|
||||
candidates in the matcher's search space means fewer opportunities for a
|
||||
look-alike false match (an actor who happens to share enough facial structure
|
||||
with someone in the film, but isn't actually in it), and the recall gain shows
|
||||
it isn't costing real detections to get there.</p>
|
||||
<p>Per-model, every single model's best-scoring combo in the full 16-way matrix is
|
||||
a <code>restricted</code> variant — visible directly in the ranking below (filled dots =
|
||||
restricted, open = full; the filled dots cluster at the top for every color):</p>
|
||||
<p><img alt="All 16 bake-off combos — filled dots (restricted) dominate the top" src="../assets/images/rep4_matrix_f1.png" /></p>
|
||||
<p>See the full table in the
|
||||
<a href="../model-bakeoff/">bake-off experiment log</a>. Two
|
||||
combos hit <strong>zero</strong> true out-of-cast misidentifications:
|
||||
<code>arcface_w600k_mbf_restricted_exp</code> (F1 76.5%) and, in full mode,
|
||||
<code>LVFace-B_Glint360K_full_noexp</code> (F1 72.4%) — restriction isn't the only way to
|
||||
reach misid=0, but it's the more reliable one.</p>
|
||||
<h2 id="why-this-isnt-the-shipped-default">Why this isn't the shipped default<a class="headerlink" href="#why-this-isnt-the-shipped-default" title="Permanent link">¶</a></h2>
|
||||
<p>Cast-restriction is implemented today only as an <strong>offline optimizer technique</strong>
|
||||
(<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/4b5557974bef8783bacc375c0869e8f589d1b0a3/scripts/optimizer/cast_restrict.py"><code>scripts/optimizer/cast_restrict.py</code></a>):
|
||||
it pre-builds a filtered gallery file
|
||||
per film, using Jellyfin's own cast list, before the benchmark ever calls the
|
||||
matcher. There's no runtime "restrict matching to this title's credited cast"
|
||||
switch in the shipped application — <code>scene_analyze</code> always matches against
|
||||
whatever single gallery file it's given.</p>
|
||||
<p>Building that as a real feature would need, at minimum:</p>
|
||||
lookalike false match, and the recall gain shows this does not cost real
|
||||
detections.</p>
|
||||
<p>Every model's best-scoring combo in the training matrix uses the
|
||||
restricted gallery:</p>
|
||||
<p><img alt="All combos ranked by training-set F1, filled dots are restricted" src="../assets/images/rep4_matrix_f1.png" /></p>
|
||||
<p>See <a href="../model-bakeoff/">the full experiment log</a> for the complete table. One
|
||||
combo reaches zero true out-of-cast misidentifications,
|
||||
<code>arcface_w600k_mbf_restricted_exp</code> (F1 76.2%), and it is a restricted one,
|
||||
consistent with restriction, not expansion, being what suppresses cross-film
|
||||
confusions.</p>
|
||||
<p>The restriction effect (+4.8pp averaged across models) is larger than the
|
||||
model-choice effect: LVFace beats r18 by 6.2pp in full mode but beats mbf by
|
||||
3.3pp. Restriction is the single strongest lever in the matrix.</p>
|
||||
<h2 id="why-this-is-not-the-shipped-default">Why this is not the shipped default<a class="headerlink" href="#why-this-is-not-the-shipped-default" title="Permanent link">¶</a></h2>
|
||||
<p>Cast restriction is implemented today only as an offline optimizer
|
||||
technique
|
||||
(<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/0bd27470698c45cab21935d636a04512360e1008/scripts/optimizer/cast_restrict.py"><code>scripts/optimizer/cast_restrict.py</code></a>):
|
||||
it pre-builds a filtered gallery file per film using Jellyfin's cast list
|
||||
before the benchmark calls the matcher. There is no runtime "restrict to
|
||||
this title's credited cast" switch in the shipped application;
|
||||
<code>scene_analyze</code> always matches against whatever single gallery file it is
|
||||
given.</p>
|
||||
<p>Building this as a real feature requires:</p>
|
||||
<ul>
|
||||
<li>A live Jellyfin cast lookup at analysis time (the title is already known —
|
||||
<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/4b5557974bef8783bacc375c0869e8f589d1b0a3/scripts/run_from_jellyfin.py"><code>scripts/run_from_jellyfin.py</code></a>
|
||||
already does this same lookup for its own
|
||||
<code>filter_gallery</code>-based restriction path, just not wired into <code>scene_analyze</code>
|
||||
itself as a first-class option).</li>
|
||||
<li>A decision on the <em>fallback</em>: what happens to a real, uncredited cameo
|
||||
(see the Germar Terrell Gardner case in the LVFace deep-dive) if the gallery
|
||||
never includes them at all?</li>
|
||||
<li>Regenerating the restricted-gallery cache whenever the title's Jellyfin cast
|
||||
list changes.</li>
|
||||
<li>A live Jellyfin cast lookup at analysis time. The title is already known,
|
||||
and <a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/0bd27470698c45cab21935d636a04512360e1008/scripts/run_from_jellyfin.py"><code>scripts/run_from_jellyfin.py</code></a>
|
||||
already performs this lookup for its own <code>filter_gallery</code>-based
|
||||
restriction path; it is not wired into <code>scene_analyze</code> as a first-class
|
||||
option.</li>
|
||||
<li>A decision on the fallback case: what happens to a real, uncredited
|
||||
cameo (see the Germar Terrell Gardner and Talia Balsam cases in the
|
||||
<a href="../lvface-deep-dive/#where-lvface-beat-x-ray">LVFace deep dive</a>) if the
|
||||
restricted gallery never includes them at all.</li>
|
||||
<li>Regenerating the restricted-gallery cache whenever a title's Jellyfin
|
||||
cast list changes.</li>
|
||||
</ul>
|
||||
<p>This is why the shipped <a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/4b5557974bef8783bacc375c0869e8f589d1b0a3/src/config.hpp"><code>src/config.hpp</code></a>
|
||||
defaults use the <code>full</code>-mode winner
|
||||
(<code>LVFace-B_Glint360K_full_exp</code>, F1 75.3% training / 67.4% held-out macro) rather
|
||||
than the higher-scoring <code>restricted_exp</code> (78.3%) — the 78.3% number describes a
|
||||
capability the app doesn't have yet, not what actually ships.</p>
|
||||
<p>The shipped <a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/0bd27470698c45cab21935d636a04512360e1008/src/config.hpp"><code>src/config.hpp</code></a> defaults use
|
||||
the full-mode winner (<code>LVFace-B_Glint360K_full_exp</code>, F1 75.3% training,
|
||||
67.4% held-out macro) rather than the higher-scoring <code>restricted_exp</code>
|
||||
(78.3%), because 78.3% describes a capability the application does not
|
||||
have yet.</p>
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user