docs: deploy from 4b55579
This commit is contained in:
+99
-28
@@ -10,6 +10,8 @@
|
||||
|
||||
|
||||
|
||||
<link rel="canonical" href="https://pages.tourolle.paris/dtourolle/scene-actor-extraction/">
|
||||
|
||||
|
||||
|
||||
<link rel="next" href="best-model/">
|
||||
@@ -49,6 +51,8 @@
|
||||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="stylesheets/extra.css">
|
||||
|
||||
<script>__md_scope=new URL(".",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
|
||||
|
||||
|
||||
@@ -65,7 +69,7 @@
|
||||
|
||||
|
||||
|
||||
<body dir="ltr" data-md-color-scheme="slate" data-md-color-primary="indigo" data-md-color-accent="indigo">
|
||||
<body dir="ltr" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="amber">
|
||||
|
||||
|
||||
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
|
||||
@@ -121,7 +125,21 @@
|
||||
|
||||
|
||||
|
||||
<input class="md-option" data-md-color-media="" data-md-color-scheme="slate" data-md-color-primary="indigo" data-md-color-accent="indigo" aria-hidden="true" type="radio" name="__palette" id="__palette_0">
|
||||
<input class="md-option" data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="amber" aria-label="Switch to light mode" type="radio" name="__palette" id="__palette_0">
|
||||
|
||||
<label class="md-header__button md-icon" title="Switch to light mode" for="__palette_1" hidden>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 7a5 5 0 0 1 5 5 5 5 0 0 1-5 5 5 5 0 0 1-5-5 5 5 0 0 1 5-5m0 2a3 3 0 0 0-3 3 3 3 0 0 0 3 3 3 3 0 0 0 3-3 3 3 0 0 0-3-3m0-7 2.39 3.42C13.65 5.15 12.84 5 12 5s-1.65.15-2.39.42zM3.34 7l4.16-.35A7.2 7.2 0 0 0 5.94 8.5c-.44.74-.69 1.5-.83 2.29zm.02 10 1.76-3.77a7.131 7.131 0 0 0 2.38 4.14zM20.65 7l-1.77 3.79a7.02 7.02 0 0 0-2.38-4.15zm-.01 10-4.14.36c.59-.51 1.12-1.14 1.54-1.86.42-.73.69-1.5.83-2.29zM12 22l-2.41-3.44c.74.27 1.55.44 2.41.44.82 0 1.63-.17 2.37-.44z"/></svg>
|
||||
</label>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<input class="md-option" data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme="default" data-md-color-primary="black" data-md-color-accent="indigo" aria-label="Switch to dark mode" type="radio" name="__palette" id="__palette_1">
|
||||
|
||||
<label class="md-header__button md-icon" title="Switch to dark mode" for="__palette_0" hidden>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m17.75 4.09-2.53 1.94.91 3.06-2.63-1.81-2.63 1.81.91-3.06-2.53-1.94L12.44 4l1.06-3 1.06 3zm3.5 6.91-1.64 1.25.59 1.98-1.7-1.17-1.7 1.17.59-1.98L15.75 11l2.06-.05L18.5 9l.69 1.95zm-2.28 4.95c.83-.08 1.72 1.1 1.19 1.85-.32.45-.66.87-1.08 1.27C15.17 23 8.84 23 4.94 19.07c-3.91-3.9-3.91-10.24 0-14.14.4-.4.82-.76 1.27-1.08.75-.53 1.93.36 1.85 1.19-.27 2.86.69 5.83 2.89 8.02a9.96 9.96 0 0 0 8.02 2.89m-1.64 2.02a12.08 12.08 0 0 1-7.8-3.47c-2.17-2.19-3.33-5-3.49-7.82-2.81 3.14-2.7 7.96.31 10.98 3.02 3.01 7.84 3.12 10.98.31"/></svg>
|
||||
</label>
|
||||
|
||||
|
||||
</form>
|
||||
@@ -246,13 +264,13 @@
|
||||
|
||||
|
||||
<li class="md-tabs__item">
|
||||
<a href="rep4-optimizer-results/" class="md-tabs__link">
|
||||
<a href="model-bakeoff/" class="md-tabs__link">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Rep4 Bake-off & Re-tune (full log)
|
||||
Model Bake-off & Re-tune (full log)
|
||||
|
||||
</a>
|
||||
</li>
|
||||
@@ -627,14 +645,14 @@
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="rep4-optimizer-results/" class="md-nav__link">
|
||||
<a href="model-bakeoff/" class="md-nav__link">
|
||||
|
||||
|
||||
|
||||
<span class="md-ellipsis">
|
||||
|
||||
|
||||
Rep4 Bake-off & Re-tune (full log)
|
||||
Model Bake-off & Re-tune (full log)
|
||||
|
||||
|
||||
|
||||
@@ -782,35 +800,64 @@
|
||||
film or TV episode — built on <a href="https://gitea.tourolle.paris/dtourolle/KPN">KPN++</a>
|
||||
(a C++20 Kahn Process Network library) for the detect → track → match → scene
|
||||
pipeline, with a Jellyfin-integrated gallery and an X-Ray-validated optimizer.</p>
|
||||
<p>This is a perfect X-Ray second, on a film the optimizer never saw:</p>
|
||||
<p><img alt="A perfect X-Ray second: three faces named at 100%, two more correctly carried off-screen" src="assets/images/lovelace_perfect_second.jpg" /></p>
|
||||
<p>Every visible face named at 100% — Chris Noth, Hank Azaria, Bobby Cannavale —
|
||||
the background extra honestly left unnamed, and the two credited cast without
|
||||
a visible face correctly carried as present off-screen by the tracker's
|
||||
presence windows. That's the pipeline exactly reproducing Amazon X-Ray's
|
||||
record for this second.</p>
|
||||
<p>It doesn't always go like that: the hardest held-out film scores 46% F1, and
|
||||
the report is honest about <em>why</em> — one tunable trade (extinction bridging at
|
||||
hard cuts), one structural ceiling (X-Ray credits people whose faces never
|
||||
appear), and a few cases where the pipeline is right and X-Ray is wrong. The
|
||||
evidence for all of it is in the pages below.</p>
|
||||
<h2 id="start-here-four-questions-this-bake-off-answers">Start here — four questions this bake-off answers<a class="headerlink" href="#start-here-four-questions-this-bake-off-answers" title="Permanent link">¶</a></h2>
|
||||
<div class="grid cards">
|
||||
<ul>
|
||||
<li><strong><a href="best-model/">Which model is best?</a></strong> — calibration curves first
|
||||
(discriminative power, independent of any threshold), then F1 on the actual
|
||||
benchmark. LVFace-B Glint360K wins both.</li>
|
||||
<li><strong><a href="gallery-scope/">Whole gallery vs. limited (cast-restricted) gallery</a></strong> —
|
||||
restricting the matcher to a film's credited cast is a clean win on every
|
||||
axis (+3.3pp F1, less than a third the misIDs), but isn't a shipped runtime
|
||||
feature yet.</li>
|
||||
<li><strong><a href="pose-expansion/">Does pose expansion help?</a></strong> — a real training-set
|
||||
effect that didn't reproduce on 5 held-out films once two methodology bugs
|
||||
were caught and fixed. An honest null result, not a forced narrative.</li>
|
||||
<li><strong><a href="lvface-deep-dive/">Deep dive: LVFace-B Glint360K</a></strong> — the winning
|
||||
model's held-out generalization gap, its two real failure modes (frozen-bbox
|
||||
"ghost tracks"), and one case where it correctly identified an actor that
|
||||
the X-Ray ground truth itself failed to credit.</li>
|
||||
<li>
|
||||
<p><span class="twemoji lg middle"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M18 2c-.9 0-2 1-2 2H8c0-1-1.1-2-2-2H2v9c0 1 1 2 2 2h2.2c.4 2 1.7 3.7 4.8 4v2.08C8 19.54 8 22 8 22h8s0-2.46-3-2.92V17c3.1-.3 4.4-2 4.8-4H20c1 0 2-1 2-2V2zM6 11H4V4h2zm14 0h-2V4h2z"/></svg></span> <strong><a href="best-model/">Which model is best?</a></strong></p>
|
||||
<hr />
|
||||
<p>Calibration curves first (discriminative power, independent of any
|
||||
threshold), then F1 on the actual benchmark. LVFace-B Glint360K wins
|
||||
both.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p><span class="twemoji lg middle"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M14 12v7.88c.04.3-.06.62-.29.83a.996.996 0 0 1-1.41 0l-2.01-2.01a.99.99 0 0 1-.29-.83V12h-.03L4.21 4.62a1 1 0 0 1 .17-1.4c.19-.14.4-.22.62-.22h14c.22 0 .43.08.62.22a1 1 0 0 1 .17 1.4L14.03 12z"/></svg></span> <strong><a href="gallery-scope/">Whole vs. cast-restricted gallery</a></strong></p>
|
||||
<hr />
|
||||
<p>Restricting the matcher to a film's credited cast is a clean win on
|
||||
every axis (+3.3pp F1, less than a third the misIDs) — but isn't a
|
||||
shipped runtime feature yet.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p><span class="twemoji lg middle"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m12 0-.66.03 3.81 3.81L16.5 2.5c3.25 1.57 5.59 4.74 5.95 8.5h1.5C23.44 4.84 18.29 0 12 0m0 4c-1.93 0-3.5 1.57-3.5 3.5S10.07 11 12 11s3.5-1.57 3.5-3.5S13.93 4 12 4M.05 13C.56 19.16 5.71 24 12 24l.66-.03-3.81-3.81L7.5 21.5c-3.25-1.56-5.59-4.74-5.95-8.5zM12 13c-3.87 0-7 1.57-7 3.5V18h14v-1.5c0-1.93-3.13-3.5-7-3.5"/></svg></span> <strong><a href="pose-expansion/">Does pose expansion help?</a></strong></p>
|
||||
<hr />
|
||||
<p>A convincing training-set effect that didn't reproduce on 5 held-out
|
||||
films once two methodology bugs were caught and fixed. An honest null
|
||||
result, not a forced narrative.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p><span class="twemoji lg middle"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M18 16h-.58l-.81-.81A7.07 7.07 0 0 0 18 11c0-3.87-3.13-7-7-7-1.5 0-3 .5-4.21 1.4-3.09 2.32-3.72 6.71-1.4 9.8s6.71 3.72 9.8 1.4l.81.81V18l5 5 2-2zm-7 0c-2.76 0-5-2.24-5-5s2.24-5 5-5 5 2.24 5 5-2.24 5-5 5M3 6 1 8V1h7L6 3H3zm18-5v7l-2-2V3h-3l-2-2zM6 19l2 2H1v-7l2 2v3z"/></svg></span> <strong><a href="lvface-deep-dive/">Deep dive: LVFace-B Glint360K</a></strong></p>
|
||||
<hr />
|
||||
<p>The held-out generalization gap, how the error budget decomposes
|
||||
(extinction bridging at hard cuts, X-Ray's scene-membership vs.
|
||||
on-screen-face ceiling), and the frames where the pipeline is right
|
||||
and the ground truth is wrong.</p>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<h2 id="the-full-technical-log">The full technical log<a class="headerlink" href="#the-full-technical-log" title="Permanent link">¶</a></h2>
|
||||
<ul>
|
||||
<li><strong><a href="rep4-optimizer-results/">Rep4 model bake-off + threshold re-tune</a></strong> —
|
||||
<li><strong><a href="model-bakeoff/">Model bake-off + threshold re-tune</a></strong> —
|
||||
the complete experiment log behind the four pages above: the ROCm teardown
|
||||
deadlock root cause and fix, DE concurrency tuning, the full 16-combo
|
||||
results table, and every caveat. This is where the shipped <code>src/config.hpp</code>
|
||||
defaults come from.</li>
|
||||
results table, and every caveat. This is where the shipped
|
||||
<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/4b5557974bef8783bacc375c0869e8f589d1b0a3/src/config.hpp"><code>src/config.hpp</code></a> defaults come from.</li>
|
||||
<li><strong><a href="optimizer-experiments/">Optimizer experiments (prior round)</a></strong> — the
|
||||
earlier scene-union-metric tuning pass, superseded by the per-second metric
|
||||
used in rep4 but kept for the ground-truth/architecture background.</li>
|
||||
used in the bake-off but kept for the ground-truth/architecture background.</li>
|
||||
<li><strong><a href="service-conversion/">Service conversion (proposal)</a></strong> — design sketch
|
||||
for an idle-GPU Docker worker, not yet built.</li>
|
||||
for a native idle-GPU worker gated on screen lock, not yet built.</li>
|
||||
</ul>
|
||||
<h2 id="reproducing-the-benchmarks">Reproducing the benchmarks<a class="headerlink" href="#reproducing-the-benchmarks" title="Permanent link">¶</a></h2>
|
||||
<p>Gallery <code>.h5</code> files, embedding dumps, the X-Ray corpus, montage frame images,
|
||||
@@ -820,8 +867,8 @@ the Gitea package registry and pulled on demand:</p>
|
||||
</span><span id="__span-0-2"><a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a>scripts/artifacts/pull_artifacts.sh<span class="w"> </span>experiment-data
|
||||
</span><span id="__span-0-3"><a id="__codelineno-0-3" name="__codelineno-0-3" href="#__codelineno-0-3"></a>scripts/artifacts/pull_artifacts.sh<span class="w"> </span>montage-frames<span class="w"> </span><film-slug>
|
||||
</span></code></pre></div>
|
||||
<p>See <code>scripts/artifacts/push_artifacts.sh</code> for the upload side (requires a
|
||||
<code>GITEA_TOKEN</code> with package write scope).</p>
|
||||
<p>See <a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/4b5557974bef8783bacc375c0869e8f589d1b0a3/scripts/artifacts/push_artifacts.sh"><code>scripts/artifacts/push_artifacts.sh</code></a>
|
||||
for the upload side (requires a <code>GITEA_TOKEN</code> with package write scope).</p>
|
||||
|
||||
|
||||
|
||||
@@ -852,6 +899,30 @@ the Gitea package registry and pulled on demand:</p>
|
||||
|
||||
<footer class="md-footer">
|
||||
|
||||
|
||||
|
||||
<nav class="md-footer__inner md-grid" aria-label="Footer" >
|
||||
|
||||
|
||||
|
||||
<a href="best-model/" class="md-footer__link md-footer__link--next" aria-label="Next: Best Model">
|
||||
<div class="md-footer__title">
|
||||
<span class="md-footer__direction">
|
||||
Next
|
||||
</span>
|
||||
<div class="md-ellipsis">
|
||||
Best Model
|
||||
</div>
|
||||
</div>
|
||||
<div class="md-footer__button md-icon">
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11z"/></svg>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
</nav>
|
||||
|
||||
|
||||
<div class="md-footer-meta md-typeset">
|
||||
<div class="md-footer-meta__inner md-grid">
|
||||
<div class="md-copyright">
|
||||
@@ -877,7 +948,7 @@ the Gitea package registry and pulled on demand:</p>
|
||||
|
||||
|
||||
|
||||
<script id="__config" type="application/json">{"annotate": null, "base": ".", "features": ["navigation.tabs", "navigation.sections", "navigation.top", "content.code.copy", "content.code.annotate"], "search": "assets/javascripts/workers/search.2c215733.min.js", "tags": null, "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": null}</script>
|
||||
<script id="__config" type="application/json">{"annotate": null, "base": ".", "features": ["navigation.tabs", "navigation.sections", "navigation.top", "navigation.footer", "content.code.copy", "content.code.annotate"], "search": "assets/javascripts/workers/search.2c215733.min.js", "tags": null, "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": null}</script>
|
||||
|
||||
|
||||
<script src="assets/javascripts/bundle.d7400e89.min.js"></script>
|
||||
|
||||
Reference in New Issue
Block a user