dtourolle/scene-actor-extraction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: deploy from 4b55579

Browse Source
This commit is contained in:
dtourolle
2026-07-19 22:28:14 +02:00
parent 252625330f
commit 36f75ba199
29 changed files with 1024 additions and 260 deletions
Download Patch File Download Diff File Expand all files Collapse all files
optimizer-experiments/index.html
+81 -15
View File
@@ -10,8 +10,10 @@
<link rel="canonical" href="https://pages.tourolle.paris/dtourolle/scene-actor-extraction/optimizer-experiments/">
<link rel="prev" href="../rep4-optimizer-results/">
<link rel="prev" href="../model-bakeoff/">
<link rel="next" href="../service-conversion/">
@@ -51,6 +53,8 @@
<link rel="stylesheet" href="../stylesheets/extra.css">
<script>__md_scope=new URL("..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
@@ -67,7 +71,7 @@
<body dir="ltr" data-md-color-scheme="slate" data-md-color-primary="indigo" data-md-color-accent="indigo">
<body dir="ltr" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="amber">
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
@@ -123,7 +127,21 @@
<input class="md-option" data-md-color-media="" data-md-color-scheme="slate" data-md-color-primary="indigo" data-md-color-accent="indigo" aria-hidden="true" type="radio" name="__palette" id="__palette_0">
<input class="md-option" data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="amber" aria-label="Switch to light mode" type="radio" name="__palette" id="__palette_0">
<label class="md-header__button md-icon" title="Switch to light mode" for="__palette_1" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 7a5 5 0 0 1 5 5 5 5 0 0 1-5 5 5 5 0 0 1-5-5 5 5 0 0 1 5-5m0 2a3 3 0 0 0-3 3 3 3 0 0 0 3 3 3 3 0 0 0 3-3 3 3 0 0 0-3-3m0-7 2.39 3.42C13.65 5.15 12.84 5 12 5s-1.65.15-2.39.42zM3.34 7l4.16-.35A7.2 7.2 0 0 0 5.94 8.5c-.44.74-.69 1.5-.83 2.29zm.02 10 1.76-3.77a7.131 7.131 0 0 0 2.38 4.14zM20.65 7l-1.77 3.79a7.02 7.02 0 0 0-2.38-4.15zm-.01 10-4.14.36c.59-.51 1.12-1.14 1.54-1.86.42-.73.69-1.5.83-2.29zM12 22l-2.41-3.44c.74.27 1.55.44 2.41.44.82 0 1.63-.17 2.37-.44z"/></svg>
</label>
<input class="md-option" data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme="default" data-md-color-primary="black" data-md-color-accent="indigo" aria-label="Switch to dark mode" type="radio" name="__palette" id="__palette_1">
<label class="md-header__button md-icon" title="Switch to dark mode" for="__palette_0" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m17.75 4.09-2.53 1.94.91 3.06-2.63-1.81-2.63 1.81.91-3.06-2.53-1.94L12.44 4l1.06-3 1.06 3zm3.5 6.91-1.64 1.25.59 1.98-1.7-1.17-1.7 1.17.59-1.98L15.75 11l2.06-.05L18.5 9l.69 1.95zm-2.28 4.95c.83-.08 1.72 1.1 1.19 1.85-.32.45-.66.87-1.08 1.27C15.17 23 8.84 23 4.94 19.07c-3.91-3.9-3.91-10.24 0-14.14.4-.4.82-.76 1.27-1.08.75-.53 1.93.36 1.85 1.19-.27 2.86.69 5.83 2.89 8.02a9.96 9.96 0 0 0 8.02 2.89m-1.64 2.02a12.08 12.08 0 0 1-7.8-3.47c-2.17-2.19-3.33-5-3.49-7.82-2.81 3.14-2.7 7.96.31 10.98 3.02 3.01 7.84 3.12 10.98.31"/></svg>
</label>
</form>
@@ -246,13 +264,13 @@
<li class="md-tabs__item">
<a href="../rep4-optimizer-results/" class="md-tabs__link">
<a href="../model-bakeoff/" class="md-tabs__link">
Rep4 Bake-off & Re-tune (full log)
Model Bake-off & Re-tune (full log)
</a>
</li>
@@ -549,14 +567,14 @@
<li class="md-nav__item">
<a href="../rep4-optimizer-results/" class="md-nav__link">
<a href="../model-bakeoff/" class="md-nav__link">
<span class="md-ellipsis">
Rep4 Bake-off & Re-tune (full log)
Model Bake-off & Re-tune (full log)
@@ -936,11 +954,15 @@ GT set = actors X-Ray lists for that scene. Per scene TP/FP/FN, then:</p>
scenes count more) → <strong>equal-weight mean across movies</strong> (macro; each film counts
the same regardless of length). This is the DE objective.</li>
</ul>
<p>Implemented in <code>scripts/optimizer/scene_score.py</code>.</p>
<p>Implemented in <code>scripts/optimizer/scene_score.py</code> — since <strong>removed</strong> along
with this metric; its per-second successor is
<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/4b5557974bef8783bacc375c0869e8f589d1b0a3/scripts/optimizer/second_score.py"><code>scripts/optimizer/second_score.py</code></a>
(see the <a href="../model-bakeoff/">bake-off round</a>).</p>
<h2 id="the-gallery-coverage-gap">The gallery coverage gap<a class="headerlink" href="#the-gallery-coverage-gap" title="Permanent link">&para;</a></h2>
<p>Diagnosing low recall: only <strong>131 of 392</strong> X-Ray cast were in the gallery (33%). Every
in-gallery actor HAD embeddings (gallery well-formed) — the gap was pure coverage.
<code>scripts/optimizer/fetch_missing_actors.py</code> recovers missing actors:
<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/4b5557974bef8783bacc375c0869e8f589d1b0a3/scripts/optimizer/fetch_missing_actors.py"><code>scripts/optimizer/fetch_missing_actors.py</code></a>
recovers missing actors:
<code>nm-id → TMDB /find external_ids → /person/{id}/images → download → embed (sae_embed)</code>,
with a <code>--wikidata</code> fallback (P345→P18 Commons photo).</p>
<ul>
@@ -958,7 +980,8 @@ credits them as cast-in-scene (incl. off-camera/background), but their face neve
appears clearly for the pipeline to detect. This is a fundamental ceiling of a
face-recognition pipeline vs X-Ray's presence semantics, not a fixable gap.</p>
<h2 id="optimizer">Optimizer<a class="headerlink" href="#optimizer" title="Permanent link">&para;</a></h2>
<p><code>scripts/optimizer/optimize.py</code> — scipy <code>differential_evolution</code> over the knob space,
<p><a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/4b5557974bef8783bacc375c0869e8f589d1b0a3/scripts/optimizer/optimize.py"><code>scripts/optimizer/optimize.py</code></a>
— scipy <code>differential_evolution</code> over the knob space,
each candidate = full replay of all films through the <strong>real</strong> C++ nodes (see the
KPN replay architecture below) scored by the metric above. Global objective (one
config for all films, not per-film).</p>
@@ -994,7 +1017,8 @@ the tightly-converged knobs were adopted as defaults.</p>
<h2 id="replay-architecture-how-the-sweep-is-cheap">Replay architecture (how the sweep is cheap)<a class="headerlink" href="#replay-architecture-how-the-sweep-is-cheap" title="Permanent link">&para;</a></h2>
<p>The optimizer never re-decodes video. <code>scene_analyze --dump-embeddings out.h5</code> runs the
expensive half once (decode→detect→align→embed) and dumps per-frame face embeddings
+ metadata to HDF5 (<code>scripts/optimizer/SCHEMA.md</code>). <code>scripts/optimizer/replay.py</code> then
+ metadata to HDF5 (<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/4b5557974bef8783bacc375c0869e8f589d1b0a3/scripts/optimizer/SCHEMA.md"><code>scripts/optimizer/SCHEMA.md</code></a>).
<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction/raw/commit/4b5557974bef8783bacc375c0869e8f589d1b0a3/scripts/optimizer/replay.py"><code>scripts/optimizer/replay.py</code></a> then
replays that dump through the <strong>real</strong> C++ <code>face_tracker → identity_matcher →
scene_tracker</code> assembled in a Python KPN network (<code>sae_kpn</code> nanobind module), varying
Config knobs freely — no GPU embedding, no decode. Verified BYTE-EXACT against
@@ -1010,10 +1034,12 @@ dump floor is 0.5).</p>
</span><span id="__span-0-6"><a id="__codelineno-0-6" name="__codelineno-0-6" href="#__codelineno-0-6"></a><span class="w"> </span>--params<span class="w"> </span>prob_threshold:0.5:0.999<span class="w"> </span>anneal_sec:1:30<span class="w"> </span>extinction_sec:1:15<span class="w"> </span><span class="se">\</span>
</span><span id="__span-0-7"><a id="__codelineno-0-7" name="__codelineno-0-7" href="#__codelineno-0-7"></a><span class="w"> </span>--popsize<span class="w"> </span><span class="m">8</span><span class="w"> </span>--maxiter<span class="w"> </span><span class="m">20</span><span class="w"> </span>--trajectory<span class="w"> </span>traj.jsonl<span class="w"> </span>--out<span class="w"> </span>opt.json
</span><span id="__span-0-8"><a id="__codelineno-0-8" name="__codelineno-0-8" href="#__codelineno-0-8"></a><span class="c1"># 4. score a fixed config / validate on a held-out set</span>
</span><span id="__span-0-9"><a id="__codelineno-0-9" name="__codelineno-0-9" href="#__codelineno-0-9"></a>python<span class="w"> </span>scripts/optimizer/score_config.py<span class="w"> </span>--manifest<span class="w"> </span>heldout.json<span class="w"> </span>--gallery<span class="w"> </span>gallery.json<span class="w"> </span><span class="se">\</span>
</span><span id="__span-0-10"><a id="__codelineno-0-10" name="__codelineno-0-10" href="#__codelineno-0-10"></a><span class="w"> </span>--config<span class="w"> </span><span class="s1">&#39;{&quot;prob_threshold&quot;:0.76,&quot;extinction_sec&quot;:1.5,&quot;anneal_sec&quot;:10}&#39;</span>
</span><span id="__span-0-9"><a id="__codelineno-0-9" name="__codelineno-0-9" href="#__codelineno-0-9"></a><span class="c1"># (historical: score_config.py and scene_score.py were removed with the</span>
</span><span id="__span-0-10"><a id="__codelineno-0-10" name="__codelineno-0-10" href="#__codelineno-0-10"></a><span class="c1"># scene-union metric — use scripts/optimizer/second_score.py, per-second)</span>
</span><span id="__span-0-11"><a id="__codelineno-0-11" name="__codelineno-0-11" href="#__codelineno-0-11"></a>python<span class="w"> </span>scripts/optimizer/second_score.py<span class="w"> </span>--help
</span></code></pre></div>
<p>See also memory: kpn-python-replay-optimizer, gallery-coverage-gap, xray-validation-*.</p>
<p>Superseded by the <a href="../model-bakeoff/">model bake-off + re-tune</a>, which
replaced this round's scene-union metric with per-second scoring.</p>
@@ -1044,6 +1070,46 @@ dump floor is 0.5).</p>
<footer class="md-footer">
<nav class="md-footer__inner md-grid" aria-label="Footer" >
<a href="../model-bakeoff/" class="md-footer__link md-footer__link--prev" aria-label="Previous: Model Bake-off &amp; Re-tune (full log)">
<div class="md-footer__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11z"/></svg>
</div>
<div class="md-footer__title">
<span class="md-footer__direction">
Previous
</span>
<div class="md-ellipsis">
Model Bake-off & Re-tune (full log)
</div>
</div>
</a>
<a href="../service-conversion/" class="md-footer__link md-footer__link--next" aria-label="Next: Service Conversion (proposal)">
<div class="md-footer__title">
<span class="md-footer__direction">
Next
</span>
<div class="md-ellipsis">
Service Conversion (proposal)
</div>
</div>
<div class="md-footer__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11z"/></svg>
</div>
</a>
</nav>
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
@@ -1069,7 +1135,7 @@ dump floor is 0.5).</p>
<script id="__config" type="application/json">{"annotate": null, "base": "..", "features": ["navigation.tabs", "navigation.sections", "navigation.top", "content.code.copy", "content.code.annotate"], "search": "../assets/javascripts/workers/search.2c215733.min.js", "tags": null, "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": null}</script>
<script id="__config" type="application/json">{"annotate": null, "base": "..", "features": ["navigation.tabs", "navigation.sections", "navigation.top", "navigation.footer", "content.code.copy", "content.code.annotate"], "search": "../assets/javascripts/workers/search.2c215733.min.js", "tags": null, "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": null}</script>
<script src="../assets/javascripts/bundle.d7400e89.min.js"></script>