docs: four focused findings pages (best model, gallery scope, expansion, deep dive)
Splits the rep4 write-up's key findings into their own linkable pages: - best-model.md: calibration curves first (discriminative power, independent of any threshold), then F1 on the benchmark — LVFace-B Glint360K wins both. - gallery-scope.md: whole vs. cast-restricted gallery, isolated from model and expansion choice — restriction wins on every axis, but isn't a shipped runtime feature yet. - pose-expansion.md: the training-set expand_gallery effect, and the held-out replication attempt that found it doesn't reproduce (5 films, 2 models, after catching and fixing a replay-timeout truncation bug and a bbox first-match-instead-of-best-match bug in the comparison harness itself). An honest null result, with the methodology errors documented since they're exactly the kind that manufacture a false "it works!" finding. - lvface-deep-dive.md: the winning model's held-out generalization gap, its two failure modes (frozen-bbox ghost tracks), and a verified case (cross- checked against Jellyfin's independent cast metadata) where LVFace correctly identified an actor that X-Ray's ground truth failed to credit. Adds a "report-highlights" artifact-registry package (scripts/artifacts/ push_artifacts.sh, pull_artifacts.sh) for hand-picked illustrative frames that aren't reproducible via the automated best/worst montage selection, and wires pulling it into scripts/docs/build_site.sh.
This commit is contained in:
@@ -0,0 +1,61 @@
|
||||
# Which embedding model is best?
|
||||
|
||||
Four candidates went into the bake-off: three ArcFace variants (w600k-R50,
|
||||
R18, w600k-MBF) and LVFace-B (Glint360K), a Vision-Transformer embedder that's
|
||||
a drop-in replacement for ArcFace's `[N,3,112,112]` input / 512-d output. The
|
||||
open question: is LVFace (455MB) actually better, or just the biggest?
|
||||
|
||||
## First signal: calibration curves
|
||||
|
||||
Each gallery carries a fitted Platt sigmoid `P(match | cosine similarity) =
|
||||
σ(a·sim + b)`, embedded directly in the gallery's HDF5 file
|
||||
(`src/gallery/gallery_calibration.hpp`). This is a property of the embedding
|
||||
space alone — computed from intra/inter-actor reference-image pairs, no
|
||||
tracking or scene logic involved — so it's a clean first read on discriminative
|
||||
power before running a single benchmark.
|
||||
|
||||

|
||||
|
||||
| model | `a` (steepness) | boundary at P=0.5 |
|
||||
|---|---|---|
|
||||
| **LVFace-B Glint360K** | **17.7** | **sim 0.228** |
|
||||
| ArcFace w600k-MBF | 16.2 | sim 0.267 |
|
||||
| ArcFace w600k-R50 | 15.4 | sim 0.301 |
|
||||
| ArcFace R18 | 15.3 | sim 0.309 |
|
||||
|
||||
LVFace has both the steepest transition and the lowest decision boundary — it
|
||||
separates same-actor from different-actor reference pairs more confidently, at
|
||||
a *lower* similarity threshold, than any ArcFace variant. That's a genuine
|
||||
head start before the tracking/scoring pipeline is even involved.
|
||||
|
||||
## Second signal: F1 on the actual benchmark
|
||||
|
||||
Best full-gallery (no cast-restriction) result per model, from the 16-combo
|
||||
rep4 matrix (`rep4-optimizer-results.md`):
|
||||
|
||||
| model | F1 | P | R | misID |
|
||||
|---|---|---|---|---|
|
||||
| **LVFace-B Glint360K** | **75.3%** | 89.7% | **65.4%** | 232 |
|
||||
| ArcFace w600k-MBF | 74.2% | 87.4% | 64.4% | 57 |
|
||||
| ArcFace R18 | 69.1% | 87.6% | 57.7% | 242 |
|
||||
| ArcFace w600k-R50 | 68.5% | 94.0% | 54.1% | 150 |
|
||||
|
||||
LVFace wins outright, with the highest recall of any full-mode combo. This
|
||||
reverses an earlier conclusion from a prior (superseded) benchmarking pass
|
||||
using a scene-union metric, which found the three models statistically
|
||||
indistinguishable (~85% each) and concluded LVFace wasn't worth its size — that
|
||||
metric hid out-of-cast false positives behind a gallery∩cast recall mask (see
|
||||
`optimizer-experiments.md`); the per-second metric used here does not.
|
||||
|
||||
Held-out validation (5 films never seen by the optimizer) confirms LVFace's
|
||||
lead holds up out of sample — see the deep-dive page for the full breakdown,
|
||||
including where it fails.
|
||||
|
||||
## Caveat: model choice is an operational change
|
||||
|
||||
Switching the default embedder isn't just flipping a config value — the
|
||||
gallery itself is model-specific (embeddings from different models aren't
|
||||
comparable), so any existing gallery built against ArcFace w600k-R50 needs to
|
||||
be rebuilt from source images against LVFace before the new default takes
|
||||
effect. `scripts/optimizer/reembed_gallery.py` does this from a reference
|
||||
gallery's cached source images without re-downloading anything.
|
||||
@@ -0,0 +1,56 @@
|
||||
# Whole gallery vs. limited (cast-restricted) gallery
|
||||
|
||||
Two ways to run the matcher: **full** scores every detected face against the
|
||||
entire library gallery (2418 actors across the 9-film benchmark set); **restricted**
|
||||
pre-filters each film's gallery down to just its Jellyfin-credited cast (typically
|
||||
~15 top-billed actors) before the matcher ever runs.
|
||||
|
||||
## The result
|
||||
|
||||
Averaged across all 4 models and both expansion settings, on the 4 rep4 training
|
||||
films:
|
||||
|
||||
| scope | F1 | P | R | total misID (8 evals) |
|
||||
|---|---|---|---|---|
|
||||
| full | 71.2% | 91.1% | 59.0% | 1073 |
|
||||
| **restricted** | **74.5%** | 92.2% | **62.9%** | **329** |
|
||||
|
||||
This is not a precision/recall trade — restriction wins on every axis at once:
|
||||
**+3.3pp F1, +3.9pp recall, and less than a third the total misIDs.** Fewer
|
||||
candidates in the matcher's search space means fewer opportunities for a
|
||||
look-alike false match (an actor who happens to share enough facial structure
|
||||
with someone in the film, but isn't actually in it), and the recall gain shows
|
||||
it isn't costing real detections to get there.
|
||||
|
||||
Per-model, every single model's best-scoring combo in the full 16-way matrix is
|
||||
a `restricted` variant — see the full table in `rep4-optimizer-results.md`. Two
|
||||
combos hit **zero** true out-of-cast misidentifications:
|
||||
`arcface_w600k_mbf_restricted_exp` (F1 76.5%) and, in full mode,
|
||||
`LVFace-B_Glint360K_full_noexp` (F1 72.4%) — restriction isn't the only way to
|
||||
reach misid=0, but it's the more reliable one.
|
||||
|
||||
## Why this isn't the shipped default
|
||||
|
||||
Cast-restriction is implemented today only as an **offline optimizer technique**
|
||||
(`scripts/optimizer/cast_restrict.py`): it pre-builds a filtered gallery file
|
||||
per film, using Jellyfin's own cast list, before the benchmark ever calls the
|
||||
matcher. There's no runtime "restrict matching to this title's credited cast"
|
||||
switch in the shipped application — `scene_analyze` always matches against
|
||||
whatever single gallery file it's given.
|
||||
|
||||
Building that as a real feature would need, at minimum:
|
||||
|
||||
- A live Jellyfin cast lookup at analysis time (the title is already known —
|
||||
`run_from_jellyfin.py` already does this same lookup for its own
|
||||
`filter_gallery`-based restriction path, just not wired into `scene_analyze`
|
||||
itself as a first-class option).
|
||||
- A decision on the *fallback*: what happens to a real, uncredited cameo
|
||||
(see the Germar Terrell Gardner case in the LVFace deep-dive) if the gallery
|
||||
never includes them at all?
|
||||
- Regenerating the restricted-gallery cache whenever the title's Jellyfin cast
|
||||
list changes.
|
||||
|
||||
This is why the shipped `src/config.hpp` defaults use the `full`-mode winner
|
||||
(`LVFace-B_Glint360K_full_exp`, F1 75.3% training / 67.4% held-out macro) rather
|
||||
than the higher-scoring `restricted_exp` (78.3%) — the 78.3% number describes a
|
||||
capability the app doesn't have yet, not what actually ships.
|
||||
+23
-5
@@ -5,12 +5,30 @@ film or TV episode — built on [KPN++](https://gitea.tourolle.paris/dtourolle/K
|
||||
(a C++20 Kahn Process Network library) for the detect → track → match → scene
|
||||
pipeline, with a Jellyfin-integrated gallery and an X-Ray-validated optimizer.
|
||||
|
||||
## Start here
|
||||
## Start here — four questions this bake-off answers
|
||||
|
||||
- **[Rep4 model bake-off + threshold re-tune](rep4-optimizer-results.md)** — the
|
||||
current experiment log: model comparison, DE threshold tuning, held-out
|
||||
validation, and the visual failure-mode evidence (frozen-bbox ghost tracks).
|
||||
This is where the shipped `src/config.hpp` defaults come from.
|
||||
- **[Which model is best?](best-model.md)** — calibration curves first
|
||||
(discriminative power, independent of any threshold), then F1 on the actual
|
||||
benchmark. LVFace-B Glint360K wins both.
|
||||
- **[Whole gallery vs. limited (cast-restricted) gallery](gallery-scope.md)** —
|
||||
restricting the matcher to a film's credited cast is a clean win on every
|
||||
axis (+3.3pp F1, less than a third the misIDs), but isn't a shipped runtime
|
||||
feature yet.
|
||||
- **[Does pose expansion help?](pose-expansion.md)** — a real training-set
|
||||
effect that didn't reproduce on 5 held-out films once two methodology bugs
|
||||
were caught and fixed. An honest null result, not a forced narrative.
|
||||
- **[Deep dive: LVFace-B Glint360K](lvface-deep-dive.md)** — the winning
|
||||
model's held-out generalization gap, its two real failure modes (frozen-bbox
|
||||
"ghost tracks"), and one case where it correctly identified an actor that
|
||||
the X-Ray ground truth itself failed to credit.
|
||||
|
||||
## The full technical log
|
||||
|
||||
- **[Rep4 model bake-off + threshold re-tune](rep4-optimizer-results.md)** —
|
||||
the complete experiment log behind the four pages above: the ROCm teardown
|
||||
deadlock root cause and fix, DE concurrency tuning, the full 16-combo
|
||||
results table, and every caveat. This is where the shipped `src/config.hpp`
|
||||
defaults come from.
|
||||
- **[Optimizer experiments (prior round)](optimizer-experiments.md)** — the
|
||||
earlier scene-union-metric tuning pass, superseded by the per-second metric
|
||||
used in rep4 but kept for the ground-truth/architecture background.
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
# Deep dive: LVFace-B Glint360K
|
||||
|
||||
LVFace won the model bake-off (see `best-model.md`) and is the shipped default
|
||||
embedder. This page is the honest accounting of how it actually performs —
|
||||
including where it's wrong, and one case where the ground truth itself is
|
||||
wrong and LVFace is right.
|
||||
|
||||
## Training vs. held-out: the generalization gap
|
||||
|
||||
The shipped config (`prob_threshold=0.754, anneal_sec=35.54,
|
||||
extinction_sec=57.43, expand_gallery=true`) was tuned against 4 films. Scored
|
||||
against the 5 films the optimizer never saw:
|
||||
|
||||
| film | F1 | P | R | TPI | FPI | misid | FN |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| Benny & Joon | 83.0% | 89.1% | 77.7% | 15125 | 1846 | 0 | 4337 |
|
||||
| Lovelace | 77.5% | 90.3% | 67.9% | 14990 | 1085 | 58 | 7085 |
|
||||
| Valerian and the City of a Thousand Planets | 74.1% | 97.1% | 60.0% | 18663 | 548 | 0 | 12467 |
|
||||
| Downton Abbey: A New Era | 56.2% | 97.8% | 39.4% | 52027 | 1173 | 0 | 80084 |
|
||||
| **The Many Saints of Newark** | **46.3%** | **54.7%** | 40.1% | 15922 | 4394 | **974** | 23791 |
|
||||
| **macro average** | **67.4%** | 85.8% | 57.0% | | | | |
|
||||
|
||||
**67.4% held-out vs. 75.3% on training** — an ~8pp drop, and a **37pp spread
|
||||
between the best and worst held-out film**. The config does not generalize
|
||||
uniformly; two films are outright failure cases, for two different reasons.
|
||||
|
||||
## Failure mode 1: frozen-bbox "ghost tracks"
|
||||
|
||||
Both Many Saints of Newark (974 misIDs) and Downton Abbey (FN=80084, the worst
|
||||
recall of the five) trace to the same root cause, verified directly against
|
||||
the raw per-frame stream and the HDF5 dump's own detection counts — not
|
||||
inferred from the score alone.
|
||||
|
||||

|
||||
|
||||
At this second, three of the four labeled boxes ("Jon Bernthal", "Joey Diaz",
|
||||
"Billy Magnussen") sit over empty background — a blurred wall, hanging
|
||||
plates — with no face in them. The real face in frame carries a second,
|
||||
colliding label from another frozen box.
|
||||
|
||||

|
||||
|
||||
This is the starkest case: **15 actors named, all wrong, over a completely
|
||||
blank closing title card.** Confirmed against the dump directly: `face_count`
|
||||
is 0 from this point onward (no detector output at all), yet the same 15
|
||||
identities keep appearing with the *exact same bounding box, unchanged to the
|
||||
pixel*, for 57+ consecutive seconds.
|
||||
|
||||
This is `SceneTrackerFunc::active_[actor_idx].last_bbox`
|
||||
(`src/nodes/scene_tracker_node.hpp`) being re-emitted unchanged — the
|
||||
extinction state machine working exactly as coded, not a bug. The film cuts
|
||||
from a packed group shot straight into 40+ seconds of blank titles/credits,
|
||||
and `extinction_sec=57.4` is comfortably long enough to bridge that entire gap
|
||||
without expiring, so the tracker faithfully reports "last known position" for
|
||||
a cast that is no longer on screen at all. `extinction_sec` was tuned toward
|
||||
long windows specifically because they bridge real gaps (occlusion, a turned
|
||||
face) in most training footage — this is the cost side of that trade,
|
||||
surfacing only when a film has a long enough faceless stretch to expose it.
|
||||
|
||||
## Failure mode 2: a genuine misID (for contrast)
|
||||
|
||||
Not every held-out failure is a ghost. This is a real face, correctly
|
||||
detected, confidently misidentified:
|
||||
|
||||
*(same many_saints_ghost_fpi.jpg frame above also shows Leslie Odom Jr.'s box
|
||||
carrying a second, colliding "Michael Gandolfini" label — two real tracks'
|
||||
frozen positions happening to overlap, not a detection error.)*
|
||||
|
||||
## Where LVFace beat X-Ray
|
||||
|
||||
Not every "misID" is actually wrong. `second_score.py` counts a name as a true
|
||||
out-of-cast misID whenever the named actor isn't in X-Ray's credited cast list
|
||||
for the film at all — but X-Ray's cast list is itself incomplete.
|
||||
|
||||

|
||||
|
||||
Germar Terrell Gardner — a real, clean, high-confidence detection — is counted
|
||||
as a misID here because he doesn't appear in X-Ray's `people.csv` for The Many
|
||||
Saints of Newark at all. But Jellyfin's independent cast metadata *does* credit
|
||||
him for this exact film (cross-checked via `experiments/manifests/
|
||||
jellyfin_casts.json`, a completely separate data source from X-Ray). This
|
||||
isn't a lookalike error or a gallery mixup — it's the pipeline correctly
|
||||
recognising a real cast member that one ground-truth source happened to omit.
|
||||
|
||||
This doesn't mean every flagged misID is secretly correct — Many Saints'
|
||||
974-count total is still overwhelmingly the frozen-bbox failure mode above,
|
||||
not uncredited-but-real cameos. But it's a reminder that the X-Ray corpus is a
|
||||
convenient, large-scale ground truth, not a perfect one, and the "misID" number
|
||||
in any of these tables has some irreducible noise floor from ground-truth gaps
|
||||
in the other direction too.
|
||||
|
||||
## Summary
|
||||
|
||||
LVFace is the right default: it wins the model comparison outright, and its
|
||||
failures are traceable, understood, and mostly attributable to one tunable
|
||||
knob (`extinction_sec`) rather than the embedder itself. The held-out
|
||||
generalization gap (75.3% → 67.4%) is real and should be treated as the honest
|
||||
expected performance, not the training-set number.
|
||||
@@ -0,0 +1,97 @@
|
||||
# Pose expansion: does "learning" new poses mid-film help?
|
||||
|
||||
`expand_gallery` (`src/gallery/track_gallery.hpp`) promotes a confidently-identified
|
||||
track's novel-pose reference views into a per-film, in-memory gallery annex — the
|
||||
idea being that once the pipeline is sure who someone is, a pose it hasn't seen
|
||||
before (turned head, different lighting) becomes a free extra reference for
|
||||
recognising that actor again later in the same film, without touching the baked
|
||||
gallery.
|
||||
|
||||
## The training-set signal
|
||||
|
||||
Averaged across all 4 models, on the 4 films used for optimization:
|
||||
|
||||
| scope | expansion | F1 | R | misID |
|
||||
|---|---|---|---|---|
|
||||
| full | off | 71.2% | 58.3% | 209 |
|
||||
| full | **on** | 71.2% | 59.7% | **864** |
|
||||
| restricted | off | 73.6% | 61.3% | 194 |
|
||||
| restricted | **on** | **75.4%** | **64.5%** | 135 |
|
||||
|
||||
In `restricted` mode (matcher's candidate set capped to the film's own credited
|
||||
cast) expansion looked like a clean win: +1.8pp F1, +3.2pp recall, misID actually
|
||||
lower. In `full` mode it looked flat-to-costly: ~0 F1 change, recall +1.4pp, but
|
||||
misID roughly quadrupled (209 → 864) — see `rep4-optimizer-results.md` for the
|
||||
per-model breakdown. That's the number that motivated this page: **does turning
|
||||
expansion on actually change what gets recognised, frame by frame, or is the
|
||||
aggregate F1 shift something else?**
|
||||
|
||||
## Held-out test: does it reproduce?
|
||||
|
||||
Same model + same tuned config, `expand_gallery` toggled on vs. off, nothing else
|
||||
changed — full gallery mode, per-second scoring against X-Ray. This isolates
|
||||
expansion from every other variable (config, model, threshold) that differs
|
||||
between the training-set `exp`/`noexp` rows above.
|
||||
|
||||
**LVFace-B Glint360K, all 5 held-out films** (films never seen by the optimizer):
|
||||
|
||||
| film | F1 (exp) | F1 (noexp) | TPI Δ | FN Δ |
|
||||
|---|---|---|---|---|
|
||||
| Benny & Joon | 83.0% | 83.0% | -2 | +2 |
|
||||
| Downton Abbey: A New Era | 56.1% | 56.2% | -7 | +7 |
|
||||
| Lovelace | 77.5% | 77.4% | +33 | -33 |
|
||||
| The Many Saints of Newark | 46.3% | 46.3% | +2 | -2 |
|
||||
| Valerian and the City of a Thousand Planets | 74.1% | 74.1% | +2 | -2 |
|
||||
|
||||
**ArcFace R18** (Benny & Joon, r18's own tuned config): F1 77.1% for both, TPI/FN
|
||||
identical, FPI differs by 2 (noise).
|
||||
|
||||
**Every film, both models tested: F1 within 0.1–0.2pp, TPI/FN swings in the tens
|
||||
out of tens of thousands.** That's noise, not a signal — expansion made no
|
||||
measurable difference to per-second onscreen identification anywhere it was
|
||||
tested on unseen data.
|
||||
|
||||
## Two bugs this required catching (this section's own methodology)
|
||||
|
||||
Getting to the clean table above took two wrong turns, both worth recording
|
||||
since they're exactly the kind of error that produces a false positive "look,
|
||||
expansion helped!" finding:
|
||||
|
||||
1. **Timeout truncation.** The first Downton Abbey `exp` replay was cut off by a
|
||||
60s subprocess timeout at ~76% through the film (5589 of 7368 expected
|
||||
seconds) — a genuinely large, silent data loss that showed up as a large,
|
||||
convincing-looking TPI gap (47938 vs 52032) purely because one run had a
|
||||
quarter of the film missing. Caught by comparing `n_seconds` between runs
|
||||
before trusting any score delta; fixed by re-running with a longer timeout.
|
||||
2. **Bbox-matching bug.** An early per-second raw-annotation diff matched each
|
||||
`exp` detection to the *first* `noexp` detection with IoU > 0.5, not the
|
||||
*best*-overlapping one. With 3 faces close together in frame, this produced
|
||||
spurious "disagreements" (e.g. "exp says Aidan Quinn, noexp says Johnny
|
||||
Depp" at the same seconds) that vanished entirely once the match picked the
|
||||
true best-IoU candidate — both configs had actually output the exact same
|
||||
three names at the exact same three boxes.
|
||||
|
||||
Both bugs independently pointed toward "expansion is doing something," and both
|
||||
were artifacts of the comparison harness, not the pipeline. Worth remembering
|
||||
when a before/after diff looks dramatic: check that the two runs actually cover
|
||||
the same seconds, and match entities by best overlap, not first-found.
|
||||
|
||||
## What this means
|
||||
|
||||
The training-set aggregate effect (particularly the ~4x misID increase in full
|
||||
mode) doesn't reproduce on held-out data — at minimum it's far smaller than the
|
||||
training-set numbers suggested, and plausibly it's sampling variation from only
|
||||
4 training films rather than a real, generalizable mechanism. This doesn't mean
|
||||
`expand_gallery` never does anything (the mechanism is real — see
|
||||
`track_gallery.hpp`'s promotion logging: tracks *do* get confirmed and views *do*
|
||||
get promoted into the annex on every film tested), only that **whatever effect
|
||||
it has on final per-second identification was too small to detect against 5
|
||||
held-out films** with this scoring method. A cleaner test would need either many
|
||||
more held-out films or a metric that can see the annex's direct contribution
|
||||
(e.g. tagging which reference embedding won each match), neither of which this
|
||||
pass had budget for.
|
||||
|
||||
**Practical takeaway**: don't treat the training-set `exp` vs `noexp` numbers in
|
||||
`rep4-optimizer-results.md` as proof that expansion changes real-world behavior
|
||||
in either direction — on the evidence gathered so far, it doesn't move the
|
||||
needle enough to see.
|
||||
+6
-1
@@ -18,7 +18,12 @@ theme:
|
||||
|
||||
nav:
|
||||
- Home: index.md
|
||||
- Rep4 Bake-off & Re-tune: rep4-optimizer-results.md
|
||||
- Findings:
|
||||
- Best Model: best-model.md
|
||||
- Gallery Scope (Full vs. Limited): gallery-scope.md
|
||||
- Pose Expansion: pose-expansion.md
|
||||
- LVFace Deep Dive: lvface-deep-dive.md
|
||||
- Rep4 Bake-off & Re-tune (full log): rep4-optimizer-results.md
|
||||
- Optimizer Experiments (prior round): optimizer-experiments.md
|
||||
- Service Conversion (proposal): service-conversion.md
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@
|
||||
# scripts/artifacts/pull_artifacts.sh galleries [version]
|
||||
# scripts/artifacts/pull_artifacts.sh montage-frames <film-slug> [version]
|
||||
# scripts/artifacts/pull_artifacts.sh experiment-data [version]
|
||||
# scripts/artifacts/pull_artifacts.sh report-highlights <name> [version]
|
||||
# version defaults to "latest" (newest uploaded version, by created_at).
|
||||
set -euo pipefail
|
||||
|
||||
@@ -74,10 +75,19 @@ pull_experiment_data() {
|
||||
rm "$tmp"
|
||||
}
|
||||
|
||||
pull_report_highlight() {
|
||||
local version="$1" name="$2"
|
||||
local dest="${REPO_ROOT}/docs/assets/images"
|
||||
mkdir -p "$dest"
|
||||
echo "=== report-highlights/${name} (version ${version}) ==="
|
||||
curl -sf "${DL_BASE}/generic/report-highlights/${version}/${name}" -o "${dest}/${name}"
|
||||
}
|
||||
|
||||
if [ $# -eq 0 ]; then
|
||||
echo "usage: $0 galleries [version]" >&2
|
||||
echo " $0 montage-frames <film-slug> [version]" >&2
|
||||
echo " $0 experiment-data [version]" >&2
|
||||
echo " $0 report-highlights <name> [version]" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -99,8 +109,14 @@ case "$TARGET" in
|
||||
[ "$VERSION" = "latest" ] && VERSION="$(resolve_latest_version experiment-data)"
|
||||
pull_experiment_data "$VERSION"
|
||||
;;
|
||||
report-highlights)
|
||||
NAME="${2:?usage: $0 report-highlights <name> [version]}"
|
||||
VERSION="${3:-latest}"
|
||||
[ "$VERSION" = "latest" ] && VERSION="$(resolve_latest_version report-highlights)"
|
||||
pull_report_highlight "$VERSION" "$NAME"
|
||||
;;
|
||||
*)
|
||||
echo "unknown target: $TARGET (expected galleries, montage-frames, or experiment-data)" >&2
|
||||
echo "unknown target: $TARGET (expected galleries, montage-frames, experiment-data, or report-highlights)" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
@@ -11,12 +11,16 @@
|
||||
# scripts/artifacts/push_artifacts.sh galleries
|
||||
# scripts/artifacts/push_artifacts.sh montage-frames
|
||||
# scripts/artifacts/push_artifacts.sh experiment-data
|
||||
# scripts/artifacts/push_artifacts.sh galleries montage-frames experiment-data
|
||||
# scripts/artifacts/push_artifacts.sh report-highlights
|
||||
# scripts/artifacts/push_artifacts.sh galleries montage-frames experiment-data report-highlights
|
||||
#
|
||||
# Package layout (owner=dtourolle, repo=scene-actor-extraction):
|
||||
# generic/galleries/<version>/gallery_<model>.h5 (one file per model)
|
||||
# generic/montage-frames/<version>/<film-slug>.zip (zipped per-film frames)
|
||||
# generic/experiment-data/<version>/experiment-data.zip (manifests/trajectories/results)
|
||||
# generic/report-highlights/<version>/<name>.jpg (individual, hand-picked
|
||||
# illustrative frames referenced by docs/*.md that aren't reproducible via
|
||||
# the automated best/worst montage selection — see scripts/docs/build_site.sh)
|
||||
# version = current git short SHA, so artifacts are traceable to the code that
|
||||
# produced them. Re-running with the same SHA overwrites that version's files.
|
||||
set -euo pipefail
|
||||
@@ -95,17 +99,28 @@ push_experiment_data() {
|
||||
upload "experiment-data" "experiment-data.zip" "$zipfile"
|
||||
}
|
||||
|
||||
push_report_highlights() {
|
||||
echo "=== report-highlights (version ${VERSION}) ==="
|
||||
local src="${REPO_ROOT}/experiments/results/holdout/montage/many_saints/out_of_cast_fpi/10_t003641.jpg"
|
||||
if [ ! -f "$src" ]; then
|
||||
echo " [warn] $src not found, skipping" >&2
|
||||
return
|
||||
fi
|
||||
upload "report-highlights" "germar_beats_xray.jpg" "$src"
|
||||
}
|
||||
|
||||
if [ $# -eq 0 ]; then
|
||||
echo "usage: $0 <galleries|montage-frames|experiment-data> [...]" >&2
|
||||
echo "usage: $0 <galleries|montage-frames|experiment-data|report-highlights> [...]" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
for target in "$@"; do
|
||||
case "$target" in
|
||||
galleries) push_galleries ;;
|
||||
montage-frames) push_montage_frames ;;
|
||||
experiment-data) push_experiment_data ;;
|
||||
*) echo "unknown target: $target (expected galleries, montage-frames, or experiment-data)" >&2; exit 1 ;;
|
||||
galleries) push_galleries ;;
|
||||
montage-frames) push_montage_frames ;;
|
||||
experiment-data) push_experiment_data ;;
|
||||
report-highlights) push_report_highlights ;;
|
||||
*) echo "unknown target: $target (expected galleries, montage-frames, experiment-data, or report-highlights)" >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
|
||||
@@ -26,6 +26,11 @@ cp -v "${FRAMES_ROOT}/many_saints/fpi/fpi_t03543.jpg" \
|
||||
cp -v "${FRAMES_ROOT}/downton_abbey/fpi/fpi_t07242.jpg" \
|
||||
"${ASSETS_DIR}/downton_abbey_ghost_fpi.jpg"
|
||||
|
||||
if [ ! -f "${ASSETS_DIR}/germar_beats_xray.jpg" ]; then
|
||||
echo "==> pulling report-highlights/germar_beats_xray.jpg..."
|
||||
scripts/artifacts/pull_artifacts.sh report-highlights germar_beats_xray.jpg
|
||||
fi
|
||||
|
||||
if [ ! -d experiments/galleries ] || [ -z "$(ls -A experiments/galleries 2>/dev/null)" ]; then
|
||||
echo "==> pulling galleries (not found locally)..."
|
||||
scripts/artifacts/pull_artifacts.sh galleries
|
||||
|
||||
Reference in New Issue
Block a user