dtourolle/scene-actor-extraction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

improved performance

Browse Source
This commit is contained in:
dtourolle
2026-06-13 22:44:44 +02:00
parent fc16d4a0e1
commit a3ba53ddf7
8 changed files with 142 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+6 -3
View File
@@ -5,7 +5,7 @@ Identifies actors in movie files and produces X-ray-style scene annotations comp
## How it works
1. **Build a gallery** — download actor headshots from TMDB/IMDB, embed them with ArcFace (`build_gallery` / `scripts/make_gallery.py`).
2. **Analyze a movie** — `scene_analyze` decodes frames at configurable FPS, detects faces (YuNet/SCRFD), tracks them across cuts, matches identities against the gallery using calibrated similarity, and writes time-window JSON.
2. **Analyze a movie** — `scene_analyze` decodes frames at configurable FPS, detects faces (SCRFD), tracks them across cuts, matches identities against the gallery using calibrated similarity, and writes time-window JSON.
3. **Output** — minimal mode produces Jellyfin-ready actor name + time-window JSON; standard mode adds per-frame bbox, similarity, and track data.
## Dependencies
@@ -13,9 +13,12 @@ Identifies actors in movie files and produces X-ray-style scene annotations comp
| Dependency | Role |
|---|---|
| KPN++ | Pipeline backbone (nodes, networks) |
| OpenCV 4 | Video decode, image ops, DNN inference, YuNet face detection |
| OpenCV 4 | Video decode, image ops, DNN inference |
| ONNX Runtime | SCRFD face detector (dynamic shape nodes unsupported by cv::dnn) |
| TensorRT + CUDA runtime + cuBLAS | Optional TRT engines for SCRFD/ArcFace (`--detector-engine`/`--arcface-engine`); identity_matcher's GPU gallery scan |
| FFmpeg (libav*) | NVDEC hardware video decode + colour conversion |
| nlohmann/json | JSON I/O |
| nanobind | Python bindings for `sae_embed` |
## Build
@@ -62,6 +65,7 @@ Models are placed in `external/`:
| `scene_analyze_debug` | Same as above + per-frame annotated JPEGs (`SAE_DEBUG=1`) |
| `scene_preview` | Live OpenCV display window while analysing |
| `build_gallery` | Offline gallery builder from a directory of images |
| `embed_faces` | CLI: image(s) → embedding JSON, used by gallery-builder scripts |
| `sae_embed` | Python module (nanobind) used by gallery-builder scripts — loads SCRFD+ArcFace once |
### `scene_analyze`
@@ -82,7 +86,6 @@ Key options:
| `--track-min-iou` | — | Minimum IoU gate for spatial assignment |
| `--track-max-embed` | — | Maximum embedding distance gate |
| `--track-max-missing` | — | Frames a track survives without a detection |
| `--track-min-frames` | 3 | Observations before a track's mean embedding is used for matching |
### Gallery builders