scene-actor-extraction/docs/service-conversion.md

# Conversion to service — a native idle-GPU worker

Status: **design / proposal**. Nothing here is built yet.

## The idea

Turn the CLI tools into a **turnkey batch worker that uses the machine's idle
GPU**: it analyses newly-added Jellyfin media when you're not using the computer
(screen locked), and stops the instant you come back. It's an overnight job on
your own Linux box.

**No Docker.** This runs on your own machine with your own drivers, so a container
buys little and costs a lot: GPU passthrough (nvidia-container-toolkit, or
`/dev/kfd`+`/dev/dri`+`video` group for ROCm) is the single most fragile part of a
containerised setup, and it exists *only* because of the container. Natively, the
GPU just works with the drivers you already have, and the media paths Jellyfin
reports are just real paths — no re-mounting. So we ship a **native installer**
instead of an image builder.

Two deliverables:

1. **An installer** — `scripts/build_install.py`. Detects your distro, ensures the
   GPU/build dependencies are present (via `dnf`/`pacman`), compiles `scene_analyze`
   for your GPU, and installs the binary + Python glue + two systemd **user**
   units under `~/.local`.
2. **A screen-lock gate** — one of those systemd units watches logind lock/unlock
   and starts/stops the worker. Lock → analyse. Unlock → stop.

## What already exists (reuse, don't rebuild)

The processing loop is already implemented — this is packaging, building, and
lock-gating, not new pipeline logic.

| Piece | Where | What it does |
|---|---|---|
| Analysis engine | `build/scene_analyze` | Video → face detect/align/embed → gallery match → result JSON |
| Backend selection | [`CMakeLists.txt`](https://REPOLINK/CMakeLists.txt) (`SAE_INFERENCE_BACKEND`, `SAE_GEMM_BACKEND`) | ORT/TRT + ROCm/CUDA, chosen **at build time** |
| New-media queue | JRay plugin → `GET /Plugins/JRay/Tasks/Pending` | Backlog of items with no results yet |
| Worker loop | [`scripts/run_from_jellyfin.py`](https://REPOLINK/scripts/run_from_jellyfin.py)` --worker` | Poll Pending → run `scene_analyze` → push results |
| Result push | `PUT /Plugins/JRay/Items/{id}/Truth` | Stores per-actor scene windows back in Jellyfin |
| Incremental gallery | [`scripts/make_jellyfin_gallery.py`](https://REPOLINK/scripts/make_jellyfin_gallery.py)` --merge` | Embeds only cast not already in the gallery |
| Secrets loader | `.env` via [`scripts/sae_env.py`](https://REPOLINK/scripts/sae_env.py) | `JELLYFIN_URL`, `JELLYFIN_API_KEY`, `TMDB_API_KEY` |

## Installer config

One file. Build-time settings (fixed when we compile) vs. run-time settings (in the
worker's `.env`, editable without recompiling).

```yaml
# install.yaml — consumed by scripts/build_install.py

platform: nvidia            # nvidia | amd | cpu → picks the cmake backend
model:
  arcface: LVFace-B_Glint360K.onnx   # embedder compiled against; gallery MUST match
schedule:
  gallery_scan_interval: 24h         # incremental --merge cadence; 0 disables the scanner
prefix: ~/.local                     # install root (bin, share, systemd user units)

# runtime (written to the worker .env, not compiled in):
runtime:
  jellyfin_url: http://localhost:8096
  # JELLYFIN_API_KEY / TMDB_API_KEY are filled into .env by hand after install
```

**Secrets never go in the repo or a build artifact** — the installer writes a
`.env` under the install prefix with blanks for the keys, and you fill them in
once. `sae_env.py` already loads it.

**Model ⇄ gallery coupling (guard, don't just document):** embeddings from
different recognition models aren't interchangeable. We compile against one
embedder; the gallery must be built with the same one. Stamp the embedder name
into `gallery.json`, and have the worker **refuse to start** if the gallery's
embedder ≠ the configured `model.arcface`, rather than silently mismatching.

## Dependencies via the system package manager

The heavy build/runtime deps (OpenCV, ffmpeg, the GPU stack) are best provided by
the distro, not vendored. The installer ships a per-distro dependency list and
either installs them or prints the exact command. Targets: **Fedora (dnf)** and
**Arch (pacman)** first.

| Dependency | Fedora (dnf) | Arch (pacman) |
|---|---|---|
| OpenCV | `opencv-devel` | `opencv` |
| ffmpeg | `ffmpeg-free`/`ffmpeg` (RPM Fusion) | `ffmpeg` |
| CMake / toolchain | `cmake gcc-c++` | `cmake gcc` |
| CUDA + TensorRT (nvidia) | NVIDIA CUDA repo + `libnvinfer-*` | `cuda`, `tensorrt` |
| ROCm (amd) | `rocm-hip-sdk` / `rocblas-devel` | `rocm-hip-sdk`, `rocblas` |
| ONNX Runtime | **not packaged** — installer fetches a pinned release tarball into the prefix | AUR `onnxruntime` (or same pinned-tarball fallback) |

So the flow is: **detect distro → check each package → install via the native
manager (or print `sudo dnf install …` / `sudo pacman -S …`)**, with ONNX Runtime
as the one known gap the installer fills itself (a pinned upstream release
extracted under the install prefix, so it doesn't depend on a system package that
may not exist). CUDA/ROCm being present is *assumed* — you already run a GPU
desktop; the installer verifies and points you at the vendor repo if not.

## What `build_install.py` does

```
build_install.py install.yaml
  │
  ├─ detect distro (dnf vs pacman) and platform from config
  ├─ ensure deps: install via manager, or print the exact command; fetch ONNX Runtime if needed
  ├─ cmake + build scene_analyze with the platform's backend flags:
  │     nvidia → -DSAE_INFERENCE_BACKEND=TRT -DSAE_GEMM_BACKEND=CUDA
  │     amd    → -DSAE_INFERENCE_BACKEND=ORT -DSAE_GEMM_BACKEND=ROCM
  │     cpu    → -DSAE_INFERENCE_BACKEND=ORT  (CPU EP; slow, for smoke tests)
  ├─ install into <prefix>:
  │     bin/sae-scene-analyze           the compiled binary
  │     share/sae-worker/               Python glue + a venv (requests, etc.), models/
  │     share/sae-worker/.env           runtime config (keys blank, url from config)
  ├─ install systemd --user units:
  │     sae-worker.service              runs the worker + gallery-scan supervisor
  │     sae-lock-gate.service           watches logind lock/unlock, start/stops the worker
  └─ print next steps (edit .env, `systemctl --user enable --now sae-lock-gate`)
```

## The worker service (supervisor)

`sae-worker.service` runs a small Python supervisor as its main process:

- starts the **worker loop** (`run_from_jellyfin.py --worker`) — the hot path,
- starts a **gallery-scan timer** — sleeps `gallery_scan_interval`, runs
  `make_jellyfin_gallery.py --merge`, repeats,
- exits cleanly on SIGTERM (see re-queue below).

## The lock gate

`sae-lock-gate.service` runs a tiny watcher that subscribes to logind
lock/unlock signals and drives the worker service:

```
screen locks   → systemctl --user start sae-worker.service
screen unlocks → systemctl --user stop  sae-worker.service   (SIGTERM)
```

**Screen-lock is the only signal — deliberately.** We don't also gate on GPU/CPU
load, because our own worker *is* the load: a load threshold would form a feedback
loop (worker starts → GPU spikes → threshold trips → worker stops → load drops →
restart → …). Lock state is external to what the worker does, so it can't
oscillate.

Signal source is desktop-dependent: logind `Lock`/`Unlock` (GNOME/KDE via
`loginctl`/D-Bus) covers most setups; a `swayidle`/`xss-lock` hook is the fallback
for wlroots/X-only compositors. The installer picks based on what's present.

## On resume: hard stop + re-queue (it's free)

Stopping the worker mid-analysis costs nothing to reschedule, because of how the
JRay queue works: **an item only leaves `/Tasks/Pending` once its results are
pushed** (`push_truth`). A worker stopped mid-`scene_analyze` simply leaves that
item Pending — next lock picks it up again. No re-queue bookkeeping.

Two small correctness requirements (the only worker changes needed):

1. **Never push a partial result.** Already true — `push_truth` runs only after
   `scene_analyze` returns; a killed run pushes nothing. ✓ (keep it that way).
2. **Clean up on signal.** `process_item` writes a temp filtered-gallery file and
   unlinks it in a `finally`; a SIGKILL skips `finally`. Fix: write temps under a
   dir the worker wipes on start, and/or a SIGTERM handler that unlinks before
   exit. Minor.

Accepted trade-off: a partially-analysed title restarts from scratch next lock.
Fine for an overnight/idle workload; no mid-video checkpointing.

## The end-to-end UX

```bash
# once: build + install for your GPU + model
./scripts/build_install.py install.yaml
    # detects Fedora/Arch, ensures deps, compiles, installs units under ~/.local

# once: set your keys, enable the gate
$EDITOR ~/.local/share/sae-worker/.env     # JELLYFIN_API_KEY, TMDB_API_KEY
systemctl --user enable --now sae-lock-gate.service

# from then on: nothing. Lock your screen → it analyses. Unlock → it stops.
```

No Docker, no GPU passthrough config, no media re-mounting — the worker sees the
same filesystem and GPU as everything else on the box.

## Implementation plan (follow-up commits)

Ordered so each step stands alone:

1. **installer skeleton** — `scripts/build_install.py`: parse `install.yaml`,
   distro detect, dependency check/print (start with cpu platform so it builds
   without a GPU), cmake+build, copy into prefix.
2. **supervisor + cleanup** — `scripts/service.py` (worker loop + gallery-scan
   timer + SIGTERM); temp-file cleanup fix in `run_from_jellyfin.py`.
3. **systemd units + lock gate** — generate/install `sae-worker.service`,
   `sae-lock-gate.service`, and the logind lock watcher.
4. **gallery/model guard** — stamp embedder into `gallery.json`; startup mismatch
   check.
5. **platform + distro matrix** — nvidia/amd backends; dnf/pacman dep lists; ONNX
   Runtime fetch fallback.
6. **docs** — README "Run on your idle GPU" section.

## Settled decisions

- **ONNX Runtime build** — the installer fetches the **ROCm ORT** release. It
  serves the `amd` platform, and its CPU execution provider covers the `cpu`
  smoke-test fallback too, so one download handles both. (nvidia uses raw TRT and
  doesn't need ORT.)
- **`dnf`/`pacman` invocation** — **auto-install.** The installer runs `sudo dnf
  install …` / `sudo pacman -S …` itself (prompting for sudo), rather than only
  printing the command. It still prints what it's about to install first.
- **Distro coverage** — **Fedora + Arch only** for now. Debian/Ubuntu (`apt`) is
  out of scope.

## Open questions

*(none blocking — the spec above is buildable as-is.)*