jray-project/CLAUDE.md

# JRay — working notes

Three components. Start with the system spec; each repo's spec implements it.

| Doc | Owns |
|---|---|
| [`SPEC.md`](SPEC.md) | **System** requirements — anything spanning more than one repo |
| [`scene-actor-extraction/docs/SPEC.md`](scene-actor-extraction/docs/SPEC.md) | Extraction pipeline software requirements |
| [`scene-actor-extraction/docs/requirements.md`](scene-actor-extraction/docs/requirements.md) | Stable requirement IDs, statuses, verification plan |
| [`scene-actor-extraction/docs/plan.md`](scene-actor-extraction/docs/plan.md) | Per-requirement implementation notes and dependencies |
| [`jRay/SPEC.md`](jRay/SPEC.md) | Truth-file format, Jellyfin plugin API |
| [`JRay-public-server/SPEC.md`](JRay-public-server/SPEC.md) | Jmanifest exchange, cut matching, audio signature |

---

## Invariants — do not violate without an explicit, recorded agreement

### Always use the calibrated probability. Never a raw cosine.

**Every similarity is converted through the sigmoid calibration before it is
used, compared, or thresholded.** This applies everywhere — identity matching,
tracking association, gallery expansion admission, cluster merging — not just at
the final identity decision.

A raw cosine threshold is an unfalsifiable magic number: it means something
different for every model, every gallery, and every face size, and it cannot be
combined with anything else. A calibrated probability means the same thing
everywhere and composes (see the per-track Bayesian accumulation in A9).

**If a raw cosine is ever used, it is an agreed exception and must be recorded as
such in the spec, with the reason.** Absent that, treat any bare cosine
comparison in the code as a defect to be fixed. The constants being retired for
this reason include `track_max_embed_dist`, `cut_revive_sim`,
`expand_novelty_sim` and `expand_track_spread_max`.

**Agreed exceptions so far** (do not "fix" these):

| Where | Why |
|---|---|
| `GR-008` — distributional outlier check on an actor's own references | The sigmoid is a monotonic squash: right for *decisions*, wrong for characterising a *distribution*, since it compresses exactly the tails where outliers live. Shape is a property of the metric space. Scope is analysis only — every match decision still goes through the calibration. |
| `calibrate_gallery` — per-actor near-duplicate dedup at `1 - 1e-7` | Two reasons, either sufficient. It asks whether two vectors are **the same vector** — at that threshold it catches one source image embedded twice, and no pair of distinct photographs lands there — so it is not a decision and there is nothing for a probability to mean. And structurally, it runs on the calibration fit's *input*: a calibrated comparison there would have to be calibrated by the fit it is feeding. |

**The invariant is now enforced, not just asserted.**
`scene-actor-extraction/scripts/ci/check_raw_cosine.py` runs in the traceability
workflow and fails the build on a bare cosine with no recorded exception. It
found one immediately — the identity matcher's no-calibration fallback, which
thresholded raw cosine distance *and* fed `max(0, cosine)` into the Bayesian
accumulation as a posterior.

Know what it does not cover: it cannot follow a cosine through a variable across
statements, and it says nothing about the GEMM similarity matrix. Those remain
conventions backed by review. A pass is evidence, not proof.

### Every model gets the input it was trained for

Cost is reduced by running a model **less often** or on **fewer regions** — never
by degrading what a single inference sees. A model run off-distribution returns
confident, plausible, wrong output, and the error is invisible without a study
that should not have been needed. (See A5: TransNetV2 must be fed at native frame
rate, not a cheaper 12 fps.)

### Presence is scene-scoped, not instantaneous

The question is *"is this actor in this scene?"*, not *"is their face visible in
this frame?"* — because that is what the X-Ray ground truth records. An actor who
turns away, is occluded, or is off-camera while the shot cuts to whoever they are
speaking to **is still present**. Reporting them absent is not a stricter answer;
it is an answer to a different question. (SR-002.)

A window ends at the **last sighting**, never after it. Gaps *between* sightings
are absorbed; time *after* the final one never is.

### The public server stores no binary content

Its abuse defence is structural — no images, no base64, no opaque blobs, no
extension points. This is what makes it safe for a volunteer to operate. Any
proposal to ship images or embeddings through it is a proposal to delete that
property, not an incremental feature. (SR-004.)

---

## Conventions

### Tag code with the requirement it implements

House format, matching [`JellyTau`](../JellyTau/docs/traces-quick-ref.md) so the
tooling ports directly. Pipe separates requirement types, comma separates IDs
within a type:

```cpp
/// TRACES: AR-012, AR-013 | SR-002
struct TrackRegistry { … };
```

- Tag the unit that **decides**, not every helper it calls.
- **Tests carry TRACES too** (`UT-nnn`, `IT-nnn`) — that is what shows a
  requirement is *verified*, not merely implemented.
- **A deliberate raw-cosine use is tagged `EXCEPTION: AR-024 <reason>`.** Per the
  invariant above, a bare cosine with no recorded exception is a defect.

Chain: `PR-nnn → SR-nnn → component requirement → TRACES`, defined in
[`SPEC.md`](SPEC.md) §6. Registers live in each component's
`docs/requirements.md`; **IDs are permanent and never reused**, since renumbering
is what creates orphan tags.

When adding a requirement, give it a parent and a register row. When adding code,
give it a tag.

### Commits name their requirements too

Every commit that implements, changes, or withdraws a requirement carries a
`TRACES:` trailer — the same token and syntax as the code tags, so one grep
pattern serves both:

```
feat(audio): v1 content-derived audio signature

Implements the §3 construction with the six previously-undefined
parameters pinned, plus a golden fixture the plugin can be written from.

TRACES: IR-004, IR-005, IR-007, IR-008 | SR-003
```

This closes the last link in the chain. Code tags say *where* a requirement
lives; commit trailers say *when and why it changed* — `git log --grep=AR-012`
then reconstructs a requirement's entire history, which no other artifact gives
you.

- Trailer last, after any `Co-Authored-By`.
- A commit serving no requirement (formatting, tooling, a typo) simply omits it —
  absence is meaningful, so do not invent a tag to satisfy the form.
- Withdrawing a requirement is a change to it: name it, so the withdrawal is
  findable later.

- **Specs are requirements + current-state deltas.** Each requirement carries
  `Current:` and `Gap:` so the document doubles as a work list. Keep that shape
  when editing.
- **State understanding before writing it into a doc.** Design here has been
  settled by discussion, and the corrections have usually been *simpler* than the
  proposal. Confirm first; do not write elaborate mechanism into a spec on an
  unverified assumption.
- Breaking format changes are **batched into one `schema_version` bump** and
  coordinated across all three repos (SR-003).