# JRay — working notes

Three components. Start with the system spec; each repo's spec implements it.

| Doc | Owns |
|---|---|
| [`SPEC.md`](SPEC.md) | **System** requirements — anything spanning more than one repo |
| [`scene-actor-extraction/docs/SPEC.md`](scene-actor-extraction/docs/SPEC.md) | Extraction pipeline software requirements |
| [`scene-actor-extraction/docs/requirements.md`](scene-actor-extraction/docs/requirements.md) | Stable requirement IDs, statuses, verification plan |
| [`scene-actor-extraction/docs/plan.md`](scene-actor-extraction/docs/plan.md) | Per-requirement implementation notes and dependencies |
| [`jRay/SPEC.md`](jRay/SPEC.md) | Truth-file format, Jellyfin plugin API |
| [`JRay-public-server/SPEC.md`](JRay-public-server/SPEC.md) | Jmanifest exchange, cut matching, audio signature |

---

## Invariants — do not violate without an explicit, recorded agreement

### Always use the calibrated probability. Never a raw cosine.

**Every similarity is converted through the sigmoid calibration before it is
used, compared, or thresholded.** This applies everywhere — identity matching,
tracking association, gallery expansion admission, cluster merging — not just at
the final identity decision.

A raw cosine threshold is an unfalsifiable magic number: it means something
different for every model, every gallery, and every face size, and it cannot be
combined with anything else. A calibrated probability means the same thing
everywhere and composes (see the per-track Bayesian accumulation in A9).

**If a raw cosine is ever used, it is an agreed exception and must be recorded as
such in the spec, with the reason.** Absent that, treat any bare cosine
comparison in the code as a defect to be fixed. The constants being retired for
this reason include `track_max_embed_dist`, `cut_revive_sim`,
`expand_novelty_sim` and `expand_track_spread_max`.

**Agreed exceptions so far** (do not "fix" these):

| Where | Why |
|---|---|
| `GR-008` — distributional outlier check on an actor's own references | The sigmoid is a monotonic squash: right for *decisions*, wrong for characterising a *distribution*, since it compresses exactly the tails where outliers live. Shape is a property of the metric space. Scope is analysis only — every match decision still goes through the calibration. |

### Every model gets the input it was trained for

Cost is reduced by running a model **less often** or on **fewer regions** — never
by degrading what a single inference sees. A model run off-distribution returns
confident, plausible, wrong output, and the error is invisible without a study
that should not have been needed. (See A5: TransNetV2 must be fed at native frame
rate, not a cheaper 12 fps.)

### Presence is scene-scoped, not instantaneous

The question is *"is this actor in this scene?"*, not *"is their face visible in
this frame?"* — because that is what the X-Ray ground truth records. An actor who
turns away, is occluded, or is off-camera while the shot cuts to whoever they are
speaking to **is still present**. Reporting them absent is not a stricter answer;
it is an answer to a different question. (SR-002.)

A window ends at the **last sighting**, never after it. Gaps *between* sightings
are absorbed; time *after* the final one never is.

### The public server stores no binary content

Its abuse defence is structural — no images, no base64, no opaque blobs, no
extension points. This is what makes it safe for a volunteer to operate. Any
proposal to ship images or embeddings through it is a proposal to delete that
property, not an incremental feature. (SR-004.)

---

## Conventions

### Tag code with the requirement it implements

House format, matching [`JellyTau`](../JellyTau/docs/traces-quick-ref.md) so the
tooling ports directly. Pipe separates requirement types, comma separates IDs
within a type:

```cpp
/// TRACES: AR-012, AR-013 | SR-002
struct TrackRegistry { … };
```

- Tag the unit that **decides**, not every helper it calls.
- **Tests carry TRACES too** (`UT-nnn`, `IT-nnn`) — that is what shows a
  requirement is *verified*, not merely implemented.
- **A deliberate raw-cosine use is tagged `EXCEPTION: AR-024 <reason>`.** Per the
  invariant above, a bare cosine with no recorded exception is a defect.

Chain: `PR-nnn → SR-nnn → component requirement → TRACES`, defined in
[`SPEC.md`](SPEC.md) §6. Registers live in each component's
`docs/requirements.md`; **IDs are permanent and never reused**, since renumbering
is what creates orphan tags.

When adding a requirement, give it a parent and a register row. When adding code,
give it a tag.

- **Specs are requirements + current-state deltas.** Each requirement carries
  `Current:` and `Gap:` so the document doubles as a work list. Keep that shape
  when editing.
- **State understanding before writing it into a doc.** Design here has been
  settled by discussion, and the corrections have usually been *simpler* than the
  proposal. Confirm first; do not write elaborate mechanism into a spec on an
  unverified assumption.
- Breaking format changes are **batched into one `schema_version` bump** and
  coordinated across all three repos (SR-003).