jray-project/CLAUDE.md

JRay — working notes

Three components. Start with the system spec; each repo's spec implements it.

Doc	Owns
SPEC.md	System requirements — anything spanning more than one repo
scene-actor-extraction/docs/SPEC.md	Extraction pipeline software requirements
scene-actor-extraction/docs/requirements.md	Stable requirement IDs, statuses, verification plan
scene-actor-extraction/docs/plan.md	Per-requirement implementation notes and dependencies
jRay/SPEC.md	Truth-file format, Jellyfin plugin API
JRay-public-server/SPEC.md	Jmanifest exchange, cut matching, audio signature

---

Invariants — do not violate without an explicit, recorded agreement

Always use the calibrated probability. Never a raw cosine.

Every similarity is converted through the sigmoid calibration before it is used, compared, or thresholded. This applies everywhere — identity matching, tracking association, gallery expansion admission, cluster merging — not just at the final identity decision.

A raw cosine threshold is an unfalsifiable magic number: it means something different for every model, every gallery, and every face size, and it cannot be combined with anything else. A calibrated probability means the same thing everywhere and composes (see the per-track Bayesian accumulation in A9).

If a raw cosine is ever used, it is an agreed exception and must be recorded as such in the spec, with the reason. Absent that, treat any bare cosine comparison in the code as a defect to be fixed. The constants being retired for this reason include track_max_embed_dist, cut_revive_sim, expand_novelty_sim and expand_track_spread_max.

Agreed exceptions so far (do not "fix" these):

Where	Why
GR-008 — distributional outlier check on an actor's own references	The sigmoid is a monotonic squash: right for decisions, wrong for characterising a distribution, since it compresses exactly the tails where outliers live. Shape is a property of the metric space. Scope is analysis only — every match decision still goes through the calibration.

Every model gets the input it was trained for

Cost is reduced by running a model less often or on fewer regions — never by degrading what a single inference sees. A model run off-distribution returns confident, plausible, wrong output, and the error is invisible without a study that should not have been needed. (See A5: TransNetV2 must be fed at native frame rate, not a cheaper 12 fps.)

Presence is scene-scoped, not instantaneous

The question is "is this actor in this scene?", not "is their face visible in this frame?" — because that is what the X-Ray ground truth records. An actor who turns away, is occluded, or is off-camera while the shot cuts to whoever they are speaking to is still present. Reporting them absent is not a stricter answer; it is an answer to a different question. (SR-002.)

A window ends at the last sighting, never after it. Gaps between sightings are absorbed; time after the final one never is.

The public server stores no binary content

Its abuse defence is structural — no images, no base64, no opaque blobs, no extension points. This is what makes it safe for a volunteer to operate. Any proposal to ship images or embeddings through it is a proposal to delete that property, not an incremental feature. (SR-004.)

---

Conventions

Tag code with the requirement it implements

House format, matching JellyTau so the tooling ports directly. Pipe separates requirement types, comma separates IDs within a type:

/// TRACES: AR-012, AR-013 | SR-002
struct TrackRegistry { … };

• Tag the unit that decides, not every helper it calls.
• Tests carry TRACES too (UT-nnn, IT-nnn) — that is what shows a requirement is verified, not merely implemented.
• A deliberate raw-cosine use is tagged EXCEPTION: AR-024 <reason>. Per the invariant above, a bare cosine with no recorded exception is a defect.

Chain: PR-nnn → SR-nnn → component requirement → TRACES, defined in SPEC.md §6. Registers live in each component's docs/requirements.md; IDs are permanent and never reused, since renumbering is what creates orphan tags.

When adding a requirement, give it a parent and a register row. When adding code, give it a tag.

• Specs are requirements + current-state deltas. Each requirement carries Current: and Gap: so the document doubles as a work list. Keep that shape when editing.
• State understanding before writing it into a doc. Design here has been settled by discussion, and the corrections have usually been simpler than the proposal. Confirm first; do not write elaborate mechanism into a spec on an unverified assumption.
• Breaking format changes are batched into one schema_version bump and coordinated across all three repos (SR-003).