jray-project/CLAUDE.md

JRay — working notes

Three components. Start with the system spec; each repo's spec implements it.

Doc	Owns
SPEC.md	System requirements — anything spanning more than one repo
scene-actor-extraction/docs/SPEC.md	Extraction pipeline software requirements
scene-actor-extraction/docs/requirements.md	Stable requirement IDs, statuses, verification plan
scene-actor-extraction/docs/plan.md	Per-requirement implementation notes and dependencies
jRay/SPEC.md	Truth-file format, Jellyfin plugin API
JRay-public-server/SPEC.md	Jmanifest exchange, cut matching, audio signature

---

Invariants — do not violate without an explicit, recorded agreement

Always use the calibrated probability. Never a raw cosine.

Every similarity is converted through the sigmoid calibration before it is used, compared, or thresholded. This applies everywhere — identity matching, tracking association, gallery expansion admission, cluster merging — not just at the final identity decision.

A raw cosine threshold is an unfalsifiable magic number: it means something different for every model, every gallery, and every face size, and it cannot be combined with anything else. A calibrated probability means the same thing everywhere and composes (see the per-track Bayesian accumulation in A9).

If a raw cosine is ever used, it is an agreed exception and must be recorded as such in the spec, with the reason. Absent that, treat any bare cosine comparison in the code as a defect to be fixed. The constants being retired for this reason include track_max_embed_dist, cut_revive_sim, expand_novelty_sim and expand_track_spread_max.

Agreed exceptions so far (do not "fix" these):

Where	Why
GR-008 — distributional outlier check on an actor's own references	The sigmoid is a monotonic squash: right for decisions, wrong for characterising a distribution, since it compresses exactly the tails where outliers live. Shape is a property of the metric space. Scope is analysis only — every match decision still goes through the calibration.
calibrate_gallery — per-actor near-duplicate dedup at 1 - 1e-7	Two reasons, either sufficient. It asks whether two vectors are the same vector — at that threshold it catches one source image embedded twice, and no pair of distinct photographs lands there — so it is not a decision and there is nothing for a probability to mean. And structurally, it runs on the calibration fit's input: a calibrated comparison there would have to be calibrated by the fit it is feeding.

The invariant is now enforced, not just asserted. scene-actor-extraction/scripts/ci/check_raw_cosine.py runs in the traceability workflow and fails the build on a bare cosine with no recorded exception. It found one immediately — the identity matcher's no-calibration fallback, which thresholded raw cosine distance and fed max(0, cosine) into the Bayesian accumulation as a posterior.

Know what it does not cover: it cannot follow a cosine through a variable across statements, and it says nothing about the GEMM similarity matrix. Those remain conventions backed by review. A pass is evidence, not proof.

Every model gets the input it was trained for

Cost is reduced by running a model less often or on fewer regions — never by degrading what a single inference sees. A model run off-distribution returns confident, plausible, wrong output, and the error is invisible without a study that should not have been needed. (See A5: TransNetV2 must be fed at native frame rate, not a cheaper 12 fps.)

Presence is scene-scoped, not instantaneous

The question is "is this actor in this scene?", not "is their face visible in this frame?" — because that is what the X-Ray ground truth records. An actor who turns away, is occluded, or is off-camera while the shot cuts to whoever they are speaking to is still present. Reporting them absent is not a stricter answer; it is an answer to a different question. (SR-002.)

A window ends at the last sighting, never after it. Gaps between sightings are absorbed; time after the final one never is.

The public server stores no binary content

Its abuse defence is structural — no images, no base64, no opaque blobs, no extension points. This is what makes it safe for a volunteer to operate. Any proposal to ship images or embeddings through it is a proposal to delete that property, not an incremental feature. (SR-004.)

---

Conventions

Tag code with the requirement it implements

House format, matching JellyTau so the tooling ports directly. Pipe separates requirement types, comma separates IDs within a type:

/// TRACES: AR-012, AR-013 | SR-002
struct TrackRegistry { … };

• Tag the unit that decides, not every helper it calls.
• Tests carry TRACES too (UT-nnn, IT-nnn) — that is what shows a requirement is verified, not merely implemented.
• A deliberate raw-cosine use is tagged EXCEPTION: AR-024 <reason>. Per the invariant above, a bare cosine with no recorded exception is a defect.

Chain: PR-nnn → SR-nnn → component requirement → TRACES, defined in SPEC.md §6. Registers live in each component's docs/requirements.md; IDs are permanent and never reused, since renumbering is what creates orphan tags.

When adding a requirement, give it a parent and a register row. When adding code, give it a tag.

Commits name their requirements too

Every commit that implements, changes, or withdraws a requirement carries a TRACES: trailer — the same token and syntax as the code tags, so one grep pattern serves both:

feat(audio): v1 content-derived audio signature

Implements the §3 construction with the six previously-undefined
parameters pinned, plus a golden fixture the plugin can be written from.

TRACES: IR-004, IR-005, IR-007, IR-008 | SR-003

This closes the last link in the chain. Code tags say where a requirement lives; commit trailers say when and why it changed — git log --grep=AR-012 then reconstructs a requirement's entire history, which no other artifact gives you.

• Trailer last, after any Co-Authored-By.

• A commit serving no requirement (formatting, tooling, a typo) simply omits it — absence is meaningful, so do not invent a tag to satisfy the form.

• Withdrawing a requirement is a change to it: name it, so the withdrawal is findable later.

• Specs are requirements + current-state deltas. Each requirement carries Current: and Gap: so the document doubles as a work list. Keep that shape when editing.

• State understanding before writing it into a doc. Design here has been settled by discussion, and the corrections have usually been simpler than the proposal. Confirm first; do not write elaborate mechanism into a spec on an unverified assumption.

• Breaking format changes are batched into one schema_version bump and coordinated across all three repos (SR-003).