dtourolleandClaude Opus 5
43368cdcdb
Gives the system specification an owner. It defines the PR-nnn project requirements and SR-nnn cross-component contracts that every component spec traces up to, and until now it lived in no repository at all. The three component repositories are linked from the README and gitignored here rather than added as submodules. A submodule pins a commit, so with feature branches and worktrees in flight across the components, every component commit would leave this repository's pointer stale. The dependency is meant to run the other way: components pull this repository in for the shared tooling and system spec, both of which change rarely. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
106 lines
5.1 KiB
Markdown
106 lines
5.1 KiB
Markdown
# JRay — working notes
|
|
|
|
Three components. Start with the system spec; each repo's spec implements it.
|
|
|
|
| Doc | Owns |
|
|
|---|---|
|
|
| [`SPEC.md`](SPEC.md) | **System** requirements — anything spanning more than one repo |
|
|
| [`scene-actor-extraction/docs/SPEC.md`](scene-actor-extraction/docs/SPEC.md) | Extraction pipeline software requirements |
|
|
| [`scene-actor-extraction/docs/requirements.md`](scene-actor-extraction/docs/requirements.md) | Stable requirement IDs, statuses, verification plan |
|
|
| [`scene-actor-extraction/docs/plan.md`](scene-actor-extraction/docs/plan.md) | Per-requirement implementation notes and dependencies |
|
|
| [`jRay/SPEC.md`](jRay/SPEC.md) | Truth-file format, Jellyfin plugin API |
|
|
| [`JRay-public-server/SPEC.md`](JRay-public-server/SPEC.md) | Jmanifest exchange, cut matching, audio signature |
|
|
|
|
---
|
|
|
|
## Invariants — do not violate without an explicit, recorded agreement
|
|
|
|
### Always use the calibrated probability. Never a raw cosine.
|
|
|
|
**Every similarity is converted through the sigmoid calibration before it is
|
|
used, compared, or thresholded.** This applies everywhere — identity matching,
|
|
tracking association, gallery expansion admission, cluster merging — not just at
|
|
the final identity decision.
|
|
|
|
A raw cosine threshold is an unfalsifiable magic number: it means something
|
|
different for every model, every gallery, and every face size, and it cannot be
|
|
combined with anything else. A calibrated probability means the same thing
|
|
everywhere and composes (see the per-track Bayesian accumulation in A9).
|
|
|
|
**If a raw cosine is ever used, it is an agreed exception and must be recorded as
|
|
such in the spec, with the reason.** Absent that, treat any bare cosine
|
|
comparison in the code as a defect to be fixed. The constants being retired for
|
|
this reason include `track_max_embed_dist`, `cut_revive_sim`,
|
|
`expand_novelty_sim` and `expand_track_spread_max`.
|
|
|
|
**Agreed exceptions so far** (do not "fix" these):
|
|
|
|
| Where | Why |
|
|
|---|---|
|
|
| `GR-008` — distributional outlier check on an actor's own references | The sigmoid is a monotonic squash: right for *decisions*, wrong for characterising a *distribution*, since it compresses exactly the tails where outliers live. Shape is a property of the metric space. Scope is analysis only — every match decision still goes through the calibration. |
|
|
|
|
### Every model gets the input it was trained for
|
|
|
|
Cost is reduced by running a model **less often** or on **fewer regions** — never
|
|
by degrading what a single inference sees. A model run off-distribution returns
|
|
confident, plausible, wrong output, and the error is invisible without a study
|
|
that should not have been needed. (See A5: TransNetV2 must be fed at native frame
|
|
rate, not a cheaper 12 fps.)
|
|
|
|
### Presence is scene-scoped, not instantaneous
|
|
|
|
The question is *"is this actor in this scene?"*, not *"is their face visible in
|
|
this frame?"* — because that is what the X-Ray ground truth records. An actor who
|
|
turns away, is occluded, or is off-camera while the shot cuts to whoever they are
|
|
speaking to **is still present**. Reporting them absent is not a stricter answer;
|
|
it is an answer to a different question. (SR-002.)
|
|
|
|
A window ends at the **last sighting**, never after it. Gaps *between* sightings
|
|
are absorbed; time *after* the final one never is.
|
|
|
|
### The public server stores no binary content
|
|
|
|
Its abuse defence is structural — no images, no base64, no opaque blobs, no
|
|
extension points. This is what makes it safe for a volunteer to operate. Any
|
|
proposal to ship images or embeddings through it is a proposal to delete that
|
|
property, not an incremental feature. (SR-004.)
|
|
|
|
---
|
|
|
|
## Conventions
|
|
|
|
### Tag code with the requirement it implements
|
|
|
|
House format, matching [`JellyTau`](../JellyTau/docs/traces-quick-ref.md) so the
|
|
tooling ports directly. Pipe separates requirement types, comma separates IDs
|
|
within a type:
|
|
|
|
```cpp
|
|
/// TRACES: AR-012, AR-013 | SR-002
|
|
struct TrackRegistry { … };
|
|
```
|
|
|
|
- Tag the unit that **decides**, not every helper it calls.
|
|
- **Tests carry TRACES too** (`UT-nnn`, `IT-nnn`) — that is what shows a
|
|
requirement is *verified*, not merely implemented.
|
|
- **A deliberate raw-cosine use is tagged `EXCEPTION: AR-024 <reason>`.** Per the
|
|
invariant above, a bare cosine with no recorded exception is a defect.
|
|
|
|
Chain: `PR-nnn → SR-nnn → component requirement → TRACES`, defined in
|
|
[`SPEC.md`](SPEC.md) §6. Registers live in each component's
|
|
`docs/requirements.md`; **IDs are permanent and never reused**, since renumbering
|
|
is what creates orphan tags.
|
|
|
|
When adding a requirement, give it a parent and a register row. When adding code,
|
|
give it a tag.
|
|
|
|
- **Specs are requirements + current-state deltas.** Each requirement carries
|
|
`Current:` and `Gap:` so the document doubles as a work list. Keep that shape
|
|
when editing.
|
|
- **State understanding before writing it into a doc.** Design here has been
|
|
settled by discussion, and the corrections have usually been *simpler* than the
|
|
proposal. Confirm first; do not write elaborate mechanism into a spec on an
|
|
unverified assumption.
|
|
- Breaking format changes are **batched into one `schema_version` bump** and
|
|
coordinated across all three repos (SR-003).
|