Compare commits

...
Author SHA1 Message Date
dtourolle eed49e43c1 docs: record the second AR-024 exception, and that the invariant is now checked
The exceptions table gains the calibration's own near-duplicate dedup.
It is not a close call in either direction: at 1 - 1e-7 it tests vector
identity rather than similarity, and it runs on the fit's input, so a
calibrated comparison there would have to be calibrated by the fit it is
feeding.

More importantly, the invariant now has the enforcement its register row
always claimed. check_raw_cosine.py blocks in CI, and it caught a live
violation on its first run -- the identity matcher's no-calibration
fallback, which thresholded raw cosine distance and then fed
max(0, cosine) into the Bayesian accumulation as a posterior, past a
contract that says in terms it cannot be handed an uncalibrated number.

The note is explicit about what a pass does not prove: the check cannot
follow a cosine through a variable across statements, and says nothing
about the GEMM similarity matrix. Both remain conventions backed by
review. Writing that down is the point -- a checker trusted for more
than it does is how the raw-cosine fallback survived being read past.

TRACES: AR-024 | SR-002
2026-08-05 17:52:34 +02:00
dtourolle e1a9aad83c chore: ignore the benchmark corpus and derived data
hero/ and dvu-hero/ are ~140 MB of film clips, DVU annotations and the
gallery built from them. All reproducible via scripts/fetch_dvu.sh plus the
DVU movie.shots set, and the replay fixtures derived from them ship through
the artifact registry — so none of it belongs in git.
2026-08-04 14:15:09 +02:00
dtourolleandClaude Opus 5 413785f8be docs: PR-005 has software rows now, and the rollup should say so
The matrix section still claimed PR-005 "has no software row at all" and could
be verified only by prohibition. jRay's register has carried four rows against
it since the schema-v2 landing: JR-038 (Done), JR-034 and JR-039 (both High,
both T1, both still Planned), and JR-040 (T4). jRay is the component that
actually performs egress, so that is where the goal became verifiable rather
than merely preserved.

Three of the four are untagged in the matrix. That is unbuilt work, not a
broken chain, and saying so here keeps the rollup from reading as an
inconsistency. The structural guarantees still hold PR-005 from the other
side; nothing about SR-004 or GR-005 changes.

jRay/SPEC.md already stated this in the past tense. The vendored copy under
jRay/scripts/vendor/jray-project is a nested checkout of this repo, so it
follows on the next vendor bump rather than needing its own edit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: JR-034, JR-039 | PR-005
2026-07-31 17:11:54 +02:00
dtourolleandClaude Opus 5 bddaddac5d docs: CC0 for the system spec and shared tooling
The README asserted "GPLv3, matching the Jellyfin plugin it serves" with no
LICENSE file behind it. That answer conflicts with the architecture the same
README describes forty lines earlier, in two ways.

This repository is vendored *into* the other three, which carry three different
licences (MIT, GPL-3.0, GPL-3.0-or-later). Copyleft here pushes obligations
downstream into repositories that did not choose them, for the sake of a build
script. The dependency also runs inward, so "matching the plugin" had the
direction backwards — this repo does not serve the plugin, the plugin consumes
it. CC0 imposes nothing on any of the three.

A specification also has to be freely implementable. The design assumes third
parties reimplement it: the audio-signature conformance fixture exists so that
"an implementation can be written from that file alone", and federation is
worthless if only one server implementation may exist. A software licence on a
specification invites the question of whether an implementation written from it
is a derivative work; CC0 removes the question rather than answering it.

Same reasoning as the CC0 licence on contributed manifests (JRay-public-server
UR-019): where an artefact's whole purpose is to be copied and reimplemented,
asserting rights over it costs more than it protects.

Text from creativecommons.org, not transcribed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:44:14 +02:00
dtourolleandClaude Opus 5 17106f3370 Adopt the config-driven extractor; project overview README
Replaces the copy taken earlier with the newer version from
scene-actor-extraction's traceability-tooling branch, which had moved on: it
takes per-repo settings from a traceability.toml rather than the CLI flags
added here, validates them, and names languages ("rust") rather than making
each repo spell out extensions. That is the better design, so the flags go and
this becomes the single source.

Two fixes on top:

- Config discovery searched from the working directory only, so --root pointed
  at another tree found no traceability.toml and failed with
  "requirement_types is empty" while a perfectly good config sat in the
  directory named. That breaks both intended callers: CI passing --root, and a
  wrapper running the vendored copy. Discovery now starts from --root.
- The test suite had not been migrated with the Config refactor and failed on
  the branch as well as here. All 53 now pass: entry points take a Config,
  ci_executable moved to the Register which owns tier policy, fixtures write a
  real traceability.toml so config discovery is exercised rather than bypassed,
  and the live-register tests take LIVE_REGISTER from the environment since the
  project home holds no component register of its own.

The README becomes a project overview rather than a table of contents: what the
problem is, why a paused-frame answer is the wrong question, why gallery data
never leaves the instance, and why the manifest server can hold no binary.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 18:38:59 +02:00
dtourolleandClaude Opus 5 fec03099bc docs: commit trailers close the traceability chain
Commits that implement, change, or withdraw a requirement carry a TRACES
trailer using the same token and syntax as the code tags, so one grep pattern
serves both.

This is the last link. Code tags say where a requirement lives; commit trailers
say when and why it changed, and `git log --grep=AR-012` then reconstructs a
requirement's whole history — which no other artifact provides.

A commit serving no requirement omits the trailer: absence is meaningful, and
inventing a tag to satisfy the form is how orphan tags get created. Withdrawing
a requirement counts as changing it, so the withdrawal stays findable.

Also updates the chain diagram, which still referenced the retired @implements
tag and the thematic A1..E8 IDs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 18:36:16 +02:00
9 changed files with 1139 additions and 710 deletions
@@ -1,107 +0,0 @@
{
"permissions": {
"allow": [
"Bash(python3 -c ' *)",
"WebFetch(domain:docs.turso.tech)",
"Bash(awk '/^```/{n++} END{print \"fences:\",n,\\(n%2==0?\"balanced\":\"UNBALANCED\"\\)}' SPEC.md)",
"Bash(python3 -c \"import h5py; print\\('h5py', h5py.__version__\\)\")",
"Bash(python3 scripts/make_jellyfin_gallery.py --help)",
"Bash(timeout 30 python3 -c ' *)",
"Bash(ps -o pid,etime,cmd -C cmake)",
"Bash(cargo check *)",
"Bash(cargo test *)",
"Bash(cargo clippy *)",
"Bash(cargo deny *)",
"Bash(rustc --version)",
"Bash(cargo metadata *)",
"Bash(python3 -c \"import json,sys; d=json.load\\(sys.stdin\\); print\\(d['packages'][0].get\\('license'\\)\\)\")",
"Bash(timeout 900 cargo install cargo-deny --locked)",
"Bash(timeout 1500 docker build -t jray-server:test .)",
"Bash(timeout 900 docker build -f /tmp/claude-1000/-home-dtourolle-Development-Jray-project/d861f764-2add-4f42-a069-0954ad1f9565/scratchpad/Dockerfile.probe -t jray-probe .)",
"Bash(docker rm *)",
"Bash(docker volume *)",
"Bash(timeout 900 docker build -t jray-server:test .)",
"Bash(docker run *)",
"Bash(xargs -I{} echo \"clippy: {}\")",
"Bash(timeout 900 docker build -q -t jray-server:test .)",
"Bash(awk '{s+=$4} END {print \"total: \" s \" passing\"}')",
"Bash(xargs -I{} echo \"clippy issues: {}\")",
"Bash(timeout 600 dotnet build Jellyfin.Plugin.JRay/Jellyfin.Plugin.JRay.csproj -v q --nologo)",
"Bash(awk '{s+=$4} END {print \"server tests: \" s \" passing\"}')",
"Bash(timeout 300 dotnet build Jellyfin.Plugin.JRay/Jellyfin.Plugin.JRay.csproj -v q --nologo)",
"Bash(cp SPEC.md /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/SPEC.md.bak)",
"Bash(perl -pi -e ' *)",
"Bash(grep -vE \"AR-|DP-|IR-|GR-|VR-|SR-|PR-|H[0-9]|[0-9]{3,}\")",
"Bash(git update-index *)",
"WebFetch(domain:github.com)",
"Bash(git -C /home/dtourolle/Development/Jray-worktrees/audio-signature status --short)",
"Bash(git -C /home/dtourolle/Development/Jray-worktrees/audio-signature branch --show-current)",
"Bash(python3 -c \"import pytest; print\\(pytest.__version__\\)\")",
"Bash(python3 -c \"import numpy; print\\('numpy', numpy.__version__\\)\")",
"Read(//usr/lib/cmake/**)",
"Read(//usr/include/**)",
"Bash(grep -n \"^#\\\\{1,3\\\\} \" JRay-public-server/SPEC.md)",
"Bash(grep -n \"^#\\\\{1,3\\\\} \" scene-actor-extraction/docs/SPEC.md)",
"Bash(cargo build *)",
"Bash(python3 gen.py tone.wav)",
"Bash(python3 ref.py tone.wav)",
"Bash(python3 -c \"import json;d=json.load\\(open\\('package.json'\\)\\);print\\(json.dumps\\({k:v for k,v in d.get\\('scripts',{}\\).items\\(\\) if 'trace' in k.lower\\(\\)},indent=2\\)\\)\")",
"Bash(python3 scripts/traceability/extract_traces.py --format json)",
"Bash(python3 scripts/test_extract_traces.py)",
"Bash(python3 -m pytest scripts/traceability/test_extract_traces.py -q)",
"Bash(python3 scripts/traceability/extract_traces.py --format coverage)",
"Bash(python3 scripts/extract_traces.py)",
"Bash(python3 scripts/extract_traces.py --format json)",
"Bash(git -C /home/dtourolle/Development/Jray-worktrees/traceability-tooling log --oneline -3)",
"Bash(python3 /home/dtourolle/Development/Jray-worktrees/traceability-tooling/scripts/traceability/extract_traces.py --root /home/dtourolle/Development/Jray-project/JRay-public-server --requirements /home/dtourolle/Development/Jray-project/JRay-public-server/docs/requirements.md --system-spec /home/dtourolle/Development/Jray-project/SPEC.md --format coverage)",
"Bash(python3 scripts/traceability/test_extract_traces.py)",
"Bash(sh scripts/traceability/traceability-gate.sh)",
"Bash(python3 -c \"import pyflakes; print\\('pyflakes', pyflakes.__version__\\)\")",
"Bash(timeout 300 dotnet build -v q --nologo)",
"Bash(git check-ignore *)",
"Bash(rm -rf /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/build-full)",
"Bash(cmake -S . -B /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/build-full -DSAE_BUILD_TESTS=ON)",
"Bash(pkg-config --cflags opencv4)",
"Bash(pkg-config --cflags opencv)",
"Bash(awk '{s+=$4} END {print \"tests: \" s}')",
"Bash(python3 -m pyflakes scripts/validation/min_face_size.py)",
"Bash(python3 -m flake8 --select=F scripts/validation/min_face_size.py)",
"Bash(git -c user.name=\"Duncan Tourolle\" -c user.email=\"duncan@tourolle.paris\" commit -q -F -)",
"Bash(cmake -S . -B /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/build-full -DSAE_BUILD_TESTS=ON -DSAE_GEMM_BACKEND=CPU)",
"Bash(python3 -c \"import h5py, numpy, requests, PIL; print\\('deps ok'\\)\")",
"Bash(python3 -m py_compile scripts/validation/min_face_size.py)",
"Bash(git remote *)",
"Bash(python3 et.py --root /home/dtourolle/Development/Jray-project/jRay --requirements /home/dtourolle/Development/Jray-project/jRay/docs/requirements.md --system-spec /home/dtourolle/Development/Jray-project/SPEC.md --format coverage)",
"Bash(git -C /home/dtourolle/Development/Jray-worktrees/traceability-tooling status --short)",
"Bash(git -C /home/dtourolle/Development/Jray-worktrees/traceability-tooling log --oneline -2 -- scripts/traceability/)",
"Bash(timeout 240 python3 scripts/stamp_gallery.py --gallery /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/g_plain.h5 --show)",
"Bash(cp /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/g_plain.h5 /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/g_mig.h5)",
"Bash(python3 scripts/stamp_gallery.py --gallery /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/g_mig.h5 --arcface /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/fake.onnx)",
"Bash(timeout 590 cmake -S . -B /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/bt -DSAE_BUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Release)",
"Bash(grep -n \"^#\\\\{1,3\\\\} \\\\|^| ID \\\\|^| Requirement \\\\|^|---\" jRay/docs/requirements.md)",
"Bash(./build-full/tests/sae_tests)",
"Bash(ls /usr/include/catch2/catch_test_macros.hpp 2>/dev/null && echo \"catch2 system\"; ls /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/build-full/_deps/ 2>/dev/null | head; ls /usr/lib/libCatch2* 2>/dev/null | head)",
"Read(//usr/lib/**)",
"Bash(find /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/build-full/_deps/catch2-build -name libCatch2*)",
"Bash(timeout 590 g++ -std=c++20 -O1 -I/home/dtourolle/Development/Jray-worktrees/gallery-model-binding/src -I/usr/include/opencv5 -I/tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/build-full/_deps/nlohmann_json-src/single_include -I/tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/build-full/_deps/catch2-src/src -I/tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/build-full/_deps/catch2-build/generated-includes '-DSAE_MODELS_DIR=\"/home/dtourolle/Development/Jray-worktrees/gallery-model-binding/models\"' -o gallery_tests /home/dtourolle/Development/Jray-worktrees/gallery-model-binding/tests/test_gallery_store.cpp /home/dtourolle/Development/Jray-worktrees/gallery-model-binding/src/gallery/gallery_store.cpp /home/dtourolle/Development/Jray-worktrees/gallery-model-binding/src/gallery/embedder_stamp.cpp /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/build-full/_deps/catch2-build/src/libCatch2Main.a /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/build-full/_deps/catch2-build/src/libCatch2.a -lhdf5_cpp -lhdf5 -lopencv_core)",
"Bash(./gallery_tests)",
"Bash(grep -viE \"^\\\\[gallery\\\\]|^ |^$|WARNING \\\\\\(GR-004\\\\\\)|\\\\*\\\\*\\\\*\\\\*\")",
"Bash(python3 scripts/traceability/extract_traces.py --root jRay --requirements jRay/docs/requirements.md --system-spec SPEC.md --types JR --suffixes .cs,.js --scan-roots Jellyfin.Plugin.JRay --format coverage)",
"Bash(python3 scripts/traceability/extract_traces.py --root /home/dtourolle/Development/Jray-project/jRay --requirements /home/dtourolle/Development/Jray-project/jRay/docs/requirements.md --system-spec /home/dtourolle/Development/Jray-project/SPEC.md --types JR --suffixes .cs,.js --scan-roots Jellyfin.Plugin.JRay --format coverage)",
"Bash(cp -r /home/dtourolle/Development/Jray-project/scene-actor-extraction/external/KPN/. external/KPN/)",
"Bash(rm -rf external/KPN/.git)",
"Bash(rm -rf /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/bt)",
"Bash(timeout 590 cmake -S . -B /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/bt -DSAE_BUILD_TESTS=ON -DSAE_GEMM_BACKEND=CPU -DCMAKE_BUILD_TYPE=Release)",
"Bash(python3 scripts/traceability/extract_traces.py --root scene-actor-extraction --requirements scene-actor-extraction/docs/requirements.md --format coverage)",
"Bash(timeout 590 cmake --build /tmp/claude-1000/-home-dtourolle-Development-Jray-project/de7e497c-da04-462d-a971-d0fcd36bda2a/scratchpad/bt --target sae_tests -j8)",
"Bash(sh -n scripts/traceability/traceability-gate.sh)",
"Bash(cd /home/dtourolle/Development/Jray-project *)"
],
"additionalDirectories": [
"/home/dtourolle/Development/Jray-worktrees/gallery-model-binding/src/gallery",
"/home/dtourolle/Development/Jray-worktrees/min-face-study/scripts/validation",
"/home/dtourolle/Development/Jray-worktrees/audio-signature/tests",
"/home/dtourolle/Development/Jray-worktrees/traceability-tooling/scripts/traceability"
]
}
}
+12
View File
@@ -18,3 +18,15 @@ traces-report.json
*.swp
*~
.DS_Store
# Benchmark corpus and derived data — distributed as artifacts, never in git.
# hero/ is ~130 MB of film clips and dvu-hero/ the DVU annotations and gallery
# built from them. Both are reproducible: scripts/fetch_dvu.sh pulls the
# annotations and mugshots, and the clips come from the DVU movie.shots set.
# Replay fixtures derived from them ship via
# scene-actor-extraction/scripts/artifacts/push_artifacts.sh replay-fixtures
hero/
dvu-hero/
# Local agent session state
.claude/
+38
View File
@@ -38,6 +38,18 @@ this reason include `track_max_embed_dist`, `cut_revive_sim`,
| Where | Why |
|---|---|
| `GR-008` — distributional outlier check on an actor's own references | The sigmoid is a monotonic squash: right for *decisions*, wrong for characterising a *distribution*, since it compresses exactly the tails where outliers live. Shape is a property of the metric space. Scope is analysis only — every match decision still goes through the calibration. |
| `calibrate_gallery` — per-actor near-duplicate dedup at `1 - 1e-7` | Two reasons, either sufficient. It asks whether two vectors are **the same vector** — at that threshold it catches one source image embedded twice, and no pair of distinct photographs lands there — so it is not a decision and there is nothing for a probability to mean. And structurally, it runs on the calibration fit's *input*: a calibrated comparison there would have to be calibrated by the fit it is feeding. |
**The invariant is now enforced, not just asserted.**
`scene-actor-extraction/scripts/ci/check_raw_cosine.py` runs in the traceability
workflow and fails the build on a bare cosine with no recorded exception. It
found one immediately — the identity matcher's no-calibration fallback, which
thresholded raw cosine distance *and* fed `max(0, cosine)` into the Bayesian
accumulation as a posterior.
Know what it does not cover: it cannot follow a cosine through a variable across
statements, and it says nothing about the GEMM similarity matrix. Those remain
conventions backed by review. A pass is evidence, not proof.
### Every model gets the input it was trained for
@@ -94,6 +106,32 @@ is what creates orphan tags.
When adding a requirement, give it a parent and a register row. When adding code,
give it a tag.
### Commits name their requirements too
Every commit that implements, changes, or withdraws a requirement carries a
`TRACES:` trailer — the same token and syntax as the code tags, so one grep
pattern serves both:
```
feat(audio): v1 content-derived audio signature
Implements the §3 construction with the six previously-undefined
parameters pinned, plus a golden fixture the plugin can be written from.
TRACES: IR-004, IR-005, IR-007, IR-008 | SR-003
```
This closes the last link in the chain. Code tags say *where* a requirement
lives; commit trailers say *when and why it changed* — `git log --grep=AR-012`
then reconstructs a requirement's entire history, which no other artifact gives
you.
- Trailer last, after any `Co-Authored-By`.
- A commit serving no requirement (formatting, tooling, a typo) simply omits it —
absence is meaningful, so do not invent a tag to satisfy the form.
- Withdrawing a requirement is a change to it: name it, so the withdrawal is
findable later.
- **Specs are requirements + current-state deltas.** Each requirement carries
`Current:` and `Gap:` so the document doubles as a work list. Keep that shape
when editing.
+121
View File
@@ -0,0 +1,121 @@
Creative Commons Legal Code
CC0 1.0 Universal
CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE
LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN
ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS
INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES
REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS
PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM
THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED
HEREUNDER.
Statement of Purpose
The laws of most jurisdictions throughout the world automatically confer
exclusive Copyright and Related Rights (defined below) upon the creator
and subsequent owner(s) (each and all, an "owner") of an original work of
authorship and/or a database (each, a "Work").
Certain owners wish to permanently relinquish those rights to a Work for
the purpose of contributing to a commons of creative, cultural and
scientific works ("Commons") that the public can reliably and without fear
of later claims of infringement build upon, modify, incorporate in other
works, reuse and redistribute as freely as possible in any form whatsoever
and for any purposes, including without limitation commercial purposes.
These owners may contribute to the Commons to promote the ideal of a free
culture and the further production of creative, cultural and scientific
works, or to gain reputation or greater distribution for their Work in
part through the use and efforts of others.
For these and/or other purposes and motivations, and without any
expectation of additional consideration or compensation, the person
associating CC0 with a Work (the "Affirmer"), to the extent that he or she
is an owner of Copyright and Related Rights in the Work, voluntarily
elects to apply CC0 to the Work and publicly distribute the Work under its
terms, with knowledge of his or her Copyright and Related Rights in the
Work and the meaning and intended legal effect of CC0 on those rights.
1. Copyright and Related Rights. A Work made available under CC0 may be
protected by copyright and related or neighboring rights ("Copyright and
Related Rights"). Copyright and Related Rights include, but are not
limited to, the following:
i. the right to reproduce, adapt, distribute, perform, display,
communicate, and translate a Work;
ii. moral rights retained by the original author(s) and/or performer(s);
iii. publicity and privacy rights pertaining to a person's image or
likeness depicted in a Work;
iv. rights protecting against unfair competition in regards to a Work,
subject to the limitations in paragraph 4(a), below;
v. rights protecting the extraction, dissemination, use and reuse of data
in a Work;
vi. database rights (such as those arising under Directive 96/9/EC of the
European Parliament and of the Council of 11 March 1996 on the legal
protection of databases, and under any national implementation
thereof, including any amended or successor version of such
directive); and
vii. other similar, equivalent or corresponding rights throughout the
world based on applicable law or treaty, and any national
implementations thereof.
2. Waiver. To the greatest extent permitted by, but not in contravention
of, applicable law, Affirmer hereby overtly, fully, permanently,
irrevocably and unconditionally waives, abandons, and surrenders all of
Affirmer's Copyright and Related Rights and associated claims and causes
of action, whether now known or unknown (including existing as well as
future claims and causes of action), in the Work (i) in all territories
worldwide, (ii) for the maximum duration provided by applicable law or
treaty (including future time extensions), (iii) in any current or future
medium and for any number of copies, and (iv) for any purpose whatsoever,
including without limitation commercial, advertising or promotional
purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each
member of the public at large and to the detriment of Affirmer's heirs and
successors, fully intending that such Waiver shall not be subject to
revocation, rescission, cancellation, termination, or any other legal or
equitable action to disrupt the quiet enjoyment of the Work by the public
as contemplated by Affirmer's express Statement of Purpose.
3. Public License Fallback. Should any part of the Waiver for any reason
be judged legally invalid or ineffective under applicable law, then the
Waiver shall be preserved to the maximum extent permitted taking into
account Affirmer's express Statement of Purpose. In addition, to the
extent the Waiver is so judged Affirmer hereby grants to each affected
person a royalty-free, non transferable, non sublicensable, non exclusive,
irrevocable and unconditional license to exercise Affirmer's Copyright and
Related Rights in the Work (i) in all territories worldwide, (ii) for the
maximum duration provided by applicable law or treaty (including future
time extensions), (iii) in any current or future medium and for any number
of copies, and (iv) for any purpose whatsoever, including without
limitation commercial, advertising or promotional purposes (the
"License"). The License shall be deemed effective as of the date CC0 was
applied by Affirmer to the Work. Should any part of the License for any
reason be judged legally invalid or ineffective under applicable law, such
partial invalidity or ineffectiveness shall not invalidate the remainder
of the License, and in such case Affirmer hereby affirms that he or she
will not (i) exercise any of his or her remaining Copyright and Related
Rights in the Work or (ii) assert any associated claims and causes of
action with respect to the Work, in either case contrary to Affirmer's
express Statement of Purpose.
4. Limitations and Disclaimers.
a. No trademark or patent rights held by Affirmer are waived, abandoned,
surrendered, licensed or otherwise affected by this document.
b. Affirmer offers the Work as-is and makes no representations or
warranties of any kind concerning the Work, express, implied,
statutory or otherwise, including without limitation warranties of
title, merchantability, fitness for a particular purpose, non
infringement, or the absence of latent or other defects, accuracy, or
the present or absence of errors, whether or not discoverable, all to
the greatest extent permissible under applicable law.
c. Affirmer disclaims responsibility for clearing rights of other persons
that may apply to the Work or any use thereof, including without
limitation any person's Copyright and Related Rights in the Work.
Further, Affirmer disclaims responsibility for obtaining any necessary
consents, permissions or other rights required for any use of the
Work.
d. Affirmer understands and acknowledges that Creative Commons is not a
party to this document and has no duty or obligation with respect to
this CC0 or use of the Work.
+141 -43
View File
@@ -4,23 +4,36 @@
see who is on screen — for their own library, on their own hardware, with nobody
else learning what they own.
This is the project home. It owns the [system specification](SPEC.md) — the
requirements that span more than one component — and the tooling shared between
them. The code lives in three separate repositories, linked below.
This is the project home. It owns the [system specification](SPEC.md), the
requirements that span more than one component, and the tooling they share. The
code lives in three repositories, linked below.
---
## The three components
## The problem
| Repository | Role | Language |
|---|---|---|
| [`scene-actor-extraction`](https://gitea.tourolle.paris/dtourolle/scene-actor-extraction) | Derives presence data from a media file | C++ / Python |
| [`jRay`](https://gitea.tourolle.paris/dtourolle/jRay) | Jellyfin plugin: surfaces it in the player, owns the truth-file format | C# |
| [`JRay-public-server`](https://gitea.tourolle.paris/dtourolle/JRay-public-server) | Exchanges presence data between instances | Rust |
You are watching a film. Someone appears and you know you have seen them
before — but pausing to search breaks the film, and the answer is rarely worth
the interruption. Amazon X-Ray solves this well, and only for Amazon's catalogue.
They ship independently — the plugin to Jellyfin's catalogue, the server as a
binary, the pipeline to a GPU host — which is why they are separate repositories
rather than one monorepo.
Doing the same for a personal library is harder than it looks:
- **Face recognition on a paused frame answers the wrong question.** In dialogue
the camera is usually on whoever is *not* speaking, so a per-frame answer
reports the other actor absent. X-Ray credits a whole scene's cast for the
scene's duration, and that is the more useful question (SR-002).
- **The compute is real.** Detecting, embedding and tracking every face in a
feature takes GPU time. Doing it once per viewer, for the same film, is waste.
- **The obvious fix leaks.** A service that identifies your library must be told
what your library contains, which recreates the thing being replaced (PR-005).
JRay's answer: extract presence data locally, share the *timings* rather than
the media or the faces, and make the shared artefact structurally incapable of
carrying anything else.
---
## How it fits together
```
media file ──► extraction ──► truth file (sidecar or pushed) ──► plugin ──► player overlay
@@ -30,9 +43,28 @@ media file ──► extraction ──► truth file (sidecar or pushed) ──
(Jellyfin + TMDB)
```
Two axes, deliberately separate: **presence data** flows outward and is
shareable, being timings against public identifiers. **Gallery data** — the
actor reference faces — is built locally and never leaves the instance (SR-005).
| Repository | Role | Language |
|---|---|---|
| [`scene-actor-extraction`](https://gitea.tourolle.paris/dtourolle/scene-actor-extraction) | Derives presence data from a media file | C++ / Python |
| [`jRay`](https://gitea.tourolle.paris/dtourolle/jRay) | Jellyfin plugin: surfaces it in the player, owns the truth-file format | C# |
| [`JRay-public-server`](https://gitea.tourolle.paris/dtourolle/JRay-public-server) | Exchanges presence data between instances | Rust |
They ship independently — the plugin to Jellyfin's catalogue, the server as a
static binary, the pipeline to a GPU host — which is why they are separate
repositories rather than one monorepo.
**Two data axes, deliberately separate.** *Presence data* flows outward and is
shareable: it is timings against public TMDB identifiers. *Gallery data* — the
actor reference faces — is built locally from your own Jellyfin instance plus
TMDB, and never leaves the machine (SR-005). Not by export, not by opt-in, not
at all: the capability is what creates the exposure, so it does not exist.
**The manifest server holds no binary content.** An accepted manifest contains
bounded numbers, regex-constrained identifiers, and references to TMDB persons.
No images, no embeddings, no free-form strings, no extension points (SR-004).
That is what makes it safe for a volunteer to operate an instance, and it is a
property preserved by prohibition — any proposal to ship blobs through it is a
proposal to delete it.
---
@@ -49,21 +81,20 @@ git clone git@gitea.tourolle.paris:dtourolle/jRay.git
git clone git@gitea.tourolle.paris:dtourolle/JRay-public-server.git
```
The component directories are `.gitignore`d here, so they sit beside the system
spec without this repository trying to track them. Each has its own README with
build instructions.
Each component has its own README with build instructions. They are
`.gitignore`d here, so they sit beside the system spec without this repository
trying to track them.
### Why not submodules for the components
### Why the components are not submodules
A submodule pins a commit. With feature branches and worktrees in flight across
the components, every component commit would leave this repository's pointer
stale and its `git status` dirty until someone committed a pointer bump — churn
that buys nothing, since the components are developed together in one directory
anyway.
stale and its `git status` dirty until someone committed a bump — churn that
buys nothing, since the components are developed together in one directory.
The dependency runs the other way instead: **components pull *this* repository
in** for the shared tooling and the system spec, both of which change rarely.
That is the asymmetry submodules suit.
The dependency runs the other way: **each component pulls *this* repository in**
as a submodule, for the system spec and the shared tooling, both of which change
rarely. That is the asymmetry submodules suit.
---
@@ -72,22 +103,19 @@ That is the asymmetry submodules suit.
| Doc | Owns |
|---|---|
| [`SPEC.md`](SPEC.md) | **System requirements** — `PR-nnn` project goals, `SR-nnn` cross-component contracts |
| [`CLAUDE.md`](CLAUDE.md) | Working notes and the invariants that must not be violated silently |
| [`CLAUDE.md`](CLAUDE.md) | Working notes, and the invariants that must not be violated silently |
| Each repo's `SPEC.md` | That component's software requirements |
| Each repo's `docs/requirements.md` | Its stable requirement IDs, status, and verification plan |
Read the system spec first. Every component requirement traces up to an
`SR-nnn`, and every `SR-nnn` to a `PR-nnn`, so the chain explains *why* a given
piece of code exists.
piece of code exists — and makes it visible when something exists for no stated
reason.
---
## Requirement traceability
Requirements are traceable **up** to the project goal and **down** to the code
implementing them. A requirement nothing traces to is either unnecessary or
unimplemented, and both are worth knowing.
```
PR-nnn project requirement (SPEC.md §1) — why the system exists
└─ SR-nnn system requirement (SPEC.md §3) — what spans components
@@ -95,29 +123,99 @@ PR-nnn project requirement (SPEC.md §1) — why the system exists
└─ TRACES tag (source)
```
Tag the code that *satisfies* a requirement:
Tag the code that *satisfies* a requirement — the unit that decides, not every
helper it calls:
```rust
/// TRACES: UR-003 | SR-004
/// TRACES: UR-003, UR-011 | SR-004
pub fn validate_manifest(m: Jmanifest) -> VResult<ValidManifest> { … }
```
A pipe separates requirement *types*; a comma separates IDs within a type. Tag
the unit that decides, not every helper it calls — a tag on every function is
noise, and rots faster than it helps.
A pipe separates requirement *types*; a comma separates IDs within a type.
Tests carry tags too (`UT-nnn`, `IT-nnn`), which is what shows a requirement is
*verified* rather than merely implemented. A deliberate departure from an
invariant is tagged `EXCEPTION:` with its reason — an untagged one is a defect.
The gate reports coverage, orphan tags (an ID no register defines), and untraced
requirements. Two rules it inherits from JellyTau, both learned the hard way:
### Running the gate
The tooling lives in [`scripts/traceability/`](scripts/traceability/) here and
is vendored into each component as a submodule, so there is **one
implementation**. Each component declares its own taxonomy in a
`traceability.toml` at its root:
```toml
requirement_types = ["UR", "DR"]
languages = ["rust"]
source_roots = ["src", "tests"]
```
```sh
scripts/traceability-gate.sh # from any component
```
It reports coverage, orphan tags (an ID no register defines), untraced
requirements, and requirements verifiable only on hardware CI lacks.
**Two rules inherited from JellyTau, both learned the hard way:**
- **Denominators are read from the register at run time, never hardcoded.** A
gate that divides by a frozen literal reported 158% coverage for months and so
could never fail — worse than no gate, because it was trusted.
gate that divided by a frozen literal reported *158% coverage* for months
while the requirement count grew, so its threshold could never trip. A gate
that cannot fail is worse than no gate, because it is trusted.
- **Coverage above 100% is a hard failure**, not a pass. It means the
computation is broken, and it is the signal that catches the above
immediately.
computation is broken, and it is the signal that catches the above at once.
The same reasoning is why a requirement whose only evidence is a test that never
runs is reported as *tagged but unexecuted*, never counted as covered.
---
## Status
| Component | State |
|---|---|
| `scene-actor-extraction` | Pipeline redesign in progress — presence follows track extent (AR-012), replacing per-frame recognition |
| `jRay` | Truth-file serving and overlay working; manifest-sharing configuration added, fetch path outstanding |
| `JRay-public-server` | Core implemented: 23/32 requirements traced, 189 tests. Audio-tier matching and federation deferred by design |
**One schema bump is pending across all three repos** (SR-003). It removes
`anneal_sec`, adds `extinction_sec` and `gallery_scope`, gives each window its
belief and identification route, and adds the audio signature. Breaking changes
are batched, so these ship together rather than piecemeal.
---
## Licence
GPLv3, matching the Jellyfin plugin it serves.
**This repository — the system spec, the working notes, and the traceability
tooling — is [CC0 1.0](LICENSE).** Public domain dedication: no attribution
required, no conditions.
That is deliberate, and it is not the licence the components use:
| Repository | Licence |
|---|---|
| **This one** (specs + shared tooling) | **CC0 1.0** |
| `scene-actor-extraction` | MIT, with a model/third-party addendum |
| `jRay` | GPL-3.0 |
| `JRay-public-server` | GPL-3.0-or-later (code) · CC0 1.0 (contributed manifests) |
Two reasons, both structural rather than philosophical:
- **This repository is vendored *into* the other three**, as described above —
and they carry three different licences. CC0 is the only choice that imposes
nothing on any of them: no attribution to propagate, no copyleft reaching into
an MIT repository, no aggregation question to answer. A copyleft licence here
would push obligations downstream into repositories that did not choose them,
for the sake of a build script.
- **A specification has to be freely implementable.** The design assumes third
parties reimplement it — the audio-signature conformance fixture exists
precisely so that "an implementation can be written from that file alone", and
federation is worthless if only one server implementation may exist. Licensing
the spec under a software licence invites the question of whether an
implementation written from it is a derivative work. CC0 removes the question
rather than answering it.
The same reasoning produced the CC0 licence on contributed manifests
(`JRay-public-server` SPEC §5b): where the artefact's whole purpose is to be
copied and reimplemented, asserting rights over it costs more than it protects.
+20 -10
View File
@@ -334,12 +334,18 @@ unimplemented, and both are worth knowing.
### The chain
```
PR-n project requirement (this document, §1) — why the system exists
└─ SR-n system requirement (this document, §3) — what spans components
└─ A1…E8 / §n software requirement (component specs) — what one repo does
└─ @implements tag (source) — where it actually is
PR-nnn project requirement (this document, §1) — why the system exists
└─ SR-nnn system requirement (this document, §3) — what spans components
└─ component requirement (component registers) — what one repo does
├─ TRACES tag (source) — where it lives
└─ TRACES trailer (commit message) — when and why it changed
```
The commit trailer is the last link and the only one that carries *history*:
`git log --grep=AR-012` reconstructs everything that ever happened to a
requirement, which no other artifact provides. Same token and syntax as the code
tag, so one grep pattern serves both.
### ID scheme
Zero-padded three digits throughout. **IDs are permanent**: a withdrawn
@@ -466,9 +472,13 @@ misread as gaps:
- **`DP-*` and `VR-*` trace to no system requirement.** Deployment and parameter
studies are single-repo concerns serving `PR-004` and `PR-002` directly. This
is correct.
- **`PR-005` (leak nothing) has no software row at all.** It is satisfied
*structurally* — `SR-004` (server holds no binary) and `GR-005` (gallery never
leaves the instance) — rather than by any component doing something. It cannot
be verified by pointing at code, and it dies the moment either prohibition is
relaxed. A goal preserved only by prohibitions needs watching precisely because
nothing traces to it.
- **`PR-005` (leak nothing) will look thinly covered, and that is the true
picture.** It had no software row anywhere; jRay is the component that
actually performs egress, so four rows now carry it — `JR-038` (exchange
off by default, Done), `JR-034` (contribution strips `movie` and
`jellyfin_id`, posts only to contribute-enabled servers) and `JR-039`
(batch `exists` capped at 100, sweeps paced), both High and T1, and
`JR-040` (the config page states the per-server exposure, T4). Three are
untagged because they are unbuilt, not because the chain is broken. The
structural guarantees — `SR-004` and `GR-005` — still hold the goal from
the other side, and it still dies the moment either prohibition is relaxed.
File diff suppressed because it is too large Load Diff
+89 -156
View File
@@ -43,6 +43,27 @@ TAG = "TRA" + "CES:"
EXC = "EXCEP" + "TION:"
# --------------------------------------------------------------------------
# Test configuration
#
# Every entry point takes a Config since the tool became shared. These helpers
# keep each test stating only the field it varies.
# --------------------------------------------------------------------------
def _config(root=".", **kw) -> "et.Config":
"""A Config with the extraction repo's shape, overridable per test."""
fields = dict(
requirement_types=("AR", "DP", "IR", "GR", "VR"),
source_suffixes=frozenset(et.LANGUAGE_SUFFIXES["cpp"]
| et.LANGUAGE_SUFFIXES["python"]),
source_roots=("src", "tests", "scripts", "experiments", "eval"),
root=Path(root),
)
fields.update(kw)
return et.Config(**fields)
# --------------------------------------------------------------------------
# Tag parsing
# --------------------------------------------------------------------------
@@ -127,11 +148,11 @@ def test_scans_cpp_and_python_but_not_vendored_or_non_source():
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
_write_tree(root)
files = et.iter_source_files(root)
files = et.iter_source_files(_config(root))
names = sorted(f.name for f in files)
assert names == ["gallery.py", "tracker.hpp"], names
scan = et.scan_files(files, root)
scan = et.scan_files(files, _config(root))
traced = sorted({i for t in scan.traces for i in t.requirements})
assert traced == ["AR-012", "AR-013", "GR-001", "SR-002", "SR-005"]
@@ -141,7 +162,7 @@ def test_context_is_found_below_a_cpp_tag_and_above_a_python_tag():
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
_write_tree(root)
scan = et.scan_files(et.iter_source_files(root), root)
scan = et.scan_files(et.iter_source_files(_config(root)), _config(root))
contexts = {t.file: t.context for t in scan.traces}
assert "TrackRegistry" in contexts["src/tracker.hpp"]
assert "build_gallery" in contexts["scripts/gallery.py"]
@@ -164,7 +185,7 @@ def test_exception_tag_is_captured_with_its_reason():
root = Path(tmp)
(root / "src").mkdir(parents=True)
(root / "src" / "outlier.cpp").write_text(EXCEPTION_SOURCE, encoding="utf-8")
scan = et.scan_files(et.iter_source_files(root), root)
scan = et.scan_files(et.iter_source_files(_config(root)), _config(root))
assert len(scan.exceptions) == 1
exc = scan.exceptions[0]
assert exc.requirement == "AR-024"
@@ -179,11 +200,11 @@ def test_exception_is_never_counted_as_coverage():
root = Path(tmp)
(root / "src").mkdir(parents=True)
(root / "src" / "outlier.cpp").write_text(EXCEPTION_SOURCE, encoding="utf-8")
scan = et.scan_files(et.iter_source_files(root), root)
scan = et.scan_files(et.iter_source_files(_config(root)), _config(root))
assert scan.traces == []
register = et.parse_register(
"| ID | Requirement | Status |\n|---|---|---|\n"
"| AR-024 | Always the calibrated probability | Planned |\n")
"| AR-024 | Always the calibrated probability | Planned |\n", _config())
cov = et.compute_coverage(
[i for t in scan.traces for i in t.requirements], register)
assert cov.covered == []
@@ -197,7 +218,7 @@ def test_exception_without_a_reason_is_reported():
(root / "src").mkdir(parents=True)
(root / "src" / "bare.cpp").write_text(
f"// {EXC} AR-024\n", encoding="utf-8")
scan = et.scan_files(et.iter_source_files(root), root)
scan = et.scan_files(et.iter_source_files(_config(root)), _config(root))
assert len(scan.exceptions) == 1
assert scan.diagnostics.exceptions_without_reason
@@ -210,7 +231,7 @@ def test_mixed_type_group_is_reported():
(root / "src").mkdir(parents=True)
(root / "src" / "a.cpp").write_text(
f"// {TAG} AR-001, SR-002\n", encoding="utf-8")
scan = et.scan_files(et.iter_source_files(root), root)
scan = et.scan_files(et.iter_source_files(_config(root)), _config(root))
assert scan.diagnostics.mixed_type_groups
@@ -225,7 +246,7 @@ def test_counts_a_well_formed_table_row_as_a_defined_requirement():
md = REGISTER_HEADER + (
"| AR-001 | Detect faces in sampled frames | SR-002 | High | Done |\n"
"| AR-002 | Minimum face size 66x66 px | SR-002 | High | Planned |\n")
register = et.parse_register(md)
register = et.parse_register(md, _config())
assert register.count("AR") == 2
assert register.count("GR") == 0
assert register.total == 2
@@ -237,7 +258,7 @@ def test_does_not_count_ids_that_appear_only_in_the_traces_to_column():
md = REGISTER_HEADER + (
"| GR-001 | Build gallery from library cast | SR-001, SR-005 | High | Done |\n"
"| GR-002 | Incremental merge refresh | PR-003 | High | Done |\n")
register = et.parse_register(md)
register = et.parse_register(md, _config())
assert register.count("GR") == 2
assert register.ids == {"GR-001", "GR-002"}
@@ -246,7 +267,7 @@ def test_does_not_count_ids_mentioned_in_prose():
md = ("Some prose explaining that AR-005 relates to GR-001 and VR-003.\n\n"
+ REGISTER_HEADER
+ "| AR-005 | Align to 112x112 | SR-002 | High | Done |\n")
register = et.parse_register(md)
register = et.parse_register(md, _config())
assert register.ids == {"AR-005"}
@@ -260,7 +281,7 @@ def test_does_not_count_the_verification_plan_table_as_definitions():
+ "| ID | Tier | Test asserts | Edge cases to cover |\n|---|---|---|---|\n"
+ "| AR-001 | T3 | Detector returns plausible boxes | smoke only |\n"
+ "| AR-099 | T1 | Something not in the register | - |\n")
register = et.parse_register(md)
register = et.parse_register(md, _config())
assert register.ids == {"AR-001"}
assert register.total == 1
@@ -271,7 +292,7 @@ def test_deduplicates_an_id_listed_in_two_definition_tables():
+ "\n"
+ REGISTER_HEADER
+ "| AR-001 | Detect faces | SR-002 | High | Done |\n")
register = et.parse_register(md)
register = et.parse_register(md, _config())
assert register.total == 1
@@ -281,7 +302,7 @@ def test_withdrawn_requirements_leave_the_denominator():
md = REGISTER_HEADER + (
"| AR-001 | Detect faces | SR-002 | High | Done |\n"
"| AR-002 | Superseded mechanism | SR-002 | High | Withdrawn |\n")
register = et.parse_register(md)
register = et.parse_register(md, _config())
assert register.ids == {"AR-001"}
assert "AR-002" in register.withdrawn
@@ -291,8 +312,8 @@ def test_the_denominator_is_live_adding_a_row_lowers_coverage():
# more requirement defined => a lower percentage, mechanically.
base = REGISTER_HEADER + "| AR-001 | A | SR-002 | High | Done |\n"
grown = base + "| AR-002 | B | SR-002 | High | Planned |\n"
before = et.compute_coverage(["AR-001"], et.parse_register(base))
after = et.compute_coverage(["AR-001"], et.parse_register(grown))
before = et.compute_coverage(["AR-001"], et.parse_register(base, _config()))
after = et.compute_coverage(["AR-001"], et.parse_register(grown, _config()))
assert before.percent == 100.0
assert after.percent == 50.0
assert after.total == 2
@@ -301,7 +322,7 @@ def test_the_denominator_is_live_adding_a_row_lowers_coverage():
def test_register_captures_the_row_fields_not_just_the_id():
md = REGISTER_HEADER + (
"| AR-012 | Presence follows track extent | **SR-002** | High | Planned |\n")
req = et.parse_register(md).requirements["AR-012"]
req = et.parse_register(md, _config()).requirements["AR-012"]
assert req.text == "Presence follows track extent"
assert req.traces_to == "**SR-002**"
assert req.status == "Planned"
@@ -328,7 +349,7 @@ def test_tier_assignment_handles_lists_ranges_and_wildcards():
"| AR-002 … AR-004 | **T2** | replay |\n"
"| AR-006 | T1 + T4 | mixed |\n"
"| VR-* | Out of CI | studies |\n")
register = et.parse_register(md)
register = et.parse_register(md, _config())
assert register.requirements["AR-001"].tiers == {"T3"}
assert register.requirements["AR-003"].tiers == {"T2"}
assert register.requirements["AR-006"].tiers == {"T1", "T4"}
@@ -337,7 +358,7 @@ def test_tier_assignment_handles_lists_ranges_and_wildcards():
def test_a_range_cannot_invent_a_requirement_the_register_lacks():
md = TIER_REGISTER + "\n" + _tier_table("| AR-001 … AR-050 | T2 | wide |\n")
register = et.parse_register(md)
register = et.parse_register(md, _config())
assert register.total == 12
assert "AR-050" not in register.ids
@@ -346,7 +367,7 @@ def test_slash_shorthand_in_the_verification_plan_expands():
md = TIER_REGISTER + "\n" + (
"| ID | Tier | Test asserts | Edge cases |\n|---|---|---|---|\n"
"| AR-009/008 | T2 | Cut shifts weighting | cut with same people |\n")
register = et.parse_register(md)
register = et.parse_register(md, _config())
assert register.requirements["AR-008"].tiers == {"T2"}
assert register.requirements["AR-009"].tiers == {"T2"}
@@ -358,15 +379,15 @@ def test_tiers_from_both_tables_are_unioned_not_overwritten():
md = TIER_REGISTER + "\n" + _tier_table("| AR-006 | T4 | GPU host only |\n") + "\n" + (
"| ID | Tier | Test asserts | Edge cases |\n|---|---|---|---|\n"
"| AR-006 | T1 + T4 | GEMM equals reference loop | small input in CI |\n")
register = et.parse_register(md)
register = et.parse_register(md, _config())
assert register.requirements["AR-006"].tiers == {"T1", "T4"}
assert register.requirements["AR-006"].ci_executable
assert register.is_ci_executable("AR-006")
def test_a_t4_only_requirement_is_not_ci_executable():
md = TIER_REGISTER + "\n" + _tier_table("| AR-007 | **T4** | GPU only |\n")
register = et.parse_register(md)
assert not register.requirements["AR-007"].ci_executable
register = et.parse_register(md, _config())
assert not register.is_ci_executable("AR-007")
assert register.unexecutable_ids() == {"AR-007"}
@@ -379,12 +400,12 @@ def test_a_requirement_tracing_up_to_nothing_is_reported():
"| AR-002 | Parent is a section | §4 | Medium | Planned |\n"
"| AR-003 | Serves nothing stated | - | Low | Planned |\n"
"| AR-004 | Blank cell | | Low | Planned |\n")
register = et.parse_register(md)
register = et.parse_register(md, _config())
assert register.parentless_ids() == {"AR-003", "AR-004"}
def test_a_requirement_with_no_tier_is_unknown_not_unexecutable():
register = et.parse_register(TIER_REGISTER)
register = et.parse_register(TIER_REGISTER, _config())
assert register.tier_unknown_ids() == register.ids
assert register.unexecutable_ids() == set()
@@ -402,7 +423,7 @@ COVERAGE_REGISTER = et.parse_register(
+ "\n"
+ _tier_table("| AR-001, AR-002 | T2 | replay |\n"
"| GR-001 | T1 | bookkeeping |\n"
"| AR-027 | **T4** | GPU host only |\n"))
"| AR-027 | **T4** | GPU host only |\n"), _config())
def test_coverage_is_the_intersection_of_traced_and_defined():
@@ -510,10 +531,30 @@ def _fixture_repo(tmp: str, source: str) -> Path:
(root / "src").mkdir(parents=True, exist_ok=True)
(root / "docs" / "requirements.md").write_text(FIXTURE_REGISTER, encoding="utf-8")
(root / "src" / "pipeline.cpp").write_text(source, encoding="utf-8")
(root / et.CONFIG_FILENAME).write_text(
'requirement_types = ["AR", "DP", "IR", "GR", "VR"]\n'
'languages = ["cpp", "python"]\n'
'source_roots = ["src", "tests", "scripts", "experiments", "eval"]\n',
encoding="utf-8")
return root
def _run_gate(root: Path, *extra: str):
"""Run the CLI against `root`, supplying the per-repo config it now needs.
The tool no longer guesses a taxonomy: a repo declares its prefixes and
languages, and the gate refuses to run without them rather than reporting a
misleading zero. So a gate test must provide them too — written as a real
`traceability.toml`, which exercises the config-discovery path rather than
bypassing it.
"""
config = root / et.CONFIG_FILENAME
if not config.exists():
config.write_text(
'requirement_types = ["AR", "DP", "IR", "GR", "VR"]\n'
'languages = ["cpp", "python"]\n'
'source_roots = ["src", "tests", "scripts", "experiments", "eval"]\n',
encoding="utf-8")
buffer = io.StringIO()
with redirect_stdout(buffer):
code = et.main(["--root", str(root), "--format", "coverage", *extra])
@@ -595,11 +636,11 @@ def test_gate_hard_fails_on_an_impossible_ratio():
# reported as a pass. Forced here by handing the reporter a poisoned value.
with tempfile.TemporaryDirectory() as tmp:
root = _fixture_repo(tmp, f"// {TAG} AR-001\nint main() {{}}\n")
register = et.read_register(root / "docs" / "requirements.md")
scan = et.scan_files(et.iter_source_files(root), root)
report = et.build_report(root, register, scan)
register = et.read_register(_config(requirements_path=str(root / "docs" / "requirements.md")))
scan = et.scan_files(et.iter_source_files(_config(root)), _config(root))
report = et.build_report(_config(root, min_coverage=50.0), register, scan)
report.coverage.percent = 158.0
text, code = et.format_coverage_report(report, 50.0)
text, code = et.format_coverage_report(report)
assert code == 1
assert "exceeds 100%" in text
@@ -610,9 +651,9 @@ def test_the_per_type_breakdown_sums_to_the_headline_figure():
with tempfile.TemporaryDirectory() as tmp:
root = _fixture_repo(
tmp, f"// {TAG} AR-001, AR-027\nint main() {{}}\n")
register = et.read_register(root / "docs" / "requirements.md")
scan = et.scan_files(et.iter_source_files(root), root)
report = et.build_report(root, register, scan)
register = et.read_register(_config(requirements_path=str(root / "docs" / "requirements.md")))
scan = et.scan_files(et.iter_source_files(_config(root)), _config(root))
report = et.build_report(_config(root), register, scan)
stats = et.per_type_stats(report)
assert sum(c for c, _, _ in stats.values()) == len(report.coverage.covered)
assert sum(u for _, u, _ in stats.values()) == len(report.coverage.unexecuted)
@@ -637,7 +678,7 @@ def test_json_and_markdown_outputs_are_written_and_consistent():
assert data["coverage"]["covered"] == 1
assert data["coverage"]["percent"] == round(100 / 3, 1)
assert data["byType"]["SR"] == ["SR-002"]
assert data["gpuOnlyRequirements"] == ["AR-027"]
assert data["unexecutableRequirements"] == ["AR-027"]
assert "AR-001" in md_out.read_text(encoding="utf-8")
@@ -662,144 +703,36 @@ def test_system_spec_parsing_enables_orphan_checks_for_pr_and_sr():
# requirements are added.
# --------------------------------------------------------------------------
#: The live-register tests below ran against this repo's own register when the
#: tool lived inside `scene-actor-extraction`. Now that it is shared, this repo
#: is the project home and holds no component register of its own, so they take
#: a path from the environment and skip when it is absent.
#:
#: LIVE_REGISTER=../scene-actor-extraction/docs/requirements.md \
#: python3 test_extract_traces.py
#:
#: They are kept rather than deleted because they assert something the synthetic
#: fixtures cannot: that a *real* register, with all its formatting accidents,
#: parses at all.
LIVE_REGISTER = os.environ.get("LIVE_REGISTER")
LIVE_REGISTER = os.environ.get('LIVE_REGISTER')
def test_the_live_register_parses_and_assigns_tiers():
if not LIVE_REGISTER:
return # skipped: no component register to point at
register = et.read_register(Path(LIVE_REGISTER))
return # skipped: the project home holds no component register
register = et.read_register(_config(root=str(Path(LIVE_REGISTER).resolve().parent.parent), requirements_path=str(LIVE_REGISTER)))
assert register.total > 0
for req_type in et.LOCAL_TYPES:
assert register.count(req_type) > 0, req_type
assert sum(register.count(t) for t in et.LOCAL_TYPES) == register.total
# A tier that no GPU-less host can run must be visible in the parse, not
# merely stated in prose.
assert register.unexecutable_ids() <= register.ids
# The GPU-less CI host must be visible in the parse, not just in prose.
assert "AR-027" in register.unexecutable_ids()
assert register.requirements["AR-027"].tiers == {"T4"}
assert register.requirements["AR-012"].tiers == {"T2"}
assert register.unexecutable_ids() < register.ids
def test_the_live_register_yields_a_gate_run_that_cannot_exceed_one_hundred():
if not LIVE_REGISTER:
return
register = et.read_register(Path(LIVE_REGISTER))
root = Path(LIVE_REGISTER).resolve().parent.parent
scan = et.scan_files(et.iter_source_files(root), root)
register = et.read_register(_config(root=str(Path(LIVE_REGISTER).resolve().parent.parent), requirements_path=str(LIVE_REGISTER)))
files = et.iter_source_files(et.REPO_ROOT, _config())
scan = et.scan_files(files, et.REPO_ROOT, _config())
cov = et.compute_coverage(
[i for t in scan.traces for i in t.requirements], register)
assert 0.0 <= cov.percent <= 100.0
assert len(cov.covered) <= cov.total
# --------------------------------------------------------------------------
# Per-repo configuration — the flags that make this tool shareable.
#
# Without them a repo whose prefixes or language differ from the defaults gets
# zero requirements and zero files, which the gate correctly refuses to report
# as coverage. These assert the overrides actually take effect.
# --------------------------------------------------------------------------
def _restore_taxonomy(fn):
"""Run `fn` with the module defaults restored afterwards."""
local, suffixes, roots = et.LOCAL_TYPES, set(et.SOURCE_SUFFIXES), et.SCAN_ROOTS
try:
fn()
finally:
et.configure_taxonomy(local)
et.configure_suffixes(suffixes)
et.configure_scan_roots(roots)
def test_types_override_changes_which_prefixes_are_counted():
def body():
register_md = (
"| ID | Requirement | Traces to | Priority | Status |\n"
"|---|---|---|---|---|\n"
"| UR-001 | A user requirement | SR-001 | High | Done |\n"
"| DR-001 | A dev requirement | PR-004 | High | Done |\n"
)
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "requirements.md"
path.write_text(register_md, encoding="utf-8")
# Under the defaults these prefixes are unknown, so nothing counts.
et.configure_taxonomy(("AR", "DP"))
assert et.read_register(path).total == 0
et.configure_taxonomy(("UR", "DR"))
register = et.read_register(path)
assert register.total == 2
assert register.count("UR") == 1
assert register.count("DR") == 1
_restore_taxonomy(body)
def test_suffix_override_changes_which_files_are_scanned():
def body():
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
(root / "src").mkdir()
(root / "src" / "lib.rs").write_text(
"/// TRACES: UR-001 | SR-004\npub fn f() {}\n", encoding="utf-8")
# Default suffixes are C++/Python, so a Rust tree scans as empty —
# the failure mode this override exists to fix.
assert et.iter_source_files(root) == []
et.configure_suffixes([".rs"])
files = et.iter_source_files(root)
assert len(files) == 1
scan = et.scan_files(files, root)
assert [i for t in scan.traces for i in t.requirements] == [
"UR-001", "SR-004"]
_restore_taxonomy(body)
def test_suffix_override_accepts_extensions_with_or_without_a_dot():
def body():
et.configure_suffixes(["rs", ".cs"])
assert et.SOURCE_SUFFIXES == {".rs", ".cs"}
_restore_taxonomy(body)
def test_scan_root_override_limits_the_walk():
def body():
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
# Not "vendor" — that is in EXCLUDED_DIR_NAMES and would be
# filtered whatever the scan roots say, testing the wrong thing.
for d in ("src", "eval"):
(root / d).mkdir()
(root / d / "f.py").write_text("# TRACES: AR-001\n", encoding="utf-8")
et.configure_scan_roots(["src"])
assert len(et.iter_source_files(root)) == 1
et.configure_scan_roots(["src", "eval"])
assert len(et.iter_source_files(root)) == 2
_restore_taxonomy(body)
def test_defaults_are_unchanged_by_the_overrides_existing():
# The extraction pipeline must keep working with no flags at all, or moving
# the tool here would have broken the repo it came from.
assert et.LOCAL_TYPES == ("AR", "DP", "IR", "GR", "VR")
assert ".py" in et.SOURCE_SUFFIXES and ".cpp" in et.SOURCE_SUFFIXES
assert ".rs" not in et.SOURCE_SUFFIXES
# --------------------------------------------------------------------------
def _main() -> int:
+39 -66
View File
@@ -1,89 +1,62 @@
#!/bin/sh
#
# Requirement traceability gate. Run locally exactly as CI runs it:
# Requirement traceability gate. Run locally exactly as CI runs it, from the
# component repo root:
#
# scripts/traceability/traceability-gate.sh
#
# Writes traces-report.json and docs/traceability.md, prints the coverage
# report, and exits non-zero when the gate fails.
# Writes the JSON report and the markdown matrix, prints the coverage report,
# and exits non-zero when the gate fails.
#
# Environment:
# MIN_COVERAGE minimum overall coverage percent (default 0 - see below)
# ALLOW_ORPHANS set to 1 to report orphan tags without failing
# TRACES_JSON JSON report path (default traces-report.json)
# TRACES_MD markdown matrix path (default docs/traceability.md)
# REPO_ROOT repository to scan. Defaults to two levels above this
# script, which is correct when the tooling lives in the repo
# it checks. **When vendored as a submodule that default is
# the submodule itself**, so a consuming repo must set this —
# its wrapper does.
# TYPES comma-separated requirement prefixes (e.g. UR,DR). Defaults
# to the extraction set; a repo whose prefixes differ parses
# to zero requirements without this.
# SUFFIXES comma-separated file extensions (e.g. .rs). Defaults to the
# C++/Python set; a repo whose language differs scans zero
# files without this.
# SCAN_ROOTS comma-separated directories to walk, relative to REPO_ROOT.
# SYSTEM_SPEC optional path to the system SPEC.md, which defines the PR/SR
# IDs; when given, PR/SR orphans are reported too. It lives in
# the project home (jray-project) — when this tooling is
# vendored from there, it is a sibling of this script.
# This script is shared by every JRay component, so it knows nothing about any
# one repo. All repo-specific settings - requirement ID prefixes, source
# suffixes, scan roots, register path, thresholds - live in `traceability.toml`
# at the component repo root. Run
#
# Threshold policy lives here and nowhere else. It is deliberately NOT
# duplicated into the workflow YAML: a threshold written in two places is a
# threshold that will disagree with itself.
# scripts/traceability/extract_traces.py --print-example-config
#
# MIN_COVERAGE defaults to 0 because almost nothing is tagged yet - tags are
# added as the pipeline is built, so a low number today is accurate rather than
# alarming. A zero threshold does NOT mean the gate cannot fail: orphan tags,
# a >100% ratio, a register that parses to nothing, and an empty source scan
# are all hard failures from day one. Raise MIN_COVERAGE as tags land; treat
# every raise as a ratchet, never a reset.
# for the annotated schema. A repo whose config is wrong parses zero
# requirements or scans zero files, and the gate refuses to report rather than
# printing a misleading 0%.
#
# Environment (all optional; each overrides the config file):
# TRACES_CONFIG path to traceability.toml
# TRACES_ROOT repo root (default: nearest dir containing traceability.toml)
# MIN_COVERAGE minimum overall coverage percent
# ALLOW_ORPHANS 1 to report orphan tags without failing
# TRACES_JSON JSON report path
# TRACES_MD markdown matrix path
# SYSTEM_SPEC SPEC.md defining PR/SR; enables PR/SR orphan checking
# PYTHON interpreter (default: python3)
#
# Threshold policy belongs in traceability.toml, not here and not in the
# workflow YAML: a threshold written in two places is a threshold that will
# disagree with itself.
#
# POSIX sh, no bashisms, no jq - the extractor does its own arithmetic and
# printing so CI needs nothing beyond python3.
# printing, so CI needs nothing beyond python3.
set -eu
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
REPO_ROOT="${REPO_ROOT:-$(CDPATH= cd -- "$SCRIPT_DIR/../.." && pwd)}"
MIN_COVERAGE="${MIN_COVERAGE:-0}"
TRACES_JSON="${TRACES_JSON:-$REPO_ROOT/traces-report.json}"
TRACES_MD="${TRACES_MD:-$REPO_ROOT/docs/traceability.md}"
PYTHON="${PYTHON:-python3}"
command -v "$PYTHON" >/dev/null 2>&1 || {
echo "FAILED: $PYTHON not found. The traceability gate needs Python 3.9+" >&2
echo "FAILED: $PYTHON not found. The traceability gate needs Python 3.9+," >&2
echo " or 3.11+ to read traceability.toml." >&2
exit 2
}
set -- \
--root "$REPO_ROOT" \
--requirements "${REQUIREMENTS:-$REPO_ROOT/docs/requirements.md}" \
--format coverage \
--json-out "$TRACES_JSON" \
--markdown-out "$TRACES_MD" \
--min-coverage "$MIN_COVERAGE"
set -- --format coverage
if [ "${ALLOW_ORPHANS:-0}" = "1" ]; then
set -- "$@" --allow-orphans
fi
if [ -n "${SYSTEM_SPEC:-}" ]; then
set -- "$@" --system-spec "$SYSTEM_SPEC"
fi
if [ -n "${TYPES:-}" ]; then
set -- "$@" --types "$TYPES"
fi
if [ -n "${SUFFIXES:-}" ]; then
set -- "$@" --suffixes "$SUFFIXES"
fi
if [ -n "${SCAN_ROOTS:-}" ]; then
set -- "$@" --scan-roots "$SCAN_ROOTS"
fi
# Explicit `if` rather than `[ ... ] && ...`, because a trailing false test in
# an && list exits under `set -e` in some POSIX shells.
if [ -n "${TRACES_CONFIG:-}" ]; then set -- "$@" --config "$TRACES_CONFIG"; fi
if [ -n "${TRACES_ROOT:-}" ]; then set -- "$@" --root "$TRACES_ROOT"; fi
if [ -n "${MIN_COVERAGE:-}" ]; then set -- "$@" --min-coverage "$MIN_COVERAGE"; fi
if [ -n "${TRACES_JSON:-}" ]; then set -- "$@" --json-out "$TRACES_JSON"; fi
if [ -n "${TRACES_MD:-}" ]; then set -- "$@" --markdown-out "$TRACES_MD"; fi
if [ -n "${SYSTEM_SPEC:-}" ]; then set -- "$@" --system-spec "$SYSTEM_SPEC"; fi
if [ "${ALLOW_ORPHANS:-0}" = "1" ]; then set -- "$@" --allow-orphans; fi
exec "$PYTHON" "$SCRIPT_DIR/extract_traces.py" "$@"