refactor(traceability): parameterise the extractor for all three components

The tool is moving into the jray-project submodule to be shared by scene-actor-extraction (C++/Python), jRay (C#) and JRay-public-server (Rust). Two constants blocked that: LOCAL_TYPES and SOURCE_SUFFIXES were hardcoded to this repo, so either sibling parsed zero requirements and scanned zero files. Both, plus the register path, scan roots, system-spec path, exclude list and CI-executable tier set, are now configuration. One implementation, parameterised. A second copy for "the other language" is how two implementations start drifting apart, so there is exactly one - the same code path now produces: scene-actor-extraction AR/DP/IR/GR/VR 59 defined 0 tagged 0.0% jRay JR 46 defined 24 covered 52.2% JRay-public-server UR/DR 32 defined 23 covered 71.9% Configuration is traceability.toml at the component repo root, CLI flags, or both (flags win). Its directory defines the repo root, so the gate works from any subdirectory. `--print-example-config` emits the annotated schema. The JSON report echoes the settings it ran with, since a shared tool's output is otherwise ambiguous about which repo it describes. The refusal behaviour is kept and sharpened, because parameterising is exactly what makes it easy to point a repo at the wrong prefixes or the wrong suffixes. Zero requirements parsed or zero files scanned is still a hard failure, and the message now names the setting that is wrong rather than printing a plausible 0%. Config errors exit 2, not 1: a broken config is not a coverage failure, and conflating them makes CI logs lie about why the job went red. Also fixed while adapting to the sibling registers, which are read but not modified here: * escaped `\|` inside a markdown cell no longer shifts every later column (the server's register contains `small-\|M\|`); * a tag above an attribute-decorated declaration attributes to the declaration, not to `[HttpGet(...)]` or `#[derive(...)]` - the gap jRay's register calls out; * Rust and C# declaration patterns for context extraction; * the missing-tier warning is suppressed for a register that assigns no tiers at all, rather than listing every requirement in it. The workflow is now component-agnostic too: the changed-file check reads its extension list out of the report the gate just wrote, so the definition of "source file" lives in one place. 79 tests, still fixture-based, now including the cross-repo cases: the same parser over JR and UR/DR registers, the same scanner over Rust and C#, and both misconfigurations failing loudly.
build: requirement traceability extractor, gate, and CI workflow
2026-07-30 18:35:56 +02:00 · 2026-07-30 18:16:58 +02:00
6 changed files with 3067 additions and 0 deletions
@@ -0,0 +1,145 @@
+name: Traceability Validation
+
+# Mirrors JellyTau's .gitea/workflows/traceability-check.yml. The extractor is
+# stdlib Python, so there is no toolchain install step and no jq.
+#
+# This workflow is component-agnostic: every repo-specific setting - which ID
+# prefixes count, which file suffixes are source, which directories to scan,
+# the threshold - lives in traceability.toml at the repo root, and the same
+# extractor is shared by all three JRay components. Copying this file into
+# another component needs no edits.
+#
+# NOTE: the runner here is an Intel N100 with no discrete GPU. This job is only
+# ever static analysis of source comments plus markdown parsing, so it is cheap;
+# the requirements it reports as "tagged but unexecuted" are the ones that need
+# a GPU host, and they are deliberately never counted as covered.
+
+on:
+  push:
+    branches:
+      - main
+      - master
+      - develop
+  pull_request:
+    branches:
+      - main
+      - master
+      - develop
+
+jobs:
+  validate-traces:
+    runs-on: linux/amd64
+    name: Check requirement traces
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          submodules: recursive
+
+      - name: Check Python is available
+        run: |
+          set -e
+          command -v python3 >/dev/null 2>&1 || {
+            echo "python3 is missing from the runner image."
+            echo "The traceability tooling is stdlib-only Python;"
+            echo "3.9+ with CLI flags, 3.11+ to read traceability.toml."
+            exit 1
+          }
+          python3 --version
+
+      # The gate's own arithmetic is the thing being trusted, so its tests run
+      # before it does. JellyTau's gate was believed for months while it was
+      # dividing by frozen literals; untested gate logic is how that happens.
+      - name: Test the extractor
+        run: python3 scripts/traceability/test_extract_traces.py
+
+      # Threshold policy and every other repo-specific setting live in
+      # traceability.toml, not here, so local runs and CI runs cannot disagree
+      # about what "passing" means. Denominators come from docs/requirements.md
+      # at run time and are never hardcoded -- in this file or anywhere else.
+      #
+      # A misconfigured run (zero requirements parsed, zero files scanned) is a
+      # hard failure rather than a plausible-looking 0%.
+      - name: Traceability gate
+        run: sh scripts/traceability/traceability-gate.sh
+
+      - name: Check modified files for traces
+        if: github.event_name == 'pull_request'
+        run: |
+          set -e
+          echo "Checking modified sources for TRACES tags..."
+
+          # The extensions come from the report the gate just wrote, which got
+          # them from traceability.toml. Restating them here would be a second
+          # place for the source-file definition to live, and the two would
+          # drift the first time a language is added.
+          PATTERN=$(python3 -c "
+          import json, re, sys
+          suffixes = json.load(open('traces-report.json'))['config']['sourceSuffixes']
+          print('(' + '|'.join(re.escape(s) + '\$' for s in suffixes) + ')')
+          ")
+          echo "Source suffixes from traceability.toml: $PATTERN"
+
+          CHANGED=$(git diff --name-only "origin/${{ github.base_ref }}...HEAD" \
+                    | grep -E "$PATTERN" || true)
+
+          if [ -z "$CHANGED" ]; then
+            echo "No source files changed."
+            exit 0
+          fi
+
+          echo "Changed files:"
+          echo "$CHANGED" | sed 's/^/   /'
+          echo ""
+
+          # Advisory by design: not every file implements a requirement, and a
+          # tag on every function is noise that rots faster than it helps
+          # (CLAUDE.md: tag the unit that decides). This step exists to prompt,
+          # not to block. The blocking checks are in the gate step above.
+          #
+          # Piped into the loop rather than a here-string, and `case` rather
+          # than `[[ == ]]`, so this works under dash as well as bash. The loop
+          # body runs in a subshell, so misses are recorded in a file.
+          MISSING=$(mktemp)
+          echo "$CHANGED" | while IFS= read -r file; do
+            case "$file" in
+              */test_*.py|*_test.py|*Tests.cs|tests/*|*/tests/*) continue ;;
+            esac
+            [ -f "$file" ] || continue
+            if ! grep -q 'TRACES:' "$file"; then
+              echo "   no TRACES tag: $file"
+              echo "$file" >> "$MISSING"
+            fi
+          done
+
+          COUNT=$(wc -l < "$MISSING" | tr -d ' ')
+          rm -f "$MISSING"
+
+          if [ "$COUNT" -gt 0 ]; then
+            echo ""
+            echo "$COUNT changed file(s) carry no requirement tag."
+            echo "Format: // TRACES: AR-012, AR-013 | SR-002"
+            echo "        (pipe separates requirement types, comma separates IDs)"
+            echo "A deliberate invariant exception is tagged separately:"
+            echo "        // EXCEPTION: AR-024 <reason>"
+            echo "See CLAUDE.md and SPEC.md section 6."
+          fi
+
+      - name: Report summary
+        if: always()
+        run: |
+          echo "Traceability matrix: docs/traceability.md"
+          echo ""
+          head -40 docs/traceability.md || true
+
+      - name: Save reports
+        if: always()
+        uses: actions/upload-artifact@v3
+        with:
+          name: traceability-reports
+          path: |
+            traces-report.json
+            docs/traceability.md
+          retention-days: 30
@@ -0,0 +1,136 @@
+# Requirements traceability matrix
+
+<!-- GENERATED FILE - do not edit by hand. -->
+<!-- Regenerate: scripts/traceability/traceability-gate.sh -->
+
+**Generated:** 2026-07-30T16:35:36+00:00
+
+Denominators are read from [`requirements.md`](requirements.md) at run time, never hardcoded. Coverage counts a requirement only when it is tagged in source **and** has a verification tier this repo's CI host can execute (`T1, T2, T3, static`).
+
+## Summary
+
+| Metric | Value |
+|---|---|
+| Source files scanned | 86 |
+| TRACES tags found | 0 |
+| EXCEPTION tags found | 0 |
+| Requirements defined | 59 |
+| Requirements covered | 0 |
+| **Coverage** | **0.0%** (0/59) |
+| Coverage of CI-executable scope | 0.0% (0/50) |
+| Tagged but unexecuted in CI | 0 |
+| Orphan tags | 0 |
+
+### By type
+
+| Type | Covered | Tagged but unexecuted | Defined |
+|---|---|---|---|
+| AR | 0 | 0 | 27 |
+| DP | 0 | 0 | 6 |
+| IR | 0 | 0 | 8 |
+| GR | 0 | 0 | 9 |
+| VR | 0 | 0 | 9 |
+
+
+## Not executable in CI
+
+These requirements have no verification tier this repo's CI host can run, so a tag on them is evidence of *intent*, not of verification. They are never counted as covered.
+
+| ID | Tiers | Tagged in source | Requirement |
+|---|---|---|---|
+| AR-027 | T4 | no | Throughput acceptable for **arbitrary** gallery size |
+| VR-001 | out-of-ci | no | HDF5 post-inference dump at the embedded-frame boundary |
+| VR-002 | out-of-ci | no | Replay drives the **real** KPN nodes, not a reimplementation |
+| VR-003 | out-of-ci | no | Scoring: micro-F1 against X-Ray, precision/recall logged at every eva… |
+| VR-004 | out-of-ci | no | Reproducible validation corpus with ground truth |
+| VR-005 | out-of-ci | no | Minimum face size study — TPI/FPI vs probe size, gallery held at nati… |
+| VR-006 | out-of-ci | no | Re-tune `scene_threshold` once native-rate decode lands |
+| VR-007 | out-of-ci | no | Expansion band, clustering threshold, and deferred-pass ablation |
+| VR-008 | out-of-ci | no | Gallery scaling benchmark — throughput vs gallery size |
+
+## Orphan tags
+
+A tag naming an ID `requirements.md` does not define. This is what renumbering produces, and what a typo produces.
+
+_None._
+
+## Requirements tracing up to nothing
+
+A register row whose `Traces to` cell names no parent. Work serving no stated goal is how scope creeps in, and it is invisible unless something looks.
+
+_None._
+
+## Recorded exceptions
+
+Deliberate, documented departures from an invariant (`EXCEPTION: XX-nnn <reason>`). Reported separately and never counted as coverage — an exception is a decision to be reviewed, not evidence a requirement is met.
+
+_None._
+
+## Register
+
+| ID | Status | Tier | Traces to | Trace state | Tagged in | Requirement |
+|---|---|---|---|---|---|---|
+| AR-001 | Done | T3 | SR-002 | untagged | - | Detect faces in sampled frames; emit bbox, confidence, 5-point landma… |
+| AR-002 | Planned | T2 | SR-002 | untagged | - | Minimum face size 66×66 px, expressed in **original** resolution (dec… |
+| AR-003 | Planned | T1, T2, T4 | SR-002 | untagged | - | No fixed per-frame face cap — crowd scenes must not lose background c… |
+| AR-004 | Planned | T1, T4 | SR-002 | untagged | - | Backpressure: unbounded faces/frame absorbed by slowing, never by dro… |
+| AR-005 | Done | T1, T3 | SR-002 | untagged | - | Align to 112×112 via ArcFace 5-point similarity transform |
+| AR-006 | Done | T3 | SR-002 | untagged | - | 512-d L2-normalised embeddings, batched |
+| AR-007 | In Progress | T2 | SR-002 | untagged | - | Associate detections by IoU + embedding, with **frame-dependent** wei… |
+| AR-008 | Planned | T2 | SR-002 | untagged | - | One track pool keyed on `last_seen`; no separate revival path |
+| AR-009 | Done | T2 | SR-002 | untagged | - | Camera-cut detection (histogram) as an association hint |
+| AR-010 | In Progress | T2 | SR-002 | untagged | - | Scene-boundary detection (TransNetV2) as an association hint |
+| AR-011 | Planned | T1, T2 | SR-002 | untagged | - | **Every model is fed the input it was trained for** — cost reduced by… |
+| AR-012 | Planned | T2 | **SR-002** | untagged | - | Presence follows **track extent**, not per-frame recognition |
+| AR-013 | Planned | T2 | SR-002 | untagged | - | `last_seen` optional state machine; window ends at last sighting, nev… |
+| AR-014 | Planned | T2 | SR-002 | untagged | - | Belief swap A→B terminates the track and starts a new one |
+| AR-015 | Planned | T2 | SR-002 | untagged | - | Two live tracks owned by one actor ⇒ treat as a detected cut, re-asso… |
+| AR-016 | Planned | T2 | SR-002 | untagged | - | All tracks closed at EOF — a film ends with faces on screen |
+| AR-017 | Planned | T1, T2 | SR-002 | untagged | - | Every presence claim carries its belief and identification route |
+| AR-018 | Planned | T1, T2 | SR-005 | untagged | - | Per-subject embedding store with banded admission (novel enough, safe… |
+| AR-019 | In Progress | T2 | SR-005 | untagged | - | Per-film gallery annex from owned tracks; acquires the non-frontal vi… |
+| AR-020 | Planned | T2 | SR-005 | untagged | - | Deferred re-identification of unknown tracks against the final expand… |
+| AR-021 | Planned | T2 | SR-005 | untagged | - | Cluster unknown tracks into one entity per person, under temporal can… |
+| AR-022 | Planned | T1, T2 | §4 | untagged | - | Capture still-unidentified tracks: embeddings, metadata, **context cr… |
+| AR-023 | Done | T1 | SR-002 | untagged | - | Fit sigmoid calibration from intra/inter similarity distributions |
+| AR-024 | Planned | T1, static | SR-002 | untagged | - | **Always the calibrated probability, never a raw cosine** — exception… |
+| AR-025 | Planned | T1 | SR-002 | untagged | - | Per-track Bayesian accumulation in log-odds, with correlated-observat… |
+| AR-026 | In Progress | T1, T4 | SR-001 | untagged | - | All similarity computed as GEMM, including annex and deferred pass |
+| AR-027 | Planned | T4 | SR-001 | untagged | - | Throughput acceptable for **arbitrary** gallery size |
+| DP-001 | Done | T1, manual | PR-004 | untagged | - | One analysis core; modes are front-ends and must not fork pipeline lo… |
+| DP-002 | Done | T1, manual | PR-004 | untagged | - | Batch CLI over one title |
+| DP-003 | Planned | T1, manual | PR-004 | untagged | - | On-demand resident service with bounded, observable queue |
+| DP-004 | Planned | T1, manual | PR-004 | untagged | - | Opportunistic/idle mode: external trigger, hard stop, implicit re-que… |
+| DP-005 | Planned | T1, manual | PR-004 | untagged | - | Native installer, no Docker; Fedora + Arch |
+| DP-006 | Planned | T1, manual | PR-003 | untagged | - | Background incremental gallery refresh on a timer |
+| IR-001 | Done | T1 | SR-003 | untagged | - | Emit the JRay truth format as sibling `.jray.json` |
+| IR-002 | Planned | T1 | SR-003 | untagged | - | Windows carry belief + route; `extraction.*` carries `extinction_sec`… |
+| IR-003 | Planned | T1 | SR-003 | untagged | - | Output written **after** the deferred pass, not at EOF |
+| IR-004 | Planned | T1 | SR-003 | untagged | - | Compute the audio signature exactly per server spec §3 |
+| IR-005 | Planned | T1 | SR-003 | untagged | - | Golden-vector fixture shared with the plugin repo to prove bit-exactn… |
+| IR-006 | Done | unset | SR-001 | untagged | - | Jellyfin round-trip: pull pending queue, push complete results only |
+| IR-007 | Planned | unset | SR-003 | untagged | - | Media < 120 s: emit no signature, apply no sync offset — identical ru… |
+| IR-008 | Planned | unset | SR-003 | untagged | - | Emit and honour the signature's own `v1:` version prefix |
+| GR-001 | Done | T1, T3 | SR-001, SR-005 | untagged | - | Build gallery from Jellyfin library cast, TMDB profile fallback |
+| GR-002 | Done | T1, T3 | PR-003 | untagged | - | Incremental `--merge` refresh without re-embedding known actors |
+| GR-003 | Planned | T1, T3 | SR-001 | untagged | - | Report coverage: zero-image actors, under-referenced actors, dedup, c… |
+| GR-004 | Planned | T1, T3 | SR-001 | untagged | - | Stamp embedder identity into the gallery; **hard startup error** on m… |
+| GR-005 | Done | T1, T3 | **SR-005** | untagged | - | Gallery data never leaves the instance |
+| GR-006 | Planned | T1 | SR-005 | untagged | - | Provenance tiers: baked / harvested / confirmed, distinguishable per … |
+| GR-007 | Planned | T1 | SR-005 | untagged | - | Persist harvested embeddings **flagged and reviewable**, never silent… |
+| GR-008 | Planned | T1 | SR-005 | untagged | - | Flag distributional outliers among an actor's references (poisoning g… |
+| GR-009 | TBD | unset | §4 | untagged | - | Human-confirmed associations persist and improve future extractions |
+| VR-001 | Done | out-of-ci | PR-002 | untagged | - | HDF5 post-inference dump at the embedded-frame boundary |
+| VR-002 | Done | out-of-ci | PR-002 | untagged | - | Replay drives the **real** KPN nodes, not a reimplementation |
+| VR-003 | Done | out-of-ci | PR-002 | untagged | - | Scoring: micro-F1 against X-Ray, precision/recall logged at every eva… |
+| VR-004 | Done | out-of-ci | PR-002 | untagged | - | Reproducible validation corpus with ground truth |
+| VR-005 | Planned | out-of-ci | PR-002 | untagged | - | Minimum face size study — TPI/FPI vs probe size, gallery held at nati… |
+| VR-006 | Planned | out-of-ci | PR-002 | untagged | - | Re-tune `scene_threshold` once native-rate decode lands |
+| VR-007 | Planned | out-of-ci | PR-002 | untagged | - | Expansion band, clustering threshold, and deferred-pass ablation |
+| VR-008 | Planned | out-of-ci | PR-002 | untagged | - | Gallery scaling benchmark — throughput vs gallery size |
+| VR-009 | Planned | T1, out-of-ci | PR-002 | untagged | - | Verify accumulated posteriors are calibrated against held-out tracks |
+
+## Detailed mapping
+
+_No TRACES tags found yet. Tags are added as code is written; an empty matrix on a new tree is the correct reading, not a failure._
+
@@ -0,0 +1,62 @@
+#!/bin/sh
+#
+# Requirement traceability gate. Run locally exactly as CI runs it, from the
+# component repo root:
+#
+#     scripts/traceability/traceability-gate.sh
+#
+# Writes the JSON report and the markdown matrix, prints the coverage report,
+# and exits non-zero when the gate fails.
+#
+# This script is shared by every JRay component, so it knows nothing about any
+# one repo. All repo-specific settings - requirement ID prefixes, source
+# suffixes, scan roots, register path, thresholds - live in `traceability.toml`
+# at the component repo root. Run
+#
+#     scripts/traceability/extract_traces.py --print-example-config
+#
+# for the annotated schema. A repo whose config is wrong parses zero
+# requirements or scans zero files, and the gate refuses to report rather than
+# printing a misleading 0%.
+#
+# Environment (all optional; each overrides the config file):
+#   TRACES_CONFIG   path to traceability.toml
+#   TRACES_ROOT     repo root (default: nearest dir containing traceability.toml)
+#   MIN_COVERAGE    minimum overall coverage percent
+#   ALLOW_ORPHANS   1 to report orphan tags without failing
+#   TRACES_JSON     JSON report path
+#   TRACES_MD       markdown matrix path
+#   SYSTEM_SPEC     SPEC.md defining PR/SR; enables PR/SR orphan checking
+#   PYTHON          interpreter (default: python3)
+#
+# Threshold policy belongs in traceability.toml, not here and not in the
+# workflow YAML: a threshold written in two places is a threshold that will
+# disagree with itself.
+#
+# POSIX sh, no bashisms, no jq - the extractor does its own arithmetic and
+# printing, so CI needs nothing beyond python3.
+
+set -eu
+
+SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
+
+PYTHON="${PYTHON:-python3}"
+command -v "$PYTHON" >/dev/null 2>&1 || {
+    echo "FAILED: $PYTHON not found. The traceability gate needs Python 3.9+," >&2
+    echo "        or 3.11+ to read traceability.toml." >&2
+    exit 2
+}
+
+set -- --format coverage
+
+# Explicit `if` rather than `[ ... ] && ...`, because a trailing false test in
+# an && list exits under `set -e` in some POSIX shells.
+if [ -n "${TRACES_CONFIG:-}" ]; then set -- "$@" --config "$TRACES_CONFIG"; fi
+if [ -n "${TRACES_ROOT:-}" ];   then set -- "$@" --root "$TRACES_ROOT"; fi
+if [ -n "${MIN_COVERAGE:-}" ];  then set -- "$@" --min-coverage "$MIN_COVERAGE"; fi
+if [ -n "${TRACES_JSON:-}" ];   then set -- "$@" --json-out "$TRACES_JSON"; fi
+if [ -n "${TRACES_MD:-}" ];     then set -- "$@" --markdown-out "$TRACES_MD"; fi
+if [ -n "${SYSTEM_SPEC:-}" ];   then set -- "$@" --system-spec "$SYSTEM_SPEC"; fi
+if [ "${ALLOW_ORPHANS:-0}" = "1" ]; then set -- "$@" --allow-orphans; fi
+
+exec "$PYTHON" "$SCRIPT_DIR/extract_traces.py" "$@"
@@ -0,0 +1,34 @@
+# Traceability configuration for scene-actor-extraction.
+#
+# Read by the shared extractor (scripts/traceability/extract_traces.py), which
+# is the same implementation every JRay component uses. Everything repo-specific
+# lives here rather than in the tool; run `extract_traces.py
+# --print-example-config` for the annotated schema.
+#
+# This file's directory is taken as the repo root, so the gate works from any
+# subdirectory.
+
+# The prefixes this repo's register defines. Nothing else enters the fraction:
+# UT/IT are evidence for requirements, PR/SR belong to the system spec.
+requirement_types = ["AR", "DP", "IR", "GR", "VR"]
+
+# C++ pipeline plus the Python tooling, optimizer and validation scripts.
+languages = ["cpp", "python"]
+
+source_roots = ["src", "tests", "scripts", "experiments", "eval"]
+
+# CI is an Intel N100 with no discrete GPU. T4 is deliberately absent: a
+# requirement verifiable only on GPU hardware is reported as tagged but
+# unexecuted and never counted as covered, because counting a test that cannot
+# run is the same failure mode as JellyTau's 158% coverage bug.
+ci_executable_tiers = ["T1", "T2", "T3", "static"]
+
+# Threshold policy. 0 today because almost nothing is tagged yet - tags land as
+# the pipeline is built. This is not a gate that cannot fail: orphan tags, a
+# >100% ratio, a register that parses to nothing and an empty source scan are
+# all hard failures already. Ratchet this up as tags land; never reset it down.
+min_coverage = 0.0
+
+# The system spec owning PR/SR is vendored per-component as a submodule. Point
+# at it once that lands to turn on PR/SR orphan checking:
+# system_spec = "scripts/vendor/jray-project/SPEC.md"