dtourolle/jellytau
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 39 Wiki Activity

docs: raise the documented traceability gate to 88%

Browse Source 
The spec review checklist still asked for >= 50%, the figure the gate sat
at before it was found to be unreachable; traceability-ci.md carried 82%
throughout. Both now read 88%, matching the ratchet, and the checklist
points at `bun run traces:coverage` rather than inviting anyone to trust a
number written in a document.

Also refreshes the two stale coverage snapshots in traceability-ci.md
(~86% from July 2026, and targets of 70% and 90% that the current 90%
already passes) and records the 50 -> 82 -> 88 ratchet history.
This commit is contained in:
dtourolle
2026-08-20 19:30:48 +02:00
parent 46a5219f8e
commit 4567c63797
2 changed files with 16 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/specs/SPEC-REVIEW-CHECKLIST.md
+2 -1
View File
@@ -50,7 +50,8 @@ Copy the boxes into the review comment (or the PR) and tick them.
- [ ] Linked to existing URs, or new URs/DRs are allocated in
[requirements.md](../requirements.md).
- [ ] Requirement-implementing code will carry `// TRACES:` comments (CLAUDE.md).
- [ ] Traceability coverage stays ≥ 50% (the CI gate).
- [ ] Traceability coverage stays ≥ 88% (the CI gate — a ratchet, so check
`bun run traces:coverage` rather than trusting this number).
## Conflicts & hygiene
docs/traceability-ci.md
+14 -13
View File
@@ -15,7 +15,7 @@ The CI/CD pipeline automatically validates that code changes are properly traced
Traceability validation lives in `.gitea/workflows/traceability-check.yml`:
- ✅ Automatic trace extraction
- ✅ Coverage validation against minimum threshold (82%, ratcheted)
- ✅ Coverage validation against minimum threshold (88%, ratcheted)
- ✅ Modified file checking
- ✅ Artifact preservation
- ✅ Summary reports
@@ -43,7 +43,7 @@ Extracts all TRACES comments from:
### 2. Coverage Thresholds
The workflow checks:
- **Minimum overall coverage:** 82% (`MIN_THRESHOLD`)
- **Minimum overall coverage:** 88% (`MIN_THRESHOLD`)
Denominators are **derived from `docs/requirements.md` at run time** — they are
never hardcoded here or in the workflow. Run `bun run traces:coverage` for the
@@ -67,9 +67,10 @@ or if it computes above 100%, which can only mean the gate is miscounting.
#### Ratchet policy
`MIN_THRESHOLD` **only ever goes up.** It is deliberately set a few points below
the coverage actually achieved (82 against a real 86%), so a genuine regression
the coverage actually achieved (88 against a real ~90%), so a genuine regression
trips it. It previously sat at 50 while true coverage was 86%: nearly half the
matrix could have rotted before CI objected.
matrix could have rotted before CI objected. It was ratcheted 50 → 82 when that
was found, and 82 → 88 once coverage had held above 88% for several releases.
When coverage rises durably, raise the threshold to just under the new figure.
**Never lower it to make a red build pass** — add the missing TRACES comments
@@ -151,13 +152,13 @@ TRACES: [UR-###, ...] | [IR-###, ...] | [DR-###, ...] | [JA-###, ...]
### On Push to Main Branch
1. ✅ Extracts all traces from code
2. ✅ Validates coverage is >= 82%
2. ✅ Validates coverage is >= 88%
3. ✅ Generates full traceability report
4. ✅ Saves report as artifact
### On Pull Request
1. ✅ Extracts all traces
2. ✅ Validates coverage >= 82%
2. ✅ Validates coverage >= 88%
3. ✅ Checks modified files for TRACES
4. ✅ Warns if new code lacks TRACES
5. ✅ Suggests proper format
@@ -165,7 +166,7 @@ TRACES: [UR-###, ...] | [IR-###, ...] | [DR-###, ...] | [JA-###, ...]
### Failure Scenarios
The workflow **fails** (blocks merge) if:
- Coverage drops below 82%
- Coverage drops below 88%
- A `TRACES:` comment names an ID `docs/requirements.md` does not define
- JSON extraction fails
- Invalid trace format
@@ -203,12 +204,12 @@ below threshold. Numbers are deliberately not pinned here; the previous snapshot
in this section (51%, 56/114) was stale by roughly 100 requirements and was what
made the broken CI arithmetic look plausible for so long.
As of July 2026 overall coverage is ~86% (182/212).
As of August 2026 overall coverage is ~90%.
### Targets
- **Short term** (Sprint): Maintain ≥82% overall (the current ratchet)
- **Medium term** (Month): Reach 70% overall coverage
- **Long term** (Release): Reach 90% coverage with focus on:
- **Short term** (Sprint): Maintain ≥88% overall (the current ratchet)
- **Medium term** (Month): Hold above 90% and ratchet the gate to match
- **Long term** (Release): Reach 95% coverage with focus on:
- IR requirements (API clients)
- JA requirements (Jellyfin API endpoints)
- Remaining UR/DR requirements
@@ -241,14 +242,14 @@ When submitting a pull request:
- [ ] All new code has TRACES comments linking to requirements
- [ ] TRACES format is correct: `// TRACES: UR-001 | DR-002`
- [ ] Workflow passes (coverage ≥ 82%)
- [ ] Workflow passes (coverage ≥ 88%)
- [ ] No coverage regressions
- [ ] Artifact traceability report was generated
## Troubleshooting
### "Coverage below minimum threshold"
**Problem:** Workflow fails with coverage < 82%
**Problem:** Workflow fails with coverage < 88%
**Solution:**
1. Run `bun run traces:json` locally