docs: raise the documented traceability gate to 88%
The spec review checklist still asked for >= 50%, the figure the gate sat at before it was found to be unreachable; traceability-ci.md carried 82% throughout. Both now read 88%, matching the ratchet, and the checklist points at `bun run traces:coverage` rather than inviting anyone to trust a number written in a document. Also refreshes the two stale coverage snapshots in traceability-ci.md (~86% from July 2026, and targets of 70% and 90% that the current 90% already passes) and records the 50 -> 82 -> 88 ratchet history.
This commit is contained in:
@@ -50,7 +50,8 @@ Copy the boxes into the review comment (or the PR) and tick them.
|
|||||||
- [ ] Linked to existing URs, or new URs/DRs are allocated in
|
- [ ] Linked to existing URs, or new URs/DRs are allocated in
|
||||||
[requirements.md](../requirements.md).
|
[requirements.md](../requirements.md).
|
||||||
- [ ] Requirement-implementing code will carry `// TRACES:` comments (CLAUDE.md).
|
- [ ] Requirement-implementing code will carry `// TRACES:` comments (CLAUDE.md).
|
||||||
- [ ] Traceability coverage stays ≥ 50% (the CI gate).
|
- [ ] Traceability coverage stays ≥ 88% (the CI gate — a ratchet, so check
|
||||||
|
`bun run traces:coverage` rather than trusting this number).
|
||||||
|
|
||||||
## Conflicts & hygiene
|
## Conflicts & hygiene
|
||||||
|
|
||||||
|
|||||||
+14
-13
@@ -15,7 +15,7 @@ The CI/CD pipeline automatically validates that code changes are properly traced
|
|||||||
Traceability validation lives in `.gitea/workflows/traceability-check.yml`:
|
Traceability validation lives in `.gitea/workflows/traceability-check.yml`:
|
||||||
|
|
||||||
- ✅ Automatic trace extraction
|
- ✅ Automatic trace extraction
|
||||||
- ✅ Coverage validation against minimum threshold (82%, ratcheted)
|
- ✅ Coverage validation against minimum threshold (88%, ratcheted)
|
||||||
- ✅ Modified file checking
|
- ✅ Modified file checking
|
||||||
- ✅ Artifact preservation
|
- ✅ Artifact preservation
|
||||||
- ✅ Summary reports
|
- ✅ Summary reports
|
||||||
@@ -43,7 +43,7 @@ Extracts all TRACES comments from:
|
|||||||
|
|
||||||
### 2. Coverage Thresholds
|
### 2. Coverage Thresholds
|
||||||
The workflow checks:
|
The workflow checks:
|
||||||
- **Minimum overall coverage:** 82% (`MIN_THRESHOLD`)
|
- **Minimum overall coverage:** 88% (`MIN_THRESHOLD`)
|
||||||
|
|
||||||
Denominators are **derived from `docs/requirements.md` at run time** — they are
|
Denominators are **derived from `docs/requirements.md` at run time** — they are
|
||||||
never hardcoded here or in the workflow. Run `bun run traces:coverage` for the
|
never hardcoded here or in the workflow. Run `bun run traces:coverage` for the
|
||||||
@@ -67,9 +67,10 @@ or if it computes above 100%, which can only mean the gate is miscounting.
|
|||||||
#### Ratchet policy
|
#### Ratchet policy
|
||||||
|
|
||||||
`MIN_THRESHOLD` **only ever goes up.** It is deliberately set a few points below
|
`MIN_THRESHOLD` **only ever goes up.** It is deliberately set a few points below
|
||||||
the coverage actually achieved (82 against a real 86%), so a genuine regression
|
the coverage actually achieved (88 against a real ~90%), so a genuine regression
|
||||||
trips it. It previously sat at 50 while true coverage was 86%: nearly half the
|
trips it. It previously sat at 50 while true coverage was 86%: nearly half the
|
||||||
matrix could have rotted before CI objected.
|
matrix could have rotted before CI objected. It was ratcheted 50 → 82 when that
|
||||||
|
was found, and 82 → 88 once coverage had held above 88% for several releases.
|
||||||
|
|
||||||
When coverage rises durably, raise the threshold to just under the new figure.
|
When coverage rises durably, raise the threshold to just under the new figure.
|
||||||
**Never lower it to make a red build pass** — add the missing TRACES comments
|
**Never lower it to make a red build pass** — add the missing TRACES comments
|
||||||
@@ -151,13 +152,13 @@ TRACES: [UR-###, ...] | [IR-###, ...] | [DR-###, ...] | [JA-###, ...]
|
|||||||
|
|
||||||
### On Push to Main Branch
|
### On Push to Main Branch
|
||||||
1. ✅ Extracts all traces from code
|
1. ✅ Extracts all traces from code
|
||||||
2. ✅ Validates coverage is >= 82%
|
2. ✅ Validates coverage is >= 88%
|
||||||
3. ✅ Generates full traceability report
|
3. ✅ Generates full traceability report
|
||||||
4. ✅ Saves report as artifact
|
4. ✅ Saves report as artifact
|
||||||
|
|
||||||
### On Pull Request
|
### On Pull Request
|
||||||
1. ✅ Extracts all traces
|
1. ✅ Extracts all traces
|
||||||
2. ✅ Validates coverage >= 82%
|
2. ✅ Validates coverage >= 88%
|
||||||
3. ✅ Checks modified files for TRACES
|
3. ✅ Checks modified files for TRACES
|
||||||
4. ✅ Warns if new code lacks TRACES
|
4. ✅ Warns if new code lacks TRACES
|
||||||
5. ✅ Suggests proper format
|
5. ✅ Suggests proper format
|
||||||
@@ -165,7 +166,7 @@ TRACES: [UR-###, ...] | [IR-###, ...] | [DR-###, ...] | [JA-###, ...]
|
|||||||
|
|
||||||
### Failure Scenarios
|
### Failure Scenarios
|
||||||
The workflow **fails** (blocks merge) if:
|
The workflow **fails** (blocks merge) if:
|
||||||
- Coverage drops below 82%
|
- Coverage drops below 88%
|
||||||
- A `TRACES:` comment names an ID `docs/requirements.md` does not define
|
- A `TRACES:` comment names an ID `docs/requirements.md` does not define
|
||||||
- JSON extraction fails
|
- JSON extraction fails
|
||||||
- Invalid trace format
|
- Invalid trace format
|
||||||
@@ -203,12 +204,12 @@ below threshold. Numbers are deliberately not pinned here; the previous snapshot
|
|||||||
in this section (51%, 56/114) was stale by roughly 100 requirements and was what
|
in this section (51%, 56/114) was stale by roughly 100 requirements and was what
|
||||||
made the broken CI arithmetic look plausible for so long.
|
made the broken CI arithmetic look plausible for so long.
|
||||||
|
|
||||||
As of July 2026 overall coverage is ~86% (182/212).
|
As of August 2026 overall coverage is ~90%.
|
||||||
|
|
||||||
### Targets
|
### Targets
|
||||||
- **Short term** (Sprint): Maintain ≥82% overall (the current ratchet)
|
- **Short term** (Sprint): Maintain ≥88% overall (the current ratchet)
|
||||||
- **Medium term** (Month): Reach 70% overall coverage
|
- **Medium term** (Month): Hold above 90% and ratchet the gate to match
|
||||||
- **Long term** (Release): Reach 90% coverage with focus on:
|
- **Long term** (Release): Reach 95% coverage with focus on:
|
||||||
- IR requirements (API clients)
|
- IR requirements (API clients)
|
||||||
- JA requirements (Jellyfin API endpoints)
|
- JA requirements (Jellyfin API endpoints)
|
||||||
- Remaining UR/DR requirements
|
- Remaining UR/DR requirements
|
||||||
@@ -241,14 +242,14 @@ When submitting a pull request:
|
|||||||
|
|
||||||
- [ ] All new code has TRACES comments linking to requirements
|
- [ ] All new code has TRACES comments linking to requirements
|
||||||
- [ ] TRACES format is correct: `// TRACES: UR-001 | DR-002`
|
- [ ] TRACES format is correct: `// TRACES: UR-001 | DR-002`
|
||||||
- [ ] Workflow passes (coverage ≥ 82%)
|
- [ ] Workflow passes (coverage ≥ 88%)
|
||||||
- [ ] No coverage regressions
|
- [ ] No coverage regressions
|
||||||
- [ ] Artifact traceability report was generated
|
- [ ] Artifact traceability report was generated
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
### "Coverage below minimum threshold"
|
### "Coverage below minimum threshold"
|
||||||
**Problem:** Workflow fails with coverage < 82%
|
**Problem:** Workflow fails with coverage < 88%
|
||||||
|
|
||||||
**Solution:**
|
**Solution:**
|
||||||
1. Run `bun run traces:json` locally
|
1. Run `bun run traces:json` locally
|
||||||
|
|||||||
Reference in New Issue
Block a user