ci: enforce the checks the contributor rules already required
Four gates that were documented but unenforced, plus the flaky test that made a full-suite run untrustworthy. Rust lint/format: CLAUDE.md has required `cargo fmt` and `cargo clippy` before every commit for as long as the rule existed, yet neither ran anywhere in CI — the requirement rested on memory alone. Both now run in build-and-test.yml and build-release.yml. rustfmt and clippy are already baked into the builder image, so nothing is installed at job time. `cargo fmt --all -- --check` is strict immediately (the tree is clean). Clippy is advisory for now: ~51 pre-existing warnings mean `-D warnings` would fail on unrelated work, so the step carries a TODO to flip the flag once the backlog clears. A compile error still fails it, so it is not a no-op. Traceability threshold: MIN_THRESHOLD sat at 50 while real coverage was 86%, so nearly half the matrix could rot before the gate objected. Ratcheted to 82 with the policy written down — it only ever goes up, and is never lowered to make a red build pass. The same figure lives in MIN_COVERAGE_PERCENT so `traces:coverage` gates locally on the same bar, and a test fails if the two drift. Dangling IDs: a TRACES comment could name any well-formed ID and the extractor accepted it silently, so typos and renames that missed a call site passed unnoticed. `bun run traces:validate` cross-checks every traced ID against the table rows in requirements.md and fails with the referencing files listed. It spans UT/IT as well, which the coverage orphan list ignores by design. This currently reports DR-189 and UT-188, which are being defined separately. Flaky offlineCatalog test: the first dynamic import of the service paid ~1s to transform its dependency graph, charged to a test body against vitest's 5s default. Alone it passed; under suite-wide contention it timed out. The import is now warmed at collection time, so no test is timing the compiler — the timeout is deliberately unchanged. The store shim also drops subscribers from module instances discarded by resetModules, which previously leaked across tests.
This commit is contained in:
+42
-10
@@ -15,7 +15,7 @@ The CI/CD pipeline automatically validates that code changes are properly traced
|
||||
Traceability validation lives in `.gitea/workflows/traceability-check.yml`:
|
||||
|
||||
- ✅ Automatic trace extraction
|
||||
- ✅ Coverage validation against minimum threshold (50%)
|
||||
- ✅ Coverage validation against minimum threshold (82%, ratcheted)
|
||||
- ✅ Modified file checking
|
||||
- ✅ Artifact preservation
|
||||
- ✅ Summary reports
|
||||
@@ -43,7 +43,7 @@ Extracts all TRACES comments from:
|
||||
|
||||
### 2. Coverage Thresholds
|
||||
The workflow checks:
|
||||
- **Minimum overall coverage:** 50%
|
||||
- **Minimum overall coverage:** 82% (`MIN_THRESHOLD`)
|
||||
|
||||
Denominators are **derived from `docs/requirements.md` at run time** — they are
|
||||
never hardcoded here or in the workflow. Run `bun run traces:coverage` for the
|
||||
@@ -61,8 +61,39 @@ a `TRACES:` comment but is not defined in `requirements.md` is reported as
|
||||
**orphaned** and does not count toward coverage. UT/IT test identifiers are a
|
||||
separate taxonomy and are excluded entirely.
|
||||
|
||||
The workflow **fails** and blocks merge if coverage drops below 50% — or if it
|
||||
computes above 100%, which can only mean the gate is miscounting.
|
||||
The workflow **fails** and blocks merge if coverage drops below the threshold —
|
||||
or if it computes above 100%, which can only mean the gate is miscounting.
|
||||
|
||||
#### Ratchet policy
|
||||
|
||||
`MIN_THRESHOLD` **only ever goes up.** It is deliberately set a few points below
|
||||
the coverage actually achieved (82 against a real 86%), so a genuine regression
|
||||
trips it. It previously sat at 50 while true coverage was 86%: nearly half the
|
||||
matrix could have rotted before CI objected.
|
||||
|
||||
When coverage rises durably, raise the threshold to just under the new figure.
|
||||
**Never lower it to make a red build pass** — add the missing TRACES comments
|
||||
instead. The same number lives in `MIN_COVERAGE_PERCENT` in
|
||||
`scripts/extract-traces.ts` (so `bun run traces:coverage` gates locally on the
|
||||
same bar); `scripts/extract-traces.test.ts` fails if the two drift apart.
|
||||
|
||||
### 2b. Dangling requirement IDs
|
||||
|
||||
```bash
|
||||
bun run traces:validate
|
||||
```
|
||||
|
||||
Every ID named by a `TRACES:` comment must be defined as a table row in
|
||||
`docs/requirements.md`. The extractor used to accept any well-formed ID
|
||||
silently, so a typo or a rename that missed a call site passed unnoticed —
|
||||
`DR-189` and `UT-188` were referenced from three source files, defined nowhere,
|
||||
for months.
|
||||
|
||||
This check spans **all six** ID types (UR/IR/DR/JA/UT/IT), unlike the coverage
|
||||
`orphaned` list above, which considers only the four requirement types so that
|
||||
UT/IT noise cannot bury a real typo in the ratio's reporting. The workflow step
|
||||
**fails the build** on any dangling ID and prints each offender with the files
|
||||
that reference it.
|
||||
|
||||
### 3. Modified File Checking
|
||||
On pull requests, the workflow:
|
||||
@@ -120,13 +151,13 @@ TRACES: [UR-###, ...] | [IR-###, ...] | [DR-###, ...] | [JA-###, ...]
|
||||
|
||||
### On Push to Main Branch
|
||||
1. ✅ Extracts all traces from code
|
||||
2. ✅ Validates coverage is >= 50%
|
||||
2. ✅ Validates coverage is >= 82%
|
||||
3. ✅ Generates full traceability report
|
||||
4. ✅ Saves report as artifact
|
||||
|
||||
### On Pull Request
|
||||
1. ✅ Extracts all traces
|
||||
2. ✅ Validates coverage >= 50%
|
||||
2. ✅ Validates coverage >= 82%
|
||||
3. ✅ Checks modified files for TRACES
|
||||
4. ✅ Warns if new code lacks TRACES
|
||||
5. ✅ Suggests proper format
|
||||
@@ -134,7 +165,8 @@ TRACES: [UR-###, ...] | [IR-###, ...] | [DR-###, ...] | [JA-###, ...]
|
||||
|
||||
### Failure Scenarios
|
||||
The workflow **fails** (blocks merge) if:
|
||||
- Coverage drops below 50%
|
||||
- Coverage drops below 82%
|
||||
- A `TRACES:` comment names an ID `docs/requirements.md` does not define
|
||||
- JSON extraction fails
|
||||
- Invalid trace format
|
||||
|
||||
@@ -174,7 +206,7 @@ made the broken CI arithmetic look plausible for so long.
|
||||
As of July 2026 overall coverage is ~86% (182/212).
|
||||
|
||||
### Targets
|
||||
- **Short term** (Sprint): Maintain ≥50% overall
|
||||
- **Short term** (Sprint): Maintain ≥82% overall (the current ratchet)
|
||||
- **Medium term** (Month): Reach 70% overall coverage
|
||||
- **Long term** (Release): Reach 90% coverage with focus on:
|
||||
- IR requirements (API clients)
|
||||
@@ -209,14 +241,14 @@ When submitting a pull request:
|
||||
|
||||
- [ ] All new code has TRACES comments linking to requirements
|
||||
- [ ] TRACES format is correct: `// TRACES: UR-001 | DR-002`
|
||||
- [ ] Workflow passes (coverage ≥ 50%)
|
||||
- [ ] Workflow passes (coverage ≥ 82%)
|
||||
- [ ] No coverage regressions
|
||||
- [ ] Artifact traceability report was generated
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Coverage below minimum threshold"
|
||||
**Problem:** Workflow fails with coverage < 50%
|
||||
**Problem:** Workflow fails with coverage < 82%
|
||||
|
||||
**Solution:**
|
||||
1. Run `bun run traces:json` locally
|
||||
|
||||
@@ -133,13 +133,14 @@ bun run traces:json | jq '.requirements."UR-005"'
|
||||
### Before Committing
|
||||
1. Ensure all new code has TRACES
|
||||
2. Format is correct: `// TRACES: ...`
|
||||
3. Requirements exist in README.md
|
||||
4. No typos in requirement IDs
|
||||
3. Requirements exist in `docs/requirements.md` — `bun run traces:validate`
|
||||
4. No typos in requirement IDs (same command catches them)
|
||||
|
||||
## CI/CD Validation
|
||||
|
||||
The workflow automatically checks:
|
||||
- ✅ Coverage stays >= 50%
|
||||
- ✅ Coverage stays >= 82% (a ratchet — raise it, never lower it)
|
||||
- ✅ Every traced ID is defined in `docs/requirements.md`
|
||||
- ✅ New files have TRACES
|
||||
- ✅ JSON format is valid
|
||||
- ✅ Reports are generated
|
||||
|
||||
Reference in New Issue
Block a user