jellytau/docs/traceability-ci.md

# Requirement Traceability CI/CD Pipeline

This document explains the automated requirement traceability validation system for JellyTau.

## Overview

The CI/CD pipeline automatically validates that code changes are properly traced to requirements. This ensures:
- ✅ Requirements are implemented with clear traceability
- ✅ No requirement coverage regressions
- ✅ Code changes are linked to specific requirements
- ✅ Quality metrics are tracked over time

## Gitea Actions Workflows

Traceability validation lives in `.gitea/workflows/traceability-check.yml`:

- ✅ Automatic trace extraction
- ✅ Coverage validation against minimum threshold (88%, ratcheted)
- ✅ Modified file checking
- ✅ Artifact preservation
- ✅ Summary reports

**Runs on:** Every push and pull request to `master`/`main`/`develop`

A second workflow, `traceability.yml`, previously duplicated this one as a
"GitHub-compatible alternative". It was removed: CI here is Gitea Actions, and
its only unique step (PR comments via `actions/github-script`) depended on the
GitHub REST client, which Gitea does not provide. To add PR comments, post to
Gitea's `/api/v1/repos/{owner}/{repo}/issues/{index}/comments` from
`traceability-check.yml` rather than reviving the old file.

## What Gets Validated

### 1. Trace Extraction
```bash
bun run traces:json > traces-report.json
```
Extracts all TRACES comments from:
- TypeScript files (`src/**/*.ts`)
- Svelte components (`src/**/*.svelte`)
- Rust code (`src-tauri/src/**/*.rs`)
- Test files

### 2. Coverage Thresholds
The workflow checks:
- **Minimum overall coverage:** 88% (`MIN_THRESHOLD`)

Denominators are **derived from `docs/requirements.md` at run time** — they are
never hardcoded here or in the workflow. Run `bun run traces:coverage` for the
current per-type breakdown; any number written into this document is a snapshot
that will drift.

> **Why this matters.** The workflow used to divide by frozen literals
> (UR/39, IR/24, DR/48, JA/3, total 114) while `requirements.md` had grown past
> 200. It reported **158%** coverage, so the 50% threshold was unreachable and
> the job could not fail regardless of how far coverage dropped. See
> The fix derives the denominators from `requirements.md` at run time.

Coverage is the *intersection* of traced and defined IDs: an ID that appears in
a `TRACES:` comment but is not defined in `requirements.md` is reported as
**orphaned** and does not count toward coverage. UT/IT test identifiers are a
separate taxonomy and are excluded entirely.

The workflow **fails** and blocks merge if coverage drops below the threshold —
or if it computes above 100%, which can only mean the gate is miscounting.

#### Ratchet policy

`MIN_THRESHOLD` **only ever goes up.** It is deliberately set a few points below
the coverage actually achieved (88 against a real ~90%), so a genuine regression
trips it. It previously sat at 50 while true coverage was 86%: nearly half the
matrix could have rotted before CI objected. It was ratcheted 50 → 82 when that
was found, and 82 → 88 once coverage had held above 88% for several releases.

When coverage rises durably, raise the threshold to just under the new figure.
**Never lower it to make a red build pass** — add the missing TRACES comments
instead. The same number lives in `MIN_COVERAGE_PERCENT` in
`scripts/extract-traces.ts` (so `bun run traces:coverage` gates locally on the
same bar); `scripts/extract-traces.test.ts` fails if the two drift apart.

### 2b. Dangling requirement IDs

```bash
bun run traces:validate
```

Every ID named by a `TRACES:` comment must be defined as a table row in
`docs/requirements.md`. The extractor used to accept any well-formed ID
silently, so a typo or a rename that missed a call site passed unnoticed —
`DR-189` and `UT-188` were referenced from three source files, defined nowhere,
for months.

This check spans **all six** ID types (UR/IR/DR/JA/UT/IT), unlike the coverage
`orphaned` list above, which considers only the four requirement types so that
UT/IT noise cannot bury a real typo in the ratio's reporting. The workflow step
**fails the build** on any dangling ID and prints each offender with the files
that reference it.

### 3. Modified File Checking
On pull requests, the workflow:
1. Detects all changed TypeScript/Svelte/Rust files
2. Warns if new/modified files lack TRACES comments
3. Suggests the TRACES format for missing comments

## How to Add Traces to New Code

When you add new code or modify existing code, include TRACES comments:

### TypeScript/Svelte Example
```typescript
// TRACES: UR-005, UR-026 | DR-029
export function handlePlayback() {
  // Implementation...
}
```

### Rust Example
```rust
/// TRACES: UR-005 | DR-001
pub fn player_state_changed(state: PlayerState) {
    // Implementation...
}
```

### Test Example
```rust
// TRACES: UR-005 | DR-001 | UT-026, UT-027
#[cfg(test)]
mod tests {
    // Tests...
}
```

## TRACES Format

```
TRACES: [UR-###, ...] | [IR-###, ...] | [DR-###, ...] | [JA-###, ...]
```

- `UR-###` - User Requirements (features users see)
- `IR-###` - Integration Requirements (API/platform integration)
- `DR-###` - Development Requirements (internal architecture)
- `JA-###` - Jellyfin API Requirements (Jellyfin API usage)

**Examples:**
- `// TRACES: UR-005` - Single requirement
- `// TRACES: UR-005, UR-026` - Multiple of same type
- `// TRACES: UR-005 | DR-029` - Multiple types
- `// TRACES: UR-005, UR-026 | DR-001, DR-029 | UT-001` - Complex

## Workflow Behavior

### On Push to Main Branch
1. ✅ Extracts all traces from code
2. ✅ Validates coverage is >= 88%
3. ✅ Generates full traceability report
4. ✅ Saves report as artifact

### On Pull Request
1. ✅ Extracts all traces
2. ✅ Validates coverage >= 88%
3. ✅ Checks modified files for TRACES
4. ✅ Warns if new code lacks TRACES
5. ✅ Suggests proper format
6. ✅ Generates report artifact

### Failure Scenarios
The workflow **fails** (blocks merge) if:
- Coverage drops below 88%
- A `TRACES:` comment names an ID `docs/requirements.md` does not define
- JSON extraction fails
- Invalid trace format

The workflow **warns** (but doesn't block) if:
- New files lack TRACES comments
- Coverage drops (but still above threshold)

## Viewing Reports

### In Gitea Actions UI
1. Go to **Actions** tab
2. Click the **Traceability Validation** workflow run
3. Download **traceability-reports** artifact
4. View:
   - `traces-report.json` - Raw trace data
   - `docs/traceability.md` - Formatted report

### Locally
```bash
# Extract current traces
bun run traces:json | jq '.byType'

# Generate full report
bun run traces:markdown
cat docs/traceability.md
```

## Coverage Goals

### Current Status

Run `bun run traces:coverage` — it prints the live figure and exits non-zero
below threshold. Numbers are deliberately not pinned here; the previous snapshot
in this section (51%, 56/114) was stale by roughly 100 requirements and was what
made the broken CI arithmetic look plausible for so long.

As of August 2026 overall coverage is ~90%.

### Targets
- **Short term** (Sprint): Maintain ≥88% overall (the current ratchet)
- **Medium term** (Month): Hold above 90% and ratchet the gate to match
- **Long term** (Release): Reach 95% coverage with focus on:
  - IR requirements (API clients)
  - JA requirements (Jellyfin API endpoints)
  - Remaining UR/DR requirements

## Improving Coverage

### For Missing User Requirements (UR)
1. Review [README.md](../README.md) for unimplemented features
2. Add TRACES to code that implements them
3. Focus on high-priority features (High/Medium priority)

### For Missing Integration Requirements (IR)
1. Add TRACES to Jellyfin API client methods
2. Add TRACES to platform-specific backends (Android/Linux)
3. Link to corresponding Jellyfin API endpoints

### For Missing Development Requirements (DR)
1. Add TRACES to UI components in `src/lib/components/`
2. Add TRACES to composables in `src/lib/composables/`
3. Add TRACES to player backend in `src-tauri/src/player/`

### For Jellyfin API Requirements (JA)
1. Add TRACES to Jellyfin API wrapper methods
2. Document which endpoints map to which requirements
3. Link to Jellyfin API documentation

## Example PR Checklist

When submitting a pull request:

- [ ] All new code has TRACES comments linking to requirements
- [ ] TRACES format is correct: `// TRACES: UR-001 | DR-002`
- [ ] Workflow passes (coverage ≥ 88%)
- [ ] No coverage regressions
- [ ] Artifact traceability report was generated

## Troubleshooting

### "Coverage below minimum threshold"
**Problem:** Workflow fails with coverage < 88%

**Solution:**
1. Run `bun run traces:json` locally
2. Check which requirements are traced
3. Add TRACES to untraced code sections
4. Re-run extraction to verify

### "New files without TRACES"
**Problem:** Workflow warns about new files lacking TRACES

**Solution:**
1. Add TRACES comments to all new code
2. Format: `// TRACES: UR-001 | DR-002`
3. Map code to specific requirements from README.md
4. Re-push

### "Invalid JSON format"
**Problem:** Trace extraction produces invalid JSON

**Solution:**
1. Check for malformed TRACES comments
2. Run locally: `bun run traces:json`
3. Look for parsing errors
4. Fix and retry

## Integration with Development

### Before Committing
```bash
# Check your traces
bun run traces:json | jq '.byType'

# Regenerate report
bun run traces:markdown

# Verify traces syntax
grep "TRACES:" src/**/*.ts src/**/*.rs
```

### In Your IDE
Add a file watcher to regenerate traces on save:
```json
{
  "fileWatcher.watchPatterns": [
    "src/**/*.ts",
    "src/**/*.svelte",
    "src-tauri/src/**/*.rs"
  ],
  "fileWatcher.command": "bun run traces:markdown"
}
```

### Git Hooks
Add a pre-push hook to validate traces:

```bash
#!/bin/bash
# .git/hooks/pre-push
bun run traces:json > /dev/null
if [ $? -ne 0 ]; then
  echo "❌ Invalid TRACES format"
  exit 1
fi
```

## References

- [Extract Traces Script](../scripts/README.md#extract-tracests)
- [Requirements Specification](../README.md#requirements-specification)
- [Traceability Matrix](./traceability.md)
- [Gitea Actions Documentation](https://docs.gitea.io/en-us/actions/)

## Support

For issues or questions:
1. Check this document
2. Review example traces in `src/lib/stores/`
3. Check existing TRACES comments for format
4. Review workflow logs in Gitea Actions

---

**Last Updated:** 2026-02-13