jellytau/docs/traceability-ci.md

Requirement Traceability CI/CD Pipeline

This document explains the automated requirement traceability validation system for JellyTau.

Overview

The CI/CD pipeline automatically validates that code changes are properly traced to requirements. This ensures:

• ✅ Requirements are implemented with clear traceability
• ✅ No requirement coverage regressions
• ✅ Code changes are linked to specific requirements
• ✅ Quality metrics are tracked over time

Gitea Actions Workflows

Traceability validation lives in .gitea/workflows/traceability-check.yml:

• ✅ Automatic trace extraction
• ✅ Coverage validation against minimum threshold (88%, ratcheted)
• ✅ Modified file checking
• ✅ Artifact preservation
• ✅ Summary reports

Runs on: Every push and pull request to master/main/develop

A second workflow, traceability.yml, previously duplicated this one as a "GitHub-compatible alternative". It was removed: CI here is Gitea Actions, and its only unique step (PR comments via actions/github-script) depended on the GitHub REST client, which Gitea does not provide. To add PR comments, post to Gitea's /api/v1/repos/{owner}/{repo}/issues/{index}/comments from traceability-check.yml rather than reviving the old file.

What Gets Validated

1. Trace Extraction

bun run traces:json > traces-report.json

Extracts all TRACES comments from:

• TypeScript files (src/**/*.ts)
• Svelte components (src/**/*.svelte)
• Rust code (src-tauri/src/**/*.rs)
• Test files

2. Coverage Thresholds

The workflow checks:

• Minimum overall coverage: 88% (MIN_THRESHOLD)

Denominators are derived from docs/requirements.md at run time — they are never hardcoded here or in the workflow. Run bun run traces:coverage for the current per-type breakdown; any number written into this document is a snapshot that will drift.

Why this matters. The workflow used to divide by frozen literals (UR/39, IR/24, DR/48, JA/3, total 114) while requirements.md had grown past 200. It reported 158% coverage, so the 50% threshold was unreachable and the job could not fail regardless of how far coverage dropped. See The fix derives the denominators from requirements.md at run time.

Coverage is the intersection of traced and defined IDs: an ID that appears in a TRACES: comment but is not defined in requirements.md is reported as orphaned and does not count toward coverage. UT/IT test identifiers are a separate taxonomy and are excluded entirely.

The workflow fails and blocks merge if coverage drops below the threshold — or if it computes above 100%, which can only mean the gate is miscounting.

Ratchet policy

MIN_THRESHOLD only ever goes up. It is deliberately set a few points below the coverage actually achieved (88 against a real ~90%), so a genuine regression trips it. It previously sat at 50 while true coverage was 86%: nearly half the matrix could have rotted before CI objected. It was ratcheted 50 → 82 when that was found, and 82 → 88 once coverage had held above 88% for several releases.

When coverage rises durably, raise the threshold to just under the new figure. Never lower it to make a red build pass — add the missing TRACES comments instead. The same number lives in MIN_COVERAGE_PERCENT in scripts/extract-traces.ts (so bun run traces:coverage gates locally on the same bar); scripts/extract-traces.test.ts fails if the two drift apart.

2b. Dangling requirement IDs

bun run traces:validate

Every ID named by a TRACES: comment must be defined as a table row in docs/requirements.md. The extractor used to accept any well-formed ID silently, so a typo or a rename that missed a call site passed unnoticed — DR-189 and UT-188 were referenced from three source files, defined nowhere, for months.

This check spans all six ID types (UR/IR/DR/JA/UT/IT), unlike the coverage orphaned list above, which considers only the four requirement types so that UT/IT noise cannot bury a real typo in the ratio's reporting. The workflow step fails the build on any dangling ID and prints each offender with the files that reference it.

3. Modified File Checking

On pull requests, the workflow:

1. Detects all changed TypeScript/Svelte/Rust files
2. Warns if new/modified files lack TRACES comments
3. Suggests the TRACES format for missing comments

How to Add Traces to New Code

When you add new code or modify existing code, include TRACES comments:

TypeScript/Svelte Example

// TRACES: UR-005, UR-026 | DR-029
export function handlePlayback() {
  // Implementation...
}

Rust Example

/// TRACES: UR-005 | DR-001
pub fn player_state_changed(state: PlayerState) {
    // Implementation...
}

Test Example

// TRACES: UR-005 | DR-001 | UT-026, UT-027
#[cfg(test)]
mod tests {
    // Tests...
}

TRACES Format

TRACES: [UR-###, ...] | [IR-###, ...] | [DR-###, ...] | [JA-###, ...]

• UR-### - User Requirements (features users see)
• IR-### - Integration Requirements (API/platform integration)
• DR-### - Development Requirements (internal architecture)
• JA-### - Jellyfin API Requirements (Jellyfin API usage)

Examples:

• // TRACES: UR-005 - Single requirement
• // TRACES: UR-005, UR-026 - Multiple of same type
• // TRACES: UR-005 | DR-029 - Multiple types
• // TRACES: UR-005, UR-026 | DR-001, DR-029 | UT-001 - Complex

Workflow Behavior

On Push to Main Branch

1. ✅ Extracts all traces from code
2. ✅ Validates coverage is >= 88%
3. ✅ Generates full traceability report
4. ✅ Saves report as artifact

On Pull Request

1. ✅ Extracts all traces
2. ✅ Validates coverage >= 88%
3. ✅ Checks modified files for TRACES
4. ✅ Warns if new code lacks TRACES
5. ✅ Suggests proper format
6. ✅ Generates report artifact

Failure Scenarios

The workflow fails (blocks merge) if:

• Coverage drops below 88%
• A TRACES: comment names an ID docs/requirements.md does not define
• JSON extraction fails
• Invalid trace format

The workflow warns (but doesn't block) if:

• New files lack TRACES comments
• Coverage drops (but still above threshold)

Viewing Reports

In Gitea Actions UI

1. Go to Actions tab
2. Click the Traceability Validation workflow run
3. Download traceability-reports artifact
4. View:
   • traces-report.json - Raw trace data
   • docs/traceability.md - Formatted report

Locally

# Extract current traces
bun run traces:json | jq '.byType'

# Generate full report
bun run traces:markdown
cat docs/traceability.md

Coverage Goals

Current Status

Run bun run traces:coverage — it prints the live figure and exits non-zero below threshold. Numbers are deliberately not pinned here; the previous snapshot in this section (51%, 56/114) was stale by roughly 100 requirements and was what made the broken CI arithmetic look plausible for so long.

As of August 2026 overall coverage is ~90%.

Targets

• Short term (Sprint): Maintain ≥88% overall (the current ratchet)
• Medium term (Month): Hold above 90% and ratchet the gate to match
• Long term (Release): Reach 95% coverage with focus on:
  • IR requirements (API clients)
  • JA requirements (Jellyfin API endpoints)
  • Remaining UR/DR requirements

Improving Coverage

For Missing User Requirements (UR)

1. Review README.md for unimplemented features
2. Add TRACES to code that implements them
3. Focus on high-priority features (High/Medium priority)

For Missing Integration Requirements (IR)

1. Add TRACES to Jellyfin API client methods
2. Add TRACES to platform-specific backends (Android/Linux)
3. Link to corresponding Jellyfin API endpoints

For Missing Development Requirements (DR)

1. Add TRACES to UI components in src/lib/components/
2. Add TRACES to composables in src/lib/composables/
3. Add TRACES to player backend in src-tauri/src/player/

For Jellyfin API Requirements (JA)

1. Add TRACES to Jellyfin API wrapper methods
2. Document which endpoints map to which requirements
3. Link to Jellyfin API documentation

Example PR Checklist

When submitting a pull request:

• All new code has TRACES comments linking to requirements
• TRACES format is correct: // TRACES: UR-001 | DR-002
• Workflow passes (coverage ≥ 88%)
• No coverage regressions
• Artifact traceability report was generated

Troubleshooting

"Coverage below minimum threshold"

Problem: Workflow fails with coverage < 88%

Solution:

1. Run bun run traces:json locally
2. Check which requirements are traced
3. Add TRACES to untraced code sections
4. Re-run extraction to verify

"New files without TRACES"

Problem: Workflow warns about new files lacking TRACES

Solution:

1. Add TRACES comments to all new code
2. Format: // TRACES: UR-001 | DR-002
3. Map code to specific requirements from README.md
4. Re-push

"Invalid JSON format"

Problem: Trace extraction produces invalid JSON

Solution:

1. Check for malformed TRACES comments
2. Run locally: bun run traces:json
3. Look for parsing errors
4. Fix and retry

Integration with Development

Before Committing

# Check your traces
bun run traces:json | jq '.byType'

# Regenerate report
bun run traces:markdown

# Verify traces syntax
grep "TRACES:" src/**/*.ts src/**/*.rs

In Your IDE

Add a file watcher to regenerate traces on save:

{
  "fileWatcher.watchPatterns": [
    "src/**/*.ts",
    "src/**/*.svelte",
    "src-tauri/src/**/*.rs"
  ],
  "fileWatcher.command": "bun run traces:markdown"
}

Git Hooks

Add a pre-push hook to validate traces:

#!/bin/bash
# .git/hooks/pre-push
bun run traces:json > /dev/null
if [ $? -ne 0 ]; then
  echo "❌ Invalid TRACES format"
  exit 1
fi

References

• Extract Traces Script
• Requirements Specification
• Traceability Matrix
• Gitea Actions Documentation

Support

For issues or questions:

1. Check this document
2. Review example traces in src/lib/stores/
3. Check existing TRACES comments for format
4. Review workflow logs in Gitea Actions

---

Last Updated: 2026-02-13