From 9d9abb227ff5e88f35b7792d7af5d0cbd775ee8b Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Sun, 30 Aug 2026 10:17:34 +0200 Subject: [PATCH 01/13] Ask where a TRACES tag sits, so the tool stops tagging its own fixtures MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The traceability tool scans `tools/`, which is its own source, and a line was taken for a tag whenever `TRACES:` appeared anywhere on it. Its unit-test fixtures are therefore tags. R1 — cross-platform output within a bounded tolerance, the requirement with no acceptance criterion at all — was reported implemented on the strength of two string literals in `context_looks_forward_then_backward`. R1 was only the visible case because it had no other coverage. The same fixtures also contributed sites to FR-CAT-1, FR-CAT-2 and NFR-P1, `gestures.rs` contributed one to FR-UI-4 from a `push_str`, and `dr-pipeline/build.rs` contributed FR-DEV-3a and FR-DEV-3c from the tag it *emits* into generated code. Those four requirements keep real tags elsewhere, so nothing but noise is lost by dropping them. The rule is about position, not about string literals. It cannot be about string literals: `schema.rs` writes six genuine tags inside Rust string literals, because the SQL it embeds is commented with `--`, and an extractor that refused those would lose more than it saved. What separates the two is where on the line the tag is. A tag written to be read is the first word of its comment; a tag quoted inside an expression never is. So `tag_body` asks for a comment opener at the start of the line and `TRACES:` immediately after it. That closes every shape but one: a multi-line literal whose lines really do begin with `///`, which no line-oriented reader can tell from source. There is one such fixture and its ids are now UT and IT, which `is_requirement` already excludes from coverage — the mechanism existed and was simply never used on the tool itself. `this_crates_own_fixtures_cannot_reach_the_register` enforces that: any requirement id below `mod tests` in this crate fails the test and says to use a UT- or IT- id instead. Tagging the tool's real code is still allowed. On `SOURCE_SUFFIXES`, which cannot reach `AndroidManifest.xml`, the Flatpak manifest, the Dockerfile or the CI workflows: it is deliberately left alone, and the reasoning is recorded beside it. A tag on a manifest asserts that a comment exists next to a line nothing checks, which is the weak form CONTRIBUTING.md warns about. The convention already in the tree — a Rust test that `include_str!`s the file and asserts what must be in it, with the tag on the test — is what a tag is supposed to mean. Co-Authored-By: Claude Opus 5 (1M context) --- docs/outstanding.md | 9 ++- tools/traceability/src/lib.rs | 146 ++++++++++++++++++++++++++++++++-- 2 files changed, 145 insertions(+), 10 deletions(-) diff --git a/docs/outstanding.md b/docs/outstanding.md index 1bc0473..b6addb2 100644 --- a/docs/outstanding.md +++ b/docs/outstanding.md @@ -320,9 +320,12 @@ fixed before spike S9", because S9 both validates R1 and calibrates what toleran The threshold was never fixed and S9 has not run, so R1 currently has no acceptance criterion at all — there is nothing a test could assert. -Worse, the matrix reports R1 as *covered*. Both of its tags are string literals inside the -traceability tool's own unit tests (`tools/traceability/src/lib.rs`), which the tool scans along with -everything else, because a fixture demonstrating tag extraction is indistinguishable from a tag. +The matrix used to report R1 as *covered*, and what covered it was two string literals: fixtures +inside the traceability tool's own unit tests, which the tool scans along with everything else, +because a fixture demonstrating tag extraction was indistinguishable from a tag. The extractor now +asks where the tag sits — a tag is the first word of a comment, not a string appearing anywhere on a +line — and R1 is untagged again, which is the honest reading while it has no acceptance criterion to +tag anything against. NFR-OPS-1 is covered the same way, from a tag on `compute_coverage` — and no rotating, size-capped on-disk log exists; logging goes to stderr and logcat. These are two of the cases [CONTRIBUTING.md](../CONTRIBUTING.md) already warns about, now named. diff --git a/tools/traceability/src/lib.rs b/tools/traceability/src/lib.rs index bbc8e29..b3ee4da 100644 --- a/tools/traceability/src/lib.rs +++ b/tools/traceability/src/lib.rs @@ -19,6 +19,20 @@ //! count as the numerator is precisely what lets a ratio exceed 100%, since //! a tag naming a deleted requirement would count as covered. Such tags are //! reported as orphans instead. +//! +//! # And why the extractor is fussy about where a tag sits +//! +//! A third rule, learned the same way. `SOURCE_ROOTS` includes `tools`, so this +//! crate scans itself, and the fixtures below demonstrating tag extraction were +//! read as tags: R1 — cross-platform output within a bounded tolerance — was +//! reported implemented on the strength of two string literals in a unit test. +//! +//! 3. **A tag is a comment whose first word is `TRACES:`**, not a line in which +//! the string appears. This is a rule about position +//! rather than about string literals, because `schema.rs` writes real tags +//! inside string literals — the SQL it embeds is commented with `--` — and +//! an extractor that refused those would lose six genuine tags to save two +//! false ones. use std::collections::{BTreeMap, BTreeSet}; use std::path::{Path, PathBuf}; @@ -182,16 +196,43 @@ pub fn is_requirement(id: &str) -> bool { REQUIREMENT_TYPES.contains(&type_of(id)) } +/// Comment openers a tag may be introduced by. +/// +/// `--` is here for the SQL embedded in `schema.rs`, which carries real tags +/// inside Rust string literals; `#` for YAML; `*` for the continuation lines of +/// a block comment. +const COMMENT_OPENERS: &[&str] = &["///", "//!", "//", "