dtourolle/JRay-public-server
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Initial implementation: core vertical slice
CI / fmt, clippy, test (push) Failing after 2m46s
Details
CI / static musl binary (push) Has been skipped
Details
CI / advisories and licences (push) Successful in 4m22s
Details

Browse Source 
Implements the core of SPEC.md — the manifest exchange, less audio-tier
matching (§3) and federation (§9a), both of which the spec sequences as
later work.

- §2 Jmanifest format and series bundles
- §3 cut matching: exact / runtime / loose tiers
- §4 API, less POST /manifests/search
- §5 rate limiting; §5a trust model, anonymous bearer tokens
- §6 upload validation, all four stages
- §7 relational storage, no JSON blob on the write path
- §8 Rust + Axum + SQLite, single serialized writer, in-process job queue
- §9a content addressing, computed on upload

Reconciled against the system spec:

- anneal_sec removed, withdrawn upstream by AR-012/AR-013. Presence follows
  track extent, so a track survives its own gaps and there is nothing to
  anneal. Its successor extinction_sec and the new gallery_scope are accepted
  and stored; scope enters the §7 ranking. A manifest still carrying
  anneal_sec is a hard 400, not silently ignored — it came from a pipeline
  whose window semantics differ from what this server assumes.
- Audio signature: media under 120 s now emits no signature at all, matching
  scene-actor-extraction IR-007. The earlier §3 draft allowed a shortened
  window under 150 s, which was the weaker rule — a caller-varying length is
  the property SR-004 forbids.
- UR IDs regularised to UR-nnn; docs/requirements.md registers 32
  requirements, each tracing to an SR-nnn or PR-nnn.

189 tests: unit, end-to-end through the real router, and an injection suite
covering SQL, JSON, header and Unicode payloads. Writing that suite found two
real gaps, both fixed here: compatibility homoglyphs passed the §5a character
class, and a one-frame audio signature was accepted on a feature-length item.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
dtourolle
2026-07-30 18:14:02 +02:00
co-authored by Claude Opus 5
commit a848750a65
38 changed files with 13014 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/requirements.md
+200
View File
@@ -0,0 +1,200 @@
# JRay-public-server — requirements register
Stable IDs for every requirement in [`../SPEC.md`](../SPEC.md), which holds the
prose. This file is the **authoritative list**; the CI gate reads its
denominators from here (see the [system spec](../../SPEC.md) §6).
**IDs are permanent.** A withdrawn requirement is marked `Withdrawn` and its
number is never reused — renumbering is what produces orphan TRACES tags.
Tag code with `// TRACES: UR-003 | SR-004`.
| Type | Scope |
|---|---|
| `UR` | User/functional — what the server does |
| `DR` | Development — how it is built and operated |
| `UT` / `IT` | Unit / integration tests |
Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
A requirement is `Done` only when it is implemented **and** has a test that
executes. Everything below runs in CI on any machine — this repo has no GPU
requirement and no fixture-generation step, unlike `scene-actor-extraction`.
---
## User requirements (UR)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| UR-001 | Cheap existence probe, separate from the fetch, returning availability and cut-match tier without payload | SR-001 | High | Done |
| UR-002 | Accept a contributed manifest for a media item | PR-006 | High | Done |
| UR-003 | Content verification: strict schema, size caps, approximate TMDB cast match | SR-004 | High | Done |
| UR-004 | Rate limiting, per token where present and per source IP otherwise | SR-004 | High | Done |
| UR-005 | Trust without accounts: not usable as a content store, nor for prank manifests | SR-004 | High | Done |
| UR-006 | Serve and accept a whole series in one operation | PR-006 | High | Done |
| UR-007 | Plugin queries an ordered, configurable list of servers | PR-005 | High | In Progress |
| UR-008 | Servers replicate manifests between each other | PR-006 | Medium | Planned |
| UR-009 | Store an audio spectral-peak signature for content-based identification | SR-003 | Medium | In Progress |
| UR-010 | Identity crossing the API boundary is TMDB/IMDB ids, never a name alone | SR-001 | High | Done |
| UR-011 | Reject any field capable of carrying binary or attacker-chosen content | SR-004 | High | Done |
| UR-012 | Never accept, store, or serve gallery data — reference faces or embeddings | SR-005 | High | Done |
| UR-013 | Windows are scene-scoped claims; never reinterpret their boundaries | SR-002 | High | Done |
| UR-014 | Reject an unknown `jmanifest_version` outright, never guess | SR-003 | High | Done |
### Notes on status
**UR-007 is `In Progress`, not `Done`.** The plugin now carries the ordered
server list and its per-server trust settings, with the community instance
pre-configured but disabled. The fetch path that consumes it does not exist yet.
**UR-009 is `In Progress`.** The server accepts, validates and stores
`cut.audio_signature`, and `content_id` correctly excludes it (§9a). What is
absent is `audio`-tier matching and `POST /manifests/search`. This is the
sequencing §3 recommends — accumulate signatures first, enable matching once
coverage is useful — not an oversight.
**UR-012 is satisfied structurally, by absence.** There is no field in the
Jmanifest capable of carrying an embedding or a crop, and no endpoint that would
accept one. Like PR-005 in the system spec, it cannot be verified by pointing at
code that does something; UT-024 verifies it by asserting that the obvious
attempts are rejected.
---
## Development requirements (DR)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| DR-001 | Strict parse boundary: unknown fields rejected structurally, not by validator code | SR-004 | High | Done |
| DR-002 | Fully relational storage — no JSON blob on the write path | SR-004 | High | Done |
| DR-003 | Single serialized writer connection, with a read pool alongside | PR-004 | High | Done |
| DR-004 | All database access behind a repository layer, not scattered through handlers | PR-004 | Medium | Done |
| DR-005 | Background work in-process, with the job queue as a table so it survives restart | PR-004 | High | Done |
| DR-006 | Rate-limit counters in process memory; no external counter store | PR-004 | Medium | Done |
| DR-007 | Ship a single static binary plus one database file; container optional | PR-004 | High | Done |
| DR-008 | `X-Forwarded-For` honoured only from explicitly configured proxies | SR-004 | High | Done |
| DR-009 | Body caps enforced while streaming, before parsing, per route | SR-004 | High | Done |
| DR-010 | Request bodies are UTF-8 only, rejected with a diagnosable error otherwise | SR-003 | Medium | Done |
| DR-011 | `content_id` canonical form is byte-stable and cross-implementation tested | SR-003 | High | Done |
| DR-012 | Dependency audit: advisories, licence policy, source policy | PR-004 | Medium | Done |
| DR-013 | API errors use the status codes the spec names, not the framework's defaults | SR-003 | Medium | Done |
| DR-014 | Portable SQL — no SQLite-specific form where a standard one exists | PR-004 | Medium | Done |
---
## Verification
**No GPU, no fixtures, no external services.** Every test here runs on any
machine in under three seconds. The TMDB dependency is the only external service,
and it is absent from tests by construction: an unconfigured client makes uploads
stay `pending`, which is the correct production failure mode (§8) and happens to
make the test suite hermetic.
| Tier | Runs in CI | What it covers |
|---|---|---|
| **T1 — unit** | Yes | Pure logic: validation, cut matching, cast-check scoring, canonicalisation, rate limiting |
| **T2 — integration** | Yes | End-to-end through the real router against a temporary on-disk database |
There is no tier that does not run. A requirement here is either verified or
visibly not.
**Integration tests use an on-disk temporary database, not `:memory:`.** DR-003
specifies one writer connection plus a read pool, and in-memory SQLite is
per-connection — the readers would see an empty database. Testing the real
topology is the point, so this is a deliberate choice rather than an oversight.
### Per-requirement verification
| ID | Tier | Test asserts | Edge cases covered |
|---|---|---|---|
| UR-001 | T2 | `exists` reports availability and tier without payload | Absent title returns `200` with `false`, not `404`; batch form is positional; one bad item does not fail the batch |
| UR-002 | T2 | Valid upload accepted as `202 pending` | Duplicate content deduplicates; same contributor resubmitting the same cut is `409` |
| UR-003 | T1 + T2 | Strict schema, caps, and cast-match thresholds | Ratio boundaries at 0.6 and 0.3 exactly; small-\|M\| all-but-one rule; missing TMDB credits flags rather than rejects |
| UR-004 | T1 + T2 | Limits engage and carry the documented headers | Window reset; a rejected request does not extend its own lockout; surfaces have independent budgets |
| UR-005 | T1 + T2 | Prank manifests rejected; no free-text channel | Uncredited cast rejected; name-only matches capped; automatic revocation needs a minimum sample |
| UR-006 | T2 | Bundle accepted per-episode, non-atomically | One bad episode rejected while its neighbours are accepted; envelope errors are whole-request `400` |
| UR-007 | — | *No server-side test.* Plugin-side; the register there will carry it | — |
| UR-009 | T1 | Signature structurally validated | Fixed length; reserved high bit; **media < 120 s must send no signature at all** |
| UR-010 | T1 + T2 | Actors persist as TMDB person ids | A name the upload invented does not round-trip |
| UR-011 | T2 | Every payload-shaped field rejected | base64, hex, markup, control characters, bidi overrides, compatibility homoglyphs |
| UR-012 | T2 | No endpoint accepts embeddings or image data | An `embedding` or `crop` field is an unknown-field `400` |
| UR-013 | T1 | Stored windows are byte-identical to those submitted | Adjacent windows never merged; a window is never trimmed to a shorter one |
| UR-014 | T1 | Unknown `jmanifest_version` rejected | Version `2` and version `0` both refused, naming the field |
| DR-001 | T1 | Unknown field at any nesting depth fails to parse | `movie` and `jellyfin_id` named in the error |
| DR-003 | T1 | Concurrent writes serialize rather than returning `SQLITE_BUSY` | Failed transaction rolls back fully |
| DR-005 | T1 | Jobs lease once, reschedule with backoff, survive restart | Stranded lease released at startup; future job not leased early |
| DR-008 | T2 | Forged `X-Forwarded-For` cannot mint a fresh budget | Untrusted peer ignored; trusted proxy honoured; client-supplied entries to the left cannot spoof |
| DR-009 | T2 | Oversized body rejected as `413` | **A lying `Content-Length` does not bypass the cap**; per-route limits differ |
| DR-010 | T1 | Non-UTF-8 rejected by name | UTF-16 with and without BOM; UTF-8 BOM; declared `charset=utf-16` |
| DR-011 | T1 | Canonical form is stable and order-independent | Accumulated float error hashes identically; `audio_signature` and `extraction` excluded; **golden vector verified against an independent Python implementation** |
| DR-013 | T1 | Schema mismatch is `400`, not the framework's `422` | §4 names `400` for a forbidden field, and a client checking for it would mishandle `422` |
Three are worth singling out, because each verifies a claim that would otherwise
be an assertion:
- **DR-009's lying-`Content-Length` case.** §6 stage 0 is explicit that the
header is a claim by the client, so the streaming cap is mandatory rather than
redundant. A test that only sends honest bodies verifies nothing.
- **DR-011's golden vector.** Two servers that validated the same upload must
reach the same `content_id`, and the plugin must reproduce it byte-identically
from a different language. The fixture is the only thing that can catch
divergence before it silently breaks federation deduplication.
- **UR-011's homoglyph case.** Writing this test found a real gap: compatibility
variants (`𝐒𝐭𝐞𝐯𝐞`, `Ａｃｔｏｒ`) are letters by Unicode category and NFC does not
fold them, so a fullwidth-digit alphabet would have reopened the encoding
channel §5a's "no digits" rule closes.
---
## Withdrawn
| ID | Requirement | Reason |
|---|---|---|
| — | `anneal_sec` in `extraction` | Withdrawn upstream (`scene-actor-extraction` AR-012/AR-013): presence follows track extent, so a track survives its own gaps and there is nothing to anneal. Ships as part of the SR-003 bump |
Deleted rather than retained at zero: a field naming a mechanism the pipeline no
longer has is actively misleading to anyone reading a manifest, and would outlive
everyone who remembers why it is zero.
No `UR`/`DR` number was ever assigned to it — it was a *field*, not a
requirement — so nothing is orphaned by its removal.
---
## Pending — the SR-003 schema bump
These are `Planned` rather than absent, because the bump is coordinated across
three repos and this register should show the work rather than imply the server
is finished.
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| UR-015 | Accept `extraction.extinction_sec` in place of `anneal_sec` | SR-003 | High | Planned |
| UR-016 | Accept and store `extraction.gallery_scope`; rank on it (§7) | SR-003 | Medium | Planned |
| UR-017 | Accept per-window belief and identification route; `scenes` becomes objects | SR-003 | High | Planned |
| UR-018 | Exclude belief from `content_id`, replicating it as an attribute | SR-003 | High | Planned |
**UR-018 is the one with a trap in it.** Belief is a producer-side estimate that
may legitimately differ between pipeline versions for identical timings, so
including it in the canonical form would give two servers different `content_id`s
for the same content — the exact failure mode §9a quantises centiseconds to
avoid. It follows `audio_signature`'s precedent: replicated as an attribute, not
part of identity.
---
## Notes on coverage
- **`DR-*` traces to `PR-004` (self-hosted) more often than to an `SR-nnn`.**
Operational simplicity is a single-repo concern serving the project goal
directly. This is correct rather than a gap: §8's whole argument for one binary
and one file is that federation only works if running an instance is easy.
- **UR-007 has no server-side test** and cannot have one — it is a requirement on
the plugin, recorded here because this spec is where it is stated. It should be
cross-referenced from the plugin's register when that is created, and until
then it is visibly unverified rather than quietly assumed.
- **UR-012 and UR-013 are preserved by prohibition**, like PR-005 in the system
spec. They cannot be verified by pointing at code that does something, only by
asserting that the attempts fail. They die the moment either prohibition is
relaxed, which is precisely why they are stated rather than left implicit.