Initial implementation: core vertical slice
CI / fmt, clippy, test (push) Failing after 2m46s
CI / static musl binary (push) Has been skipped
CI / advisories and licences (push) Successful in 4m22s

Implements the core of SPEC.md — the manifest exchange, less audio-tier
matching (§3) and federation (§9a), both of which the spec sequences as
later work.

- §2 Jmanifest format and series bundles
- §3 cut matching: exact / runtime / loose tiers
- §4 API, less POST /manifests/search
- §5 rate limiting; §5a trust model, anonymous bearer tokens
- §6 upload validation, all four stages
- §7 relational storage, no JSON blob on the write path
- §8 Rust + Axum + SQLite, single serialized writer, in-process job queue
- §9a content addressing, computed on upload

Reconciled against the system spec:

- anneal_sec removed, withdrawn upstream by AR-012/AR-013. Presence follows
  track extent, so a track survives its own gaps and there is nothing to
  anneal. Its successor extinction_sec and the new gallery_scope are accepted
  and stored; scope enters the §7 ranking. A manifest still carrying
  anneal_sec is a hard 400, not silently ignored — it came from a pipeline
  whose window semantics differ from what this server assumes.
- Audio signature: media under 120 s now emits no signature at all, matching
  scene-actor-extraction IR-007. The earlier §3 draft allowed a shortened
  window under 150 s, which was the weaker rule — a caller-varying length is
  the property SR-004 forbids.
- UR IDs regularised to UR-nnn; docs/requirements.md registers 32
  requirements, each tracing to an SR-nnn or PR-nnn.

189 tests: unit, end-to-end through the real router, and an injection suite
covering SQL, JSON, header and Unicode payloads. Writing that suite found two
real gaps, both fixed here: compatibility homoglyphs passed the §5a character
class, and a one-frame audio signature was accepted on a feature-length item.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-30 18:14:02 +02:00
co-authored by Claude Opus 5
commit a848750a65
38 changed files with 13014 additions and 0 deletions
+220
View File
@@ -0,0 +1,220 @@
# JRay public server
A community manifest exchange for JRay. Jellyfin servers running the JRay plugin
pull actor-timeline manifests ("Jmanifests") for titles they own instead of
running the CV pipeline locally, and optionally contribute the manifests they
generate back.
See [SPEC.md](SPEC.md) for the design. Section references throughout the code
point at it.
The community instance is **`https://jray.tourolle.paris`**. The JRay plugin
ships with it pre-configured but **disabled** — §9 requires that no traffic leave
an installation until an admin opts in, so the default entry exists to save the
admin from typing a URL, not to enable sharing on their behalf. Set
`JRAY_SERVER_ID=jray.tourolle.paris` when deploying that instance: it becomes the
`origin` stamped on manifests it first accepts (§9a) and the salt for report IP
hashes.
## Status
First implementation pass: the **core vertical slice**, reconciled against the
[system spec](../SPEC.md).
Per-requirement status is in [`docs/requirements.md`](docs/requirements.md) —
32 requirements (`UR-001..018`, `DR-001..014`), each tracing up to an `SR-nnn`
or `PR-nnn`. `UR-015..018` are the pending SR-003 schema bump and are marked
`Planned` rather than omitted.
Implemented:
- §2 Jmanifest format and series bundles
- §3 cut matching — `exact` / `runtime` / `loose` tiers
- §4 the API surface, less `POST /manifests/search`
- §5 rate limiting, in-process fixed-window counters
- §5a trust model — anonymous bearer tokens, closed-vocabulary storage,
automatic revocation
- §6 upload validation, all four stages
- §7 relational storage, no JSON blobs on the write path
- §8 Rust + Axum + SQLite, single serialized writer, in-process job queue
- §9a content addressing (`content_id`), computed on upload
Reconciled with the system spec (see `docs/requirements.md` for the detail):
- **`anneal_sec` removed.** Withdrawn upstream by AR-012/AR-013 — presence now
follows track extent, so a track survives its own gaps and there is nothing to
anneal. Its successor `extinction_sec` and the new `gallery_scope` are
accepted, stored, and (for scope) used in §7 ranking. A manifest still
carrying `anneal_sec` is now a hard `400`, not silently ignored: it was
produced by a pipeline whose window semantics differ from what this server
assumes.
- **Audio signature: the 120 s rule now matches both producers.** An earlier
draft of §3 allowed a shortened window for items under 150 s; that conflicted
with `scene-actor-extraction` IR-007 and was the weaker rule, since a
caller-varying length is the property SR-004 forbids. Items under 120 s now
send no signature at all.
Deferred:
- §3 audio signatures — the field is **accepted, validated and stored**, and
`content_id` already excludes it, but `audio`-tier matching and
`POST /manifests/search` are not wired up. This follows §3's own recommended
sequencing: ship the plugin-side computation first, let signatures accumulate,
then enable matching once coverage is useful.
- §9a federation endpoints (`/federation/*`) and the pull worker. The schema
columns (`content_id`, `origin`, `ingested_from`, `peers`) are in place, and
`ingest::persist` is already the shared path a pull would reuse.
## Running
```sh
cargo run
```
Configuration is entirely environment variables:
| Variable | Default | Purpose |
|---|---|---|
| `JRAY_BIND` | `127.0.0.1:8080` | Listen address. Terminate TLS at the operator's proxy (§8) |
| `JRAY_DB` | `jray.db` | SQLite path. WAL mode, created on first run |
| `JRAY_TMDB_API_KEY` | — | **Hard dependency for UR-3.** Without it, uploads stay `pending` and are never listed |
| `JRAY_TMDB_BASE_URL` | `https://api.themoviedb.org/3` | Override for testing |
| `JRAY_TRUSTED_PROXIES` | — | Comma-separated proxy IPs whose `X-Forwarded-For` is honoured. **Not default-on**: §5 rate limiting and report attribution key on client IP, so a spoofable header defeats both |
| `JRAY_SERVER_ID` | `localhost` | This server's identity, used as manifest `origin` and as the report IP-hash salt |
| `JRAY_REQUEST_TIMEOUT_SEC` | `30` | Request timeout so a slow bundle query fails fast |
| `JRAY_JOB_BATCH` | `8` | Cast-check jobs leased per worker tick |
| `JRAY_JOB_POLL_SEC` | `5` | Worker poll interval |
| `JRAY_LOG` | `info` | `tracing` filter |
Contributing requires a token (§5a) — an anonymous bearer capability, not an
account. Self-issue one:
```sh
curl -sX POST -H 'content-type: application/json' -d '{}' \
http://127.0.0.1:8080/api/v1/tokens
```
## Deployment
§8's deployment notes are requirements, not suggestions:
- Enforce the body cap at **both** the proxy and the app. `client_max_body_size`
(nginx) / `request_body max_size` (Caddy) should match §6 stage 1, so oversized
uploads are dropped at the edge and never occupy an application worker. The app
must also be safe when run without a proxy, which it is.
- Set `JRAY_TRUSTED_PROXIES` to the proxy's address, or `X-Forwarded-For` is
ignored and every client behind it shares one rate-limit bucket.
- **Back up the SQLite file with `VACUUM INTO` or the backup API** — never a
plain file copy of a live WAL database. Manifests represent real CV compute.
## Tests
```sh
cargo test # 178 tests
cargo deny check # advisories, licences, bans, sources
```
Unit tests per module, plus two integration suites:
- `tests/api.rs` — end-to-end through the real router: status codes, headers, and
the properties that only hold if the layers compose correctly (per-route body
caps, rate-limit surfaces, the strict schema actually reaching uploads).
- `tests/injection.rs` — that hostile input cannot escape its layer: SQL payloads
in query parameters, path segments, JSON bodies, bearer tokens and report notes;
JSON structure abuse; CRLF header injection; path traversal; and Unicode tricks
against the §5a character class.
Security-relevant properties are asserted rather than assumed —
`movie`/`jellyfin_id` rejection, the §5a character class defeating base64/hex
smuggling, per-route body caps, a lying `Content-Length` not bypassing the cap,
and a forged `X-Forwarded-For` not resetting a rate-limit budget.
**On injection specifically.** Two independent defences apply, and they fail
differently, so both are tested:
1. **Parameterised queries.** Every value reaches SQLite through `params![]`. The
only `format!`-built SQL interpolates two compile-time constants (a column list
and a status literal) — no runtime input ever becomes SQL syntax. This is what
actually prevents injection.
2. **Closed-vocabulary validation.** Identifiers are regex-constrained and free
text is limited to a closed character class, so most payloads never reach the
query layer at all.
The injection suite would still pass on defence 1 alone, which is deliberate: if
validation were ever loosened, the tests should not silently start depending on it.
Writing that suite found two real gaps, both since fixed:
- Compatibility homoglyphs (`𝐒𝐭𝐞𝐯𝐞`, `Actor`) passed the §5a class. They are
letters by Unicode category and NFC does not fold them — only NFKC would. Beyond
name spoofing, a fullwidth-digit alphabet would have reopened the encoding
channel the "no digits" rule exists to close.
- A one-frame `audio_signature` was accepted on a feature-length manifest, making
the field the variable-length container §3 explicitly forbids. The length floor
is now derived from the declared runtime, keeping §3's genuine short-item
exception without trusting the client's length.
Two tests worth knowing about:
- `content_id::tests::golden_vector_hash_is_stable` locks the §9a canonical form.
**The JRay plugin must reproduce it byte-identically**; §8 notes the extraction
side is Python, so this can no longer be one shared implementation and must be
cross-tested instead. `GOLDEN_VECTORS` is that fixture, and its hash was
verified against an independent Python implementation.
- The integration tests use an on-disk temporary database, not `:memory:`,
because §8's topology is one writer connection plus a read pool — in-memory
SQLite is per-connection, so the readers would see an empty database.
## CI and container image
`.gitea/workflows/ci.yml` runs on Gitea Actions (and unmodified on GitHub
Actions), gated fastest-first: `fmt` → `clippy` → `test` → `cargo deny` → static
musl build. The dependency audit also runs weekly, since advisories appear without
any code changing.
The `Dockerfile` is the optional convenience, not the intended deployment path —
§8 is explicit that neither Docker nor Compose should be *required*. It builds
against musl and runs from `scratch` as uid 65534: **8.7 MB**, no libc, no shell,
no package manager. `rusqlite` bundles SQLite and `reqwest` uses rustls, so there
is nothing left to link.
```sh
docker build -t jray-server .
docker run --rm -p 8080:8080 -v jray-data:/data -e JRAY_TMDB_API_KEY=... jray-server
```
Two things that are easy to get wrong, so they are handled explicitly:
- **Static linkage is asserted with `file`, not `ldd`.** The musl target produces
a *static-PIE*, and `ldd` prints the musl loader for one — an `ldd`-based check
reports a perfectly static binary as dynamic.
- **`/data` is staged with the runtime uid's ownership.** Docker seeds a named
volume from the image directory, ownership included, so a non-root server can
create the database on first run. A **bind mount is not seeded this way** — the
host directory keeps its own ownership, so `chown 65534:65534` it first or the
server exits with "unable to open database file".
Two tests are worth knowing about:
- `content_id::tests::golden_vector_hash_is_stable` locks the §9a canonical form.
**The JRay plugin must reproduce it byte-identically**; §8 notes the extraction
side is Python, so this can no longer be one shared implementation and must be
cross-tested instead. `GOLDEN_VECTORS` is that fixture, and its hash was
verified against an independent Python implementation.
- The integration tests use an on-disk temporary database, not `:memory:`,
because §8's topology is one writer connection plus a read pool — in-memory
SQLite is per-connection, so the readers would see an empty database.
## Known gaps
- **The §6 stage-3 thresholds are still §10's guesses.** §10 (5) is explicit that
running the check over the 331 real corpus files would give the true
distribution of honest-upload match ratios, and is "the single cheapest way to
de-risk UR-3 and UR-5". The scoring logic is deliberately pure functions in
`castcheck.rs` so that retuning is a test-data exercise, not a code change.
- `POST /manifests/{id}/report` records reports but nothing consumes them yet.
§5a's divergence detection and the operator kill switch are not implemented;
delisting is currently a manual `UPDATE`, which §5a does note is the intended
shape ("one UPDATE", not a moderation queue).
- No admin surface. Revocation is automatic (§5a), but an operator has no
endpoint for the deliberate kill-switch case.