JRay-public-server/README.md

# JRay public server

A community manifest exchange for JRay. Jellyfin servers running the JRay plugin
pull actor-timeline manifests ("Jmanifests") for titles they own instead of
running the CV pipeline locally, and optionally contribute the manifests they
generate back.

See [SPEC.md](SPEC.md) for the design. Section references throughout the code
point at it. If you are considering **running an instance**, read
[`docs/legal-posture.md`](docs/legal-posture.md) first: it states what an
instance holds, what it structurally cannot do, and the operator checklist.

## Licensing — two licences, do not conflate them

| | Licence | Where |
|---|---|---|
| **Code** | `GPL-3.0-or-later` | [`LICENSE`](LICENSE), declared in `Cargo.toml` |
| **Contributed manifests** | **CC0 1.0 Universal** | [`LICENSE-DATA`](LICENSE-DATA), specified in [SPEC.md](SPEC.md) §5b |

Both files ship with any distribution. They answer different questions, and a
package carrying only `LICENSE` leaves the one that matters for federation
unanswered.

The data licence is the one that matters operationally, because manifests are
what replicate between instances (§9a) — a code licence grants nothing over them.
`POST /tokens` returns the licence and its terms with every issued token, so the
grant is one contributors actually make rather than one the server announces.

The grant covers **the manifest only** — timings, identifiers, audio signature.
It does not, and cannot, license the underlying work.

The community instance is **`https://jray.tourolle.paris`**. The JRay plugin
ships with it pre-configured but **disabled** — §9 requires that no traffic leave
an installation until an admin opts in, so the default entry exists to save the
admin from typing a URL, not to enable sharing on their behalf. Set
`JRAY_SERVER_ID=jray.tourolle.paris` when deploying that instance: it becomes the
`origin` stamped on manifests it first accepts (§9a) and the salt for report IP
hashes.

## Status

First implementation pass: the **core vertical slice**, reconciled against the
[system spec](../SPEC.md).

Per-requirement status is in [`docs/requirements.md`](docs/requirements.md) —
32 requirements (`UR-001..018`, `DR-001..014`), each tracing up to an `SR-nnn`
or `PR-nnn`. `UR-015..018` are the pending SR-003 schema bump and are marked
`Planned` rather than omitted.

Implemented:

- §2 Jmanifest format and series bundles
- §3 cut matching — `exact` / `runtime` / `loose` tiers
- §4 the API surface, less `POST /manifests/search`
- §5 rate limiting, in-process fixed-window counters
- §5a trust model — anonymous bearer tokens, closed-vocabulary storage,
  automatic revocation
- §6 upload validation, all four stages
- §7 relational storage, no JSON blobs on the write path
- §8 Rust + Axum + SQLite, single serialized writer, in-process job queue
- §9a content addressing (`content_id`), computed on upload
- §9a federation — change feed, fetch by `content_id`, batch `have`, peer
  directory, capabilities, and a pull worker that **re-derives judgement rather
  than inheriting it**: every pulled manifest runs the full §6 validation and
  this server's own cast check, and the body is verified to hash to the
  `content_id` requested before it is stored

**Schema version 2** (SR-003). `jmanifest_version` moved to 2 in lockstep with
the truth file's `schema_version` — breaking changes are batched and ship
together across all three repos. It is a **flag day**: version 1 is rejected
outright rather than carried alongside, because a v1 read path would be the one
nobody exercises and so the one that rots.

Reconciled with the system spec (see `docs/requirements.md` for the detail):

- **`anneal_sec` removed.** Withdrawn upstream by AR-012/AR-013 — presence now
  follows track extent, so a track survives its own gaps and there is nothing to
  anneal. Its successor `extinction_sec` and the new `gallery_scope` are
  accepted, stored, and (for scope) used in §7 ranking. A manifest still
  carrying `anneal_sec` is now a hard `400`, not silently ignored: it was
  produced by a pipeline whose window semantics differ from what this server
  assumes.
- **Windows carry belief and route.** `scenes` are objects rather than float
  pairs: `{ "start", "end", "belief", "route" }`, where belief is the posterior
  that justified the claim and route is `live`/`deferred`/`pooled`. Both are
  **excluded from `content_id`** — belief is a producer-side estimate that may
  differ between pipeline versions for identical timings, so hashing it would
  give two servers different ids for the same content. `src/content_id.rs` is
  unchanged by the bump and its golden vector still passes, which is the
  evidence rather than the claim.
- **Audio signature: the 120 s rule now matches both producers.** An earlier
  draft of §3 allowed a shortened window for items under 150 s; that conflicted
  with `scene-actor-extraction` IR-007 and was the weaker rule, since a
  caller-varying length is the property SR-004 forbids. Items under 120 s now
  send no signature at all.

Deferred:

- §3 audio signatures — the field is **accepted, validated and stored**, and
  `content_id` already excludes it, but `audio`-tier matching and
  `POST /manifests/search` are not wired up. This follows §3's own recommended
  sequencing: ship the plugin-side computation first, let signatures accumulate,
  then enable matching once coverage is useful.
(Federation landed — see below.)

## Running

```sh
cargo run
```

Configuration is entirely environment variables:

| Variable | Default | Purpose |
|---|---|---|
| `JRAY_BIND` | `127.0.0.1:8080` | Listen address. Terminate TLS at the operator's proxy (§8) |
| `JRAY_DB` | `jray.db` | SQLite path. WAL mode, created on first run |
| `JRAY_TMDB_API_KEY` | — | **Hard dependency for UR-3.** Without it, uploads stay `pending` and are never listed |
| `JRAY_TMDB_BASE_URL` | `https://api.themoviedb.org/3` | Override for testing |
| `JRAY_TRUSTED_PROXIES` | — | Comma-separated proxy IPs whose `X-Forwarded-For` is honoured. **Not default-on**: §5 rate limiting and report attribution key on client IP, so a spoofable header defeats both |
| `JRAY_SERVER_ID` | `localhost` | This server's identity, used as manifest `origin` and as the report IP-hash salt |
| `JRAY_REQUEST_TIMEOUT_SEC` | `30` | Request timeout so a slow bundle query fails fast |
| `JRAY_JOB_BATCH` | `8` | Cast-check jobs leased per worker tick |
| `JRAY_JOB_POLL_SEC` | `5` | Worker poll interval |
| `JRAY_LOG` | `info` | `tracing` filter |

Contributing requires a token (§5a) — an anonymous bearer capability, not an
account. Self-issue one:

```sh
curl -sX POST -H 'content-type: application/json' -d '{}' \
  http://127.0.0.1:8080/api/v1/tokens
```

## Deployment

### Install from the apt repository

```sh
sudo install -d -m0755 /etc/apt/keyrings
curl -fsSL https://gitea.tourolle.paris/api/packages/dtourolle/debian/repository.key \
  | sudo gpg --dearmor -o /etc/apt/keyrings/gitea-dtourolle.gpg
echo "deb [signed-by=/etc/apt/keyrings/gitea-dtourolle.gpg] \
https://gitea.tourolle.paris/api/packages/dtourolle/debian stable main" \
  | sudo tee /etc/apt/sources.list.d/jray-server.list
sudo apt update && sudo apt install jray-server
```

The install **asks** for the settings that matter — public hostname, listen
address, trusted proxies, TMDB key, contact — because two of them fail silently
when unset rather than loudly (see §8, DR-016). Change them later with:

```sh
sudo dpkg-reconfigure jray-server
```

The package ships one static binary, the systemd unit, and an example nginx
site at `/usr/share/doc/jray-server/examples/nginx-jray-server.conf`. The
example is **not** installed into nginx: the proxy usually runs on a different
host, so copy it to whichever machine that is.

Prefer not to add a third-party apt source? Every tagged release also carries
the `.deb` as an asset — `sudo dpkg -i jray-server_*.deb` does the same thing,
minus upgrades.

### Notes

§8's deployment notes are requirements, not suggestions:

- Enforce the body cap at **both** the proxy and the app. `client_max_body_size`
  (nginx) / `request_body max_size` (Caddy) should match §6 stage 1, so oversized
  uploads are dropped at the edge and never occupy an application worker. The app
  must also be safe when run without a proxy, which it is.
- Set `JRAY_TRUSTED_PROXIES` to the proxy's address, or `X-Forwarded-For` is
  ignored and every client behind it shares one rate-limit bucket.
- **Back up the SQLite file with `VACUUM INTO` or the backup API** — never a
  plain file copy of a live WAL database. Manifests represent real CV compute.

## Tests

```sh
cargo test                      # 212 tests
cargo deny check                # advisories, licences, bans, sources
scripts/traceability-gate.sh    # requirement coverage
```

The traceability gate needs the shared tooling submodule:

```sh
git submodule update --init --recursive
```

It reports coverage against [`docs/requirements.md`](docs/requirements.md),
flags orphan tags (an ID no register defines), and fails on a >100% ratio — the
signal that the computation itself is broken. Currently **25/32 (78%)**; the
untraced remainder is UR-007, which is plugin-side.

Unit tests per module, plus two integration suites:

- `tests/api.rs` — end-to-end through the real router: status codes, headers, and
  the properties that only hold if the layers compose correctly (per-route body
  caps, rate-limit surfaces, the strict schema actually reaching uploads).
- `tests/injection.rs` — that hostile input cannot escape its layer: SQL payloads
  in query parameters, path segments, JSON bodies, bearer tokens and report notes;
  JSON structure abuse; CRLF header injection; path traversal; and Unicode tricks
  against the §5a character class.

Security-relevant properties are asserted rather than assumed —
`movie`/`jellyfin_id` rejection, the §5a character class defeating base64/hex
smuggling, per-route body caps, a lying `Content-Length` not bypassing the cap,
and a forged `X-Forwarded-For` not resetting a rate-limit budget.

**On injection specifically.** Two independent defences apply, and they fail
differently, so both are tested:

1. **Parameterised queries.** Every value reaches SQLite through `params![]`. The
   only `format!`-built SQL interpolates two compile-time constants (a column list
   and a status literal) — no runtime input ever becomes SQL syntax. This is what
   actually prevents injection.
2. **Closed-vocabulary validation.** Identifiers are regex-constrained and free
   text is limited to a closed character class, so most payloads never reach the
   query layer at all.

The injection suite would still pass on defence 1 alone, which is deliberate: if
validation were ever loosened, the tests should not silently start depending on it.

Writing that suite found two real gaps, both since fixed:

- Compatibility homoglyphs (`𝐒𝐭𝐞𝐯𝐞`, `Ａｃｔｏｒ`) passed the §5a class. They are
  letters by Unicode category and NFC does not fold them — only NFKC would. Beyond
  name spoofing, a fullwidth-digit alphabet would have reopened the encoding
  channel the "no digits" rule exists to close.
- A one-frame `audio_signature` was accepted on a feature-length manifest, making
  the field the variable-length container §3 explicitly forbids. The length floor
  is now derived from the declared runtime, keeping §3's genuine short-item
  exception without trusting the client's length.

Two tests worth knowing about:

- `content_id::tests::golden_vector_hash_is_stable` locks the §9a canonical form.
  **The JRay plugin must reproduce it byte-identically**; §8 notes the extraction
  side is Python, so this can no longer be one shared implementation and must be
  cross-tested instead. `GOLDEN_VECTORS` is that fixture, and its hash was
  verified against an independent Python implementation.
- The integration tests use an on-disk temporary database, not `:memory:`,
  because §8's topology is one writer connection plus a read pool — in-memory
  SQLite is per-connection, so the readers would see an empty database.

## CI and container image

`.gitea/workflows/ci.yml` runs on Gitea Actions (and unmodified on GitHub
Actions), gated fastest-first: `fmt` → `clippy` → `test` → `cargo deny` → static
musl build. The dependency audit also runs weekly, since advisories appear without
any code changing.

The `Dockerfile` is the optional convenience, not the intended deployment path —
§8 is explicit that neither Docker nor Compose should be *required*. It builds
against musl and runs from `scratch` as uid 65534: **8.7 MB**, no libc, no shell,
no package manager. `rusqlite` bundles SQLite and `reqwest` uses rustls, so there
is nothing left to link.

```sh
docker build -t jray-server .
docker run --rm -p 8080:8080 -v jray-data:/data -e JRAY_TMDB_API_KEY=... jray-server
```

Two things that are easy to get wrong, so they are handled explicitly:

- **Static linkage is asserted with `file`, not `ldd`.** The musl target produces
  a *static-PIE*, and `ldd` prints the musl loader for one — an `ldd`-based check
  reports a perfectly static binary as dynamic.
- **`/data` is staged with the runtime uid's ownership.** Docker seeds a named
  volume from the image directory, ownership included, so a non-root server can
  create the database on first run. A **bind mount is not seeded this way** — the
  host directory keeps its own ownership, so `chown 65534:65534` it first or the
  server exits with "unable to open database file".

Two tests are worth knowing about:

- `content_id::tests::golden_vector_hash_is_stable` locks the §9a canonical form.
  **The JRay plugin must reproduce it byte-identically**; §8 notes the extraction
  side is Python, so this can no longer be one shared implementation and must be
  cross-tested instead. `GOLDEN_VECTORS` is that fixture, and its hash was
  verified against an independent Python implementation.
- The integration tests use an on-disk temporary database, not `:memory:`,
  because §8's topology is one writer connection plus a read pool — in-memory
  SQLite is per-connection, so the readers would see an empty database.

## Known gaps

- **The §6 stage-3 thresholds are still §10's guesses.** §10 (5) is explicit that
  running the check over the 331 real corpus files would give the true
  distribution of honest-upload match ratios, and is "the single cheapest way to
  de-risk UR-3 and UR-5". The scoring logic is deliberately pure functions in
  `castcheck.rs` so that retuning is a test-data exercise, not a code change.
- `POST /manifests/{id}/report` records reports but nothing consumes them yet.
  §5a's divergence detection and the operator kill switch are not implemented;
  delisting is currently a manual `UPDATE`, which §5a does note is the intended
  shape ("one UPDATE", not a moderation queue).
- No admin surface. Revocation is automatic (§5a), but an operator has no
  endpoint for the deliberate kill-switch case.