jRay/SPEC.md

# JRay truth file format

JRay reads "truth" files produced offline by the
[scene-actor-extraction](https://github.com/dtourolle/scene-actor-extraction)
pipeline (`result_sink_node`, `Verbosity::minimal`, `schema_version: 1`).

## File location

For a media file `Movie.mkv`, the pipeline writes a sibling file
`Movie.jray.json` (suffix configurable in the plugin settings, default
`.jray.json`). The plugin resolves this path from the Jellyfin item's media
source path by stripping the extension and appending the suffix.

## JSON schema (schema_version 1, minimal verbosity)

```json
{
  "schema_version": 1,
  "movie": "/path/to/Movie.mkv",
  "sample_fps": 1,
  "anneal_sec": 2,
  "actors": [
    {
      "name": "Tom Hanks",
      "imdb_id": "nm0000158",
      "tmdb_id": "31",
      "jellyfin_id": "abc123-guid",
      "scenes": [[12.0, 45.0], [102.5, 150.0]]
    }
  ]
}
```

- `schema_version`: integer, bump on breaking changes. JRay should refuse (or
  warn) on a version it doesn't understand.
- `movie`: absolute path to the source media file at extraction time (informational only).
- `sample_fps`: frames-per-second the pipeline sampled at.
- `anneal_sec`: gap (in seconds) below which consecutive detections of the
  same actor were merged into a single scene window.
- `actors[]`: one entry per actor detected anywhere in the film.
  - `name`: display name from the gallery.
  - `imdb_id` / `tmdb_id` / `jellyfin_id`: identity keys, each `""` if not
    resolved. JRay should prefer `jellyfin_id` (a Jellyfin Person item GUID)
    when non-empty, and otherwise resolve `imdb_id`/`tmdb_id` against the
    item's People `ProviderIds`.
  - `scenes`: list of `[start_sec, end_sec]` windows (inclusive) during which
    the actor is on screen.

## Querying "who's on screen at time t"

For a given timestamp `t` (seconds), an actor is visible if any of their
`scenes` windows satisfies `start <= t <= end`.

## API

All read endpoints below require an authenticated Jellyfin user token (passed as
`X-Emby-Token` or `Authorization: MediaBrowser Token="..."`); the admin endpoints
additionally require the **Administrator** role. Only `GET /Plugins/JRay/ClientScript`
is anonymous.

### `GET /Plugins/JRay/Items/{itemId}/Timeline`

Returns the full truth file (schema above) for an item, or `404` if no truth
data exists (neither a managed upload nor a sidecar file). Requires an
authenticated user token.

### `GET /Plugins/JRay/Items/{itemId}/jray?t={seconds}`

Returns an extensible "context at time t" envelope, or `404` if no truth
data exists for the item. Requires an authenticated user token:

```json
{
  "actors": [
    { "name": "Tom Hanks", "imdb_id": "nm0000158", "tmdb_id": "31", "jellyfin_id": "abc123-guid" }
  ]
}
```

Future fields (e.g. `locations`, `trivia`) will be added to this object
without changing the route, so clients should ignore unknown keys.

### `PUT /Plugins/JRay/Items/{itemId}/Truth`

For servers that cannot run the extraction pipeline locally, a remote worker
may push truth data directly. Requires an administrator API key. Body is a
truth file (schema above). Returns `204` on success, or `400` if
`schema_version` is not `1`.

This "managed" truth data takes precedence over any sidecar
`Movie.jray.json` file for the same item, and is stored independently of the
media library filesystem.

### `DELETE /Plugins/JRay/Items/{itemId}/Truth`

Removes managed truth data for an item (idempotent, always returns `204`).
The item falls back to its sidecar truth file, if any, on subsequent reads.
Requires an administrator API key.

### `GET /Plugins/JRay/ClientScript`

Serves the pause-overlay script that JRay injects into the web client's
`index.html` (see below). Anonymous access.

### `GET /Plugins/JRay/Tasks/Pending?limit=10`

Lets a remote extraction worker discover what to work on next. Returns a
random sample (default 10, max 100) of movies/episodes in the library that
have no truth data yet (neither a managed upload nor a sidecar file):

```json
[
  { "item_id": "abc123-guid", "path": "/data/movies/Movie.mkv", "name": "Movie" }
]
```

Requires an administrator API key. The sample is random, so repeated polling
naturally spreads work across the backlog without needing server-side task
tracking; an empty array means there's nothing left to do (or every remaining
item is a virtual/missing-path item that JRay can't process).

**Prioritise/ignore rules apply here.** Items covered by an **ignore** rule are
never returned. Items covered by a **prioritise** rule are returned ahead of
un-prioritised items (still randomised within each tier). See
[Prioritise / ignore rules](#prioritise--ignore-rules) below. Rules only affect
this work-discovery endpoint — they never change the overlay or the read
endpoints, so an item you ignore for extraction still shows its overlay if truth
data happens to exist for it.

## Prioritise / ignore rules

Admins can steer the work-discovery queue with a small set of **rules**. Each
rule targets a **genre**, a **series**, or a single **item**, and either
**prioritises** (moves matching items to the front of `Tasks/Pending`) or
**ignores** them (hides them from `Tasks/Pending` entirely). This is how you
say "never extract anime", "process this series first", or "skip this one
movie".

Rule resolution for an item picks the **most specific** matching scope:
`Item` overrides `Series`, which overrides `Genre`. A rule is uniquely keyed by
its scope + value, and setting a rule for an existing scope+value **replaces**
it — so a single target can never be both prioritised and ignored. (An item can
still be pulled in two directions across scopes, e.g. a prioritised series in an
ignored genre; specificity resolves that — the series rule wins.)

Rules are persisted to `policy.json` under the plugin's configuration directory.
A rule object:

```json
{ "scope": "Genre", "value": "Anime", "action": "Ignore", "label": "Anime" }
```

- `scope`: `"Genre"`, `"Series"`, or `"Item"`.
- `value`: a genre name (for `Genre`), a series id GUID (for `Series`), or an
  item id GUID (for `Item`). Genre matching is case-insensitive.
- `action`: `"Prioritise"` or `"Ignore"`.
- `label`: optional human-readable label shown in the config UI (informational).

All endpoints below require an **Administrator** API key.

### `GET /Plugins/JRay/Policy/Rules`

Returns all configured rules as a JSON array of rule objects.

### `PUT /Plugins/JRay/Policy/Rules`

Adds or replaces a rule (body is a single rule object). Replaces any existing
rule with the same `scope` + `value`. Returns `204`, or `400` if `value` is
empty.

### `DELETE /Plugins/JRay/Policy/Rules?scope={scope}&value={value}`

Removes the rule matching `scope` + `value`. Idempotent, always returns `204`.

## Coverage overview

### `GET /Plugins/JRay/Coverage`

Returns how much of the library has truth data, overall and broken down by
media type (Film vs TV) and by genre. Requires an **Administrator** API key.

```json
{
  "total":  { "total": 1200, "covered": 300, "pending": 850, "prioritised": 40, "ignored": 50 },
  "by_media_type": [
    { "label": "Film", "counts": { "total": 400, "covered": 200, "pending": 190, "prioritised": 10, "ignored": 10 } },
    { "label": "TV",   "counts": { "total": 800, "covered": 100, "pending": 660, "prioritised": 30, "ignored": 40 } }
  ],
  "by_genre": [
    { "label": "Anime", "counts": { "total": 120, "covered": 0, "pending": 0, "prioritised": 0, "ignored": 120 } }
  ]
}
```

Each `counts` object buckets items as: `covered` (has truth data),
`pending` (needs processing and not ignored; `prioritised` is the subset of
`pending` under a prioritise rule), and `ignored` (excluded by an ignore rule).
`total` is the sum. A useful "percent done" is `covered / (total - ignored)`,
so ignoring a genre or series does **not** drag the percentage down — ignored
items are treated as intentionally out of scope.

An item counts toward every genre it carries, so genre rows can overlap and
their totals need not sum to the library total.

### Pickers for the config UI

Three helper endpoints populate the rule editor's dropdowns (all require an
**Administrator** API key, all return `[{ "value": ..., "label": ... }]`):

- `GET /Plugins/JRay/Coverage/Genres` — distinct genres present on movies/episodes.
- `GET /Plugins/JRay/Coverage/Series` — series in the library (`value` is the series id).
- `GET /Plugins/JRay/Coverage/Items?search={term}&limit={n}` — movies/episodes
  whose name matches `term` (`value` is the item id; `limit` default 25, max
  100). An empty/absent `search` returns `[]`.

## Client: pushing results from a remote extraction worker

A worker that runs the extraction pipeline on a different machine than
Jellyfin (i.e. it cannot write a `Movie.jray.json` sidecar next to the media
file) can push results directly over HTTP.

To find work, poll `GET /Plugins/JRay/Tasks/Pending?limit=10` (see above) for
a random batch of items that still need processing, instead of walking the
whole library and checking each item's `Timeline`/sidecar yourself.

### 1. Authenticate

Create an **Administrator** API key in Jellyfin (Dashboard → API Keys), and
send it on every request as either:

```
X-Emby-Token: <api-key>
```

or:

```
Authorization: MediaBrowser Token="<api-key>"
```

### 2. Resolve the Jellyfin item id

The push endpoint is keyed by the Jellyfin item GUID, not by file path. To
find it for `Movie.mkv`:

```
GET /Items?Recursive=true&Fields=Path&IncludeItemTypes=Movie,Episode
```

(use `&ParentId=<library-id>` to narrow the search if the library is large).
Each returned item DTO has `Id` (the GUID) and `Path`. Match `Path` against
the absolute path of the file you just processed — note this requires the
worker to see the file at the *same path* Jellyfin does (same mount/share);
translate paths first if the worker mounts the library elsewhere.

This mapping is stable until the file is moved/re-scanned, so the worker
should cache `path -> itemId` and only re-resolve on a cache miss.

### 3. Push the truth file

```
PUT /Plugins/JRay/Items/{itemId}/Truth
Content-Type: application/json

<truth file JSON, schema_version 1, as produced by result_sink_node>
```

- `204 No Content` — stored. Takes effect immediately (any cached read for
  this item is invalidated server-side).
- `400 Bad Request` — `schema_version` is not `1`.
- `401`/`403` — API key missing or not an administrator.

The `PUT` is idempotent (replaces any existing managed truth for the item),
so the worker can safely retry on network errors.

### 4. (Optional) Remove pushed data

```
DELETE /Plugins/JRay/Items/{itemId}/Truth
```

Always returns `204`. The item falls back to a sidecar `Movie.jray.json` (if
any) on the next read.

## Web client pause overlay

Since Jellyfin has no plugin hook for player UI, JRay injects
`<script defer src="/Plugins/JRay/ClientScript"></script>` into the web
client's `index.html` on startup (idempotent, marked with
`<!-- jray-overlay -->`). The injected script listens for the video player's
pause event, calls `jray?t=` for the current item and timestamp, and renders
a small overlay listing on-screen actors. This can be disabled via the
plugin's "Enable pause overlay" setting, which also removes the injected
script.