jRay/SPEC.md

JRay truth file format

JRay reads "truth" files produced offline by the scene-actor-extraction pipeline (result_sink_node, Verbosity::minimal, schema_version: 1).

File location

For a media file Movie.mkv, the pipeline writes a sibling file Movie.jray.json (suffix configurable in the plugin settings, default .jray.json). The plugin resolves this path from the Jellyfin item's media source path by stripping the extension and appending the suffix.

JSON schema (schema_version 1, minimal verbosity)

{
  "schema_version": 1,
  "movie": "/path/to/Movie.mkv",
  "sample_fps": 1,
  "anneal_sec": 2,
  "actors": [
    {
      "name": "Tom Hanks",
      "imdb_id": "nm0000158",
      "tmdb_id": "31",
      "jellyfin_id": "abc123-guid",
      "scenes": [[12.0, 45.0], [102.5, 150.0]]
    }
  ]
}

• schema_version: integer, bump on breaking changes. JRay should refuse (or warn) on a version it doesn't understand.
• movie: absolute path to the source media file at extraction time (informational only).
• sample_fps: frames-per-second the pipeline sampled at.
• anneal_sec: gap (in seconds) below which consecutive detections of the same actor were merged into a single scene window.
• actors[]: one entry per actor detected anywhere in the film.
  • name: display name from the gallery.
  • imdb_id / tmdb_id / jellyfin_id: identity keys, each "" if not resolved. JRay should prefer jellyfin_id (a Jellyfin Person item GUID) when non-empty, and otherwise resolve imdb_id/tmdb_id against the item's People ProviderIds.
  • scenes: list of [start_sec, end_sec] windows (inclusive) during which the actor is on screen.

Querying "who's on screen at time t"

For a given timestamp t (seconds), an actor is visible if any of their scenes windows satisfies start <= t <= end.

API

GET /Plugins/JRay/Items/{itemId}/Timeline

Returns the full truth file (schema above) for an item, or 404 if no truth data exists (neither a managed upload nor a sidecar file).

GET /Plugins/JRay/Items/{itemId}/jray?t={seconds}

Returns an extensible "context at time t" envelope, or 404 if no truth data exists for the item:

{
  "actors": [
    { "name": "Tom Hanks", "imdb_id": "nm0000158", "tmdb_id": "31", "jellyfin_id": "abc123-guid" }
  ]
}

Future fields (e.g. locations, trivia) will be added to this object without changing the route, so clients should ignore unknown keys.

PUT /Plugins/JRay/Items/{itemId}/Truth

For servers that cannot run the extraction pipeline locally, a remote worker may push truth data directly. Requires an administrator API key. Body is a truth file (schema above). Returns 204 on success, or 400 if schema_version is not 1.

This "managed" truth data takes precedence over any sidecar Movie.jray.json file for the same item, and is stored independently of the media library filesystem.

DELETE /Plugins/JRay/Items/{itemId}/Truth

Removes managed truth data for an item (idempotent, always returns 204). The item falls back to its sidecar truth file, if any, on subsequent reads. Requires an administrator API key.

GET /Plugins/JRay/ClientScript

Serves the pause-overlay script that JRay injects into the web client's index.html (see below). Anonymous access.

Web client pause overlay

Since Jellyfin has no plugin hook for player UI, JRay injects <script defer src="/Plugins/JRay/ClientScript"></script> into the web client's index.html on startup (idempotent, marked with <!-- jray-overlay -->). The injected script listens for the video player's pause event, calls jray?t= for the current item and timestamp, and renders a small overlay listing on-screen actors. This can be disabled via the plugin's "Enable pause overlay" setting, which also removes the injected script.