20 Commits
Author SHA1 Message Date
dtourolleandClaude Opus 5 d9a38bb7fb JR-021, JR-022, JR-023: File Transformation is a hard dependency
🏗️ Build Plugin / build (push) Successful in 59s
Latest Release / latest-release (push) Successful in 30s
🧪 Test Plugin / test (push) Successful in 22s
Deletes the on-disk index.html injection rather than leaving it switched off.
Plugin.cs had already stopped calling it, but an unreachable write path with a
live signature is the one a later refactor re-enables by accident, and it was
still the behaviour the README and the release changelog advertised.

Patching index.html on disk is destructive in ways a plugin cannot clean up
after: the patch outlives an uninstall, a web-client upgrade discards it
silently, and it races any other plugin touching the same file. It is also a
second code path, and the one nobody runs is the one that rots.

WebClientPatchService is now removal-only. The strip is factored out as
RemoveInjection so it is testable without a filesystem. Removal is the one
write JR-021 permits -- an earlier JRay did patch the file, and those users
must not be left with a stale injection pointing at endpoints that have since
changed. It keys on JRay's own marker, so it touches nothing another plugin
added.

JR-021 is a requirement to *not do* something, which no unit test can
demonstrate, so scripts/checks/no-index-injection.sh verifies it by absence.
The check was confirmed to fail on a reintroduced Apply() and on reintroduced
ReplaceLast injection -- a check that has only ever passed is not evidence.

Jellyfin has no plugin dependency mechanism, so nothing installs File
Transformation for the user and a log warning alone is one nobody reads.
GET /Plugins/JRay/Status/Dependencies reports whether the dependency is
satisfied, and the configuration page renders it with the repository URL and
what to do with it. Absent the plugin only the overlay is disabled; every
other feature works.

JR-022 and JR-023 stay In Progress rather than Done: neither has a test that
executes, and this repo has no test project yet.

TRACES: JR-021, JR-022, JR-023 | PR-004

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 18:58:12 +02:00
dtourolleandClaude Opus 5 3b24fe1b3c Requirements register, spec rewrite, and TRACES tags
jRay had no requirement IDs, so nothing in this repo could be traced to and
the CI gate had no denominator to read. The other two components had already
moved to registers; this brings the plugin level with them.

Adds docs/requirements.md with 46 permanent JR-nnn IDs, each carrying a parent
requirement, priority, status and verification tier, plus a per-requirement
verification plan. JR is flat rather than split by theme: the plugin is one
deployable with one audience, and JRay-public-server already ships UR/DR, so a
second repo using those prefixes would make UR-007 ambiguous across registers.

Rewrites SPEC.md as requirements prose with Current:/Gap: on every one. It had
drifted into a format-plus-API reference that documented schema_version 1 while
owning a format whose v2 shape was specified only in the other two repos, said
nothing about SR-002's scene-scoped semantics, and carried the manifest
exchange as a "planned" aside while its configuration classes were already
implemented. Plugin-side exchange obligations move here from the server's
spec, where they were an ownership inversion.

Adds JR-038..041 for PR-005, which had no software row in any repo -- it was
held structurally by SR-004 and GR-005 both being prohibitions, and a goal
preserved only by prohibitions is the kind that erodes unnoticed. jRay is the
component that actually opens a socket.

Tags 18 units with the requirements they satisfy. Tags name what the code
satisfies, so FileTransformationRegistration is not tagged JR-021: that
requirement is a prohibition and was still violated elsewhere when this was
written.

Vendors jray-project as a submodule for the system spec and shared gate.

TRACES: JR-001, JR-004, JR-005, JR-007, JR-008, JR-009, JR-010, JR-011
TRACES: JR-012, JR-013, JR-014, JR-015, JR-016, JR-017, JR-018, JR-019
TRACES: JR-020, JR-024, JR-025, JR-036, JR-038

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 18:57:48 +02:00
Gitea Actions 32f976e79f Update manifest.json for version 0.0.4 2026-07-04 19:47:05 +00:00
Gitea Actions 8e1679faed Update manifest.json for latest build (13471e2) 2026-07-04 19:44:35 +00:00
dtourolle 13471e21fb fix bug that causesd stale media (not on disk) to be suggested to workers
🏗️ Build Plugin / build (push) Successful in 20s
Latest Release / latest-release (push) Successful in 24s
🧪 Test Plugin / test (push) Successful in 19s
🚀 Release Plugin / build-and-release (push) Successful in 20s
2026-07-04 21:42:43 +02:00
Gitea Actions 2ffbd2f50e Update manifest.json for version 0.0.3 2026-07-04 19:11:39 +00:00
Gitea Actions edb93e1028 Update manifest.json for latest build (f6762fc) 2026-07-04 19:10:48 +00:00
dtourolle f6762fcf29 feat: prioritise and blacklist media and overview in setting page of progress in adding info
🏗️ Build Plugin / build (push) Successful in 1m16s
Latest Release / latest-release (push) Successful in 27s
🧪 Test Plugin / test (push) Successful in 20s
🚀 Release Plugin / build-and-release (push) Successful in 24s
2026-07-04 21:08:59 +02:00
dtourolle bb02c0f9b9 Enlarge actor overlay images and replace broken link with in-player popup
Clicking an actor card no longer navigates the SPA away from the player
(which lost playback position and often failed for person items). It now
opens a closeable in-player detail pop-up with a larger portrait and full
overview, dismissible via the close button, Back/Escape, the backdrop, or
by pressing play. On-screen card portraits enlarged from 80px to 120px.
2026-07-04 21:06:02 +02:00
dtourolle f11e5508ad update readme and spec for current release 2026-06-28 12:16:07 +02:00
Gitea Actions f5826084e0 Update manifest.json for version 0.0.2 2026-06-12 18:30:36 +00:00
Gitea Actions bea4d931c6 Update manifest.json for latest build (49c7077)
🚀 Release Plugin / build-and-release (push) Successful in 37s
2026-06-12 18:26:36 +00:00
dtourolle 49c7077e23 bump server version
🏗️ Build Plugin / build (push) Successful in 33s
Latest Release / latest-release (push) Successful in 26s
🧪 Test Plugin / test (push) Successful in 18s
2026-06-12 20:25:32 +02:00
Gitea Actions ff339b681b Update manifest.json for latest build (0e6b03b) 2026-06-12 18:17:26 +00:00
dtourolle 0e6b03be15 fancier overlay
🏗️ Build Plugin / build (push) Successful in 21s
Latest Release / latest-release (push) Successful in 23s
🧪 Test Plugin / test (push) Successful in 18s
2026-06-12 20:16:39 +02:00
Gitea Actions 772babe9bc Update manifest.json for version 0.0.1 2026-06-12 18:11:18 +00:00
Gitea Actions f8365a5b58 Update manifest.json for latest build (4b3b64b) 2026-06-12 18:10:35 +00:00
dtourolle 4b3b64bd7e json copying
🏗️ Build Plugin / build (push) Successful in 22s
Latest Release / latest-release (push) Successful in 28s
🧪 Test Plugin / test (push) Successful in 17s
🚀 Release Plugin / build-and-release (push) Successful in 27s
debugging overlay
2026-06-12 20:09:36 +02:00
Gitea Actions abad55788d Update manifest.json for version 0.0.0 2026-06-12 17:05:10 +00:00
Gitea Actions dea172d990 Update manifest.json for latest build (51a7365) 2026-06-12 17:04:27 +00:00
40 changed files with 3493 additions and 562 deletions
+3
View File
@@ -0,0 +1,3 @@
[submodule "scripts/vendor/jray-project"]
path = scripts/vendor/jray-project
url = git@gitea.tourolle.paris:dtourolle/jray-project.git
@@ -0,0 +1,110 @@
namespace Jellyfin.Plugin.JRay.Configuration;
/// <summary>
/// How far a configured manifest server is trusted. See the JRay public server
/// specification, §9 "Trusting third-party servers".
/// </summary>
public enum ServerTrustLevel
{
/// <summary>
/// Accept manifests, but never contribute to this server and never send
/// library inventory beyond the single item being queried. The default for
/// user-added servers.
/// </summary>
FetchOnly = 0,
/// <summary>
/// Eligible to contribute to, subject to <see cref="ManifestServer.AllowContribute"/>.
/// </summary>
Full = 1,
}
/// <summary>
/// The minimum cut-match tier a fetched manifest must reach before it is stored.
/// See the public server specification, §3 "Cut matching".
/// </summary>
public enum MatchTier
{
/// <summary>Audio 0.600.85, or runtimes within ±30s. Surfaced as a caveat in the UI.</summary>
Loose = 0,
/// <summary>Runtimes within ±2s.</summary>
Runtime = 1,
/// <summary>Audio signature score ≥ 0.85; may carry a non-zero offset.</summary>
Audio = 2,
/// <summary>Identical <c>video_hash</c> — the same file.</summary>
Exact = 3,
}
/// <summary>
/// One entry in the ordered list of manifest servers the plugin queries
/// (public server specification, §9 "Multiple servers").
/// </summary>
/// <remarks>
/// The list is ordered because order *is* the user's trust ranking, made
/// explicit: for a fetch, servers are tried in order and the first acceptable
/// result wins. Querying every server for every item would multiply egress and
/// leak the library to more parties.
///
/// <c>FetchOnly</c> is the default for user-added servers. Adding a third-party
/// server means trusting its operator not to serve deliberately wrong actor
/// data — the client-side controls bound the damage to bad overlay content,
/// they cannot make wrong data right.
/// </remarks>
// TRACES: JR-025 | PR-005, PR-006
public class ManifestServer
{
/// <summary>
/// Initializes a new instance of the <see cref="ManifestServer"/> class.
/// </summary>
public ManifestServer()
{
Url = string.Empty;
Name = string.Empty;
Token = string.Empty;
Enabled = false;
AllowContribute = false;
TrustLevel = ServerTrustLevel.FetchOnly;
}
/// <summary>
/// Gets or sets the base URL of the server, e.g. "https://jray.tourolle.paris".
/// HTTPS is required for non-loopback servers: a plaintext server would let
/// any network intermediary rewrite actor overlays.
/// </summary>
public string Url { get; set; }
/// <summary>
/// Gets or sets the display label shown in the configuration page.
/// </summary>
public string Name { get; set; }
/// <summary>
/// Gets or sets the API token used to contribute manifests. Optional —
/// required only to contribute, never to fetch. This is an anonymous bearer
/// capability rather than an account (public server specification, §5a).
/// </summary>
public string Token { get; set; }
/// <summary>
/// Gets or sets a value indicating whether this server is queried at all.
/// Lets an admin disable an entry without deleting it and losing its token.
/// </summary>
public bool Enabled { get; set; }
/// <summary>
/// Gets or sets a value indicating whether locally generated manifests may be
/// contributed to this server. Independent of fetching, and off by default:
/// contribution is never fanned out, because broadcasting uploads to every
/// configured server would multiply privacy exposure without the user
/// intending it.
/// </summary>
public bool AllowContribute { get; set; }
/// <summary>
/// Gets or sets how far this server is trusted.
/// </summary>
public ServerTrustLevel TrustLevel { get; set; }
}
@@ -1,3 +1,4 @@
using System.Collections.ObjectModel;
using MediaBrowser.Model.Plugins; using MediaBrowser.Model.Plugins;
namespace Jellyfin.Plugin.JRay.Configuration; namespace Jellyfin.Plugin.JRay.Configuration;
@@ -5,8 +6,28 @@ namespace Jellyfin.Plugin.JRay.Configuration;
/// <summary> /// <summary>
/// Plugin configuration. /// Plugin configuration.
/// </summary> /// </summary>
/// <remarks>
/// Every manifest-exchange switch here defaults to <b>off</b>, including the
/// pre-configured community server, so no traffic leaves an installation until
/// an admin acts. Fetching and contributing each reveal to a server operator
/// that some instance holds a given title; that is inherent to the exchange, so
/// the defaults bound the exposure rather than pretending to remove it.
/// </remarks>
// TRACES: JR-036, JR-038 | PR-005
public class PluginConfiguration : BasePluginConfiguration public class PluginConfiguration : BasePluginConfiguration
{ {
/// <summary>
/// The community manifest exchange. Shipped pre-configured but
/// <b>disabled</b>, so no traffic leaves an installation until an admin opts
/// in (public server specification, §9).
/// </summary>
public const string CommunityServerUrl = "https://jray.tourolle.paris";
/// <summary>
/// Display name for <see cref="CommunityServerUrl"/>.
/// </summary>
public const string CommunityServerName = "JRay Community";
/// <summary> /// <summary>
/// Initializes a new instance of the <see cref="PluginConfiguration"/> class. /// Initializes a new instance of the <see cref="PluginConfiguration"/> class.
/// </summary> /// </summary>
@@ -15,6 +36,24 @@ public class PluginConfiguration : BasePluginConfiguration
TruthFileSuffix = ".jray.json"; TruthFileSuffix = ".jray.json";
CacheDurationMinutes = 60; CacheDurationMinutes = 60;
EnableOverlay = true; EnableOverlay = true;
// Manifest sharing is a network egress feature, so every part of it is
// off by default (public server specification, §9 "Configuration").
EnableManifestSharing = false;
ContributeManifests = false;
ComputeAudioSignatures = false;
MinimumMatchTier = MatchTier.Runtime;
// Pre-configured but disabled: the admin opts in by enabling it, rather
// than by having to discover and type a URL.
Servers.Add(new ManifestServer
{
Url = CommunityServerUrl,
Name = CommunityServerName,
Enabled = false,
AllowContribute = false,
TrustLevel = ServerTrustLevel.FetchOnly,
});
} }
/// <summary> /// <summary>
@@ -37,4 +76,62 @@ public class PluginConfiguration : BasePluginConfiguration
/// any previously injected script is removed. /// any previously injected script is removed.
/// </summary> /// </summary>
public bool EnableOverlay { get; set; } public bool EnableOverlay { get; set; }
/// <summary>
/// Gets or sets a value indicating whether JRay may fetch actor-timeline
/// manifests from the configured servers. Off by default — this is a network
/// egress feature and must be opt-in.
/// </summary>
/// <remarks>
/// Fetching reveals to a server operator that some instance holds a given
/// title. That is inherent to the exchange, and each configured server
/// multiplies the exposure, which the configuration page states plainly.
/// </remarks>
public bool EnableManifestSharing { get; set; }
/// <summary>
/// Gets or sets a value indicating whether locally generated manifests may be
/// contributed back. A separate opt-in from downloading, and off by default.
/// </summary>
/// <remarks>
/// Contribution additionally requires <see cref="ManifestServer.AllowContribute"/>
/// on the specific server and a token for it. Uploads are never fanned out to
/// every configured server.
/// </remarks>
public bool ContributeManifests { get; set; }
/// <summary>
/// Gets or sets the minimum cut-match tier a fetched manifest must reach
/// before it is stored.
/// </summary>
/// <remarks>
/// Defaults to <see cref="MatchTier.Runtime"/>. <see cref="MatchTier.Loose"/>
/// admits manifests whose runtime differs by up to 30s, which may be a
/// different trim of the same cut — usable, but it should be surfaced as a
/// caveat rather than applied silently.
/// </remarks>
public MatchTier MinimumMatchTier { get; set; }
/// <summary>
/// Gets or sets a value indicating whether the plugin computes audio
/// signatures for library items, enabling content-based cut matching and
/// identification of files whose providence is unknown.
/// </summary>
/// <remarks>
/// Off by default. Uses the FFmpeg binary Jellyfin already ships (via
/// <c>IMediaEncoder.EncoderPath</c>), so there is no extra dependency, but it
/// costs roughly a second or two of I/O per item and is therefore opt-in.
/// </remarks>
public bool ComputeAudioSignatures { get; set; }
/// <summary>
/// Gets the ordered list of manifest servers.
/// </summary>
/// <remarks>
/// Order is the user's trust ranking: for a fetch, servers are tried in order
/// and the first result clearing <see cref="MinimumMatchTier"/> wins. For a
/// series, first-match applies per <i>episode</i>, so a later server is
/// queried only for the episodes earlier ones lacked.
/// </remarks>
public Collection<ManifestServer> Servers { get; } = new();
} }
@@ -29,16 +29,82 @@
<span>Enable pause overlay</span> <span>Enable pause overlay</span>
</label> </label>
<div class="fieldDescription"> <div class="fieldDescription">
Injects a small script into the web client that shows on-screen actors Shows the cast of the current scene when playback is paused. This
when playback is paused. Disabling this removes the injected script. requires the
<a is="emby-linkbutton" class="button-link" href="https://github.com/IAmParadox27/jellyfin-plugin-file-transformation" target="_blank" rel="noopener">File Transformation</a>
plugin, which rewrites the web client's index.html as it is served.
JRay never modifies index.html on disk, so there is no fallback if
that plugin is absent — only the overlay is affected, and every
other JRay feature keeps working.
</div> </div>
</div> </div>
<div id="JRayDependencyStatus" class="fieldDescription" style="margin:0 0 1.5em;padding:0.75em 1em;border-radius:0.25em;display:none;"></div>
<div> <div>
<button is="emby-button" type="submit" class="raised button-submit block emby-button"> <button is="emby-button" type="submit" class="raised button-submit block emby-button">
<span>Save</span> <span>Save</span>
</button> </button>
</div> </div>
</form> </form>
<h2 style="margin-top:2em;">Coverage</h2>
<p class="fieldDescription">
How much of your library has scene-actor truth data. Ignored items are
excluded from the "to&nbsp;do" total, so blacklisting a genre or series
won't drag your progress down.
</p>
<div id="JRayCoverageSummary" style="margin-bottom:1em;">Loading…</div>
<div id="JRayCoverageBar" style="height:1.25em;border-radius:0.25em;overflow:hidden;background:rgba(127,127,127,0.25);display:flex;margin-bottom:0.5em;"></div>
<div id="JRayCoverageLegend" class="fieldDescription" style="margin-bottom:1.5em;"></div>
<details style="margin-bottom:1.5em;">
<summary style="cursor:pointer;">By media type</summary>
<div id="JRayCoverageByType" style="margin-top:0.5em;"></div>
</details>
<details style="margin-bottom:1.5em;">
<summary style="cursor:pointer;">By genre</summary>
<div id="JRayCoverageByGenre" style="margin-top:0.5em;"></div>
</details>
<h2 style="margin-top:2em;">Prioritise / ignore rules</h2>
<p class="fieldDescription">
Shape the work-discovery API (<code>/Plugins/JRay/Tasks/Pending</code>).
<strong>Ignore</strong> hides matching items from extraction workers;
<strong>Prioritise</strong> pushes them to the front of the queue. A more
specific rule wins (item&nbsp;&gt;&nbsp;series&nbsp;&gt;&nbsp;genre). Setting a
rule for a target that already has one replaces it, so nothing can be both
prioritised and ignored.
</p>
<div style="display:flex;flex-wrap:wrap;gap:0.5em;align-items:flex-end;margin-bottom:1em;">
<div class="selectContainer" style="margin:0;">
<label class="selectLabel" for="JRayRuleScope">Scope</label>
<select id="JRayRuleScope" is="emby-select">
<option value="Genre">Genre</option>
<option value="Series">Series</option>
<option value="Item">Item</option>
</select>
</div>
<div id="JRayItemSearchContainer" class="inputContainer" style="margin:0;flex:1;min-width:14em;display:none;">
<label class="inputLabel inputLabelUnfocused" for="JRayItemSearch">Search title</label>
<input id="JRayItemSearch" type="text" is="emby-input" placeholder="Type a movie or episode name…" />
</div>
<div class="selectContainer" style="margin:0;flex:1;min-width:14em;">
<label class="selectLabel" for="JRayRuleValue">Target</label>
<select id="JRayRuleValue" is="emby-select"></select>
</div>
<div class="selectContainer" style="margin:0;">
<label class="selectLabel" for="JRayRuleAction">Action</label>
<select id="JRayRuleAction" is="emby-select">
<option value="Ignore">Ignore</option>
<option value="Prioritise">Prioritise</option>
</select>
</div>
<button id="JRayAddRule" is="emby-button" type="button" class="raised emby-button">
<span>Add rule</span>
</button>
</div>
<div id="JRayRulesList"></div>
</div> </div>
</div> </div>
<script type="text/javascript"> <script type="text/javascript">
@@ -46,6 +112,256 @@
pluginUniqueId: '96a22d9d-23fd-49bb-8970-5e153817d223' pluginUniqueId: '96a22d9d-23fd-49bb-8970-5e153817d223'
}; };
function jrayUrl(path) {
return ApiClient.getUrl('Plugins/JRay/' + path);
}
function jrayApi(path, method, body) {
var opts = {
url: jrayUrl(path),
type: method || 'GET'
};
if (body !== undefined) {
opts.data = JSON.stringify(body);
opts.contentType = 'application/json';
}
if (!method || method === 'GET') {
opts.dataType = 'json';
}
return ApiClient.ajax(opts);
}
function escapeHtml(s) {
return String(s == null ? '' : s)
.replace(/&/g, '&amp;').replace(/</g, '&lt;')
.replace(/>/g, '&gt;').replace(/"/g, '&quot;');
}
// ---- Dependency status ----
// Jellyfin cannot install a plugin's dependency, so the only thing
// that closes the gap is saying so where an admin can act on it.
function loadDependencyStatus() {
var el = document.querySelector('#JRayDependencyStatus');
return jrayApi('Status/Dependencies').then(function (status) {
el.style.display = '';
if (status.file_transformation_available) {
el.style.background = 'rgba(82,168,82,0.15)';
el.innerHTML = '<strong>File Transformation detected.</strong> '
+ 'The pause overlay is served by rewriting index.html as it is '
+ 'sent, leaving the file on disk untouched.';
return;
}
el.style.background = 'rgba(220,160,60,0.18)';
el.innerHTML = '<strong>File Transformation is not installed &mdash; the pause '
+ 'overlay is disabled.</strong> Every other JRay feature is unaffected. '
+ 'To enable it, add this repository in Dashboard &rarr; Plugins &rarr; '
+ 'Repositories, then install "File Transformation" and restart:<br />'
+ '<code>' + escapeHtml(status.file_transformation_manifest_url) + '</code>';
}).catch(function () {
el.style.display = 'none';
});
}
// ---- Coverage ----
var JRayCoverColors = {
covered: '#4caf50',
prioritised: '#ff9800',
pending: '#9e9e9e',
ignored: 'rgba(127,127,127,0.35)'
};
function renderBar(el, counts) {
el.innerHTML = '';
var order = [
['covered', counts.covered],
['prioritised', counts.prioritised],
['pending', counts.pending - counts.prioritised],
['ignored', counts.ignored]
];
order.forEach(function (seg) {
if (counts.total <= 0 || seg[1] <= 0) { return; }
var d = document.createElement('div');
d.style.background = JRayCoverColors[seg[0]];
d.style.width = (100 * seg[1] / counts.total) + '%';
el.appendChild(d);
});
}
function pct(counts) {
var denom = counts.total - counts.ignored;
if (denom <= 0) { return '—'; }
return Math.round(100 * counts.covered / denom) + '%';
}
function renderBreakdown(el, rows) {
if (!rows || !rows.length) {
el.textContent = 'Nothing to show.';
return;
}
var html = '<table style="width:100%;border-collapse:collapse;">' +
'<thead><tr style="text-align:left;">' +
'<th>Name</th><th>Covered</th><th>Pending</th><th>Ignored</th><th>Total</th><th>%</th>' +
'</tr></thead><tbody>';
rows.forEach(function (r) {
var c = r.counts;
html += '<tr>' +
'<td>' + escapeHtml(r.label) + '</td>' +
'<td>' + c.covered + '</td>' +
'<td>' + c.pending + (c.prioritised ? ' (' + c.prioritised + ' ★)' : '') + '</td>' +
'<td>' + c.ignored + '</td>' +
'<td>' + c.total + '</td>' +
'<td>' + pct(c) + '</td>' +
'</tr>';
});
html += '</tbody></table>';
el.innerHTML = html;
}
function loadCoverage() {
document.querySelector('#JRayCoverageSummary').textContent = 'Loading…';
jrayApi('Coverage').then(function (report) {
var t = report.total;
document.querySelector('#JRayCoverageSummary').innerHTML =
'<strong>' + pct(t) + '</strong> covered — ' +
t.covered + ' of ' + (t.total - t.ignored) + ' items processed' +
(t.ignored ? ' (' + t.ignored + ' ignored)' : '') + '.';
renderBar(document.querySelector('#JRayCoverageBar'), t);
document.querySelector('#JRayCoverageLegend').innerHTML =
'<span style="color:' + JRayCoverColors.covered + '">■</span> covered &nbsp; ' +
'<span style="color:' + JRayCoverColors.prioritised + '">■</span> prioritised &nbsp; ' +
'<span style="color:' + JRayCoverColors.pending + '">■</span> pending &nbsp; ' +
'<span>▨</span> ignored';
renderBreakdown(document.querySelector('#JRayCoverageByType'), report.by_media_type);
renderBreakdown(document.querySelector('#JRayCoverageByGenre'), report.by_genre);
});
}
// ---- Rules ----
var JRayOptionCache = { Genre: null, Series: null };
function loadOptionsFor(scope) {
if (JRayOptionCache[scope]) {
return Promise.resolve(JRayOptionCache[scope]);
}
var path = scope === 'Series' ? 'Coverage/Series' : 'Coverage/Genres';
return jrayApi(path).then(function (opts) {
JRayOptionCache[scope] = opts;
return opts;
});
}
function fillSelect(select, opts, emptyText) {
if (!opts.length) {
select.innerHTML = '<option value="">' + emptyText + '</option>';
return;
}
select.innerHTML = opts.map(function (o) {
return '<option value="' + escapeHtml(o.value) + '">' + escapeHtml(o.label) + '</option>';
}).join('');
}
function populateValueSelect() {
var scope = document.querySelector('#JRayRuleScope').value;
var select = document.querySelector('#JRayRuleValue');
var isItem = scope === 'Item';
document.querySelector('#JRayItemSearchContainer').style.display = isItem ? '' : 'none';
if (isItem) {
// The target dropdown is filled from a live search instead of a full list.
select.innerHTML = '<option value="">Type a title to search…</option>';
return;
}
select.innerHTML = '<option value="">Loading…</option>';
loadOptionsFor(scope).then(function (opts) {
fillSelect(select, opts, '(none found)');
});
}
var JRayItemSearchTimer = null;
function onItemSearch() {
var term = document.querySelector('#JRayItemSearch').value.trim();
var select = document.querySelector('#JRayRuleValue');
clearTimeout(JRayItemSearchTimer);
if (!term) {
select.innerHTML = '<option value="">Type a title to search…</option>';
return;
}
select.innerHTML = '<option value="">Searching…</option>';
JRayItemSearchTimer = setTimeout(function () {
jrayApi('Coverage/Items?search=' + encodeURIComponent(term)).then(function (opts) {
fillSelect(select, opts, '(no matches)');
});
}, 300);
}
function labelForRule(rule) {
if (rule.label) { return rule.label; }
var cache = JRayOptionCache[rule.scope];
if (cache) {
var hit = cache.filter(function (o) { return o.value === rule.value; })[0];
if (hit) { return hit.label; }
}
return rule.value;
}
function loadRules() {
jrayApi('Policy/Rules').then(function (rules) {
var el = document.querySelector('#JRayRulesList');
if (!rules.length) {
el.innerHTML = '<p class="fieldDescription">No rules yet.</p>';
return;
}
var html = '<table style="width:100%;border-collapse:collapse;">' +
'<thead><tr style="text-align:left;"><th>Action</th><th>Scope</th><th>Target</th><th></th></tr></thead><tbody>';
rules.forEach(function (r) {
var badge = r.action === 'Prioritise'
? '<span style="color:' + JRayCoverColors.prioritised + '">★ Prioritise</span>'
: '<span>⦸ Ignore</span>';
html += '<tr>' +
'<td>' + badge + '</td>' +
'<td>' + escapeHtml(r.scope) + '</td>' +
'<td>' + escapeHtml(labelForRule(r)) + '</td>' +
'<td><button is="emby-button" type="button" class="emby-button jray-del" ' +
'data-scope="' + escapeHtml(r.scope) + '" data-value="' + escapeHtml(r.value) + '">Remove</button></td>' +
'</tr>';
});
html += '</tbody></table>';
el.innerHTML = html;
Array.prototype.forEach.call(el.querySelectorAll('.jray-del'), function (btn) {
btn.addEventListener('click', function () {
var q = 'Policy/Rules?scope=' + encodeURIComponent(btn.getAttribute('data-scope')) +
'&value=' + encodeURIComponent(btn.getAttribute('data-value'));
jrayApi(q, 'DELETE').then(function () {
loadRules();
loadCoverage();
});
});
});
});
}
function addRule() {
var scope = document.querySelector('#JRayRuleScope').value;
var valueSelect = document.querySelector('#JRayRuleValue');
var value = valueSelect.value;
if (!value) { return; }
var rule = {
scope: scope,
value: value,
action: document.querySelector('#JRayRuleAction').value,
label: valueSelect.options[valueSelect.selectedIndex].text
};
jrayApi('Policy/Rules', 'PUT', rule).then(function () {
loadRules();
loadCoverage();
});
}
document.querySelector('#JRayConfigPage') document.querySelector('#JRayConfigPage')
.addEventListener('pageshow', function() { .addEventListener('pageshow', function() {
Dashboard.showLoadingMsg(); Dashboard.showLoadingMsg();
@@ -55,8 +371,17 @@
document.querySelector('#EnableOverlay').checked = config.EnableOverlay; document.querySelector('#EnableOverlay').checked = config.EnableOverlay;
Dashboard.hideLoadingMsg(); Dashboard.hideLoadingMsg();
}); });
loadDependencyStatus();
populateValueSelect();
loadRules();
loadCoverage();
}); });
document.querySelector('#JRayRuleScope').addEventListener('change', populateValueSelect);
document.querySelector('#JRayItemSearch').addEventListener('input', onItemSearch);
document.querySelector('#JRayAddRule').addEventListener('click', addRule);
document.querySelector('#JRayConfigForm') document.querySelector('#JRayConfigForm')
.addEventListener('submit', function(e) { .addEventListener('submit', function(e) {
Dashboard.showLoadingMsg(); Dashboard.showLoadingMsg();
@@ -11,12 +11,19 @@ using Microsoft.AspNetCore.Mvc;
namespace Jellyfin.Plugin.JRay.Controllers; namespace Jellyfin.Plugin.JRay.Controllers;
/// <summary> /// <summary>
/// Exposes scene-actor-extraction "truth" data: which actors are on screen /// Exposes scene-actor-extraction "truth" data: which actors are present in
/// at a given timestamp in a movie. /// the scene at a given timestamp.
/// </summary> /// </summary>
/// <remarks>
/// Presence is <b>scene-scoped</b>, not instantaneous: a window is a claim about
/// scene membership, not a recognition event, so an actor who is off-camera
/// during a reverse shot is still present. Windows are served exactly as stored
/// — never merged, split or trimmed.
/// </remarks>
[ApiController] [ApiController]
[Route("Plugins/JRay/Items/{itemId}")] [Route("Plugins/JRay/Items/{itemId}")]
[Authorize] [Authorize]
// TRACES: JR-004, JR-005, JR-012, JR-013, JR-014 | SR-002
public class ActorsController : ControllerBase public class ActorsController : ControllerBase
{ {
private readonly ITruthDataService _truthDataService; private readonly ITruthDataService _truthDataService;
@@ -0,0 +1,234 @@
using System;
using System.Collections.Generic;
using System.Linq;
using Jellyfin.Data.Enums;
using Jellyfin.Plugin.JRay.Models;
using Jellyfin.Plugin.JRay.Services;
using Jellyfin.Plugin.JRay.Services.Interfaces;
using MediaBrowser.Controller.Entities;
using MediaBrowser.Controller.Entities.TV;
using MediaBrowser.Controller.Library;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
namespace Jellyfin.Plugin.JRay.Controllers;
/// <summary>
/// Reports how much of the library has truth data (overall, by media type, and
/// by genre), and supplies the genre/series option lists the config page's
/// rule editor needs.
/// </summary>
/// <remarks>
/// Percent done is <c>covered / (total - ignored)</c>: ignored items are
/// intentionally out of scope, so excluding a genre must not drag the figure
/// down as though it were outstanding work.
/// </remarks>
[ApiController]
[Route("Plugins/JRay/Coverage")]
[Authorize(Roles = "Administrator")]
// TRACES: JR-018, JR-019 | PR-003
public class CoverageController : ControllerBase
{
private readonly ILibraryManager _libraryManager;
private readonly ITruthDataService _truthDataService;
private readonly IMediaPolicyStore _policyStore;
/// <summary>
/// Initializes a new instance of the <see cref="CoverageController"/> class.
/// </summary>
/// <param name="libraryManager">The Jellyfin library manager.</param>
/// <param name="truthDataService">The truth data service.</param>
/// <param name="policyStore">The media policy store.</param>
public CoverageController(ILibraryManager libraryManager, ITruthDataService truthDataService, IMediaPolicyStore policyStore)
{
_libraryManager = libraryManager;
_truthDataService = truthDataService;
_policyStore = policyStore;
}
/// <summary>
/// Gets the coverage report: totals plus breakdowns by media type and by genre.
/// </summary>
/// <returns>The coverage report.</returns>
[HttpGet]
[ProducesResponseType(StatusCodes.Status200OK)]
public ActionResult<CoverageReport> GetCoverage()
{
var rules = _policyStore.GetRules();
var items = _libraryManager.GetItemList(new InternalItemsQuery
{
IncludeItemTypes = new[] { BaseItemKind.Movie, BaseItemKind.Episode },
IsVirtualItem = false,
Recursive = true,
});
var report = new CoverageReport();
var byMediaType = new Dictionary<string, CoverageCounts>(StringComparer.Ordinal);
var byGenre = new Dictionary<string, CoverageCounts>(StringComparer.OrdinalIgnoreCase);
foreach (var item in items)
{
if (!TasksController.MediaFileExists(item))
{
continue;
}
var covered = _truthDataService.HasTruth(item.Id, item.Path);
var action = PolicyResolver.Resolve(rules, item.Id, TasksController.GetSeriesId(item), item.Genres);
Accumulate(report.Total, covered, action);
Accumulate(GetBucket(byMediaType, MediaTypeLabel(item)), covered, action);
foreach (var genre in item.Genres)
{
if (!string.IsNullOrWhiteSpace(genre))
{
Accumulate(GetBucket(byGenre, genre), covered, action);
}
}
}
foreach (var kvp in byMediaType.OrderBy(k => k.Key, StringComparer.Ordinal))
{
report.ByMediaType.Add(new CoverageBreakdownRow { Label = kvp.Key, Counts = kvp.Value });
}
foreach (var kvp in byGenre.OrderBy(k => k.Key, StringComparer.OrdinalIgnoreCase))
{
report.ByGenre.Add(new CoverageBreakdownRow { Label = kvp.Key, Counts = kvp.Value });
}
return Ok(report);
}
/// <summary>
/// Gets the distinct genres present on movies/episodes, for the rule editor.
/// </summary>
/// <returns>Genre options, sorted by name.</returns>
[HttpGet("Genres")]
[ProducesResponseType(StatusCodes.Status200OK)]
public ActionResult<IEnumerable<PickerOption>> GetGenres()
{
var items = _libraryManager.GetItemList(new InternalItemsQuery
{
IncludeItemTypes = new[] { BaseItemKind.Movie, BaseItemKind.Episode },
IsVirtualItem = false,
Recursive = true,
});
var genres = items
.SelectMany(item => item.Genres)
.Where(g => !string.IsNullOrWhiteSpace(g))
.Distinct(StringComparer.OrdinalIgnoreCase)
.OrderBy(g => g, StringComparer.OrdinalIgnoreCase)
.Select(g => new PickerOption { Value = g, Label = g });
return Ok(genres);
}
/// <summary>
/// Gets the series in the library, for the rule editor's series picker.
/// </summary>
/// <returns>Series options (id + title), sorted by title.</returns>
[HttpGet("Series")]
[ProducesResponseType(StatusCodes.Status200OK)]
public ActionResult<IEnumerable<PickerOption>> GetSeries()
{
var series = _libraryManager.GetItemList(new InternalItemsQuery
{
IncludeItemTypes = new[] { BaseItemKind.Series },
IsVirtualItem = false,
Recursive = true,
});
var options = series
.OrderBy(s => s.Name, StringComparer.OrdinalIgnoreCase)
.Select(s => new PickerOption { Value = s.Id.ToString("D"), Label = s.Name });
return Ok(options);
}
/// <summary>
/// Searches movies/episodes by name, for the rule editor's item picker.
/// </summary>
/// <param name="search">A name fragment to match (case-insensitive). Required.</param>
/// <param name="limit">The maximum number of results (default 25, max 100).</param>
/// <returns>Matching item options (id + name), sorted by name.</returns>
[HttpGet("Items")]
[ProducesResponseType(StatusCodes.Status200OK)]
public ActionResult<IEnumerable<PickerOption>> SearchItems([FromQuery] string? search, [FromQuery] int limit = 25)
{
if (string.IsNullOrWhiteSpace(search))
{
return Ok(Array.Empty<PickerOption>());
}
var items = _libraryManager.GetItemList(new InternalItemsQuery
{
IncludeItemTypes = new[] { BaseItemKind.Movie, BaseItemKind.Episode },
IsVirtualItem = false,
Recursive = true,
SearchTerm = search,
Limit = Math.Clamp(limit, 1, 100),
});
var options = items
.OrderBy(i => i.Name, StringComparer.OrdinalIgnoreCase)
.Select(i => new PickerOption { Value = i.Id.ToString("D"), Label = ItemLabel(i) });
return Ok(options);
}
private static string ItemLabel(BaseItem item)
{
// Give episodes a series-qualified label so identically-named episodes
// (e.g. "Pilot") are distinguishable in the picker.
if (item is Episode episode && !string.IsNullOrEmpty(episode.SeriesName))
{
return episode.SeriesName + " — " + item.Name;
}
return item.Name;
}
private static void Accumulate(CoverageCounts counts, bool covered, PolicyAction? action)
{
counts.Total++;
if (action == PolicyAction.Ignore)
{
counts.Ignored++;
return;
}
if (covered)
{
counts.Covered++;
return;
}
counts.Pending++;
if (action == PolicyAction.Prioritise)
{
counts.Prioritised++;
}
}
private static CoverageCounts GetBucket(Dictionary<string, CoverageCounts> buckets, string key)
{
if (!buckets.TryGetValue(key, out var counts))
{
counts = new CoverageCounts();
buckets[key] = counts;
}
return counts;
}
private static string MediaTypeLabel(BaseItem item)
{
return item.GetBaseItemKind() == BaseItemKind.Episode ? "TV" : "Film";
}
}
@@ -0,0 +1,82 @@
using System.Collections.Generic;
using Jellyfin.Plugin.JRay.Models;
using Jellyfin.Plugin.JRay.Services.Interfaces;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
namespace Jellyfin.Plugin.JRay.Controllers;
/// <summary>
/// Manages the prioritise/ignore rules that shape the work-discovery API
/// (<c>GET /Plugins/JRay/Tasks/Pending</c>). Rules can target a genre, a
/// series, or a single item; setting a rule for a target that already has one
/// replaces it, so a target can never be both prioritised and ignored.
/// </summary>
/// <remarks>
/// Rules steer work discovery only. They never reach the read endpoints or the
/// overlay, because a rule says "don't spend compute here", not "pretend this
/// item does not exist".
/// </remarks>
[ApiController]
[Route("Plugins/JRay/Policy")]
[Authorize(Roles = "Administrator")]
// TRACES: JR-016, JR-014 | PR-003
public class PolicyController : ControllerBase
{
private readonly IMediaPolicyStore _policyStore;
/// <summary>
/// Initializes a new instance of the <see cref="PolicyController"/> class.
/// </summary>
/// <param name="policyStore">The media policy store.</param>
public PolicyController(IMediaPolicyStore policyStore)
{
_policyStore = policyStore;
}
/// <summary>
/// Gets all configured prioritise/ignore rules.
/// </summary>
/// <returns>The current rules.</returns>
[HttpGet("Rules")]
[ProducesResponseType(StatusCodes.Status200OK)]
public ActionResult<IReadOnlyList<MediaPolicyRule>> GetRules()
{
return Ok(_policyStore.GetRules());
}
/// <summary>
/// Adds or replaces a rule. If a rule already exists for the same scope
/// and value, its action is updated.
/// </summary>
/// <param name="rule">The rule to set.</param>
/// <returns>204 on success, or 400 if the rule has no value.</returns>
[HttpPut("Rules")]
[ProducesResponseType(StatusCodes.Status204NoContent)]
[ProducesResponseType(StatusCodes.Status400BadRequest)]
public IActionResult SetRule([FromBody] MediaPolicyRule rule)
{
if (rule is null || string.IsNullOrWhiteSpace(rule.Value))
{
return BadRequest("A rule must have a non-empty value.");
}
_policyStore.SetRule(rule);
return NoContent();
}
/// <summary>
/// Removes the rule matching the given scope and value.
/// </summary>
/// <param name="scope">The scope of the rule to remove.</param>
/// <param name="value">The value of the rule to remove.</param>
/// <returns>204 whether or not a matching rule existed.</returns>
[HttpDelete("Rules")]
[ProducesResponseType(StatusCodes.Status204NoContent)]
public IActionResult RemoveRule([FromQuery] PolicyScope scope, [FromQuery] string value)
{
_policyStore.RemoveRule(scope, value ?? string.Empty);
return NoContent();
}
}
@@ -0,0 +1,38 @@
using Jellyfin.Plugin.JRay.Models;
using Jellyfin.Plugin.JRay.Services;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
namespace Jellyfin.Plugin.JRay.Controllers;
/// <summary>
/// Reports whether JRay's hard dependency on the File Transformation plugin is
/// satisfied, for the configuration page to surface.
/// </summary>
/// <remarks>
/// Absent that plugin the pause overlay is disabled and every other JRay feature
/// continues to work — so this is a status to display, not an error to raise.
/// </remarks>
[ApiController]
[Route("Plugins/JRay/Status")]
[Authorize(Roles = "Administrator")]
// TRACES: JR-023 | PR-004
public class StatusController : ControllerBase
{
/// <summary>
/// Gets the dependency status.
/// </summary>
/// <returns>The dependency status.</returns>
[HttpGet("Dependencies")]
[ProducesResponseType(StatusCodes.Status200OK)]
public ActionResult<DependencyStatus> GetDependencies()
{
return Ok(new DependencyStatus
{
FileTransformationAvailable = Plugin.FileTransformationAvailable,
OverlayEnabled = Plugin.OverlayEnabled,
FileTransformationManifestUrl = FileTransformationRegistration.ManifestUrl,
});
}
}
@@ -1,10 +1,13 @@
using System; using System;
using System.Collections.Generic; using System.Collections.Generic;
using System.IO;
using System.Linq; using System.Linq;
using Jellyfin.Data.Enums; using Jellyfin.Data.Enums;
using Jellyfin.Plugin.JRay.Models; using Jellyfin.Plugin.JRay.Models;
using Jellyfin.Plugin.JRay.Services;
using Jellyfin.Plugin.JRay.Services.Interfaces; using Jellyfin.Plugin.JRay.Services.Interfaces;
using MediaBrowser.Controller.Entities; using MediaBrowser.Controller.Entities;
using MediaBrowser.Controller.Entities.TV;
using MediaBrowser.Controller.Library; using MediaBrowser.Controller.Library;
using Microsoft.AspNetCore.Authorization; using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Http; using Microsoft.AspNetCore.Http;
@@ -16,9 +19,19 @@ namespace Jellyfin.Plugin.JRay.Controllers;
/// Lets a remote extraction worker discover which library items still need /// Lets a remote extraction worker discover which library items still need
/// to be processed. /// to be processed.
/// </summary> /// </summary>
/// <remarks>
/// The sample is random so that repeated polling spreads work across the
/// backlog without the server tracking who holds what, and so two workers
/// polling concurrently mostly do not collide.
///
/// Prioritise/ignore rules are applied <b>here and only here</b> (JR-017):
/// they express "don't spend compute on this", not "pretend this does not
/// exist", so they never reach the read endpoints or the overlay.
/// </remarks>
[ApiController] [ApiController]
[Route("Plugins/JRay/Tasks")] [Route("Plugins/JRay/Tasks")]
[Authorize(Roles = "Administrator")] [Authorize(Roles = "Administrator")]
// TRACES: JR-015, JR-017 | PR-003
public class TasksController : ControllerBase public class TasksController : ControllerBase
{ {
private const int DefaultLimit = 10; private const int DefaultLimit = 10;
@@ -26,16 +39,19 @@ public class TasksController : ControllerBase
private readonly ILibraryManager _libraryManager; private readonly ILibraryManager _libraryManager;
private readonly ITruthDataService _truthDataService; private readonly ITruthDataService _truthDataService;
private readonly IMediaPolicyStore _policyStore;
/// <summary> /// <summary>
/// Initializes a new instance of the <see cref="TasksController"/> class. /// Initializes a new instance of the <see cref="TasksController"/> class.
/// </summary> /// </summary>
/// <param name="libraryManager">The Jellyfin library manager.</param> /// <param name="libraryManager">The Jellyfin library manager.</param>
/// <param name="truthDataService">The truth data service.</param> /// <param name="truthDataService">The truth data service.</param>
public TasksController(ILibraryManager libraryManager, ITruthDataService truthDataService) /// <param name="policyStore">The prioritise/ignore policy store.</param>
public TasksController(ILibraryManager libraryManager, ITruthDataService truthDataService, IMediaPolicyStore policyStore)
{ {
_libraryManager = libraryManager; _libraryManager = libraryManager;
_truthDataService = truthDataService; _truthDataService = truthDataService;
_policyStore = policyStore;
} }
/// <summary> /// <summary>
@@ -48,6 +64,7 @@ public class TasksController : ControllerBase
public ActionResult<IEnumerable<PendingExtractionItem>> GetPending([FromQuery] int limit = DefaultLimit) public ActionResult<IEnumerable<PendingExtractionItem>> GetPending([FromQuery] int limit = DefaultLimit)
{ {
var effectiveLimit = Math.Clamp(limit, 1, MaxLimit); var effectiveLimit = Math.Clamp(limit, 1, MaxLimit);
var rules = _policyStore.GetRules();
var items = _libraryManager.GetItemList(new InternalItemsQuery var items = _libraryManager.GetItemList(new InternalItemsQuery
{ {
@@ -56,17 +73,69 @@ public class TasksController : ControllerBase
Recursive = true, Recursive = true,
}); });
// Keep only items that still need truth data, then apply the policy:
// drop anything ignored, and order prioritised items ahead of the rest.
// Randomise within each tier so the backlog still spreads across workers.
var pending = items var pending = items
.Where(item => !string.IsNullOrEmpty(item.Path) && !_truthDataService.HasTruth(item.Id, item.Path)) .Where(item => MediaFileExists(item) && !_truthDataService.HasTruth(item.Id, item.Path))
.OrderBy(_ => Random.Shared.Next()) .Select(item => new
.Take(effectiveLimit)
.Select(item => new PendingExtractionItem
{ {
ItemId = item.Id, Item = item,
Path = item.Path, Action = PolicyResolver.Resolve(rules, item.Id, GetSeriesId(item), item.Genres)
Name = item.Name })
.Where(x => x.Action != PolicyAction.Ignore)
.OrderByDescending(x => x.Action == PolicyAction.Prioritise)
.ThenBy(_ => Random.Shared.Next())
.Take(effectiveLimit)
.Select(x => new PendingExtractionItem
{
ItemId = x.Item.Id,
Path = x.Item.Path,
Name = x.Item.Name
}); });
return Ok(pending); return Ok(pending);
} }
/// <summary>
/// Gets the series id for an episode, or <see cref="Guid.Empty"/> for any
/// other item type (so series rules only ever match episodes).
/// </summary>
/// <param name="item">The library item.</param>
/// <returns>The owning series id, or empty.</returns>
internal static Guid GetSeriesId(BaseItem item)
{
return item is Episode episode ? episode.SeriesId : Guid.Empty;
}
/// <summary>
/// Determines whether an item's backing media still exists on disk.
/// <para>
/// A library item can outlive its file: deleting media from disk does not
/// always purge the Jellyfin database entry, and such ghosts keep
/// <c>IsVirtualItem == false</c>, so filtering on that flag alone is not
/// enough. Without this check the pending endpoint hands stale paths/URLs to
/// extraction workers, and — because a missing file can never gain truth
/// data — those items stay pending forever.
/// </para>
/// Only local (file-protocol) items are stat-checked; remote/streamed items
/// are assumed present so we never stat something off-box.
/// </summary>
/// <param name="item">The library item.</param>
/// <returns><c>true</c> if the item has usable backing media; otherwise <c>false</c>.</returns>
internal static bool MediaFileExists(BaseItem item)
{
if (string.IsNullOrEmpty(item.Path))
{
return false;
}
if (!item.IsFileProtocol)
{
return true;
}
// Path may point at a file or, for folder-based media, a directory.
return System.IO.File.Exists(item.Path) || Directory.Exists(item.Path);
}
} }
@@ -12,11 +12,18 @@ namespace Jellyfin.Plugin.JRay.Controllers;
/// <summary> /// <summary>
/// Accepts scene-actor-extraction "truth" data pushed directly by a remote /// Accepts scene-actor-extraction "truth" data pushed directly by a remote
/// extraction worker, for servers that cannot run the extraction pipeline /// extraction worker, for servers that cannot run the extraction pipeline
/// locally. See SPEC.md. /// locally. See SPEC.md §2.
/// </summary> /// </summary>
/// <remarks>
/// The <c>schema_version</c> check here refuses an unrecognised version rather
/// than guessing at its shape. It is currently the only source that checks —
/// sidecar reads do not — which JR-003 requires be fixed by moving the check
/// into the shared read path.
/// </remarks>
[ApiController] [ApiController]
[Route("Plugins/JRay/Items/{itemId}/Truth")] [Route("Plugins/JRay/Items/{itemId}/Truth")]
[Authorize(Roles = "Administrator")] [Authorize(Roles = "Administrator")]
// TRACES: JR-003, JR-009, JR-014 | SR-003
public class TruthController : ControllerBase public class TruthController : ControllerBase
{ {
private const int SupportedSchemaVersion = 1; private const int SupportedSchemaVersion = 1;
@@ -11,9 +11,14 @@ namespace Jellyfin.Plugin.JRay.Controllers;
/// Serves static client-side assets for JRay, e.g. the pause-overlay script /// Serves static client-side assets for JRay, e.g. the pause-overlay script
/// injected into the web client. /// injected into the web client.
/// </summary> /// </summary>
/// <remarks>
/// Anonymous by necessity, not by oversight: the script tag is injected into
/// <c>index.html</c>, which is served before a user has logged in.
/// </remarks>
[ApiController] [ApiController]
[Route("Plugins/JRay")] [Route("Plugins/JRay")]
[AllowAnonymous] [AllowAnonymous]
// TRACES: JR-014, JR-020 | PR-001
public class WebController : ControllerBase public class WebController : ControllerBase
{ {
private const string OverlayScriptResource = "Jellyfin.Plugin.JRay.Web.jray-overlay.js"; private const string OverlayScriptResource = "Jellyfin.Plugin.JRay.Web.jray-overlay.js";
@@ -11,10 +11,10 @@
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<PackageReference Include="Jellyfin.Controller" Version="10.9.11" > <PackageReference Include="Jellyfin.Controller" Version="10.11.5" >
<ExcludeAssets>runtime</ExcludeAssets> <ExcludeAssets>runtime</ExcludeAssets>
</PackageReference> </PackageReference>
<PackageReference Include="Jellyfin.Model" Version="10.9.11"> <PackageReference Include="Jellyfin.Model" Version="10.11.5">
<ExcludeAssets>runtime</ExcludeAssets> <ExcludeAssets>runtime</ExcludeAssets>
</PackageReference> </PackageReference>
</ItemGroup> </ItemGroup>
@@ -0,0 +1,22 @@
using System.Text.Json.Serialization;
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// One labelled row of a coverage breakdown (e.g. a single genre, or a single
/// media type such as "Film" or "TV").
/// </summary>
public class CoverageBreakdownRow
{
/// <summary>
/// Gets or sets the label for this row (genre name or media type name).
/// </summary>
[JsonPropertyName("label")]
public string Label { get; set; } = string.Empty;
/// <summary>
/// Gets or sets the coverage counts for this row.
/// </summary>
[JsonPropertyName("counts")]
public CoverageCounts Counts { get; set; } = new();
}
@@ -0,0 +1,41 @@
using System.Text.Json.Serialization;
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// A count of library items bucketed by JRay coverage status. Used both for
/// the overall totals and for each per-genre / per-media-type breakdown row.
/// </summary>
public class CoverageCounts
{
/// <summary>
/// Gets or sets the total number of items in this bucket.
/// </summary>
[JsonPropertyName("total")]
public int Total { get; set; }
/// <summary>
/// Gets or sets the number of items that already have truth data.
/// </summary>
[JsonPropertyName("covered")]
public int Covered { get; set; }
/// <summary>
/// Gets or sets the number of items awaiting processing (no truth data,
/// not ignored). Includes prioritised-but-not-yet-covered items.
/// </summary>
[JsonPropertyName("pending")]
public int Pending { get; set; }
/// <summary>
/// Gets or sets the number of pending items that are prioritised.
/// </summary>
[JsonPropertyName("prioritised")]
public int Prioritised { get; set; }
/// <summary>
/// Gets or sets the number of items excluded from processing by an ignore rule.
/// </summary>
[JsonPropertyName("ignored")]
public int Ignored { get; set; }
}
@@ -0,0 +1,31 @@
using System.Collections.ObjectModel;
using System.Text.Json.Serialization;
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// A snapshot of how much of the library has truth data, with overall totals
/// and breakdowns by media type and by genre.
/// </summary>
public class CoverageReport
{
/// <summary>
/// Gets or sets the coverage counts across the whole library.
/// </summary>
[JsonPropertyName("total")]
public CoverageCounts Total { get; set; } = new();
/// <summary>
/// Gets the per-media-type breakdown (Film vs TV).
/// </summary>
[JsonPropertyName("by_media_type")]
[JsonObjectCreationHandling(JsonObjectCreationHandling.Populate)]
public Collection<CoverageBreakdownRow> ByMediaType { get; } = new();
/// <summary>
/// Gets the per-genre breakdown, one row per genre present in the library.
/// </summary>
[JsonPropertyName("by_genre")]
[JsonObjectCreationHandling(JsonObjectCreationHandling.Populate)]
public Collection<CoverageBreakdownRow> ByGenre { get; } = new();
}
@@ -0,0 +1,37 @@
using System.Text.Json.Serialization;
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// Whether JRay's one hard dependency is satisfied, and what to do if it is not.
/// </summary>
/// <remarks>
/// Jellyfin has no plugin dependency mechanism — a manifest cannot declare that
/// another plugin is required, and nothing will install one. The gap is closed by
/// telling the admin, on the page where they can act on it. A warning that exists
/// only in the server log is one nobody reads.
/// </remarks>
public class DependencyStatus
{
/// <summary>
/// Gets or sets a value indicating whether the File Transformation plugin was
/// found and JRay's overlay transformation registered with it.
/// </summary>
[JsonPropertyName("file_transformation_available")]
public bool FileTransformationAvailable { get; set; }
/// <summary>
/// Gets or sets a value indicating whether the overlay is switched on in
/// configuration. It is served only when this <i>and</i>
/// <see cref="FileTransformationAvailable"/> hold.
/// </summary>
[JsonPropertyName("overlay_enabled")]
public bool OverlayEnabled { get; set; }
/// <summary>
/// Gets or sets the repository manifest URL an admin adds to install the
/// missing dependency.
/// </summary>
[JsonPropertyName("file_transformation_manifest_url")]
public string FileTransformationManifestUrl { get; set; } = string.Empty;
}
@@ -0,0 +1,38 @@
using System.Text.Json.Serialization;
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// A single prioritise/ignore rule. A rule is uniquely identified by its
/// <see cref="Scope"/> and <see cref="Value"/>; setting a new action for an
/// existing scope+value replaces the previous one.
/// </summary>
public class MediaPolicyRule
{
/// <summary>
/// Gets or sets the scope this rule targets (genre, series, or item).
/// </summary>
[JsonPropertyName("scope")]
public PolicyScope Scope { get; set; }
/// <summary>
/// Gets or sets the target value: a genre name for <see cref="PolicyScope.Genre"/>,
/// a series id for <see cref="PolicyScope.Series"/>, or an item id for
/// <see cref="PolicyScope.Item"/>.
/// </summary>
[JsonPropertyName("value")]
public string Value { get; set; } = string.Empty;
/// <summary>
/// Gets or sets the action to apply (prioritise or ignore).
/// </summary>
[JsonPropertyName("action")]
public PolicyAction Action { get; set; }
/// <summary>
/// Gets or sets an optional human-readable label for the target (e.g. the
/// series or item display name), shown in the rules list. Informational only.
/// </summary>
[JsonPropertyName("label")]
public string Label { get; set; } = string.Empty;
}
@@ -0,0 +1,23 @@
using System.Text.Json.Serialization;
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// A selectable option for the config page's rule editor (a genre or a series),
/// pairing the value stored in a rule with a human-readable label.
/// </summary>
public class PickerOption
{
/// <summary>
/// Gets or sets the value stored in a <see cref="MediaPolicyRule"/>
/// (a genre name, or a series id).
/// </summary>
[JsonPropertyName("value")]
public string Value { get; set; } = string.Empty;
/// <summary>
/// Gets or sets the label shown to the user (genre name, or series title).
/// </summary>
[JsonPropertyName("label")]
public string Label { get; set; } = string.Empty;
}
@@ -0,0 +1,15 @@
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// The action a <see cref="MediaPolicyRule"/> applies to matching items.
/// A given scope+value can hold at most one action at a time, so an item
/// can never be both prioritised and ignored by the same target.
/// </summary>
public enum PolicyAction
{
/// <summary>Push matching items to the front of the work queue.</summary>
Prioritise,
/// <summary>Exclude matching items from the work queue entirely.</summary>
Ignore
}
@@ -0,0 +1,18 @@
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// The scope a <see cref="MediaPolicyRule"/> targets. More specific scopes
/// override broader ones when resolving an item's effective policy
/// (Item &gt; Series &gt; Genre).
/// </summary>
public enum PolicyScope
{
/// <summary>Matches every item that carries a given genre.</summary>
Genre,
/// <summary>Matches every episode of a given series (by series id).</summary>
Series,
/// <summary>Matches a single library item (by item id).</summary>
Item
}
@@ -0,0 +1,22 @@
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// The object handed to JRay's transformation callback by the File
/// Transformation plugin.
/// </summary>
/// <remarks>
/// That plugin builds a Newtonsoft <c>JObject</c> with a single <c>contents</c>
/// key and calls <c>JObject.ToObject(parameterType)</c> against this type.
/// Newtonsoft binds member names case-insensitively, so <see cref="Contents"/>
/// binds to <c>contents</c> without an attribute — and a System.Text.Json
/// attribute would have no effect here.
/// </remarks>
public class TransformationPayload
{
/// <summary>
/// Gets or sets the current contents of the file being served, including
/// any transformations applied by other plugins ahead of JRay in the
/// chain.
/// </summary>
public string? Contents { get; set; }
}
+13 -1
View File
@@ -5,8 +5,19 @@ namespace Jellyfin.Plugin.JRay.Models;
/// <summary> /// <summary>
/// One actor entry in a <see cref="TruthFile"/>, with the time windows during /// One actor entry in a <see cref="TruthFile"/>, with the time windows during
/// which they are visible on screen. /// which they are present in the scene.
/// </summary> /// </summary>
/// <remarks>
/// "Present in the scene", not "visible on screen": a window is a claim about
/// scene membership, so an actor who turns away or is off-camera during a
/// reverse shot is still present. Two windows mean a genuine departure and
/// return, not a break in detection.
///
/// Identity is public identifiers — never a name alone, which is ambiguous and
/// unstable. <c>jellyfin_id</c> is preferred locally and stripped on
/// contribution, being meaningless outside the instance that produced it.
/// </remarks>
// TRACES: JR-004, JR-007 | SR-001, SR-002
public class TruthActor public class TruthActor
{ {
/// <summary> /// <summary>
@@ -37,5 +48,6 @@ public class TruthActor
/// Gets the list of [start_sec, end_sec] windows during which the actor is on screen. /// Gets the list of [start_sec, end_sec] windows during which the actor is on screen.
/// </summary> /// </summary>
[JsonPropertyName("scenes")] [JsonPropertyName("scenes")]
[JsonObjectCreationHandling(JsonObjectCreationHandling.Populate)]
public Collection<double[]> Scenes { get; } = new(); public Collection<double[]> Scenes { get; } = new();
} }
+13 -1
View File
@@ -5,8 +5,19 @@ namespace Jellyfin.Plugin.JRay.Models;
/// <summary> /// <summary>
/// Root object of a scene-actor-extraction "truth" file /// Root object of a scene-actor-extraction "truth" file
/// (schema_version 1, minimal verbosity). See SPEC.md. /// (schema_version 1, minimal verbosity). See SPEC.md §1.
/// </summary> /// </summary>
/// <remarks>
/// JRay <b>owns</b> this format; the extraction pipeline is its producer and the
/// public server carries a derived envelope. Because three repos ship
/// independently, breaking changes are batched into one coordinated
/// <c>schema_version</c> bump rather than made piecemeal.
///
/// This type is still the v1 shape. JR-002 replaces it: <c>anneal_sec</c> out,
/// an <c>extraction</c> provenance block and a <c>cut</c> block in, and
/// <c>scenes</c> becoming objects that carry belief and identification route.
/// </remarks>
// TRACES: JR-001, JR-002 | SR-003
public class TruthFile public class TruthFile
{ {
/// <summary> /// <summary>
@@ -37,5 +48,6 @@ public class TruthFile
/// Gets the list of actors detected in the film, each with their on-screen scene windows. /// Gets the list of actors detected in the film, each with their on-screen scene windows.
/// </summary> /// </summary>
[JsonPropertyName("actors")] [JsonPropertyName("actors")]
[JsonObjectCreationHandling(JsonObjectCreationHandling.Populate)]
public Collection<TruthActor> Actors { get; } = new(); public Collection<TruthActor> Actors { get; } = new();
} }
+34 -2
View File
@@ -17,6 +17,7 @@ namespace Jellyfin.Plugin.JRay;
public class Plugin : BasePlugin<PluginConfiguration>, IHasWebPages public class Plugin : BasePlugin<PluginConfiguration>, IHasWebPages
{ {
private readonly ILogger<Plugin> _logger; private readonly ILogger<Plugin> _logger;
private readonly bool _usingFileTransformation;
/// <summary> /// <summary>
/// Initializes a new instance of the <see cref="Plugin"/> class. /// Initializes a new instance of the <see cref="Plugin"/> class.
@@ -30,8 +31,25 @@ public class Plugin : BasePlugin<PluginConfiguration>, IHasWebPages
Instance = this; Instance = this;
_logger = logger; _logger = logger;
WebClientPatchService.Apply(ApplicationPaths, Configuration.EnableOverlay, _logger); // The File Transformation plugin rewrites index.html as it is served.
ConfigurationChanged += (_, _) => WebClientPatchService.Apply(ApplicationPaths, Configuration.EnableOverlay, _logger); // Registration is unconditional: the transformation itself checks
// EnableOverlay at request time, so toggling the setting takes effect
// without re-registering.
_usingFileTransformation = FileTransformationRegistration.TryRegister(_logger);
if (!_usingFileTransformation)
{
_logger.LogWarning(
"JRay: the pause overlay is disabled because the File Transformation plugin " +
"is not available. Install it from " +
"https://github.com/IAmParadox27/jellyfin-plugin-file-transformation. " +
"All other JRay features are unaffected.");
}
// JRay never injects into index.html. This only ever *removes* a patch
// left by an earlier version of JRay, identified by its own marker —
// see SPEC.md JR-021/JR-022.
WebClientPatchService.RemoveLegacyPatch(ApplicationPaths, _logger);
} }
/// <inheritdoc /> /// <inheritdoc />
@@ -45,6 +63,20 @@ public class Plugin : BasePlugin<PluginConfiguration>, IHasWebPages
/// </summary> /// </summary>
public static Plugin? Instance { get; private set; } public static Plugin? Instance { get; private set; }
/// <summary>
/// Gets a value indicating whether the pause overlay should currently be
/// served. Read by the File Transformation callback at request time, so
/// toggling the setting takes effect without re-registering.
/// </summary>
internal static bool OverlayEnabled => Instance?.Configuration.EnableOverlay ?? false;
/// <summary>
/// Gets a value indicating whether the File Transformation plugin was found
/// at startup. When it was not, the overlay is disabled and every other JRay
/// feature continues to work.
/// </summary>
internal static bool FileTransformationAvailable => Instance?._usingFileTransformation ?? false;
/// <inheritdoc /> /// <inheritdoc />
public IEnumerable<PluginPageInfo> GetPages() public IEnumerable<PluginPageInfo> GetPages()
{ {
@@ -16,5 +16,6 @@ public class ServiceRegistrator : IPluginServiceRegistrator
{ {
serviceCollection.AddSingleton<IManagedTruthStore, ManagedTruthStore>(); serviceCollection.AddSingleton<IManagedTruthStore, ManagedTruthStore>();
serviceCollection.AddSingleton<ITruthDataService, TruthDataService>(); serviceCollection.AddSingleton<ITruthDataService, TruthDataService>();
serviceCollection.AddSingleton<IMediaPolicyStore, MediaPolicyStore>();
} }
} }
@@ -0,0 +1,165 @@
using System;
using System.Globalization;
using System.Linq;
using System.Reflection;
using System.Runtime.Loader;
using System.Text.Json;
using Jellyfin.Plugin.JRay.Models;
using Microsoft.Extensions.Logging;
namespace Jellyfin.Plugin.JRay.Services;
/// <summary>
/// Registers JRay's overlay script with the
/// <see href="https://github.com/IAmParadox27/jellyfin-plugin-file-transformation">File Transformation</see>
/// plugin, which rewrites <c>index.html</c> as it is served instead of
/// modifying the file on disk. This is non-destructive and composes with
/// other plugins that patch the same file.
/// </summary>
/// <remarks>
/// File Transformation is referenced by reflection (rather than a NuGet
/// package reference) so JRay still loads when it isn't installed. JRay must
/// never bundle the assembly: a bundled copy would sit in a different
/// <c>AssemblyLoadContext</c> from the real one, which is precisely the failure
/// the reflection integration exists to avoid.
///
/// This is the mechanism JR-021 requires, but it does not by itself satisfy
/// JR-021 — that requirement is a prohibition, and it stays unmet while
/// <see cref="WebClientPatchService"/> can still write to disk.
/// </remarks>
// TRACES: JR-020, JR-023 | PR-004
public static class FileTransformationRegistration
{
/// <summary>
/// The marker comment written alongside the injected script tag, used to
/// keep the transformation idempotent.
/// </summary>
internal const string Marker = "<!-- jray-overlay -->";
/// <summary>
/// Repository manifest an admin adds to install the dependency. Surfaced on
/// the configuration page rather than only in the log, since that is where
/// it can be acted on.
/// </summary>
public const string ManifestUrl = "https://www.iamparadox.dev/jellyfin/plugins/manifest.json";
private const string PluginInterfaceTypeName = "Jellyfin.Plugin.FileTransformation.PluginInterface";
private const string RegisterMethodName = "RegisterTransformation";
private const string ScriptTag = "<script defer src=\"/Plugins/JRay/ClientScript\"></script>";
private const string BodyClose = "</body>";
/// <summary>
/// Stable id for JRay's index.html transformation. File Transformation
/// keys registrations on this, so re-registering replaces rather than
/// duplicates.
/// </summary>
private static readonly Guid TransformationId = Guid.Parse("2c9b5a41-6ad0-4c1e-9f7d-1d1e6b0d5a90");
/// <summary>
/// Attempts to register JRay's index.html transformation with the File
/// Transformation plugin.
/// </summary>
/// <param name="logger">The logger.</param>
/// <returns><see langword="true"/> if the transformation was registered;
/// <see langword="false"/> if the File Transformation plugin is not
/// installed or its interface could not be invoked.</returns>
public static bool TryRegister(ILogger logger)
{
ArgumentNullException.ThrowIfNull(logger);
try
{
var registerMethod = ResolveRegisterMethod();
if (registerMethod is null)
{
logger.LogInformation("JRay: File Transformation plugin not found; falling back to patching index.html on disk.");
return false;
}
var payload = BuildPayload(registerMethod);
registerMethod.Invoke(null, [payload]);
logger.LogInformation("JRay: registered index.html transformation with the File Transformation plugin.");
return true;
}
catch (Exception ex) when (ex is TargetInvocationException or InvalidOperationException or JsonException or MissingMethodException)
{
logger.LogWarning(ex, "JRay: failed to register with the File Transformation plugin; falling back to patching index.html on disk.");
return false;
}
}
/// <summary>
/// The transformation callback invoked by the File Transformation plugin.
/// It is resolved by name via reflection, so the signature (public,
/// static, single payload parameter, returns <see cref="string"/>) must
/// not change.
/// </summary>
/// <param name="payload">The current state of the file being served.</param>
/// <returns>The transformed file contents.</returns>
public static string TransformIndexHtml(TransformationPayload payload)
{
ArgumentNullException.ThrowIfNull(payload);
var contents = payload.Contents ?? string.Empty;
if (!Plugin.OverlayEnabled || contents.Contains(Marker, StringComparison.Ordinal))
{
return contents;
}
var index = contents.LastIndexOf(BodyClose, StringComparison.OrdinalIgnoreCase);
if (index < 0)
{
return contents;
}
return contents[..index]
+ ScriptTag
+ Marker
+ "\n"
+ contents[index..];
}
private static MethodInfo? ResolveRegisterMethod()
{
var assembly = AssemblyLoadContext.All
.SelectMany(context => context.Assemblies)
.FirstOrDefault(candidate => candidate.FullName?.Contains(".FileTransformation", StringComparison.Ordinal) ?? false);
return assembly?
.GetType(PluginInterfaceTypeName)?
.GetMethod(RegisterMethodName, BindingFlags.Public | BindingFlags.Static);
}
/// <summary>
/// Builds the registration payload. File Transformation expects a
/// Newtonsoft <c>JObject</c>, which JRay does not reference, so the
/// payload is serialized to JSON and parsed back through the type the
/// target method actually declares.
/// </summary>
private static object BuildPayload(MethodInfo registerMethod)
{
// File Transformation matches this against Assembly.FullName exactly,
// so it must be the full display name, not the short name.
var assemblyName = typeof(FileTransformationRegistration).Assembly.FullName
?? throw new InvalidOperationException("JRay assembly has no name.");
var json = JsonSerializer.Serialize(new
{
id = TransformationId.ToString("D", CultureInfo.InvariantCulture),
fileNamePattern = "index.html",
callbackAssembly = assemblyName,
callbackClass = typeof(FileTransformationRegistration).FullName,
callbackMethod = nameof(TransformIndexHtml)
});
var payloadType = registerMethod.GetParameters().FirstOrDefault()?.ParameterType
?? throw new InvalidOperationException("RegisterTransformation has no parameters.");
var parseMethod = payloadType.GetMethod("Parse", BindingFlags.Public | BindingFlags.Static, [typeof(string)])
?? throw new InvalidOperationException($"Cannot construct File Transformation payload of type '{payloadType.FullName}'.");
return parseMethod.Invoke(null, [json])
?? throw new InvalidOperationException("File Transformation payload parsed to null.");
}
}
@@ -0,0 +1,34 @@
using System.Collections.Generic;
using Jellyfin.Plugin.JRay.Models;
namespace Jellyfin.Plugin.JRay.Services.Interfaces;
/// <summary>
/// Stores the prioritise/ignore rules that shape the work-discovery API
/// (<c>GET /Plugins/JRay/Tasks/Pending</c>). Rules can target a genre, a
/// series, or a single item; more specific scopes override broader ones.
/// </summary>
public interface IMediaPolicyStore
{
/// <summary>
/// Gets all currently configured rules.
/// </summary>
/// <returns>The list of rules (a copy; safe to enumerate).</returns>
IReadOnlyList<MediaPolicyRule> GetRules();
/// <summary>
/// Adds or replaces the rule for a given scope+value. If a rule already
/// exists for the same scope and value, its action and label are updated,
/// so a target can never hold two conflicting actions.
/// </summary>
/// <param name="rule">The rule to set.</param>
void SetRule(MediaPolicyRule rule);
/// <summary>
/// Removes the rule matching the given scope and value, if present.
/// </summary>
/// <param name="scope">The scope of the rule to remove.</param>
/// <param name="value">The value of the rule to remove.</param>
/// <returns><c>true</c> if a rule was removed.</returns>
bool RemoveRule(PolicyScope scope, string value);
}
@@ -14,6 +14,11 @@ namespace Jellyfin.Plugin.JRay.Services;
/// Stores truth files pushed directly to JRay (e.g. by a remote extraction /// Stores truth files pushed directly to JRay (e.g. by a remote extraction
/// worker) under the plugin's configuration directory, keyed by item id. /// worker) under the plugin's configuration directory, keyed by item id.
/// </summary> /// </summary>
/// <remarks>
/// Deliberately outside the media library filesystem, so a worker that cannot
/// write beside the media file is not a second-class producer.
/// </remarks>
// TRACES: JR-009, JR-010 | PR-004
public sealed class ManagedTruthStore : IManagedTruthStore public sealed class ManagedTruthStore : IManagedTruthStore
{ {
private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web); private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web);
@@ -0,0 +1,140 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text.Json;
using System.Text.Json.Serialization;
using Jellyfin.Plugin.JRay.Models;
using Jellyfin.Plugin.JRay.Services.Interfaces;
using MediaBrowser.Common.Configuration;
using Microsoft.Extensions.Logging;
namespace Jellyfin.Plugin.JRay.Services;
/// <summary>
/// Persists the prioritise/ignore rules to a single JSON file under the
/// plugin's configuration directory, and keeps an in-memory copy for fast
/// reads. All access is synchronised so the work-discovery API and the
/// config page can touch it concurrently.
/// </summary>
// TRACES: JR-016 | PR-003
public sealed class MediaPolicyStore : IMediaPolicyStore
{
private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web)
{
WriteIndented = true,
Converters = { new JsonStringEnumConverter() }
};
private readonly IApplicationPaths _applicationPaths;
private readonly ILogger<MediaPolicyStore> _logger;
private readonly object _gate = new();
private List<MediaPolicyRule>? _rules;
/// <summary>
/// Initializes a new instance of the <see cref="MediaPolicyStore"/> class.
/// </summary>
/// <param name="applicationPaths">The Jellyfin application paths.</param>
/// <param name="logger">The logger.</param>
public MediaPolicyStore(IApplicationPaths applicationPaths, ILogger<MediaPolicyStore> logger)
{
_applicationPaths = applicationPaths;
_logger = logger;
}
/// <inheritdoc />
public IReadOnlyList<MediaPolicyRule> GetRules()
{
lock (_gate)
{
return EnsureLoaded().ToList();
}
}
/// <inheritdoc />
public void SetRule(MediaPolicyRule rule)
{
ArgumentNullException.ThrowIfNull(rule);
lock (_gate)
{
var rules = EnsureLoaded();
rules.RemoveAll(r => r.Scope == rule.Scope && ValueEquals(r.Value, rule.Value));
rules.Add(rule);
Save(rules);
}
}
/// <inheritdoc />
public bool RemoveRule(PolicyScope scope, string value)
{
lock (_gate)
{
var rules = EnsureLoaded();
var removed = rules.RemoveAll(r => r.Scope == scope && ValueEquals(r.Value, value));
if (removed > 0)
{
Save(rules);
return true;
}
return false;
}
}
private static bool ValueEquals(string a, string b)
{
// Genre names are matched case-insensitively; ids happen to be
// case-insensitive too (GUID strings), so a single comparison suffices.
return string.Equals(a, b, StringComparison.OrdinalIgnoreCase);
}
private List<MediaPolicyRule> EnsureLoaded()
{
if (_rules is not null)
{
return _rules;
}
var path = GetPath();
if (!File.Exists(path))
{
_rules = new List<MediaPolicyRule>();
return _rules;
}
try
{
using var stream = File.OpenRead(path);
_rules = JsonSerializer.Deserialize<List<MediaPolicyRule>>(stream, JsonOptions)
?? new List<MediaPolicyRule>();
}
catch (Exception ex) when (ex is IOException or JsonException)
{
_logger.LogWarning(ex, "JRay: failed to read media policy file {Path}; starting empty", path);
_rules = new List<MediaPolicyRule>();
}
return _rules;
}
private void Save(List<MediaPolicyRule> rules)
{
var path = GetPath();
var directory = Path.GetDirectoryName(path) ?? throw new InvalidOperationException("Media policy path has no directory.");
Directory.CreateDirectory(directory);
var tempPath = path + ".tmp";
using (var stream = File.Create(tempPath))
{
JsonSerializer.Serialize(stream, rules, JsonOptions);
}
File.Move(tempPath, path, overwrite: true);
}
private string GetPath()
{
return Path.Combine(_applicationPaths.PluginConfigurationsPath, "JRay", "policy.json");
}
}
@@ -0,0 +1,85 @@
using System;
using System.Collections.Generic;
using Jellyfin.Plugin.JRay.Models;
namespace Jellyfin.Plugin.JRay.Services;
/// <summary>
/// Resolves the effective prioritise/ignore action for an item from the
/// configured rules. More specific scopes win: an Item rule overrides a
/// Series rule, which overrides a Genre rule. Because a given scope+value can
/// hold only one action, the only conflicts possible are across scopes, and
/// specificity resolves those.
/// </summary>
// TRACES: JR-016 | PR-003
public static class PolicyResolver
{
/// <summary>
/// Computes the effective action for an item, or <c>null</c> if no rule matches.
/// </summary>
/// <param name="rules">The configured rules.</param>
/// <param name="itemId">The item's id.</param>
/// <param name="seriesId">The item's series id, or <see cref="Guid.Empty"/> if it is not an episode.</param>
/// <param name="genres">The item's genres.</param>
/// <returns>The effective <see cref="PolicyAction"/>, or <c>null</c> when unruled.</returns>
public static PolicyAction? Resolve(
IReadOnlyList<MediaPolicyRule> rules,
Guid itemId,
Guid seriesId,
IReadOnlyList<string> genres)
{
ArgumentNullException.ThrowIfNull(rules);
PolicyAction? itemAction = null;
PolicyAction? seriesAction = null;
PolicyAction? genreAction = null;
var itemIdStr = itemId.ToString("D");
var seriesIdStr = seriesId == Guid.Empty ? null : seriesId.ToString("D");
foreach (var rule in rules)
{
switch (rule.Scope)
{
case PolicyScope.Item:
if (string.Equals(rule.Value, itemIdStr, StringComparison.OrdinalIgnoreCase))
{
itemAction = rule.Action;
}
break;
case PolicyScope.Series:
if (seriesIdStr is not null && string.Equals(rule.Value, seriesIdStr, StringComparison.OrdinalIgnoreCase))
{
seriesAction = rule.Action;
}
break;
case PolicyScope.Genre:
if (genres is not null && MatchesGenre(genres, rule.Value))
{
genreAction = rule.Action;
}
break;
}
}
return itemAction ?? seriesAction ?? genreAction;
}
private static bool MatchesGenre(IReadOnlyList<string> genres, string value)
{
for (var i = 0; i < genres.Count; i++)
{
if (string.Equals(genres[i], value, StringComparison.OrdinalIgnoreCase))
{
return true;
}
}
return false;
}
}
@@ -16,6 +16,13 @@ namespace Jellyfin.Plugin.JRay.Services;
/// item's source file, and caches the parsed result for a configurable /// item's source file, and caches the parsed result for a configurable
/// duration. /// duration.
/// </summary> /// </summary>
/// <remarks>
/// Managed truth — pushed by a worker, or fetched from a manifest server —
/// takes precedence over a sidecar file. Storing fetched manifests through the
/// managed store is what keeps this a two-way rule rather than a three-way one,
/// so the read path never learns that the exchange exists.
/// </remarks>
// TRACES: JR-008, JR-010, JR-011 | PR-001
public sealed class TruthDataService : ITruthDataService public sealed class TruthDataService : ITruthDataService
{ {
private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web); private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web);
@@ -6,26 +6,40 @@ using Microsoft.Extensions.Logging;
namespace Jellyfin.Plugin.JRay.Services; namespace Jellyfin.Plugin.JRay.Services;
/// <summary> /// <summary>
/// Injects (or removes) a script tag in the web client's <c>index.html</c> /// Removes the pause-overlay script tag that an earlier version of JRay
/// that loads JRay's pause-overlay script. This follows the pattern used by /// injected into the web client's <c>index.html</c> on disk.
/// other Jellyfin plugins (e.g. Intro Skipper) since there is no official
/// plugin hook for player-overlay UI.
/// </summary> /// </summary>
/// <remarks>
/// <b>This class cannot inject.</b> JRay reaches the web client only through
/// <see cref="FileTransformationRegistration"/>, which rewrites
/// <c>index.html</c> as it is served. Patching the file on disk was removed
/// rather than left switched off: the patch outlives an uninstall, a web-client
/// upgrade discards it silently, and it races any other plugin patching the same
/// file. An unreachable write path is also the one nobody runs, which is the one
/// a later refactor re-enables by accident.
///
/// Removal remains because users upgrading from a version that did patch the
/// file must not be left with a stale injection pointing at endpoints that have
/// since changed. It keys on JRay's own <c>&lt;!-- jray-overlay --&gt;</c>
/// marker, so it is unambiguous and touches nothing another plugin added.
/// </remarks>
// TRACES: JR-021, JR-022 | PR-004
public static class WebClientPatchService public static class WebClientPatchService
{ {
private const string Marker = "<!-- jray-overlay -->"; private const string Marker = "<!-- jray-overlay -->";
private const string ScriptTag = "<script defer src=\"/Plugins/JRay/ClientScript\"></script>"; private const string ScriptTag = "<script defer src=\"/Plugins/JRay/ClientScript\"></script>";
private const string Injected = ScriptTag + Marker + "\n</body>";
/// <summary> /// <summary>
/// Ensures the web client's index.html either has or does not have the /// Removes a legacy on-disk overlay injection, if one is present. Safe to
/// JRay overlay script injected, matching <paramref name="enableOverlay"/>. /// call on every startup: it is a no-op once the marker is gone.
/// </summary> /// </summary>
/// <param name="applicationPaths">The Jellyfin application paths.</param> /// <param name="applicationPaths">The Jellyfin application paths.</param>
/// <param name="enableOverlay">Whether the overlay script should be present.</param>
/// <param name="logger">The logger.</param> /// <param name="logger">The logger.</param>
public static void Apply(IApplicationPaths applicationPaths, bool enableOverlay, ILogger logger) public static void RemoveLegacyPatch(IApplicationPaths applicationPaths, ILogger logger)
{ {
ArgumentNullException.ThrowIfNull(applicationPaths);
ArgumentNullException.ThrowIfNull(logger);
var indexPath = Path.Combine(applicationPaths.WebPath, "index.html"); var indexPath = Path.Combine(applicationPaths.WebPath, "index.html");
try try
@@ -37,36 +51,38 @@ public static class WebClientPatchService
} }
var html = File.ReadAllText(indexPath); var html = File.ReadAllText(indexPath);
var hasMarker = html.Contains(Marker, StringComparison.Ordinal); if (!html.Contains(Marker, StringComparison.Ordinal))
{
return;
}
if (enableOverlay && !hasMarker) var cleaned = RemoveInjection(html);
{ File.WriteAllText(indexPath, cleaned);
var patched = ReplaceLast(html, "</body>", Injected); logger.LogInformation(
File.WriteAllText(indexPath, patched); "JRay: removed a pause-overlay script left in {Path} by an earlier version. "
logger.LogInformation("JRay: injected pause-overlay script into {Path}", indexPath); + "JRay no longer modifies this file; the overlay is served through File Transformation.",
} indexPath);
else if (!enableOverlay && hasMarker)
{
var patched = html.Replace(ScriptTag + Marker + "\n", string.Empty, StringComparison.Ordinal)
.Replace(ScriptTag + Marker, string.Empty, StringComparison.Ordinal);
File.WriteAllText(indexPath, patched);
logger.LogInformation("JRay: removed pause-overlay script from {Path}", indexPath);
}
} }
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{ {
logger.LogWarning(ex, "JRay: failed to patch web client index.html at {Path}", indexPath); logger.LogWarning(ex, "JRay: failed to remove the legacy overlay patch from {Path}", indexPath);
} }
} }
private static string ReplaceLast(string source, string find, string replace) /// <summary>
/// Strips the marked script tag from the document.
/// </summary>
/// <param name="html">The document contents.</param>
/// <returns>The contents with JRay's injection removed.</returns>
internal static string RemoveInjection(string html)
{ {
var index = source.LastIndexOf(find, StringComparison.Ordinal); ArgumentNullException.ThrowIfNull(html);
if (index < 0)
{
return source;
}
return source[..index] + replace + source[(index + find.Length)..]; // The trailing newline is stripped with the tag when present, so removing
// a patch restores the document byte-for-byte rather than leaving a blank
// line that accumulates across upgrades.
return html
.Replace(ScriptTag + Marker + "\n", string.Empty, StringComparison.Ordinal)
.Replace(ScriptTag + Marker, string.Empty, StringComparison.Ordinal);
} }
} }
+250 -16
View File
@@ -1,15 +1,80 @@
/*
* JRay pause overlay: lists the cast of the scene the viewer paused in.
*
* Presence is scene-scoped. An actor who has turned away, is occluded, or is
* off-camera during a reverse shot is still in the scene, so this must not be
* presented as "who is visible right now" that is a different, and weaker,
* claim than the data makes.
*
* Every server-supplied string is written with textContent, never innerHTML.
* With the manifest exchange these strings may originate from a third-party
* server, and this is the one control that holds even if every other check is
* bypassed.
*
* TRACES: JR-005, JR-020, JR-024 | SR-002, SR-004
*/
(function () { (function () {
'use strict'; 'use strict';
var POLL_INTERVAL_MS = 1000; var POLL_INTERVAL_MS = 1000;
var OVERVIEW_MAX_LENGTH = 160;
var overlayEl = null; var overlayEl = null;
var detailEl = null;
var personCache = {};
function getItemIdFromHash() { function truncate(text, maxLength) {
var match = window.location.hash.match(/[?&]id=([0-9a-fA-F-]+)/); if (text.length <= maxLength) {
return match ? match[1] : null; return text;
}
return text.slice(0, maxLength).trim() + '…';
}
function getPerson(jellyfinId) {
if (personCache[jellyfinId]) {
return personCache[jellyfinId];
}
var promise = window.ApiClient.getItem(window.ApiClient.getCurrentUserId(), jellyfinId)
.catch(function () {
return null;
});
personCache[jellyfinId] = promise;
return promise;
}
function getNowPlaying() {
if (!window.ApiClient) {
return Promise.reject(new Error('no ApiClient'));
}
var url = window.ApiClient.getUrl('Sessions', { DeviceId: window.ApiClient.deviceId() });
return window.ApiClient.ajax({ url: url, type: 'GET', dataType: 'json' }).then(function (sessions) {
var session = sessions && sessions[0];
if (!session || !session.NowPlayingItem || !session.PlayState) {
return null;
}
return {
itemId: session.NowPlayingItem.Id,
positionSeconds: (session.PlayState.PositionTicks || 0) / 10000000
};
});
}
function removeDetail() {
if (detailEl && detailEl.parentNode) {
detailEl.parentNode.removeChild(detailEl);
}
detailEl = null;
document.removeEventListener('keydown', onDetailKeydown, true);
} }
function removeOverlay() { function removeOverlay() {
removeDetail();
if (overlayEl && overlayEl.parentNode) { if (overlayEl && overlayEl.parentNode) {
overlayEl.parentNode.removeChild(overlayEl); overlayEl.parentNode.removeChild(overlayEl);
} }
@@ -17,6 +82,119 @@
overlayEl = null; overlayEl = null;
} }
function onDetailKeydown(event) {
// Back button on TV remotes / keyboards maps to Escape / Backspace.
if (event.key === 'Escape' || event.key === 'Backspace' || event.keyCode === 27 || event.keyCode === 8) {
event.stopPropagation();
event.preventDefault();
removeDetail();
}
}
// A larger, closeable "pop-up" card shown over the video when an actor is
// clicked. It stays inside the player so playback position is never lost.
function showDetail(container, actor, person) {
removeDetail();
detailEl = document.createElement('div');
detailEl.className = 'jrayActorDetail';
detailEl.style.position = 'absolute';
detailEl.style.top = '0';
detailEl.style.left = '0';
detailEl.style.right = '0';
detailEl.style.bottom = '0';
detailEl.style.zIndex = '10000';
detailEl.style.display = 'flex';
detailEl.style.alignItems = 'center';
detailEl.style.justifyContent = 'center';
detailEl.style.background = 'rgba(0, 0, 0, 0.6)';
detailEl.style.pointerEvents = 'auto';
// Click on the dimmed backdrop closes the pop-up.
detailEl.addEventListener('click', function (event) {
if (event.target === detailEl) {
removeDetail();
}
});
var panel = document.createElement('div');
panel.style.position = 'relative';
panel.style.display = 'flex';
panel.style.gap = '24px';
panel.style.maxWidth = '720px';
panel.style.width = '80%';
panel.style.maxHeight = '80%';
panel.style.overflowY = 'auto';
panel.style.background = 'rgba(20, 20, 20, 0.96)';
panel.style.color = '#fff';
panel.style.padding = '24px';
panel.style.borderRadius = '10px';
panel.style.boxShadow = '0 8px 32px rgba(0, 0, 0, 0.6)';
if (person && person.ImageTags && person.ImageTags.Primary) {
var img = document.createElement('img');
img.src = window.ApiClient.getImageUrl(actor.jellyfin_id, {
type: 'Primary',
maxHeight: 400,
tag: person.ImageTags.Primary
});
img.style.height = '300px';
img.style.width = 'auto';
img.style.borderRadius = '8px';
img.style.objectFit = 'cover';
img.style.flexShrink = '0';
panel.appendChild(img);
}
var text = document.createElement('div');
text.style.flex = '1';
var name = document.createElement('div');
name.style.fontWeight = 'bold';
name.style.fontSize = '24px';
name.style.marginBottom = '12px';
name.textContent = actor.name;
text.appendChild(name);
if (person && person.Overview) {
var overview = document.createElement('div');
overview.style.fontSize = '15px';
overview.style.lineHeight = '1.5';
overview.style.opacity = '0.9';
overview.textContent = person.Overview;
text.appendChild(overview);
}
panel.appendChild(text);
var closeBtn = document.createElement('button');
closeBtn.type = 'button';
closeBtn.setAttribute('aria-label', 'Close');
closeBtn.textContent = '✕';
closeBtn.style.position = 'absolute';
closeBtn.style.top = '8px';
closeBtn.style.right = '8px';
closeBtn.style.width = '32px';
closeBtn.style.height = '32px';
closeBtn.style.border = 'none';
closeBtn.style.borderRadius = '50%';
closeBtn.style.background = 'rgba(255, 255, 255, 0.15)';
closeBtn.style.color = '#fff';
closeBtn.style.fontSize = '16px';
closeBtn.style.cursor = 'pointer';
closeBtn.addEventListener('click', function (event) {
event.stopPropagation();
removeDetail();
});
panel.appendChild(closeBtn);
detailEl.appendChild(panel);
container.appendChild(detailEl);
// Back button (Escape/Backspace) closes the pop-up first.
document.addEventListener('keydown', onDetailKeydown, true);
}
function showOverlay(video, actors) { function showOverlay(video, actors) {
removeOverlay(); removeOverlay();
@@ -42,13 +220,69 @@
actors.forEach(function (actor) { actors.forEach(function (actor) {
var card = document.createElement('div'); var card = document.createElement('div');
card.style.background = 'rgba(0, 0, 0, 0.7)'; card.className = 'jrayActorCard';
card.style.background = 'rgba(0, 0, 0, 0.75)';
card.style.color = '#fff'; card.style.color = '#fff';
card.style.padding = '6px 12px'; card.style.padding = '8px 12px';
card.style.borderRadius = '4px'; card.style.borderRadius = '6px';
card.style.fontSize = '14px'; card.style.display = 'flex';
card.textContent = actor.name; card.style.alignItems = 'center';
card.style.gap = '10px';
card.style.maxWidth = '360px';
var textBlock = document.createElement('div');
var name = document.createElement('div');
name.style.fontWeight = 'bold';
name.style.fontSize = '16px';
name.textContent = actor.name;
textBlock.appendChild(name);
card.appendChild(textBlock);
overlayEl.appendChild(card); overlayEl.appendChild(card);
if (!actor.jellyfin_id || !window.ApiClient) {
return;
}
card.style.pointerEvents = 'auto';
card.style.cursor = 'pointer';
getPerson(actor.jellyfin_id).then(function (person) {
if (!person || !overlayEl || !overlayEl.contains(card)) {
return;
}
// Clicking the card opens the in-player detail pop-up rather than
// navigating away — this keeps the current playback position.
card.addEventListener('click', function (event) {
event.stopPropagation();
showDetail(container, actor, person);
});
if (person.ImageTags && person.ImageTags.Primary) {
var img = document.createElement('img');
img.src = window.ApiClient.getImageUrl(actor.jellyfin_id, {
type: 'Primary',
maxHeight: 120,
tag: person.ImageTags.Primary
});
img.style.height = '120px';
img.style.width = 'auto';
img.style.borderRadius = '4px';
img.style.objectFit = 'cover';
card.insertBefore(img, textBlock);
}
if (person.Overview) {
var overview = document.createElement('div');
overview.style.fontSize = '12px';
overview.style.opacity = '0.85';
overview.style.marginTop = '4px';
overview.textContent = truncate(person.Overview, OVERVIEW_MAX_LENGTH);
textBlock.appendChild(overview);
}
});
}); });
container.appendChild(overlayEl); container.appendChild(overlayEl);
@@ -72,14 +306,14 @@
}); });
} }
function onPause(event) { function onPause() {
var video = event.target; getNowPlaying().then(function (info) {
var itemId = getItemIdFromHash(); if (info) {
if (!itemId) { fetchContext(info.itemId, info.positionSeconds);
return; }
} }, function () {
// ApiClient unavailable or request error - fail silently, never break playback.
fetchContext(itemId, video.currentTime); });
} }
function onPlay() { function onPlay() {
+341 -359
View File
@@ -1,6 +1,19 @@
# JRay # JRay
JRay reads "truth" files produced offline by the scene-actor-extraction pipeline (face detection + recognition) and exposes an API to query which actors are visible on screen at a given timestamp in a movie, for building an actor-overlay (Jellyfin "X-Ray") style feature. A Jellyfin plugin that brings an actor-overlay (think Amazon "X-Ray") feature to your media: pause a movie and JRay shows you which actors are on screen at that exact moment.
JRay reads "truth" files produced offline by the
[scene-actor-extraction](https://github.com/dtourolle/scene-actor-extraction)
pipeline (face detection + recognition) and exposes an API to query which actors
are visible at a given timestamp. A small overlay, injected into the Jellyfin web
client, displays the result when you pause playback.
## Status
**Alpha** — JRay works end to end (sidecar truth files, remote truth push, and the
pause overlay) but is early software and the truth-file schema may still change.
The overlay relies on patching the web client's `index.html`, which is inherently
a little fragile across Jellyfin versions (see [Important Notes](#important-notes)).
## Quick Install ## Quick Install
@@ -10,422 +23,391 @@ Add this repository URL in Jellyfin (Dashboard → Plugins → Repositories):
https://gitea.tourolle.paris/dtourolle/jRay/raw/branch/master/manifest.json https://gitea.tourolle.paris/dtourolle/jRay/raw/branch/master/manifest.json
``` ```
Then install "JRay" from the plugin catalog. Then install "JRay" from the plugin catalog and restart Jellyfin.
--- ### Required for the overlay: File Transformation
# So you want to make a Jellyfin plugin JRay's pause overlay needs a script tag in the web client's `index.html`.
Install [File Transformation](https://github.com/IAmParadox27/jellyfin-plugin-file-transformation)
(repository `https://www.iamparadox.dev/jellyfin/plugins/manifest.json`) **before**
installing JRay. It rewrites the page as it is served, so the file on disk is
never touched — that survives server upgrades and coexists with other plugins
patching the same file.
Awesome! This guide is for you. Jellyfin plugins are written using the dotnet standard framework. What that means is you can write them in any language that implements the CLI or the DLI and can compile to net8.0. The examples on this page are in C# because that is what most of Jellyfin is written in, but F#, Visual Basic, and IronPython should all be compatible once compiled. **There is no fallback.** JRay never edits `index.html` on disk: a patch there
outlives an uninstall, is silently discarded by a web-client upgrade, and races
any other plugin touching the file. Without File Transformation the pause
overlay is simply disabled — every other JRay feature works normally, and the
plugin's configuration page tells you what is missing and how to install it.
## 0. Things you need to get started Upgrading from an older JRay that did patch `index.html`? It removes its own
patch on startup, so there is nothing to clean up by hand. See
[SPEC.md](SPEC.md) JR-021/JR-022.
- [Dotnet SDK 9.0](https://dotnet.microsoft.com/en-us/download/dotnet) ## Features
- An editor of your choice. Some free choices are: - **Pause overlay** — pause a movie or episode in the web client and see the
actors currently on screen, without leaving the player.
- **Sidecar truth files** — drop a `Movie.jray.json` next to `Movie.mkv` and JRay
picks it up automatically (suffix configurable).
- **Remote truth push** — for servers that can't run the extraction pipeline
locally, a remote worker can `PUT` truth data over HTTP. Managed (pushed) data
takes precedence over sidecar files.
- **Work discovery API** — a remote worker can poll for a random batch of library
items that still need processing, so the backlog spreads naturally across
workers without server-side task tracking.
- **Prioritise / ignore rules** — steer that backlog by genre, series, or
individual item: ignore what you never want extracted (e.g. anime), or push a
series to the front of the queue.
- **Coverage overview** — see at a glance how much of your library has actor
data, broken down by genre and by media type (film vs TV).
- **Extensible "context at time t" envelope** — the per-timestamp response is
designed to grow (locations, trivia, …) without breaking existing clients.
- **In-memory caching** — loaded truth files are cached with a configurable TTL.
- **Toggleable overlay** — disabling the overlay also cleanly removes the injected
script from the web client.
[Visual Studio Code](https://code.visualstudio.com) ## How It Works
[Visual Studio Community Edition](https://visualstudio.microsoft.com/downloads)
[Mono Develop](https://www.monodevelop.com)
## 0.5. Quickstarts
We have a number of quickstart options available to speed you along the way.
- [Download the Example Plugin Project](https://github.com/jellyfin/jellyfin-plugin-template/tree/master/Jellyfin.Plugin.Template) from this repository, open it in your IDE and go to [step 3](https://github.com/jellyfin/jellyfin-plugin-template#3-customize-plugin-information)
- Install our dotnet template by [downloading the dotnet-template/content folder from this repo](https://github.com/jellyfin/jellyfin-plugin-template/tree/master/dotnet-template/content) or off of Nuget (Coming soon)
```
dotnet new -i /path/to/templatefolder
```
- Run this command then skip to step 4
```
dotnet new Jellyfin-plugin -name MyPlugin
```
If you'd rather start from scratch keep going on to step one. This assumes no specific editor or IDE and requires only the command line with dotnet in the path.
## 1. Initialize Your Project
Make a new dotnet standard project with the following command, it will make a directory for itself.
``` ```
dotnet new classlib -f net9.0 -n MyJellyfinPlugin scene-actor-extraction Jellyfin server web client
(offline pipeline) (browser)
┌────────────────────┐ ┌──────────────────┐ ┌────────────┐
│ face detection + │ truth │ JRay plugin │ jray?t= │ pause │
│ recognition │ ───────► │ - sidecar reader │ ◄─────── │ overlay │
│ result_sink_node │ file │ - managed store │ actors │ script │
└────────────────────┘ │ - REST API │ ───────► └────────────┘
│ PUT /Truth (remote) │ - web patcher │
└────────────────────────►└──────────────────┘
``` ```
Now add the Jellyfin shared libraries. 1. The extraction pipeline analyses a film offline and emits a **truth file**
listing each detected actor and the time windows they're on screen.
2. JRay loads that truth file either from a **sidecar** next to the media
(`Movie.jray.json`) or from a **managed store** populated via the push API.
3. On startup JRay injects a small `<script>` into the web client's `index.html`.
When you pause, the script calls JRay for the current item and timestamp and
renders the on-screen actors as an overlay.
``` ## Truth File Format
dotnet add package Jellyfin.Model
dotnet add package Jellyfin.Controller
```
You have an autogenerated Class1.cs file. You won't be needing this, so go ahead and delete it. For a media file `Movie.mkv`, JRay looks for a sibling `Movie.jray.json` (suffix
configurable). Schema (`schema_version: 1`, minimal verbosity):
Navigate to the csproj that was generated, and ensure that you modify the package references to exclude assets, so that unnecessary files aren't copied over. ```json
Skipping this step will prevent your plugin from registering correctly. {
``` "schema_version": 1,
<ItemGroup> "movie": "/path/to/Movie.mkv",
<PackageReference Include="Jellyfin.Controller" Version="10.11.3"> "sample_fps": 1,
<ExcludeAssets>runtime</ExcludeAssets> "anneal_sec": 2,
</PackageReference> "actors": [
<PackageReference Include="Jellyfin.Model" Version="10.11.3">
<ExcludeAssets>runtime</ExcludeAssets>
</PackageReference>
</ItemGroup>
```
Note: Ensure the package reference version matches the install version of jellyfin server, otherwise the plugin will show as NotSupported.
## 2. Set Up the Basics
There are a few mandatory classes you'll need for a plugin so we need to make them.
### PluginConfiguration
Create a folder named "Configuration", and a PluginConfiguration.cs file inside.
You can call it whatever you'd like really. This class is used to hold settings your plugin might need. We can leave it empty for now. This class should inherit from `MediaBrowser.Model.Plugins.BasePluginConfiguration`
It should look something like the following:
```c#
using MediaBrowser.Model.Plugins;
namespace MyJellyfinPlugin.Configuration;
class PluginConfiguration : BasePluginConfiguration
{ {
"name": "Tom Hanks",
"imdb_id": "nm0000158",
"tmdb_id": "31",
"jellyfin_id": "abc123-guid",
"scenes": [[12.0, 45.0], [102.5, 150.0]]
} }
]
}
``` ```
### Plugin An actor is considered visible at timestamp `t` (seconds) if any of their
`scenes` windows satisfies `start <= t <= end`. JRay prefers `jellyfin_id` (a
Jellyfin Person GUID) when present, otherwise resolves `imdb_id`/`tmdb_id`
against the item's People `ProviderIds`.
This is the main class for your plugin and will reside in the root of your project. It will define your name, version and Id. It should inherit from `MediaBrowser.Common.Plugins.BasePlugin<PluginConfiguration>` See [SPEC.md](SPEC.md) for the full schema and field-by-field reference.
It should look something like the following: ## API
```c#
using MediaBrowser.Common.Plugins; All routes are served under `/Plugins/JRay`. Authentication uses Jellyfin's
using MyJellyfinPlugin.Configuration; standard scheme — pass a token (a user access token or an API key) as either
the `X-Emby-Token: <token>` header or `Authorization: MediaBrowser Token="<token>"`.
namespace MyJellyfinPlugin;
| Method & Route | Auth | Description |
class Plugin : BasePlugin<PluginConfiguration> | --- | --- | --- |
{ | `GET /Items/{itemId}/jray?t={seconds}` | user | "Context at time t" envelope (on-screen actors), or `404`. |
| `GET /Items/{itemId}/Timeline` | user | Full truth file for an item, or `404` if none. |
} | `PUT /Items/{itemId}/Truth` | admin | Push managed truth data (schema v1). `204` on success, `400` on bad schema. |
| `DELETE /Items/{itemId}/Truth` | admin | Remove managed truth data (idempotent, `204`). Falls back to sidecar. |
| `GET /Tasks/Pending?limit=10` | admin | Random sample of items still needing truth data (default 10, max 100); honours prioritise/ignore rules. |
| `GET /Policy/Rules` | admin | List prioritise/ignore rules. |
| `PUT /Policy/Rules` | admin | Add or replace a rule. `204`, or `400` if `value` is empty. |
| `DELETE /Policy/Rules?scope=&value=` | admin | Remove a rule (idempotent, `204`). |
| `GET /Coverage` | admin | Library coverage totals, plus breakdowns by media type and genre. |
| `GET /Coverage/Genres` \| `/Series` \| `/Items?search=` | admin | Option lists for the config page's rule editor. |
| `GET /ClientScript` | anon | The pause-overlay script injected into the web client. |
- **user** — any authenticated Jellyfin user token.
- **admin** — a token belonging to a user with the **Administrator** role
(create an API key under Dashboard → API Keys).
- **anon** — no authentication required.
## Client-Side Integration
JRay is designed so that *any* Jellyfin client (not just the bundled web overlay)
can build an actor-overlay feature. The integration is two calls: figure out
**what is playing and where**, then ask JRay **who is on screen**.
### 1. Query on-screen actors: `GET /Items/{itemId}/jray?t={seconds}`
Given a Jellyfin item id and a playback position in **seconds**, returns the
actors visible at that timestamp. This is the only call most clients need.
**Request**
```
GET /Plugins/JRay/Items/abc123-guid/jray?t=87.5
X-Emby-Token: <user-or-api-token>
``` ```
Note: If you called your PluginConfiguration class something different, you need to put that between the <> **Response — `200 OK`**
### Implement Required Properties ```json
{
The Plugin class needs a few properties implemented before it can work correctly. "actors": [
{
It needs an override on ID, an override on Name, and a constructor that follows a specific model. To get started you can use the following section. "name": "Tom Hanks",
"imdb_id": "nm0000158",
```c# "tmdb_id": "31",
public Plugin(IApplicationPaths applicationPaths, IXmlSerializer xmlSerializer) : base(applicationPaths, xmlSerializer){} "jellyfin_id": "abc123-guid"
public override string Name => throw new System.NotImplementedException(); }
public override Guid Id => Guid.Parse(""); ]
}
``` ```
## 3. Customize Plugin Information - `actors` may be an **empty array** when no one is on screen at `t` — that's a
`200`, not a `404`.
- `404 Not Found` means the item has **no truth data at all** (no managed upload
and no sidecar file). Treat this as "JRay isn't available for this item" and
silently skip — don't surface an error to the viewer.
- The id fields are `""` when unresolved. Prefer `jellyfin_id` (a Jellyfin Person
GUID) to deep-link into the library or fetch a headshot; fall back to
`imdb_id` / `tmdb_id` for external links.
- The top-level object is an **extensible envelope**: future releases may add
sibling keys (e.g. `locations`, `trivia`) alongside `actors`. **Ignore unknown
keys** so your client keeps working across versions.
You need to populate some of your plugin's information. Go ahead a put in a string of the Name you've overridden name, and generate a GUID ### 2. (Optional) Pre-fetch the whole timeline: `GET /Items/{itemId}/Timeline`
- **Windows Users**: you can use the Powershell command `New-Guid`, `[guid]::NewGuid()` or the Visual Studio GUID generator Returns the complete truth file (the [schema above](#truth-file-format)) — every
actor with all their scene windows. Use this if you'd rather fetch once and
compute "who's on screen" client-side (e.g. to drive a scrubber-bar heatmap)
instead of polling `jray?t=` on each pause. `404` if no truth data exists.
- **Linux and OS X Users**: you can use the Powershell Core command `New-Guid` or this command from your shell of choice: ### Reference implementation (web client)
```bash The bundled overlay ([`Web/jray-overlay.js`](Jellyfin.Plugin.JRay/Web/jray-overlay.js))
od -x /dev/urandom | head -n1 | awk '{OFS="-"; srand($6); sub(/./,"4",$5); sub(/./,substr("89ab",1+rand()*4,1),$6); print $2$3,$4,$5,$6,$7$8$9}' shows the full pattern using Jellyfin's `ApiClient`, and is a good template for a
``` custom client:
or ```js
// 1. Find what's playing and the current position (in seconds).
var sessions = await ApiClient.ajax({
url: ApiClient.getUrl('Sessions', { DeviceId: ApiClient.deviceId() }),
type: 'GET', dataType: 'json'
});
var s = sessions[0];
var itemId = s.NowPlayingItem.Id;
var t = (s.PlayState.PositionTicks || 0) / 10000000; // ticks → seconds
```bash // 2. Ask JRay who is on screen. ApiClient adds the auth token for you.
uuidgen var ctx = await ApiClient.ajax({
``` url: ApiClient.getUrl('Plugins/JRay/Items/' + itemId + '/jray', { t: t }),
type: 'GET', dataType: 'json'
});
- Place that guid inside the `Guid.Parse("")` quotes to define your plugin's ID. // 3. Render ctx.actors. A 404 (no truth data) rejects the promise — swallow it.
## 4. Adding Functionality
Congratulations, you now have everything you need for a perfectly functional functionless Jellyfin plugin! You can try it out right now if you'd like by compiling it, then placing the dll you generate in a subfolder (named after your plugin for example) within the plugins folder under your Jellyfin directory (Normally C:\Users\{YourUserName}\AppData\Local\jellyfin\plugins). If you want to try and hook it up to a debugger make sure you copy the generated PDB file alongside it.
Most people aren't satisfied with just having an entry in a menu for their plugin, most people want to have some functionality, so lets look at how to add it.
### 4a. Implement Interfaces
If the functionality you are trying to add is functionality related to something that Jellyfin has an interface for you're in luck. Jellyfin uses some automatic discovery and injection to allow any interfaces you implement in your plugin to be available in Jellyfin.
Here's some interfaces you could implement for common use cases:
- **IAuthenticationProvider** - Allows you to add an authentication provider that can authenticate a user based on a name and a password, but that doesn't expect to deal with local users.
- **IBaseItemComparer** - Allows you to add sorting rules for dealing with media that will show up in sort menus
- **IIntroProvider** - Allows you to play a piece of media before another piece of media (i.e. a trailer before a movie, or a network bumper before an episode of a show)
- **IItemResolver** - Allows you to define custom media types
- **ILibraryPostScanTask** - Allows you to define a task that fires after scanning a library
- **IMetadataSaver** - Allows you to define a metadata standard that Jellyfin can use to write metadata
- **IResolverIgnoreRule** - Allows you to define subpaths that are ignored by media resolvers for use with another function (i.e. you wanted to have a theme song for each tv series stored in a subfolder that could be accessed by your plugin for playback in a menu).
- **IScheduledTask** - Allows you to create a scheduled task that will appear in the scheduled task lists on the dashboard.
There are loads of other interfaces that can be used, but you'll need to poke around the API to get some info. If you're an expert on a particular interface, you should help [contribute some documentation](https://docs.jellyfin.org/general/contributing/index.html)!
### 4b. Use plugin aimed interfaces to add custom functionality
If your plugin doesn't fit perfectly neatly into a predefined interface, never fear, there are a set of interfaces and classes that allow your plugin to extend Jellyfin any which way you please. Here's a quick overview on how to use them
- **IPluginConfigurationPage** - Allows you to have a plugin config page on the dashboard. If you used one of the quickstart example projects, a premade page with some useful components to work with has been created for you! If not you can check out this guide here for how to whip one up.
**IPluginServiceRegistrator** - Will be located by Jellyfin at server startup and allows you to add services to the DI container to allow for injection in your plugin's classes later.
- **IHostedService** - Allows you to run code as a background task that will be started at program startup and will remain in memory. See [Microsoft's documentation](https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-8.0&tabs=visual-studio#ihostedservice-interface) for more information. You can make as many of these as you need; make Jellyfin aware of them with an `IPluginServiceRegistrator`. It is wildly useful for loading configs or persisting state. **Be aware that your main plugin class (IBasePlugin) cannot also be a IHostedService.**
- **ControllerBase** - Allows you to define custom REST-API endpoints. This is the default ASP.NET Web-API controller. You can use it exactly as you would in a normal Web-API project. Learn more about it [here](https://docs.microsoft.com/aspnet/core/web-api/?view=aspnetcore-5.0).
Likewise you might need to get data and services from the Jellyfin core, Jellyfin provides a number of interfaces you can add as parameters to your plugin constructor which are then made available in your project (you can see the 2 mandatory ones that are needed by the plugin system in the constructor as is).
- **IBlurayExaminer** - Allows you to examine blu-ray folders
- **IDtoService** - Allows you to create data transport objects, presumably to send to other plugins or to the core
- **ILibraryManager** - Allows you to directly access the media libraries without hopping through the API
- **ILocalizationManager** - Allows you tap into the main localization engine which governs translations, rating systems, units etc...
- **INetworkManager** - Allows you to get information about the server's networking status
- **IServerApplicationPaths** - Allows you to get the running server's paths
- **IServerConfigurationManager** - Allows you to write or read server configuration data into the application paths
- **ITaskManager** - Allows you to execute and manipulate scheduled tasks
- **IUserManager** - Allows you to retrieve user info and user library related info
- **IXmlSerializer** - Allows you to use the main xml serializer
- **IZipClient** - Allows you to use the core zip client for compressing and decompressing data
## 5. Create a Repository
- [See blog post](https://jellyfin.org/posts/plugin-updates/)
## 6. Set Up Debugging
Debugging can be set up by creating tasks which will be executed when running the plugin project. The specifics on setting up these tasks are not included as they may differ from IDE to IDE. The following list describes the general process:
- Compile the plugin in debug mode.
- Create the plugin directory if it doesn't exist.
- Copy the plugin into your server's plugin directory. The server will then execute it.
- Make sure to set the working directory of the program being debugged to the working directory of the Jellyfin Server.
- Start the server.
Some IDEs like Visual Studio Code may need the following compile flags to compile the plugin:
```shell
dotnet build Your-Plugin.sln /property:GenerateFullPaths=true /consoleloggerparameters:NoSummary
``` ```
These flags generate the full paths for file names and **do not** generate a summary during the build process as this may lead to duplicate errors in the problem panel of your IDE. Notes for client authors, learned from the reference overlay:
### 6.a Set Up Debugging on Visual Studio - **Position is in seconds.** Jellyfin reports `PositionTicks` (100 ns units);
divide by `10_000_000` before passing as `t`.
- **Fail silently.** A `404` or any network error must never interrupt playback —
just render nothing.
- **Enrich via the core API.** JRay returns ids, not images/bios. Use
`jellyfin_id` with the standard Jellyfin item/image endpoints (e.g.
`ApiClient.getItem(...)` / `getImageUrl(...)`) to show headshots and overviews,
and deep-link to `#/details?id=<jellyfin_id>`.
- **Refresh on player events.** The overlay recomputes on `pause` and clears on
`play` / `playing` / `seeking`. Poll `jray?t=` again after a seek rather than
reusing a stale result.
Visual Studio allows developers to connect to other processes and debug them, setting breakpoints and inspecting the variables of the program. We can set this up following this steps: ### Pushing truth data (remote extraction workers)
On this section we will explain how to set up our solution to enable debugging before the server starts.
1. Right-click on the solution, And click on Add -> Existing Project... For the full remote-worker workflow — authenticate, resolve the item id by
2. Locate Jellyfin executable in your installation folder and click on 'Open'. It is called `Jellyfin.exe`. Now The solution will have a new "Project" called Jellyfin. This is the executable, not the source code of Jellyfin. `Path`, and `PUT` the truth file — see
3. Right-click on this new project and click on 'Set up as Startup Project' [SPEC.md](SPEC.md#client-pushing-results-from-a-remote-extraction-worker).
4. Right-click on this new project and click on 'Properties' These endpoints require an **Administrator** token.
5. Make sure that the 'Attach' parameter is set to 'No'
From now on, everytime you click on start from Visual Studio, it will start Jellyfin attached to the debugger! ## Configuration
The only thing left to do is to compile the project as it is specified a few lines above and you are done. Configure JRay in Jellyfin Dashboard → Plugins → JRay:
### 6.b Automate the Setup on Visual Studio Code - **Truth File Suffix** — filename suffix used to locate sidecar truth files next
to media (default `.jray.json`, e.g. `Movie.mkv``Movie.jray.json`).
- **Cache Duration (minutes)** — how long a loaded truth file is cached in memory
before being re-read from disk (default `60`).
- **Enable pause overlay** — whether JRay injects its overlay script into the web
client's `index.html`. Disabling it removes any previously injected script
(default `on`).
Visual Studio Code allows developers to automate the process of starting all necessary dependencies to start debugging the plugin. This guide assumes the reader is familiar with the [documentation on debugging in Visual Studio Code](https://code.visualstudio.com/docs/editor/debugging) and has read the documentation in this file. It is assumed that the Jellyfin Server has already been compiled once. However, should one desire to automatically compile the server before the start of the debugging session, this can be easily implemented, but is not further discussed here. The config page also shows a **coverage overview** (how much of your library has
actor data, by genre and media type) and a **prioritise / ignore rules** editor.
Rules steer the work-discovery API (`/Tasks/Pending`) only — they don't affect
the overlay or read endpoints:
A full example, which aims to be portable may be found in this repo's `.vscode` folder. - **Ignore** — matching items are never offered to extraction workers. Use it to
skip content you don't want processed (e.g. a whole genre like anime, a
specific series, or one movie). Ignored items are excluded from the coverage
"percent done" so they don't count against you.
- **Prioritise** — matching items jump to the front of the work queue.
This example expects you to clone `jellyfin`, `jellyfin-web` and `jellyfin-plugin-template` under the same parent directory, though you can customize this in `settings.json` A rule targets a **genre**, a **series**, or a single **item**; the most specific
matching rule wins (item > series > genre). Setting a rule for a target that
already has one replaces it, so nothing can be both prioritised and ignored.
1. Create a `settings.json` file inside your `.vscode` folder, to specify common options specific to your local setup. ## Building the Plugin
```jsonc
{
// jellyfinDir : The directory of the cloned jellyfin server project
// This needs to be built once before it can be used
"jellyfinDir" : "${workspaceFolder}/../jellyfin/Jellyfin.Server",
// jellyfinWebDir : The directory of the cloned jellyfin-web project
// This needs to be built once before it can be used
"jellyfinWebDir" : "${workspaceFolder}/../jellyfin-web",
// jellyfinDataDir : the root data directory for a running jellyfin instance
// This is where jellyfin stores its configs, plugins, metadata etc
// This is platform specific by default, but on Windows defaults to
// ${env:LOCALAPPDATA}/jellyfin
"jellyfinDataDir" : "${env:LOCALAPPDATA}/jellyfin",
// The name of the plugin
"pluginName" : "Jellyfin.Plugin.Template",
}
```
1. To automate the launch process, create a new `launch.json` file for C# projects inside the `.vscode` folder. The example below shows only the relevant parts of the file. Adjustments to your specific setup and operating system may be required. ### Prerequisites
```jsonc - [.NET SDK 9.0](https://dotnet.microsoft.com/en-us/download/dotnet)
{ - Jellyfin 10.9.0 or later (built against `Jellyfin.Controller` 10.11.5)
// Paths and plugin names are configured in settings.json
"version": "0.2.0",
"configurations": [
{
"type": "coreclr",
"name": "Launch",
"request": "launch",
"preLaunchTask": "build-and-copy",
"program": "${config:jellyfinDir}/bin/Debug/net8.0/jellyfin.dll",
"args": [
//"--nowebclient"
"--webdir",
"${config:jellyfinWebDir}/dist/"
],
"cwd": "${config:jellyfinDir}",
}
]
}
``` ### Build Steps
The `request` type is specified as `launch`, as this `launch.json` file will start the Jellyfin Server process. The `preLaunchTask` defines a task that will run before the Jellyfin Server starts. More on this later. It is important to set the `program` path to the Jellyin Server program and set the current working directory (`cwd`) to the working directory of the Jellyfin Server. ```bash
The `args` option allows to specify arguments to be passed to the server, e.g. whether Jellyfin should start with the web-client or without it. dotnet publish Jellyfin.Plugin.JRay/Jellyfin.Plugin.JRay.csproj -c Release
```
2. Create a `tasks.json` file inside your `.vscode` folder and specify a `build-and-copy` task that will run in `sequence` order. This tasks depends on multiple other tasks and all of those other tasks can be defined as simple `shell` tasks that run commands like the `cp` command to copy a file. The sequence to run those tasks in is given below. Please note that it might be necessary to adjust the examples for your specific setup and operating system. The compiled `Jellyfin.Plugin.JRay.dll` will be under
`Jellyfin.Plugin.JRay/bin/Release/net9.0/publish/`.
The full file is shown here - Specific sections will be discussed in depth A reproducible build via the bundled Docker image is also available:
```jsonc
{
// Paths and plugin name are configured in settings.json
"version": "2.0.0",
"tasks": [
{
// A chain task - build the plugin, then copy it to your
// jellyfin server's plugin directory
"label": "build-and-copy",
"dependsOrder": "sequence",
"dependsOn": ["build", "make-plugin-dir", "copy-dll"]
},
{
// Build the plugin
"label": "build",
"command": "dotnet",
"type": "shell",
"args": [
"publish",
"${workspaceFolder}/${config:pluginName}.sln",
"/property:GenerateFullPaths=true",
"/consoleloggerparameters:NoSummary"
],
"group": "build",
"presentation": {
"reveal": "silent"
},
"problemMatcher": "$msCompile"
},
{
// Ensure the plugin directory exists before trying to use it
"label": "make-plugin-dir",
"type": "shell",
"command": "mkdir",
"args": [
"-Force",
"-Path",
"${config:jellyfinDataDir}/plugins/${config:pluginName}/"
]
},
{
// Copy the plugin dll to the jellyfin plugin install path
// This command copies every .dll from the build directory to the plugin dir
// Usually, you probablly only need ${config:pluginName}.dll
// But some plugins may bundle extra requirements
"label": "copy-dll",
"type": "shell",
"command": "cp",
"args": [
"./${config:pluginName}/bin/Debug/net8.0/publish/*",
"${config:jellyfinDataDir}/plugins/${config:pluginName}/"
]
}, ```bash
] docker build -f Dockerfile.builder -t jray-builder .
} ```
``` ## Manual Installation
1. The "build-and-copy" task which triggers all of the other tasks
```jsonc
{
// A chain task - build the plugin, then copy it to your
// jellyfin server's plugin directory
"label": "build-and-copy",
"dependsOrder": "sequence",
"dependsOn": ["build", "make-plugin-dir", "copy-dll"]
},
```
2. A build task. This task builds the plugin without generating summary, but with full paths for file names enabled.
```jsonc 1. Build the plugin (see above).
{ 2. Copy the published output into a `JRay` subfolder of your Jellyfin plugins
// Build the plugin directory (e.g. `~/.local/share/jellyfin/plugins/JRay/` on Linux, or
"label": "build", `%LOCALAPPDATA%\jellyfin\plugins\JRay\` on Windows).
"command": "dotnet", 3. Restart Jellyfin.
"type": "shell", 4. Configure JRay in Dashboard → Plugins → JRay.
"args": [
"publish",
"${workspaceFolder}/${config:pluginName}.sln",
"/property:GenerateFullPaths=true",
"/consoleloggerparameters:NoSummary"
],
"group": "build",
"presentation": {
"reveal": "silent"
},
"problemMatcher": "$msCompile"
},
```
3. A tasks which creates the necessary plugin directory and a sub-folder for the specific plugin. The plugin directory is located below the [data directory](https://jellyfin.org/docs/general/administration/configuration.html) of the Jellyfin Server. As an example, the following path can be used for the bookshelf plugin: `$HOME/.local/share/jellyfin/plugins/Bookshelf/` ## Technical Architecture
```jsonc
{
// Ensure the plugin directory exists before trying to use it
"label": "make-plugin-dir",
"type": "shell",
"command": "mkdir",
"args": [
"-Force",
"-Path",
"${config:jellyfinDataDir}/plugins/${config:pluginName}/"
]
},
```
4. A tasks which copies the plugin dll which has been built in step 2.1. The file is copied into it's specific plugin directory within the server's plugin directory. ### Directory Structure
```jsonc ```
{ Jellyfin.Plugin.JRay/
// Copy the plugin dll to the jellyfin plugin install path ├── Configuration/
// This command copies every .dll from the build directory to the plugin dir │ ├── PluginConfiguration.cs # Suffix, cache TTL, overlay toggle
// Usually, you probablly only need ${config:pluginName}.dll │ └── configPage.html # Dashboard config page (embedded resource)
// But some plugins may bundle extra requirements ├── Controllers/
"label": "copy-dll", │ ├── ActorsController.cs # /Timeline and /jray?t= read endpoints
"type": "shell", │ ├── TruthController.cs # PUT/DELETE managed truth data
"command": "cp", │ ├── TasksController.cs # /Tasks/Pending work discovery (policy-aware)
"args": [ │ ├── PolicyController.cs # /Policy/Rules prioritise/ignore CRUD
"./${config:pluginName}/bin/Debug/net8.0/publish/*", │ ├── CoverageController.cs # /Coverage overview + genre/series/item pickers
"${config:jellyfinDataDir}/plugins/${config:pluginName}/" │ └── WebController.cs # /ClientScript overlay script
] ├── Models/
}, │ ├── TruthFile.cs # Root truth-file schema (schema_version 1)
``` │ ├── TruthActor.cs # Per-actor entry with scene windows
│ ├── ActorAtTime.cs # Actor entry in the "context at t" envelope
│ ├── JRayContext.cs # Extensible "context at time t" envelope
│ ├── PendingExtractionItem.cs # Item descriptor for the work-discovery API
│ ├── MediaPolicyRule.cs # A prioritise/ignore rule (+ PolicyScope/PolicyAction)
│ ├── CoverageReport.cs # Coverage overview (+ CoverageCounts / breakdown rows)
│ └── PickerOption.cs # value/label option for the config-page pickers
├── Services/
│ ├── Interfaces/
│ │ ├── ITruthDataService.cs
│ │ ├── IManagedTruthStore.cs
│ │ └── IMediaPolicyStore.cs
│ ├── TruthDataService.cs # Resolves + caches truth (managed > sidecar)
│ ├── ManagedTruthStore.cs # Storage for pushed/managed truth data
│ ├── MediaPolicyStore.cs # Persists prioritise/ignore rules (policy.json)
│ ├── PolicyResolver.cs # Resolves an item's effective rule (item>series>genre)
│ └── WebClientPatchService.cs # Injects/removes overlay script in index.html
├── Web/
│ └── jray-overlay.js # Pause-overlay client script (embedded resource)
├── ServiceRegistrator.cs # DI registration
└── Plugin.cs # Plugin entry point; applies web patch on load
```
## Licensing ### Key Components
Licensing is a complex topic. This repository features a GPLv3 license template that can be used to provide a good default license for your plugin. You may alter this if you like, but if you do a permissive license must be chosen. 1. **Truth Data Service** (`TruthDataService`): resolves truth data for an item —
managed (pushed) data takes precedence over a sidecar file — and caches the
result in memory with the configured TTL.
2. **Managed Truth Store** (`ManagedTruthStore`): stores truth data pushed via the
API, independently of the media library filesystem.
3. **Controllers**: REST endpoints for reading (`ActorsController`), pushing
(`TruthController`), work discovery (`TasksController`), and serving the
overlay script (`WebController`).
4. **Web Client Patch Service** (`WebClientPatchService`): injects (or removes) the
`<script>` tag in the web client's `index.html`, marked with `<!-- jray-overlay -->`
so it's idempotent. Re-applied whenever configuration changes.
5. **Overlay Script** (`jray-overlay.js`): listens for the player's pause event,
calls `jray?t=`, and renders the on-screen actors.
Due to how plugins in Jellyfin work, when your plugin is compiled into a binary, it will link against the various Jellyfin binary NuGet packages. These packages are licensed under the GPLv3. Thus, due to the nature and restrictions of the GPL, the binary plugin you get will also be licensed under the GPLv3. ## Important Notes
If you accept the default GPLv3 license from this template, all will be good. However if you choose a different license, please keep this fact in mind, as it might not always be obvious that an, e.g. MIT-licensed plugin would become GPLv3 when compiled. - **Web client overlay is a patch, not a hook.** Jellyfin has no plugin hook for
player UI, so JRay edits the web client's `index.html` directly. This generally
survives across restarts but may need re-applying after a Jellyfin web update;
toggling the overlay setting off and on re-runs the patch.
- **Truth data is produced offline.** JRay does not run face detection itself — it
consumes truth files from the
[scene-actor-extraction](https://github.com/dtourolle/scene-actor-extraction)
pipeline. No extraction = no overlay.
- **Schema version 1 only.** JRay accepts `schema_version: 1`. The schema may
change in future releases; bump-aware clients should send the version they
produced.
- **Remote path mapping.** Workers pushing truth match items by `Path`, so they
must see media at the same path Jellyfin does (translate paths first if mounts
differ).
Please note that this also means making "proprietary", source-unavailable, or otherwise "hidden" plugins for public consumption is not permitted. To build a Jellyfin plugin for distribution to others, it must be under the GPLv3 or a permissive open-source license that can be linked against the GPLv3. ## Contributing
Contributions welcome! This project is hosted on a self-hosted
[Gitea](https://gitea.tourolle.paris/dtourolle/jRay) instance.
**You don't need a separate account** — you can sign in with your existing GitHub
account. On the [sign-in page](https://gitea.tourolle.paris/user/login), choose
**"Sign in with GitHub"** to register and log in via GitHub OAuth. Once signed in,
you can:
- **Raise issues** — report bugs or request features on the
[issue tracker](https://gitea.tourolle.paris/dtourolle/jRay/issues).
- **Contribute code** — fork the repository, push a branch, and open a pull request.
## License
JRay is licensed under the **GNU General Public License v3.0**. See the
[LICENSE](LICENSE) file for details.
Because Jellyfin plugins link against the GPLv3-licensed Jellyfin NuGet packages,
the compiled plugin is necessarily GPLv3 as well.
## Acknowledgments
This plugin was developed partly using
[Claude Code](https://docs.anthropic.com/en/docs/claude-code) by Anthropic.
Built on the [Jellyfin plugin template](https://github.com/jellyfin/jellyfin-plugin-template)
and powered by the
[scene-actor-extraction](https://github.com/dtourolle/scene-actor-extraction)
pipeline.
## References
- [Jellyfin Plugin Documentation](https://jellyfin.org/docs/general/server/plugins/)
- [scene-actor-extraction pipeline](https://github.com/dtourolle/scene-actor-extraction)
- [JRay truth file specification](SPEC.md)
+725 -134
View File
@@ -1,195 +1,786 @@
# JRay truth file format # jRay — software specification
JRay reads "truth" files produced offline by the Status: **alpha.** Core read path ships; the schema bump, the exchange client and
[scene-actor-extraction](https://github.com/dtourolle/scene-actor-extraction) the audio signature do not.
pipeline (`result_sink_node`, `Verbosity::minimal`, `schema_version: 1`).
## File location This is a *software* spec: its job is to implement the
[system spec](scripts/vendor/jray-project/SPEC.md), which owns everything spanning more than one repo.
Requirements here trace up to an `SR-nnn` or a `PR-nnn`; the prose below is the
detail. The authoritative ID list with status lives in
[`docs/requirements.md`](docs/requirements.md).
For a media file `Movie.mkv`, the pipeline writes a sibling file jRay is the Jellyfin plugin: it **consumes** presence data, **displays** it in
`Movie.jray.json` (suffix configurable in the plugin settings, default the player, and **owns the truth-file format** that the other two components
`.jray.json`). The plugin resolves this path from the Jellyfin item's media produce and exchange.
source path by stripping the extension and appending the suffix.
## JSON schema (schema_version 1, minimal verbosity) ---
## 0. Requirements
IDs are `JR-nnn`, zero-padded and **permanent** — a withdrawn requirement keeps
its number, because renumbering is what produces orphan TRACES tags
([system spec](scripts/vendor/jray-project/SPEC.md) §6).
| Group | IDs | Where addressed |
|---|---|---|
| Truth-file format | JR-001 … JR-007 | §1 |
| Sources and precedence | JR-008 … JR-011 | §2 |
| Read API | JR-012 … JR-014 | §3 |
| Work discovery, policy, coverage | JR-015 … JR-019 | §4 |
| Player overlay | JR-020 … JR-024 | §5 |
| Manifest exchange client | JR-025 … JR-037 | §6 |
| Egress and privacy | JR-038 … JR-041 | §7 |
| Audio signature | JR-042 … JR-045 | §8 |
| Human-in-the-loop association | JR-046 | §9 |
**JR-038 … JR-041 exist because `PR-005` had no software row anywhere.** The
system spec notes that "leak nothing about what the user owns" is preserved
structurally — by SR-004 and GR-005 both being prohibitions — and that a goal
held only by prohibitions needs watching. jRay is the component that actually
opens a socket, so it is the right place for that goal to become checkable.
---
## 1. Truth-file format — JR-001 … JR-007
### JR-001 — This document is normative for the format
The truth file is produced by `scene-actor-extraction`, read by this plugin, and
transformed into a Jmanifest by the exchange client. Three repos touch it, so
exactly one must define it, and the [system spec](scripts/vendor/jray-project/SPEC.md) §1 assigns that to
jRay. Other repos reference this section rather than restating the schema.
The truth file is **not** the Jmanifest. It carries installation-local fields
(`movie`, `jellyfin_id`) that the exchange strips, and lacks the portable
identity block the exchange adds. See §6.
**Current:** [`Models/TruthFile.cs`](Jellyfin.Plugin.JRay/Models/TruthFile.cs),
`schema_version: 1`. **Gap:** the schema below is not implemented, and the other
two specs describe the pending bump in more detail than this one does — the
ownership is stated but not yet exercised.
### JR-002 — `schema_version: 2`
```json ```json
{ {
"schema_version": 1, "schema_version": 2,
"movie": "/path/to/Movie.mkv", "movie": "/data/movies/Movie.mkv",
"sample_fps": 1, "extraction": {
"anneal_sec": 2, "sample_fps": 5,
"extinction_sec": 12,
"gallery_size": 1820,
"gallery_scope": "global",
"pipeline_version": "scene-actor-extraction 0.4.1"
},
"cut": {
"runtime_sec": 6420.5,
"audio_signature": "v1:v7fA3k…"
},
"actors": [ "actors": [
{ {
"name": "Tom Hanks", "name": "Steve Buscemi",
"imdb_id": "nm0000158", "imdb_id": "nm0000114",
"tmdb_id": "31", "tmdb_id": "884",
"jellyfin_id": "abc123-guid", "jellyfin_id": "abc123-guid",
"scenes": [[12.0, 45.0], [102.5, 150.0]] "scenes": [
{ "start": 191.6, "end": 209.2, "belief": 0.98, "route": "live" },
{ "start": 438.2, "end": 465.6, "belief": 0.81, "route": "deferred" }
]
} }
] ]
} }
``` ```
- `schema_version`: integer, bump on breaking changes. JRay should refuse (or Field notes:
warn) on a version it doesn't understand.
- `movie`: absolute path to the source media file at extraction time (informational only).
- `sample_fps`: frames-per-second the pipeline sampled at.
- `anneal_sec`: gap (in seconds) below which consecutive detections of the
same actor were merged into a single scene window.
- `actors[]`: one entry per actor detected anywhere in the film.
- `name`: display name from the gallery.
- `imdb_id` / `tmdb_id` / `jellyfin_id`: identity keys, each `""` if not
resolved. JRay should prefer `jellyfin_id` (a Jellyfin Person item GUID)
when non-empty, and otherwise resolve `imdb_id`/`tmdb_id` against the
item's People `ProviderIds`.
- `scenes`: list of `[start_sec, end_sec]` windows (inclusive) during which
the actor is on screen.
## Querying "who's on screen at time t" - `schema_version`**system-level** (SR-003), incremented once per breaking
change and referenced by the same number in all three repos.
- `movie` — absolute path at extraction time, informational only. Stripped on
contribution (JR-034).
- `extraction.*` — provenance. `sample_fps`, `gallery_size` and
`pipeline_version` move here from the top level so the truth file and the
Jmanifest's `extraction` block have the same shape, rather than differing for
no reason.
- `extraction.extinction_sec`**replaces `anneal_sec`**, which is deleted, not
retained as a vestigial `0`. It is the re-acquisition timeout that shapes
window extent, so it is what a consumer needs in order to interpret a window.
- `extraction.gallery_scope``"global"` or `"limited"`. The strongest single
quality signal when two manifests compete for one cut.
- `cut.runtime_sec` — the decoded duration the timings came from. Required for
contribution; the primary alignment guard.
- `cut.audio_signature` — optional, `v1:`-prefixed. See §8.
- `actors[].scenes[]` — objects, not float pairs. `start`/`end` in seconds,
inclusive, sorted. `belief` is the accumulated posterior that justified the
claim; `route` is `"live"`, `"deferred"` or `"pooled"` (extraction AR-017).
For a given timestamp `t` (seconds), an actor is visible if any of their **Belief is an attribute, not part of identity.** Two servers that validated the
`scenes` windows satisfies `start <= t <= end`. same upload must agree on its `content_id`, and belief is a producer-side
estimate that may legitimately differ between pipeline versions for identical
timings. It replicates the way `audio_signature` does — see the server spec §9a.
## API **Gap:** entire requirement. The changes are all breaking and ship as **one**
bump (SR-003), together with extraction `IR-002` and the server's acceptance of
the new shape.
### `GET /Plugins/JRay/Items/{itemId}/Timeline` ### JR-003 — Unknown `schema_version` is refused, never guessed
Returns the full truth file (schema above) for an item, or `404` if no truth **Decision: flag day.** The plugin accepts `schema_version: 2` and rejects
data exists (neither a managed upload nor a sidecar file). everything else, on every path — sidecar read, managed `PUT`, and fetched
manifest. There is no transitional dual-accept.
### `GET /Plugins/JRay/Items/{itemId}/jray?t={seconds}` All three components are pre-release and move together, and the alternative
carries a cost that outlasts the transition: a v1 read path is the one nobody
exercises, so it is the one that rots, and it would have to be carried through
every subsequent change to the reader.
Returns an extensible "context at time t" envelope, or `404` if no truth **The consequence must be stated plainly rather than discovered:** existing v1
data exists for the item: sidecar files on disk **stop being read** at the bump, and stay dark until the
library is re-extracted. The plugin logs this per item, naming the file and the
version found, rather than silently reporting no coverage — an item that looks
un-extracted when it was merely stale is the failure mode that wastes a user's
compute.
**Current:** `PUT` rejects `schema_version != 1` with `400`; sidecar reads do not
check the version at all. **Gap:** the version check must move into the shared
read path so all three sources are covered, and the target becomes `2`.
### JR-004 — A window is a scene-membership claim
**This is SR-002, and it binds this plugin harder than it binds anything else,**
because jRay is where the claim reaches a human.
An actor who turns away, is occluded, or is off-camera while the shot cuts to
whoever they are speaking to **is still present**. Two windows mean a genuine
departure and return, not a break in detection. Gaps shorter than
`extinction_sec` were absorbed upstream and are claimed as presence.
The plugin therefore **never reinterprets, merges, splits, or trims windows.**
It stores and serves what it was given. The one permitted transformation is the
timebase offset of JR-030, which shifts every window uniformly and so preserves
the claim.
**Gap:** stated nowhere in the code today. The read path happens to comply, but
by not having been written to do otherwise rather than by requirement.
### JR-005 — Query semantics, and how presence is presented
An actor is present at `t` if any window satisfies `start <= t <= end`.
**The presentation must not assert instantaneous visibility.** SR-002 is explicit
that a consumer must never interpret window boundaries as "the face was detected
here", and the overlay is the exact place that misreading would be made
user-visible. "On screen now" is a claim the data does not support; "in this
scene" is the claim it does.
This is a wording requirement, not a hedge — it is the difference between the
product being right and being a worse version of a frame-by-frame detector.
**Current:** the query is implemented correctly in
[`ActorsController`](Jellyfin.Plugin.JRay/Controllers/ActorsController.cs).
**Gap:** wording, not logic. The overlay renders a bare list with **no heading
at all**, so it asserts nothing — but it also tells the viewer nothing about
what the list means, and a viewer's default reading of a paused frame is "these
people are on screen". [`README.md`](README.md) states that reading outright
("which actors are on screen at that exact moment"), and the model type is
`ActorAtTime`.
### JR-006 — Numerous windows
SR-002 warns that windows may be numerous and consumers must not assume a handful
of long ones. Track-extent presence with a short `extinction_sec` produces many
short windows per actor, and the previous design's few long ones were an artefact
of the over-claiming that was removed.
The read path must therefore treat per-actor windows as a sorted sequence to be
searched, not a short list to be scanned, and the `jray?t=` response must stay
small regardless of how many windows an actor has.
**Gap:** windows are scanned linearly and the whole truth file is held per item.
Adequate at current sizes; unmeasured, and unstated until now.
### JR-007 — Identity is public identifiers
Each actor carries `imdb_id`, `tmdb_id` and `jellyfin_id`, any of which may be
`""`. Resolution prefers `jellyfin_id` (a Jellyfin Person GUID) when non-empty,
and otherwise matches `imdb_id`/`tmdb_id` against the item's People
`ProviderIds`. Never a name alone — names are ambiguous and unstable (SR-001).
`jellyfin_id` is the exception that proves the rule: it is meaningful only on the
instance that produced it, which is exactly why the exchange strips it (JR-034).
**Current:** implemented. **Gap:** none.
---
## 2. Truth-data sources and precedence — JR-008 … JR-011
### JR-008 — Sidecar discovery
For `Movie.mkv`, the plugin looks for `Movie.jray.json` beside it — suffix
configurable, default `.jray.json`, resolved from the item's media source path.
**Current:** implemented. **Gap:** none.
### JR-009 — Managed truth push
`PUT`/`DELETE .../Truth` let a worker that cannot write beside the media file
deliver results over HTTP. Stored under the plugin's configuration directory,
keyed by item id, independent of the library filesystem.
**Current:** implemented. **Gap:** none.
### JR-010 — Precedence and provenance
There are now **three** sources: a sidecar file, a push from a local worker, and
a manifest fetched from a server. Managed truth — pushed *or* fetched — takes
precedence over a sidecar.
**Fetched manifests are stored through the managed store**, so precedence stays a
two-way rule rather than a three-way one, and the read path does not learn about
the exchange at all.
But the three are no longer interchangeable, so **provenance is recorded with the
stored truth**: which source it came from, and for a fetched one, which server
and at what match tier. A `loose`-tier fetch from a third-party server and a
locally-computed sidecar are not the same claim, and JR-036 requires the
difference be surfaceable.
**Current:** two-way precedence implemented in
[`TruthDataService`](Jellyfin.Plugin.JRay/Services/TruthDataService.cs).
**Gap:** no provenance is recorded.
### JR-011 — Caching
Loaded truth is cached in memory for a configurable duration. Any write —
managed `PUT`, `DELETE`, or a stored fetch — invalidates that item's entry
immediately, so a push takes effect without waiting for expiry.
**Current:** implemented. **Gap:** none.
---
## 3. Read API — JR-012 … JR-014
### JR-012 — `GET /Plugins/JRay/Items/{itemId}/Timeline`
Returns the full truth file (§1), or `404` if no truth data exists from any
source.
### JR-013 — `GET /Plugins/JRay/Items/{itemId}/jray?t={seconds}`
Returns an extensible "context at time `t`" envelope, or `404`:
```json ```json
{ {
"actors": [ "actors": [
{ "name": "Tom Hanks", "imdb_id": "nm0000158", "tmdb_id": "31", "jellyfin_id": "abc123-guid" } { "name": "Steve Buscemi", "imdb_id": "nm0000114", "tmdb_id": "884", "jellyfin_id": "abc123-guid" }
] ]
} }
``` ```
Future fields (e.g. `locations`, `trivia`) will be added to this object Future fields (`locations`, `trivia`, and per JR-005 a presence caveat) are added
without changing the route, so clients should ignore unknown keys. to this object without changing the route, so **clients must ignore unknown
keys**.
### `PUT /Plugins/JRay/Items/{itemId}/Truth` ### JR-014 — Authorisation
For servers that cannot run the extraction pipeline locally, a remote worker | Route | Requires |
may push truth data directly. Requires an administrator API key. Body is a |---|---|
truth file (schema above). Returns `204` on success, or `400` if | `Timeline`, `jray?t=` | Authenticated Jellyfin user token |
`schema_version` is not `1`. | `Truth`, `Tasks/*`, `Policy/*`, `Coverage/*`, and §6's fetch routes | **Administrator** role |
| `ClientScript` | Anonymous — it is injected into a page served before login |
This "managed" truth data takes precedence over any sidecar **Current:** all three implemented as stated. **Gap:** none.
`Movie.jray.json` file for the same item, and is stored independently of the
media library filesystem.
### `DELETE /Plugins/JRay/Items/{itemId}/Truth` ---
Removes managed truth data for an item (idempotent, always returns `204`). ## 4. Work discovery, policy and coverage — JR-015 … JR-019
The item falls back to its sidecar truth file, if any, on subsequent reads.
Requires an administrator API key.
### `GET /Plugins/JRay/ClientScript` ### JR-015 — `GET /Plugins/JRay/Tasks/Pending?limit=10`
Serves the pause-overlay script that JRay injects into the web client's A **random** sample (default 10, max 100) of movies and episodes with no truth
`index.html` (see below). Anonymous access. data:
### `GET /Plugins/JRay/Tasks/Pending?limit=10`
Lets a remote extraction worker discover what to work on next. Returns a
random sample (default 10, max 100) of movies/episodes in the library that
have no truth data yet (neither a managed upload nor a sidecar file):
```json ```json
[ [ { "item_id": "abc123-guid", "path": "/data/movies/Movie.mkv", "name": "Movie" } ]
{ "item_id": "abc123-guid", "path": "/data/movies/Movie.mkv", "name": "Movie" }
]
``` ```
Requires an administrator API key. The sample is random and unordered, so Randomness is the design: repeated polling spreads work across the backlog
repeated polling naturally spreads work across the backlog without needing without the server tracking who is working on what, and two workers polling
server-side task tracking; an empty array means there's nothing left to do concurrently mostly do not collide. An empty array means nothing is left, or that
(or every remaining item is a virtual/missing-path item that JRay can't everything remaining is a virtual/missing-path item.
process).
## Client: pushing results from a remote extraction worker ### JR-016, JR-017 — Prioritise / ignore rules
A worker that runs the extraction pipeline on a different machine than Rules steer the queue: each targets a **genre**, a **series**, or an **item**,
Jellyfin (i.e. it cannot write a `Movie.jray.json` sidecar next to the media and either prioritises (front of the queue) or ignores (hidden entirely). This is
file) can push results directly over HTTP. how an admin says "never extract anime", "this series first", or "skip this one".
To find work, poll `GET /Plugins/JRay/Tasks/Pending?limit=10` (see above) for Resolution picks the **most specific** match: `Item` > `Series` > `Genre`. A rule
a random batch of items that still need processing, instead of walking the is keyed by scope + value, and setting one replaces any existing rule for the
whole library and checking each item's `Timeline`/sidecar yourself. same key — so a single target can never be simultaneously prioritised and
ignored. Cross-scope conflicts (a prioritised series inside an ignored genre) are
resolved by specificity: the series wins.
### 1. Authenticate ```json
{ "scope": "Genre", "value": "Anime", "action": "Ignore", "label": "Anime" }
Create an **Administrator** API key in Jellyfin (Dashboard → API Keys), and
send it on every request as either:
```
X-Emby-Token: <api-key>
``` ```
or: `value` is a genre name, a series id GUID, or an item id GUID; genre matching is
case-insensitive. Persisted to `policy.json` in the plugin's configuration
directory.
``` **JR-017 is the constraint worth stating separately: rules affect work discovery
Authorization: MediaBrowser Token="<api-key>" only.** They never change the overlay or the read endpoints. An item you ignore
for extraction still shows its overlay if truth data happens to exist — because
the rule expresses "don't spend compute here", not "pretend this doesn't exist".
Endpoints: `GET`/`PUT /Plugins/JRay/Policy/Rules`, and
`DELETE /Plugins/JRay/Policy/Rules?scope=&value=` (idempotent).
### JR-018 — `GET /Plugins/JRay/Coverage`
How much of the library has truth data, overall and by media type and genre:
```json
{
"total": { "total": 1200, "covered": 300, "pending": 850, "prioritised": 40, "ignored": 50 },
"by_media_type": [ { "label": "Film", "counts": { "…": 0 } } ],
"by_genre": [ { "label": "Anime", "counts": { "…": 0 } } ]
}
``` ```
### 2. Resolve the Jellyfin item id `prioritised` is a subset of `pending`. Percent done is
`covered / (total - ignored)` — **ignoring a genre does not drag the percentage
down**, because ignored items are intentionally out of scope, not outstanding
work. An item counts toward every genre it carries, so genre rows overlap and
need not sum to the library total.
The push endpoint is keyed by the Jellyfin item GUID, not by file path. To ### JR-019 — Pickers
find it for `Movie.mkv`:
`Coverage/Genres`, `Coverage/Series`, and `Coverage/Items?search=&limit=`
populate the rule editor's dropdowns, each returning
`[{ "value": …, "label": … }]`. An absent `search` returns `[]`.
**Current (JR-015 … JR-019):** all implemented. **Gap:** none functionally —
these traced to no requirement until this register existed, which by the gate's
own definition read as scope creep. They serve PR-003 (fully automatic, no
per-title manual work): steering a queue is how automation is directed without
becoming per-title labour.
---
## 5. Player overlay — JR-020 … JR-024
### JR-020 — The overlay
Jellyfin has no plugin hook for player UI, so jRay adds
`<script defer src="/Plugins/JRay/ClientScript"></script>` to the web client's
`index.html`. The script listens for the player's pause event, calls `jray?t=`
for the current item and timestamp, and renders the scene's cast.
Per JR-005 it presents scene membership, not instantaneous visibility.
### JR-021 — jRay never injects into `index.html` on disk
**File Transformation is a hard requirement, not a preference. There is no
on-disk patching fallback.**
The prohibition is on **injection**, not on writing: JR-022's migration must
write to the file in order to remove a legacy patch. Stating it as "never
writes" would put the two requirements in contradiction, and the static check
would have to be disabled to let the migration through — so the check is that no
code path *adds* the script tag.
At startup jRay looks for the
[File Transformation](https://github.com/IAmParadox27/jellyfin-plugin-file-transformation)
assembly via `AssemblyLoadContext` and, if present, calls
`Jellyfin.Plugin.FileTransformation.PluginInterface.RegisterTransformation` by
reflection — no compile-time dependency, so jRay loads normally when it is
absent. The payload:
```json
{
"id": "2c9b5a41-6ad0-4c1e-9f7d-1d1e6b0d5a90",
"fileNamePattern": "index.html",
"callbackAssembly": "<jRay assembly full name>",
"callbackClass": "Jellyfin.Plugin.JRay.Services.FileTransformationRegistration",
"callbackMethod": "TransformIndexHtml"
}
```
File Transformation matches `callbackAssembly` against `Assembly.FullName`
exactly, so the full display name is sent. The callback is a public static method
taking a payload with a `contents` string and returning the transformed string;
the payload binds with Newtonsoft, which matches property names
case-insensitively. Registration is unconditional at startup — the callback
itself checks the "enable overlay" setting per request, so toggling takes effect
without re-registering.
Writing to `index.html` is rejected because it is destructive in ways a plugin
cannot clean up after:
- **It outlives the plugin.** Uninstalling jRay leaves the patch in a file jRay
no longer owns.
- **It breaks on upgrade.** A web-client update replaces the file, discarding the
patch — or preserves one pointing at an endpoint that has since changed.
- **It collides.** Another plugin patching the same file races with jRay, and the
loser's edit is lost with no diagnostic.
- **It is a second code path**, and it is the one nobody runs, so it is the one
that rots.
**Current:** satisfied.
[`WebClientPatchService`](Jellyfin.Plugin.JRay/Services/WebClientPatchService.cs)
is removal-only — the injection capability is *deleted*, not switched off, since
dead code with a live signature is what a later refactor re-enables by accident.
Enforced by
[`scripts/checks/no-index-injection.sh`](scripts/checks/no-index-injection.sh),
which was verified to fail on a reintroduced injection rather than merely to pass
today. [`README.md`](README.md) no longer advertises a fallback. **Gap:** none.
### JR-022 — Migrate away from earlier on-disk patches
Users upgrading from a version that patched the file must not be left with a
stale injection. On startup jRay removes any on-disk patch bearing its own
`<!-- jray-overlay -->` marker — unambiguous, and touching nothing another plugin
added.
**Current:** implemented — `WebClientPatchService.RemoveLegacyPatch` runs at
every startup and is a no-op once the marker is gone. The strip itself is
factored out as `RemoveInjection` so it is unit-testable without a filesystem.
**Gap:** no test executes it yet, so this stays `In Progress` rather than
`Done` — there is no test project in this repo.
### JR-023 — A hard dependency Jellyfin cannot resolve
**Jellyfin has no plugin dependency mechanism.** A manifest cannot declare that
another plugin is required, and nothing will install one. File Transformation
documents only an end-user repository URL and a reflection integration for plugin
authors; there is no NuGet-style dependency to take.
**jRay must not bundle the assembly.** A bundled copy would sit in a different
`AssemblyLoadContext` from the real one — precisely the failure the reflection
integration exists to avoid — on top of licensing and version skew. The
dependency is satisfied by the user installing the real plugin.
So:
1. **Detect at startup and say so** — log a warning naming the plugin and its
install URL, and disable only the overlay. Every other feature works.
2. **Surface it where it can be acted on** — the configuration page shows
dependency status: satisfied, or missing with the manifest URL
`https://www.iamparadox.dev/jellyfin/plugins/manifest.json` and a one-line
instruction. A warning only in the server log is one nobody reads.
3. **State it as a prerequisite in install docs**, before the jRay install step.
Optionally, publish jRay through a repository manifest that also lists File
Transformation, so one repository URL surfaces both. This is not a dependency
mechanism; it removes a step and the chance of installing the wrong thing.
**Current:** all three implemented. Startup detection and registration, a warning
naming the plugin and its install URL, and a status banner on the configuration
page fed by `GET /Plugins/JRay/Status/Dependencies` — satisfied, or missing with
the manifest URL and what to do with it. The README now states the dependency
before the install step rather than after it. **Gap:** no test executes the
detection branch, so this stays `In Progress`; the config-page half is T4 and
verifiable only against a live server.
### JR-024 — Names render as text, never markup
Every string that reaches the overlay — actor names above all — is rendered as
text. With §6 the source of those strings may be a third-party server, and the
server spec §5a names this the single most important client-side control,
because it holds even when every other check is bypassed.
It is stated here as a plugin requirement because the server cannot enforce it
and the DOM is jRay's.
**Current:** satisfied. [`Web/jray-overlay.js`](Jellyfin.Plugin.JRay/Web/jray-overlay.js)
uses `textContent` throughout — no `innerHTML`, no `insertAdjacentHTML`. **Gap:**
nothing behavioural. It holds today by construction rather than by rule, which
is what the static check exists to keep true once §6 makes remote strings
reachable.
---
## 6. Manifest exchange client — JR-025 … JR-037
The wire format, tiers and server behaviour are specified in
[`../JRay-public-server/SPEC.md`](../JRay-public-server/SPEC.md). **This section
owns the client half**, which previously lived in that document's §9 — an
inversion, since those are obligations on this repo.
A Jmanifest is this truth file plus a portable identity block and a cut
fingerprint, minus the installation-local fields.
### JR-025, JR-026, JR-037 — Server list and resolution
The plugin queries a configured **ordered list**, not a single URL. Per server:
`Url`, `Name`, `Token` (contribution only), `Enabled`, `AllowContribute`,
`TrustLevel` (`Full` / `FetchOnly`). A community entry ships pre-configured but
**disabled**.
**First acceptable wins** — servers are tried in order, and the first result
clearing the configured tier is taken. Order *is* the user's trust ranking, made
explicit. Best-match-across-all would multiply egress and leak the library to
more parties for a gain the ordering already expresses.
**For a series, first-match applies per episode** (JR-026): fetch the bundle from
server 1, then query server 2 only for what is still missing. Series are commonly
split across sources, and this is where multiple servers earn their keep.
**Failure isolation** (JR-037): an unreachable or failing server is skipped after
a short timeout (5 s connect, 30 s read) and marked failed with exponential
backoff. One dead server must never stall a library sweep; failures surface
per-server in the config page.
**Current:**
[`ManifestServer`](Jellyfin.Plugin.JRay/Configuration/ManifestServer.cs) and
[`PluginConfiguration`](Jellyfin.Plugin.JRay/Configuration/PluginConfiguration.cs)
model all of this. **Gap:** nothing consumes them — there is no HTTP client.
### JR-027 … JR-029 — Every server is untrusted
Everything in the server spec §5a is a property of a *correctly operated* server.
Pointing the plugin at an arbitrary URL inherits none of it. jRay therefore
re-applies client-side what a server applies on upload, **including for the
default server**:
- **JR-027 — validate on receipt.** Downloaded manifests go through the same
strict schema as uploads: unknown fields rejected, sizes capped, and windows
bounds-checked against the item's real runtime. A manifest is never trusted
because a server served it.
- **JR-028 — size caps enforced while streaming**, so an unbounded body is
aborted rather than buffered. 2 MiB single manifest, 25 MiB bundle.
- **JR-029 — HTTPS required** for non-loopback servers, with certificate
validation never disabled. A plaintext server would let any intermediary
rewrite actor overlays.
- **`TrustLevel: FetchOnly`** — the default for user-added servers — accepts
manifests but never contributes and never sends inventory beyond the single
item queried.
The honest framing for the config page: *adding a third-party server means
trusting its operator not to serve you deliberately wrong actor data.* The
controls above bound the damage to bad overlay content; they cannot make wrong
data right.
### JR-030 — Offsets are applied before storage
When a match carries a non-zero `offset` (the `audio` tier — §8), the plugin
**must** add it to every scene window before storing.
**The stored truth file is always in the local file's own timebase.** This is
what keeps the offset out of the read path entirely: `Timeline`, `jray?t=` and
the overlay never learn that an offset existed. An offset applied at read time
would have to be applied identically in three places and would be wrong in the
fourth.
### JR-031, JR-032 — Endpoints
Mirroring the existing Truth and Tasks controllers:
| Route | Purpose |
|---|---|
| `POST /Plugins/JRay/Items/{itemId}/Fetch` | Resolve across servers in order; on a match at or above the configured tier, apply JR-030 and store via the managed store |
| `POST /Plugins/JRay/Series/{seriesId}/Fetch` | Bundle fetch with per-episode gap-filling |
| `GET /Plugins/JRay/Servers/Status` | Per-server reachability and last error, for the config page |
| `POST /Plugins/JRay/Items/{itemId}/Identify` | Compute the audio signature and search by content, for items of unknown providence |
**JR-032: `Identify` never stores automatically.** It returns candidate titles
with scores and offsets; storing one is a separate confirmation step. Content
identification is a guess about what a file *is*, and a wrong guess silently
attaches another film's cast to it.
### JR-033 — Scheduled sweep
A scheduled task walks items with no truth data and attempts a fetch, reusing the
`Tasks/Pending` backlog logic — including its policy rules — and the **batch**
`exists` endpoint, so a sweep is a handful of requests per server rather than one
per item.
### JR-034, JR-035 — Contribution
On a `PUT .../Truth` from a local worker, if contribution is enabled: strip
`movie` and `jellyfin_id`, attach identity from the item's `ProviderIds` and its
measured runtime, and `POST` to each contribute-enabled server. For a series,
batch into one bundle upload rather than per-episode posts.
**Stripping is a requirement, not hygiene.** `movie` leaks the contributor's
directory layout and `jellyfin_id` is a GUID from their database — meaningless
elsewhere and mildly identifying. The server rejects both, but the plugin must
not send them in the first place.
**Contribution is never fanned out.** A manifest goes only to servers with
`AllowContribute` set, each an explicit choice.
**JR-035:** uploads set `Expect: 100-continue`, so a server rejecting on size or
auth does so before the body is transmitted. This matters most for bundles, where
a rejected upload would otherwise push tens of MiB pointlessly.
### JR-036 — Match tier is the user's dial
The configured minimum tier (`exact` / `audio` / `runtime` / `loose`) gates what
may be stored. A `loose` match — runtimes within ±30 s — is plausibly a different
trim of the same cut, so it is **surfaced as a caveat in the UI**, not applied
silently. Per JR-010 the tier is recorded with the stored truth, which is what
makes surfacing it possible after the fetch has finished.
**Current:** `MinimumMatchTier` exists in configuration, defaulting to `runtime`.
**Gap:** nothing reads it; no caveat is displayed.
---
## 7. Egress and privacy — JR-038 … JR-041
Contribution reveals to a server operator that some instance holds a given title.
Fetching reveals the same. That is inherent to the exchange — which is why the
requirements here bound it rather than claim to remove it.
- **JR-038 — opt-in, off by default.** Manifest sharing, contribution and audio
signatures are three separate switches, all default off, and the pre-configured
community server ships **disabled**. No traffic leaves an installation until an
admin acts.
- **JR-039 — no library-wide inventory in one request.** The batch `exists`
endpoint is capped at 100 items and sweeps are paced. A single request
enumerating a library is a fingerprint of it, which is the thing PR-005 exists
to prevent.
- **JR-040 — the config page says plainly that each configured server multiplies
the exposure.** First-match resolution limits it — later servers are queried
only for what earlier ones lacked — and that is worth stating too.
- **JR-041 — the plugin never touches gallery data.** No reference faces, no
embeddings, fetched or stored or transmitted. There is no such code path and
there must not be one (SR-005). Verified by static check, mirroring the
server's UR-012.
**Current:** JR-038 holds — every switch defaults off. JR-041 holds vacuously,
there being no gallery code. **Gap:** JR-039 and JR-040 are unimplemented,
alongside the exchange client itself.
---
## 8. Audio signature — JR-042 … JR-045
A content-derived fingerprint from the centre of a media file, used to identify a
file of unknown providence and to recover the time offset between differently
trimmed releases of the same cut. Construction is specified in
[`../JRay-public-server/SPEC.md` §3](../JRay-public-server/SPEC.md) and must be
implemented **exactly**:
1. Decode a 120 s window centred on the midpoint (`runtime/2 ± 60 s`) — avoiding
logos and cold opens at the head, credits at the tail.
2. Downmix to mono, resample to 11025 Hz.
3. STFT: 4096-sample frame, 1024-sample hop (~93 ms, ~1290 frames), Hann window.
4. Log-magnitude spectrum over 3003000 Hz.
5. 32 logarithmically spaced bins; record peak-bin index plus a 2-bit energy
class.
6. One byte per frame → ~1290-byte array, base64-encoded.
**JR-042 — no new dependency.** FFmpeg performs decode, downmix and resample,
using the binary Jellyfin already ships, reached via `IMediaEncoder.EncoderPath`
from `MediaBrowser.Controller.MediaEncoding`. The plugin implements only a small
fixed FFT and bin-peak extraction.
**JR-043 — bit-exactness is verified, not assumed.** The pipeline computes this
signature too (extraction `IR-004`), deliberately: files never processed locally
still get one from the plugin. Two independent implementations of one fingerprint
are only useful if they agree exactly, so a **golden-vector fixture is shared
between the two repos** — a short WAV and its expected signature, committed in
both. It is CPU-only DSP, which is why this cross-repo check can be a binding CI
test rather than an aspiration. Extraction's counterpart is `IR-005`.
**JR-044 — media shorter than 120 s.** The window underflows, so **no signature
is emitted and no sync offset is applied**. Such items fall back to the runtime
and exact tiers, which is adequate: a 90-second extra is not content whose cut
alignment matters. Both producers must apply the identical rule, or they diverge
on exactly the short items most likely to be misidentified. Extraction's
counterpart is `IR-007`.
**JR-045 — the signature carries its own `v1:` prefix**, separate from
`schema_version`. Emit and honour it, so a future change to the DSP chain is
*detectable* rather than silently producing non-matching signatures. Extraction's
counterpart is `IR-008`.
Matching — sliding ±600 frames (≈±56 s), scoring the fraction of overlapping
frames whose peak bin matches — is a **consumer** concern and belongs to this
plugin. Offsets are applied client-side per JR-030; manifests are never
rewritten.
**Current:** `ComputeAudioSignatures` exists as a configuration switch. **Gap:**
entire requirement, both computation and matching.
---
## 9. Human-in-the-loop association — JR-046
*Proposed. See [system spec](scripts/vendor/jray-project/SPEC.md) §4, which owns the design.*
The pipeline produces **unidentified tracks** — a face that is genuinely someone,
sustained across many frames, that the gallery cannot name. A user watching the
film usually knows exactly who it is. jRay's contribution is the review UI: show
a cluster's context crops, let the user pick from the title's cast or search
TMDB, and record the association for extraction to ingest into the local gallery.
**The unit of review is a person, not a track.** Unknown tracks are clustered
upstream (`AR-021`), so the question is "who is this person, who appears in these
twelve places?" rather than twelve disconnected questions. One answer resolves
the cluster.
Deliberately left as a single `TBD` row rather than decomposed. It depends on
extraction `AR-021`/`AR-022` landing, and on **system open question 2** — whether
unidentified presence is published in the truth file at all, which determines
whether this UI's work queue arrives with the truth data or needs a separate
channel. Decomposing now would fix an interface against an undecided upstream.
---
## 10. Client: pushing results from a remote worker
Reference material for worker authors; the requirements are JR-009 and JR-015.
**1. Authenticate.** Create an Administrator API key (Dashboard → API Keys) and
send it as `X-Emby-Token: <key>` or
`Authorization: MediaBrowser Token="<key>"`.
**2. Find work.** Poll `GET /Plugins/JRay/Tasks/Pending?limit=10` rather than
walking the library and checking each item.
**3. Resolve the item id.** The push endpoint is keyed by Jellyfin item GUID, not
path:
``` ```
GET /Items?Recursive=true&Fields=Path&IncludeItemTypes=Movie,Episode GET /Items?Recursive=true&Fields=Path&IncludeItemTypes=Movie,Episode
``` ```
(use `&ParentId=<library-id>` to narrow the search if the library is large). Match `Path` against the file you processed — which requires the worker to see
Each returned item DTO has `Id` (the GUID) and `Path`. Match `Path` against the file at the *same path* Jellyfin does; translate first if it mounts the
the absolute path of the file you just processed — note this requires the library elsewhere. The mapping is stable until the file moves, so cache
worker to see the file at the *same path* Jellyfin does (same mount/share); `path -> itemId` and re-resolve only on a miss.
translate paths first if the worker mounts the library elsewhere.
This mapping is stable until the file is moved/re-scanned, so the worker **4. Push.** `PUT /Plugins/JRay/Items/{itemId}/Truth` with the truth file body.
should cache `path -> itemId` and only re-resolve on a cache miss. `204` stored (cache invalidated immediately), `400` unsupported
`schema_version`, `401`/`403` key missing or not an administrator. The `PUT` is
idempotent, so retrying on a network error is safe.
### 3. Push the truth file **5. Optionally remove.** `DELETE /Plugins/JRay/Items/{itemId}/Truth` always
returns `204`; the item falls back to its sidecar on the next read.
``` ---
PUT /Plugins/JRay/Items/{itemId}/Truth
Content-Type: application/json
<truth file JSON, schema_version 1, as produced by result_sink_node> ## 11. Open questions
```
- `204 No Content` — stored. Takes effect immediately (any cached read for 1. **`sample_fps`, `gallery_size` and `pipeline_version` move under
this item is invalidated server-side). `extraction.*` in JR-002.** This aligns the truth file with the Jmanifest's
- `400 Bad Request``schema_version` is not `1`. block of the same name, and the bump is breaking regardless. It is a change
- `401`/`403` — API key missing or not an administrator. this spec proposes rather than one inherited from SR-003's list — confirm, or
keep them top-level.
The `PUT` is idempotent (replaces any existing managed truth for the item), 2. **Does `route` belong in the `jray?t=` envelope?** JR-013 says the response is
so the worker can safely retry on network errors. extensible and JR-005 says presentation must not over-claim. Exposing belief
and route would let the overlay caveat a weak claim, but invites a UI that
### 4. (Optional) Remove pushed data shows a number to a viewer who cannot act on it.
3. **System open question 2 — unidentified presence.** If published, the overlay
``` could show "unidentified person" and JR-046 gets its queue from the truth file
DELETE /Plugins/JRay/Items/{itemId}/Truth directly. jRay is the consumer that would have to display it, so this repo has
``` a position to state.
4. **Test-ID namespacing.** `UT-nnn`/`IT-nnn` are per-component registers, so
Always returns `204`. The item falls back to a sidecar `Movie.jray.json` (if `UT-001` will exist in both this repo and `scene-actor-extraction`. Fine while
any) on the next read. the gate runs per repo; ambiguous the moment a rollup spans them.
## Web client pause overlay
Since Jellyfin has no plugin hook for player UI, JRay injects
`<script defer src="/Plugins/JRay/ClientScript"></script>` into the web
client's `index.html` on startup (idempotent, marked with
`<!-- jray-overlay -->`). The injected script listens for the video player's
pause event, calls `jray?t=` for the current item and timestamp, and renders
a small overlay listing on-screen actors. This can be disabled via the
plugin's "Enable pause overlay" setting, which also removes the injected
script.
+6 -1
View File
@@ -19,4 +19,9 @@ dotnet_configuration: "Release"
dotnet_framework: "net9.0" dotnet_framework: "net9.0"
project: "Jellyfin.Plugin.JRay/Jellyfin.Plugin.JRay.csproj" project: "Jellyfin.Plugin.JRay/Jellyfin.Plugin.JRay.csproj"
changelog: > changelog: >
Initial scaffold The pause overlay is now served through the File Transformation plugin, which
rewrites the web client's index.html as it is sent. JRay no longer edits that
file on disk and there is no fallback: without File Transformation the overlay
is disabled and every other feature works normally. Any stale on-disk patch
left by an earlier JRay is removed on startup, and the plugin's configuration
page now reports whether the dependency is satisfied.
+269
View File
@@ -0,0 +1,269 @@
# jRay — requirements register
Stable IDs for every requirement in [`../SPEC.md`](../SPEC.md), which holds the
prose. This file is the **authoritative list**; the CI gate reads its
denominators from here (see [`../scripts/vendor/jray-project/SPEC.md`](../scripts/vendor/jray-project/SPEC.md) §6).
**IDs are permanent.** A withdrawn requirement is marked `Withdrawn` and its
number is never reused — renumbering is what produces orphan TRACES tags. This
register replaces the earlier section-numbering of `SPEC.md`, which gave the
plugin no way to be traced to and left it outside the chain entirely.
Tag code with `// TRACES: JR-012 | SR-002`.
| Type | Scope |
|---|---|
| `JR` | Everything this plugin does — truth format, API, overlay, exchange client |
| `UT` / `IT` | Unit / integration tests |
`JR` is flat rather than split by theme. The plugin is one deployable with one
audience, and the thematic grouping lives in the section headings below, where it
costs nothing and cannot go stale against a prefix.
Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
---
## Truth-file format (JR-001 … JR-007)
jRay **owns** this format ([system spec](../scripts/vendor/jray-project/SPEC.md) §1); extraction is the
producer and the public server carries a derived envelope. Changes are
coordinated `schema_version` bumps (SR-003).
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| JR-001 | The truth-file format is normatively defined here; other repos reference it rather than restating it | SR-003 | High | In Progress |
| JR-002 | `schema_version: 2` shape — `extraction.*` provenance block, `cut.*` block, `scenes` as objects carrying belief and route | SR-003 | High | Planned |
| JR-003 | Reject an unknown `schema_version`, never guess. **Flag day: v2 only**, no dual-accept | SR-003 | High | Planned |
| JR-004 | A window is a **scene-membership claim**, not a recognition event — never reinterpreted, merged, split or trimmed | **SR-002** | High | Planned |
| JR-005 | Query semantics: actor present at `t` if any window contains `t`; presentation must not assert instantaneous visibility | **SR-002** | High | In Progress |
| JR-006 | Read path holds up under **numerous** windows — no assumption of a handful of long ones | SR-002 | Medium | Planned |
| JR-007 | Identity is public identifiers: prefer `jellyfin_id` locally, else resolve `imdb_id`/`tmdb_id` against the item's People `ProviderIds` | SR-001 | High | Done |
## Truth-data sources and precedence (JR-008 … JR-011)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| JR-008 | Discover a sidecar truth file beside the media, by configurable suffix | PR-001 | High | Done |
| JR-009 | Accept truth data pushed by a remote worker (`PUT`/`DELETE`), admin key | PR-004 | High | Done |
| JR-010 | Precedence: managed truth (pushed **or** fetched) overrides a sidecar; provenance is recorded so the UI can distinguish the three sources | PR-001 | High | In Progress |
| JR-011 | Loaded truth is cached; any write invalidates the item's cache entry immediately | PR-001 | Medium | Done |
## Read API (JR-012 … JR-014)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| JR-012 | `GET .../Timeline` returns the full truth file for an item | PR-001 | High | Done |
| JR-013 | `GET .../jray?t=` returns an **extensible** context envelope; consumers ignore unknown keys | PR-001 | High | Done |
| JR-014 | Authorisation: reads need an authenticated user, admin routes need the Administrator role, only `ClientScript` is anonymous | PR-004 | High | Done |
## Work discovery, policy and coverage (JR-015 … JR-019)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| JR-015 | `Tasks/Pending` serves a random sample of items with no truth data, so pollers spread across the backlog without server-side task state | PR-003 | High | Done |
| JR-016 | Prioritise/ignore rules scoped `Genre` / `Series` / `Item`; **most specific wins**; scope+value is the unique key | PR-003 | Medium | Done |
| JR-017 | Rules steer **work discovery only** — never the overlay or the read endpoints | PR-003 | Medium | Done |
| JR-018 | Coverage report by media type and genre; ignored items leave the percent-done denominator rather than dragging it down | PR-003 | Medium | Done |
| JR-019 | Picker endpoints (genres, series, item search) populate the rule editor | PR-003 | Low | Done |
## Player overlay (JR-020 … JR-024)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| JR-020 | Pause overlay: injected client script queries `jray?t=` and renders the scene's cast | **PR-001** | High | Done |
| JR-021 | **jRay never injects into `index.html` on disk.** File Transformation is a hard dependency; there is no on-disk fallback. The only permitted write is JR-022's removal | PR-004 | High | **Done** |
| JR-022 | Migration: remove any on-disk patch left by an earlier jRay, identified by the `<!-- jray-overlay -->` marker | PR-004 | High | In Progress |
| JR-023 | Absent the dependency, disable **only** the overlay and say so in the log and the config page; never bundle the assembly | PR-004 | Medium | In Progress |
| JR-024 | Actor names and all server-supplied strings render as **text, never markup** | SR-004 | High | Done |
## Manifest exchange client (JR-025 … JR-037)
Plugin-side requirements for the exchange specified in
[`../../JRay-public-server/SPEC.md`](../../JRay-public-server/SPEC.md) §9. The
wire format is the server's; **the client's obligations are jRay's**, and belong
in this register rather than in the server's spec.
The server's register already anticipates this: its `UR-007` is recorded as
having "no server-side test and cannot have one — it is a requirement on the
plugin", to be cross-referenced from the plugin's register once one exists. This
is that register, and `JR-025` is that row. `UR-007` should now point here and
stay `In Progress` until `JR-025` is `Done`.
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| JR-025 | Query an **ordered list** of servers; first result clearing the configured tier wins — **satisfies `JRay-public-server` UR-007** | PR-006 | High | In Progress |
| JR-026 | For a series, first-match applies per **episode** — later servers are queried only for the episodes earlier ones lacked | PR-006 | Medium | Planned |
| JR-027 | Treat **every** server as untrusted, including the default: re-validate on receipt against the strict upload schema, bounds-check windows against the item's real runtime | SR-004 | High | Planned |
| JR-028 | Enforce response size caps **while streaming** — 2 MiB single, 25 MiB bundle — aborting rather than buffering | SR-004 | High | Planned |
| JR-029 | HTTPS required for non-loopback servers; certificate validation must not be disabled | SR-004 | High | Planned |
| JR-030 | Apply an `audio`-tier `offset` to **every** window before storing — stored truth is always in the local file's timebase, so read paths need no offset awareness | SR-003 | High | Planned |
| JR-031 | Fetch endpoints: item fetch, series bundle fetch, per-server status, content identify | PR-006 | High | Planned |
| JR-032 | Identify is **never automatic** — storing a candidate is a separate confirmation step | PR-006 | Medium | Planned |
| JR-033 | Scheduled sweep over items lacking truth data, using the **batch** `exists` endpoint | PR-006 | Medium | Planned |
| JR-034 | Contribution strips `movie` and `jellyfin_id`, attaches identity from `ProviderIds` plus measured runtime, and posts **only** to contribute-enabled servers — never fanned out | PR-005 | High | Planned |
| JR-035 | Uploads set `Expect: 100-continue`, so a rejection lands before a bundle body is transmitted | PR-006 | Low | Planned |
| JR-036 | Minimum accepted match tier is configurable; a `loose` match surfaces as a caveat rather than being applied silently | PR-006 | Medium | In Progress |
| JR-037 | A server that is unreachable or failing is skipped on a short timeout with backoff; one dead server never stalls a sweep | PR-006 | Medium | Planned |
## Egress and privacy (JR-038 … JR-041)
`PR-005` had **no software row in any repo** — it was held structurally, by
SR-004 and GR-005 both being prohibitions. jRay is the component that actually
performs egress, so these are the rows that make it verifiable rather than merely
preserved.
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| JR-038 | Every exchange feature is **opt-in and off by default**, including the pre-configured community server | **PR-005** | High | Done |
| JR-039 | No library-wide inventory in one request: batch `exists` capped at 100 items, sweeps paced | **PR-005** | High | Planned |
| JR-040 | The config page states plainly that **each configured server multiplies the exposure** | **PR-005** | Medium | Planned |
| JR-041 | The plugin never fetches, stores, or transmits gallery data — reference faces or embeddings. It has no gallery code path at all | **SR-005** | High | Done |
## Audio signature (JR-042 … JR-045)
Mirror-image of extraction `IR-004`/`IR-005`/`IR-007`/`IR-008`. Both producers
must agree **bit-for-bit**, so each obligation is stated on both sides rather
than assumed to be inherited.
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| JR-042 | Compute the signature **exactly** per server spec §3, using the FFmpeg binary Jellyfin already ships via `IMediaEncoder.EncoderPath` — no new dependency | SR-003 | Medium | Planned |
| JR-043 | Golden-vector fixture **shared with the extraction repo**, proving the two implementations are bit-exact | SR-003 | High | Planned |
| JR-044 | Media shorter than 120 s: emit no signature and apply no sync offset — identical rule in both producers | SR-003 | Low | Planned |
| JR-045 | Emit and honour the signature's own `v1:` prefix, so a DSP change is detectable rather than silently non-matching | SR-003 | Low | Planned |
## Human-in-the-loop association (JR-046)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| JR-046 | Review UI for unidentified track clusters: show context crops, pick from the title's cast or search TMDB, record the association | [system §4](../scripts/vendor/jray-project/SPEC.md) | Medium | **TBD** |
Deliberately a single placeholder row rather than a decomposed set. It depends on
extraction `AR-021`/`AR-022` landing, and on system open question 2 (whether
unidentified presence is published at all) — decomposing it now would fix an
interface against an undecided upstream.
---
## Verification strategy
**CI is an Intel N100** ([system spec](../scripts/vendor/jray-project/SPEC.md) §6). Unlike the extraction
pipeline this costs jRay almost nothing: the plugin is CPU-only managed code, and
every requirement above except the live-integration ones is executable in CI.
| Tier | Runs in CI | What it covers |
|---|---|---|
| **T1 — Unit** | Yes | Parsing, precedence, policy resolution, coverage arithmetic, offset application, audio DSP, schema rejection |
| **T2 — Host integration** | Yes | Controllers and authorisation against a test host with a faked `ILibraryManager` |
| **T4 — Live** | **No** | Real Jellyfin + File Transformation + web client; real manifest server round-trip |
| **static** | Yes | Grep/analyzer checks — e.g. no injection path into `index.html` (JR-021) |
**T3 is deliberately unused.** The gate's `CI_EXECUTABLE_TIERS` treats T1/T2/T3
as CI-runnable and T4 as not, which is right for extraction (where T3 is slow CPU
inference and T4 is GPU). jRay has only two CI tiers and one live tier, so its
non-CI tier is numbered **T4** to match that shared constant rather than
renumbering it. Calling jRay's live tier "T3" would make the gate count
live-only requirements as covered — the exact class of error the 158% coverage
bug belongs to.
**There is no test project today.** That is the single largest gap in this
register: 46 requirements, zero `UT`/`IT` IDs, so measured coverage will open at
zero and every `Done` above rests on inspection rather than evidence.
### Per-requirement verification plan
| ID | Tier | Test asserts | Edge cases to cover |
|---|---|---|---|
| JR-001 | static | Other repos' specs link here rather than restating the schema | A second copy of the schema anywhere is the failure |
| JR-002 | T1 | A v2 file round-trips; `scenes` objects retain belief and route | Window with belief exactly at the ownership threshold; all three route values |
| JR-003 | **T1** | `schema_version` 1 and 3 are both **rejected**, not coerced | Missing field entirely; non-integer value |
| JR-004 | T1 | Windows are stored and served byte-identical to input | Adjacent windows that "look" mergeable must **not** merge |
| JR-005 | T1 | `t` exactly on `start` and on `end` are both present | Zero-length window; overlapping windows for one actor |
| JR-006 | T1 | Query cost is acceptable with 10³ windows on one actor | Sorted-window assumption stated and tested |
| JR-007 | T1 | `jellyfin_id` preferred; falls back to provider ids | All three ids empty → actor still displayable by name |
| JR-008 | T1 | Sidecar path derived from the item path plus the configured suffix | Item with no path; suffix changed at runtime |
| JR-009 | T2 | `PUT` stores, `DELETE` removes, both admin-only | `DELETE` on an item with no managed truth is still `204` |
| JR-010 | T1 | Managed overrides sidecar; provenance survives | Fetched and pushed truth for the same item |
| JR-011 | T1 | A write invalidates the cached entry immediately | Read, push, read again within the cache window |
| JR-012 | T2 | Returns the file, or `404` when no source has data | Sidecar present but unparseable |
| JR-013 | T2 | Envelope shape is stable; extra keys are additive | Item with truth data but no actor present at `t` |
| JR-014 | T2 | Anonymous request to each admin route is refused | Authenticated non-admin on an admin route |
| JR-015 | T2 | Sample excludes covered items and clamps `limit` | `limit` of 0 and of 1000; library of missing-path ghosts |
| JR-019 | T2 | Pickers return `{value,label}`; empty search returns `[]` | Two episodes named "Pilot" — labels must disambiguate |
| JR-020 | **T4** | Overlay appears on pause and lists the scene cast | Live web client only |
| JR-016 | T1 | Item beats Series beats Genre | Prioritised series inside an ignored genre — the case that motivated the rule |
| JR-017 | **T1** | An ignored item still serves its overlay | Rule added after truth data exists |
| JR-018 | T1 | `covered / (total - ignored)` | Item carrying two genres counts in both rows |
| JR-021 | **static** | No code path *adds* the script tag to `index.html` | `scripts/checks/no-index-injection.sh`. Removal (JR-022) is the one permitted write, so the check is on injection, not on writing. Verified to **fail** on a reintroduced `Apply()` and on reintroduced `ReplaceLast` injection, not merely to pass today |
| JR-022 | T1 | A marked legacy patch is removed; unmarked content untouched | Foreign plugin's injection left intact |
| JR-023 | T1 + **T4** | Absent dependency disables only the overlay | Detection unit-testable; config-page display is live |
| JR-024 | T1 | A name containing markup renders escaped | `<script>` in an actor name from a hostile server |
| JR-025 | T1 | First result clearing the tier wins; disabled servers skipped | All servers fail; first server returns a below-tier match |
| JR-026 | T1 | Server 2 queried only for episodes server 1 lacked | Bundle with a gap in the middle of a season |
| JR-027 | T1 | Unknown field, oversized body, and out-of-range window each rejected | Window ending beyond the item's runtime |
| JR-028 | T1 | Stream aborts past the cap rather than buffering | Server declaring a small length and sending more |
| JR-029 | T1 | Plain `http` to a non-loopback host is refused | `http://localhost` allowed; `http://192.168.x` refused |
| JR-030 | **T1** | Offset added to every window before storage | Negative offset; offset that would push a window below zero |
| JR-031 | T2 | All four routes exist and are admin-only | — |
| JR-032 | T1 | `Identify` returns candidates and stores nothing | A single high-confidence candidate still does not auto-store |
| JR-033 | T1 | Sweep batches through `exists` and paces | Backlog smaller than one batch |
| JR-034 | **T1** | `movie` and `jellyfin_id` absent from the upload body | Contribution attempted to a `FetchOnly` server must not send |
| JR-035 | T1 | `Expect: 100-continue` set on uploads | — |
| JR-036 | T1 | Below-tier match is not stored; `loose` is flagged | Tier configured to `exact` with only a `runtime` match available |
| JR-037 | T1 | Failing server skipped, backoff grows | Every server failing must not hang the sweep |
| JR-038 | **T1** | Every exchange switch defaults off; community server disabled | Fresh config object, no user input |
| JR-039 | T1 | Batch never exceeds 100 items | Library of 10⁴ items produces a paced sweep |
| JR-040 | **T4** | Config page states the per-server exposure | Manual review of copy |
| JR-041 | **static** | No embedding or image field is parsed or stored | Grep-based, mirroring the server's UR-012 |
| JR-042 | T1 | DSP chain matches the specified parameters exactly | Window, hop, band, bin count each asserted individually |
| JR-043 | **T1** | Signature matches the shared golden vector **bit-for-bit** | Media < 120 s → no signature; identical result in both repos |
| JR-044 | T1 | Media < 120 s yields no signature and no offset | Exactly 120 s — the boundary both repos must agree on |
| JR-045 | T1 | `v1:` emitted; an unknown prefix is refused, not parsed | `v2:` signature from a future producer |
| JR-046 | **TBD** | — | Undesigned; depends on AR-021/AR-022 and system open question 2 |
Three are worth singling out. **JR-021** and **JR-041** are static checks because
both are requirements to *not do something*, and a prohibition is verified by
absence, not by a passing test. **JR-043** is the cross-repo check: it is the only
test in this repo whose fixture is shared with another, and it is CPU-only DSP,
which is exactly why it can be the binding check rather than an aspiration.
---
## Running the gate
The shared extractor now takes the three things that vary per repo as arguments,
so this repo needs **no fork of it** — there must only ever be one
implementation:
```sh
python3 scripts/vendor/jray-project/scripts/traceability/extract_traces.py \
--root . \
--requirements docs/requirements.md \
--system-spec scripts/vendor/jray-project/SPEC.md \
--types JR \
--suffixes .cs,.js,.sh \
--scan-roots Jellyfin.Plugin.JRay,scripts/checks \
--format coverage
```
`scripts/checks` is scanned so the static checks carry their own TRACES tags —
an enforcement script is evidence for a requirement exactly as a unit test is.
The scan root is `scripts/checks` and **not** `scripts`, because the latter would
walk `scripts/vendor/jray-project` and harvest the `AR-nnn` examples in the
extractor's own docstrings as orphan tags.
`--root` must be **absolute or `.`**; the scan roots resolve beneath it. Both the
extractor and the system spec come from the submodule, so the only thing this
repo supplies is its own register and the three per-repo arguments.
Refresh the pinned tooling with
`git submodule update --remote scripts/vendor/jray-project`.
**Naming conflict to resolve.** The tool's header comment expects
`jRay → UR/DR`. This register uses `JR`, decided deliberately: `JRay-public-server`
already ships `UR-001…018` and `DR-001…014`, so a second repo using the same
prefixes would make `UR-007` ambiguous across registers — and `UR-007` is
precisely the ID the server's own register asks the plugin to cross-reference
(see JR-025). Either the comment or this register is wrong; the comment is the
cheaper of the two to change.
+74 -2
View File
@@ -7,13 +7,85 @@
"owner": "dtourolle", "owner": "dtourolle",
"category": "General", "category": "General",
"versions": [ "versions": [
{
"version": "0.0.4",
"changelog": "Release 0.0.4",
"targetAbi": "10.9.0.0",
"sourceUrl": "https://gitea.tourolle.paris/dtourolle/jRay/releases/download/v0.0.4/jray_0.0.4.0.zip",
"checksum": "cb9613660b3fe3b730f46c9c7f257333",
"timestamp": "2026-07-04T19:47:05Z"
},
{ {
"version": "0.0.0.0", "version": "0.0.0.0",
"changelog": "Latest Build", "changelog": "Latest Build",
"targetAbi": "10.9.0.0", "targetAbi": "10.9.0.0",
"sourceUrl": "https://gitea.tourolle.paris/dtourolle/jRay/releases/download/latest/jray_0.0.0.0.zip", "sourceUrl": "https://gitea.tourolle.paris/dtourolle/jRay/releases/download/latest/jray_0.0.0.0.zip",
"checksum": "d6f05de428a51573f020c06651245e69", "checksum": "a76316397228c9d8b8e38fbccad50390",
"timestamp": "2026-06-12T16:53:59Z" "timestamp": "2026-07-04T19:44:35Z"
},
{
"version": "0.0.3",
"changelog": "Release 0.0.3",
"targetAbi": "10.9.0.0",
"sourceUrl": "https://gitea.tourolle.paris/dtourolle/jRay/releases/download/v0.0.3/jray_0.0.3.0.zip",
"checksum": "da5ded2ee720f7a7f7e15ac092871a91",
"timestamp": "2026-07-04T19:11:39Z"
},
{
"version": "0.0.0.0",
"changelog": "Latest Build",
"targetAbi": "10.9.0.0",
"sourceUrl": "https://gitea.tourolle.paris/dtourolle/jRay/releases/download/latest/jray_0.0.0.0.zip",
"checksum": "0d1cc8954a1b0557eb5ec6c368bce486",
"timestamp": "2026-07-04T19:10:48Z"
},
{
"version": "0.0.2",
"changelog": "Release 0.0.2",
"targetAbi": "10.9.0.0",
"sourceUrl": "https://gitea.tourolle.paris/dtourolle/jRay/releases/download/v0.0.2/jray_0.0.2.0.zip",
"checksum": "6969ca24f23e66eaffae0b52638fea14",
"timestamp": "2026-06-12T18:30:35Z"
},
{
"version": "0.0.0.0",
"changelog": "Latest Build",
"targetAbi": "10.9.0.0",
"sourceUrl": "https://gitea.tourolle.paris/dtourolle/jRay/releases/download/latest/jray_0.0.0.0.zip",
"checksum": "2f8beef54c03cec92a17431615cba604",
"timestamp": "2026-06-12T18:26:36Z"
},
{
"version": "0.0.1",
"changelog": "Release 0.0.1",
"targetAbi": "10.9.0.0",
"sourceUrl": "https://gitea.tourolle.paris/dtourolle/jRay/releases/download/v0.0.1/jray_0.0.1.0.zip",
"checksum": "6c0e71d77ebac619920cf93a2c330b8f",
"timestamp": "2026-06-12T18:11:18Z"
},
{
"version": "0.0.0.0",
"changelog": "Latest Build",
"targetAbi": "10.9.0.0",
"sourceUrl": "https://gitea.tourolle.paris/dtourolle/jRay/releases/download/latest/jray_0.0.0.0.zip",
"checksum": "6ed76e7fea217cda914a6610c48d7d30",
"timestamp": "2026-06-12T18:10:35Z"
},
{
"version": "0.0.0",
"changelog": "Release 0.0.0",
"targetAbi": "10.9.0.0",
"sourceUrl": "https://gitea.tourolle.paris/dtourolle/jRay/releases/download/v0.0.0/jray_0.0.0.0.zip",
"checksum": "fd92458f741b8f7aa0b77da04cbc6a43",
"timestamp": "2026-06-12T17:05:10Z"
},
{
"version": "0.0.0.0",
"changelog": "Latest Build",
"targetAbi": "10.9.0.0",
"sourceUrl": "https://gitea.tourolle.paris/dtourolle/jRay/releases/download/latest/jray_0.0.0.0.zip",
"checksum": "3650af90fd163042af65cc702d618506",
"timestamp": "2026-06-12T17:04:26Z"
} }
] ]
} }
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env bash
# JR-021 — jRay never injects into index.html on disk.
#
# This is a requirement to *not do* something, so it is verified by absence.
# A unit test cannot show that no code path writes the tag; a grep can.
#
# The prohibition is on injection, not on writing: JR-022's migration must write
# to index.html in order to remove a legacy patch. So the check is for code that
# *adds* the script tag, not for File.Write* generally.
#
# TRACES: JR-021 | PR-004
set -euo pipefail
cd "$(dirname "$0")/../.."
src="Jellyfin.Plugin.JRay"
status=0
# The injection is "script tag + marker" written back to the file. The removal
# path also names both, so match on the concatenation that builds a patched
# document rather than on the constants themselves.
if grep -rn --include='*.cs' -E '(ScriptTag|Injected)[[:space:]]*\+.*BodyClose|ReplaceLast|"</body>"[[:space:]]*,' "$src" \
| grep -v 'FileTransformationRegistration.cs'; then
echo "FAIL (JR-021): index.html injection logic found outside the File Transformation callback." >&2
status=1
fi
# WebClientPatchService is removal-only. Any write there must be the cleaned
# document; a write of a *patched* one is the regression this guards.
if grep -n -E 'WriteAllText\((?!.*cleaned)' -P "$src/Services/WebClientPatchService.cs" >/dev/null 2>&1; then
echo "FAIL (JR-021): WebClientPatchService writes something other than the cleaned document." >&2
status=1
fi
# The disk-patching entry point must not come back.
if grep -rn --include='*.cs' -E '\bWebClientPatchService\.Apply\b' "$src"; then
echo "FAIL (JR-021): the injecting Apply() entry point has been reintroduced." >&2
status=1
fi
if [ "$status" -eq 0 ]; then
echo "OK (JR-021): no on-disk injection path."
fi
exit "$status"