jellyfin-srfPlay/TESTING_GUIDE.md

Testing Guide for SRF Play Plugin

This guide explains how to run tests and set up the nightly API validation for the Jellyfin SRF Play plugin.

Overview

The plugin now has a comprehensive test suite:

1. Unit Tests - Fast, isolated tests for individual components
2. Integration Tests (API Spec Tests) - Real API calls to validate the SRF Play API is working correctly
3. Nightly CI Tests - Automated nightly runs to detect API changes

Prerequisites

Quick Setup

Run the setup script to check your environment:

./setup-tests.sh

Required Software

.NET 8.0 SDK (Required - matches Jellyfin requirements)

# Arch Linux/CachyOS
sudo pacman -S dotnet-sdk-8.0 aspnet-runtime-8.0

# Or download from:
# https://dotnet.microsoft.com/download/dotnet/8.0

Verify Installation:

dotnet --list-runtimes
# Should show:
# Microsoft.NETCore.App 8.x.x

Other Requirements:

• Internet connection (for integration/API tests)

Running Tests Locally

All Tests

dotnet test

Unit Tests Only

dotnet test --filter "Category!=Integration&Category!=APISpec"

API Spec Tests Only

dotnet test --filter "Category=APISpec"

With Code Coverage

dotnet test --collect:"XPlat Code Coverage"

With Detailed Output

dotnet test --logger "console;verbosity=detailed"

Test Structure

Unit Tests (Jellyfin.Plugin.SRFPlay.Tests/UnitTests/)

• StreamUrlResolverTests.cs - Tests stream URL resolution, DRM filtering, expiration checking
• MetadataCacheTests.cs - Tests metadata caching, expiration, thread safety

Characteristics:

• Fast execution (milliseconds)
• No external dependencies
• Run on every commit/PR

Integration Tests (Jellyfin.Plugin.SRFPlay.Tests/IntegrationTests/)

• SRFApiSpecTests.cs - Validates SRF Play API compliance
  • Tests all business units (SRF, RTS, RSI, RTR, SWI)
  • Validates response schemas
  • Tests API endpoints accessibility
  • Validates HLS stream availability
  • Performance monitoring

Characteristics:

• Slower execution (seconds to minutes)
• Makes real API calls
• Run nightly via CI

Continuous Integration

Unit Tests Workflow

File: .github/workflows/unit-tests.yaml

Triggers:

• Push to master branch
• Pull requests to master
• Manual trigger

Features:

• Runs all unit tests
• Generates code coverage reports
• Posts coverage summary on PRs
• Fails if tests fail

Nightly API Spec Tests Workflow

File: .github/workflows/nightly-api-tests.yaml

Schedule: Every night at 2 AM UTC

Features:

• Validates SRF Play API is still working
• Tests all business units
• Validates response schemas
• Automatically creates a GitHub issue if tests fail
• Provides detailed test reports

What happens when tests fail:

• A GitHub issue is automatically created with:
  • Link to the failed workflow run
  • Description of what likely changed
  • Suggested actions to take
• Labels: bug, api, nightly-test-failure

Adding New Tests

Unit Test Example

using Xunit;
using FluentAssertions;

namespace Jellyfin.Plugin.SRFPlay.Tests.UnitTests;

public class MyServiceTests
{
    [Fact]
    public void MyMethod_WithValidInput_ReturnsExpectedResult()
    {
        // Arrange
        var service = new MyService();

        // Act
        var result = service.MyMethod("test");

        // Assert
        result.Should().Be("expected");
    }
}

Integration Test Example

using Xunit;
using FluentAssertions;

namespace Jellyfin.Plugin.SRFPlay.Tests.IntegrationTests;

[Trait("Category", "Integration")]
[Trait("Category", "APISpec")]
public class MyApiTests
{
    [Fact]
    public async Task ApiCall_ReturnsValidData()
    {
        // Arrange
        var client = new SRFApiClient(loggerFactory);

        // Act
        var result = await client.GetDataAsync("srf", cancellationToken);

        // Assert
        result.Should().NotBeNull();
        result.Should().NotBeEmpty();
    }
}

Test Naming Conventions

Follow the pattern: MethodName_Scenario_ExpectedBehavior

Good Examples:

• GetStreamUrl_WithDrmProtectedOnly_ReturnsNull
• IsContentExpired_WithPastValidTo_ReturnsTrue
• GetAllShows_SRF_ReturnsShows

What to Do When Tests Fail

Unit Tests Fail

1. Check the error message in the test output
2. Review recent code changes
3. Fix the bug or update the test if behavior changed intentionally
4. Run tests locally before pushing

API Spec Tests Fail (Nightly)

1. Check the GitHub issue created automatically

2. Review the workflow logs for detailed error messages

3. Common causes:

   • SRF Play API schema changed
   • New authentication requirements
   • Endpoints moved or deprecated
   • Rate limiting or temporary outages

4. Actions to take:

   • Update API models in Jellyfin.Plugin.SRFPlay/Api/Models/
   • Update API client in SRFApiClient.cs
   • Update tests to match new behavior
   • Document any breaking changes

Temporary API Outages

If the API is temporarily down:

1. Monitor the issue - it will auto-close on next successful run
2. No action needed unless failures persist for multiple days

Code Coverage

Code coverage reports are generated automatically for unit tests in CI.

View coverage locally:

dotnet test --collect:"XPlat Code Coverage"
# Coverage reports will be in TestResults/*/coverage.cobertura.xml

Target: Aim for >70% coverage for core services

Performance Benchmarks

API spec tests include performance validation:

• API calls should complete within 30 seconds
• Failures indicate potential performance degradation

Best Practices

1. Write tests first (TDD) when fixing bugs
2. Keep unit tests fast - under 100ms per test
3. Use descriptive test names that explain what's being tested
4. One assertion per test for clear failure messages
5. Clean up resources with IDisposable
6. Mock external dependencies in unit tests
7. Use real APIs only in integration tests

Troubleshooting

Tests won't run locally

# Ensure .NET 8.0 SDK is installed
dotnet --list-sdks

# If not installed, download from:
# https://dotnet.microsoft.com/download/dotnet/8.0

Integration tests fail with network errors

• Check internet connectivity
• Check if SRF Play API is accessible from your location
• Some regions may have geo-restrictions

Build succeeds but tests won't execute

# Clean and rebuild
dotnet clean
dotnet build
dotnet test

Legacy Tests

The project still contains legacy console-based tests:

• Program.cs
• TestPlayV3Api.cs

These are kept for manual testing but are not run by CI. To run them:

cd Jellyfin.Plugin.SRFPlay.Tests
dotnet run

Future Improvements

• Add more unit tests for remaining services
• Add tests for scheduled task functionality
• Add tests for proxy configuration
• Increase code coverage to >80%
• Add mutation testing
• Add performance benchmarks

Questions?

• Review Test Documentation
• Check GitHub Actions for CI results
• Look at existing tests for examples

Summary

With this testing infrastructure:

• ✅ Developers get immediate feedback on code changes
• ✅ Maintainers are automatically notified of API changes
• ✅ Users benefit from more reliable plugin
• ✅ Contributors have clear examples to follow