jellyfin-srfPlay/TESTING_GUIDE.md

# Testing Guide for SRF Play Plugin

This guide explains how to run tests and set up the nightly API validation for the Jellyfin SRF Play plugin.

## Overview

The plugin now has a comprehensive test suite:

1. **Unit Tests** - Fast, isolated tests for individual components
2. **Integration Tests (API Spec Tests)** - Real API calls to validate the SRF Play API is working correctly
3. **Nightly CI Tests** - Automated nightly runs to detect API changes

## Prerequisites

### Quick Setup

Run the setup script to check your environment:

```bash
./setup-tests.sh
```

### Required Software

**.NET 8.0 SDK** (Required - matches Jellyfin requirements)

```bash
# Arch Linux/CachyOS
sudo pacman -S dotnet-sdk-8.0 aspnet-runtime-8.0

# Or download from:
# https://dotnet.microsoft.com/download/dotnet/8.0
```

**Verify Installation:**
```bash
dotnet --list-runtimes
# Should show:
# Microsoft.NETCore.App 8.x.x
```

**Other Requirements:**
- Internet connection (for integration/API tests)

## Running Tests Locally

### All Tests
```bash
dotnet test
```

### Unit Tests Only
```bash
dotnet test --filter "Category!=Integration&Category!=APISpec"
```

### API Spec Tests Only
```bash
dotnet test --filter "Category=APISpec"
```

### With Code Coverage
```bash
dotnet test --collect:"XPlat Code Coverage"
```

### With Detailed Output
```bash
dotnet test --logger "console;verbosity=detailed"
```

## Test Structure

### Unit Tests ([Jellyfin.Plugin.SRFPlay.Tests/UnitTests/](Jellyfin.Plugin.SRFPlay.Tests/UnitTests/))

- **StreamUrlResolverTests.cs** - Tests stream URL resolution, DRM filtering, expiration checking
- **MetadataCacheTests.cs** - Tests metadata caching, expiration, thread safety

**Characteristics:**
- Fast execution (milliseconds)
- No external dependencies
- Run on every commit/PR

### Integration Tests ([Jellyfin.Plugin.SRFPlay.Tests/IntegrationTests/](Jellyfin.Plugin.SRFPlay.Tests/IntegrationTests/))

- **SRFApiSpecTests.cs** - Validates SRF Play API compliance
  - Tests all business units (SRF, RTS, RSI, RTR, SWI)
  - Validates response schemas
  - Tests API endpoints accessibility
  - Validates HLS stream availability
  - Performance monitoring

**Characteristics:**
- Slower execution (seconds to minutes)
- Makes real API calls
- Run nightly via CI

## Continuous Integration

### Unit Tests Workflow

**File:** [.github/workflows/unit-tests.yaml](.github/workflows/unit-tests.yaml)

**Triggers:**
- Push to master branch
- Pull requests to master
- Manual trigger

**Features:**
- Runs all unit tests
- Generates code coverage reports
- Posts coverage summary on PRs
- Fails if tests fail

### Nightly API Spec Tests Workflow

**File:** [.github/workflows/nightly-api-tests.yaml](.github/workflows/nightly-api-tests.yaml)

**Schedule:** Every night at 2 AM UTC

**Features:**
- Validates SRF Play API is still working
- Tests all business units
- Validates response schemas
- **Automatically creates a GitHub issue if tests fail**
- Provides detailed test reports

**What happens when tests fail:**
- A GitHub issue is automatically created with:
  - Link to the failed workflow run
  - Description of what likely changed
  - Suggested actions to take
- Labels: `bug`, `api`, `nightly-test-failure`

## Adding New Tests

### Unit Test Example

```csharp
using Xunit;
using FluentAssertions;

namespace Jellyfin.Plugin.SRFPlay.Tests.UnitTests;

public class MyServiceTests
{
    [Fact]
    public void MyMethod_WithValidInput_ReturnsExpectedResult()
    {
        // Arrange
        var service = new MyService();

        // Act
        var result = service.MyMethod("test");

        // Assert
        result.Should().Be("expected");
    }
}
```

### Integration Test Example

```csharp
using Xunit;
using FluentAssertions;

namespace Jellyfin.Plugin.SRFPlay.Tests.IntegrationTests;

[Trait("Category", "Integration")]
[Trait("Category", "APISpec")]
public class MyApiTests
{
    [Fact]
    public async Task ApiCall_ReturnsValidData()
    {
        // Arrange
        var client = new SRFApiClient(loggerFactory);

        // Act
        var result = await client.GetDataAsync("srf", cancellationToken);

        // Assert
        result.Should().NotBeNull();
        result.Should().NotBeEmpty();
    }
}
```

## Test Naming Conventions

Follow the pattern: `MethodName_Scenario_ExpectedBehavior`

**Good Examples:**
- `GetStreamUrl_WithDrmProtectedOnly_ReturnsNull`
- `IsContentExpired_WithPastValidTo_ReturnsTrue`
- `GetAllShows_SRF_ReturnsShows`

## What to Do When Tests Fail

### Unit Tests Fail

1. Check the error message in the test output
2. Review recent code changes
3. Fix the bug or update the test if behavior changed intentionally
4. Run tests locally before pushing

### API Spec Tests Fail (Nightly)

1. **Check the GitHub issue** created automatically
2. **Review the workflow logs** for detailed error messages
3. **Common causes:**
   - SRF Play API schema changed
   - New authentication requirements
   - Endpoints moved or deprecated
   - Rate limiting or temporary outages

4. **Actions to take:**
   - Update API models in [Jellyfin.Plugin.SRFPlay/Api/Models/](Jellyfin.Plugin.SRFPlay/Api/Models/)
   - Update API client in [SRFApiClient.cs](Jellyfin.Plugin.SRFPlay/Api/SRFApiClient.cs)
   - Update tests to match new behavior
   - Document any breaking changes

### Temporary API Outages

If the API is temporarily down:
1. Monitor the issue - it will auto-close on next successful run
2. No action needed unless failures persist for multiple days

## Code Coverage

Code coverage reports are generated automatically for unit tests in CI.

**View coverage locally:**
```bash
dotnet test --collect:"XPlat Code Coverage"
# Coverage reports will be in TestResults/*/coverage.cobertura.xml
```

**Target:** Aim for >70% coverage for core services

## Performance Benchmarks

API spec tests include performance validation:
- API calls should complete within 30 seconds
- Failures indicate potential performance degradation

## Best Practices

1. **Write tests first** (TDD) when fixing bugs
2. **Keep unit tests fast** - under 100ms per test
3. **Use descriptive test names** that explain what's being tested
4. **One assertion per test** for clear failure messages
5. **Clean up resources** with IDisposable
6. **Mock external dependencies** in unit tests
7. **Use real APIs** only in integration tests

## Troubleshooting

### Tests won't run locally

```bash
# Ensure .NET 8.0 SDK is installed
dotnet --list-sdks

# If not installed, download from:
# https://dotnet.microsoft.com/download/dotnet/8.0
```

### Integration tests fail with network errors

- Check internet connectivity
- Check if SRF Play API is accessible from your location
- Some regions may have geo-restrictions

### Build succeeds but tests won't execute

```bash
# Clean and rebuild
dotnet clean
dotnet build
dotnet test
```

## Legacy Tests

The project still contains legacy console-based tests:
- [Program.cs](Jellyfin.Plugin.SRFPlay.Tests/Program.cs)
- [TestPlayV3Api.cs](Jellyfin.Plugin.SRFPlay.Tests/TestPlayV3Api.cs)

These are kept for manual testing but are not run by CI. To run them:

```bash
cd Jellyfin.Plugin.SRFPlay.Tests
dotnet run
```

## Future Improvements

- [ ] Add more unit tests for remaining services
- [ ] Add tests for scheduled task functionality
- [ ] Add tests for proxy configuration
- [ ] Increase code coverage to >80%
- [ ] Add mutation testing
- [ ] Add performance benchmarks

## Questions?

- Review [Test Documentation](Jellyfin.Plugin.SRFPlay.Tests/README.md)
- Check [GitHub Actions](../../actions) for CI results
- Look at existing tests for examples

## Summary

With this testing infrastructure:
- ✅ **Developers** get immediate feedback on code changes
- ✅ **Maintainers** are automatically notified of API changes
- ✅ **Users** benefit from more reliable plugin
- ✅ **Contributors** have clear examples to follow