Add the v1 audio signature to the pipeline (IR-004, IR-005, IR-007, IR-008)
Implements the content-derived spectral-peak signature from JRay-public-server/SPEC.md §3 so a truth file is self-identifying: 120 s window centred on the media midpoint, mono at 11025 Hz, 4096/1024 Hann STFT, 32 log-spaced bins over 300-3000 Hz, one byte per frame (5-bit peak band + 2-bit energy class), base64, `v1:` prefix. Audio decode is a second stream from the FFmpeg libraries the pipeline already links for video; libswresample is added to the existing ffmpeg_libs interface target. The FFT is written out rather than pulled from a library for the same reason the plugin vendors one: the output has to be bit-identical across two languages, so a dependency whose version could change the numerics is a liability. The server spec fixes the geometry but not enough to reproduce a byte stream — Hann periodicity, band aggregation, the energy-class definition, tie-breaking and the base64 alphabet are all unconstrained by it. Those are pinned in audio_signature.hpp and mirrored in the golden fixture, so the plugin can be implemented from the fixture alone. IR-005: tests/fixtures/audio/ carries a deterministic 120 s tone (FLAC — lossless, so identical PCM to the WAV make_fixture.py emits, and 3.5x smaller in git) plus the signature it must produce, the decoded-PCM checksum and the full parameter contract. That directory is the artefact shared with the plugin repo; the PCM checksum is separate from the signature so a codec-level difference is distinguishable from a DSP one. IR-007: media under 120 s emits no signature. Same for a file with no audio stream or one that will not open — UR-9 is an enhancement and must never be able to break a fetch. Verified against an independent Python reference implementation: same bytes. All 32 bands and all 4 energy classes appear in the golden vector, and the window-centring test wraps the fixture in 90 s of silence either side and requires the golden value back. Not wired into the truth-file output yet — that is the schema_version bump under IR-002/IR-003 and is deliberately out of scope here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -21,21 +21,29 @@ add_executable(sae_tests
|
||||
test_face_utils.cpp
|
||||
test_track_gallery.cpp
|
||||
test_face_tracker.cpp
|
||||
test_audio_signature.cpp
|
||||
${CMAKE_SOURCE_DIR}/src/backends/gemm_backend.cpp
|
||||
${CMAKE_SOURCE_DIR}/src/gallery/gallery_store.cpp
|
||||
${CMAKE_SOURCE_DIR}/src/audio_signature.cpp
|
||||
)
|
||||
target_include_directories(sae_tests PRIVATE ${CMAKE_SOURCE_DIR}/src)
|
||||
# SAE_GEMM_CPU: build the CPU reference GEMM regardless of the main backend.
|
||||
# SAE_MODELS_DIR: config.hpp (pulled in by track_gallery.hpp) bakes model paths.
|
||||
# SAE_TEST_FIXTURES_DIR: the audio golden vector is read from the source tree,
|
||||
# not copied, so the file the plugin repo shares is the file under test.
|
||||
target_compile_definitions(sae_tests PRIVATE
|
||||
SAE_GEMM_CPU
|
||||
SAE_MODELS_DIR="${SAE_MODELS_DIR}")
|
||||
SAE_MODELS_DIR="${SAE_MODELS_DIR}"
|
||||
SAE_TEST_FIXTURES_DIR="${CMAKE_CURRENT_SOURCE_DIR}/fixtures")
|
||||
# gallery_store.cpp + gallery_calibration.hpp use nlohmann/json and HDF5
|
||||
# (galleries are HDF5-native, see src/gallery/gallery_store.cpp); face_utils.hpp
|
||||
# and the calibration GEMM pull in OpenCV (calib3d/imgproc/core) via types.hpp.
|
||||
# ffmpeg_libs: audio_signature.cpp decodes the golden fixture (avformat/avcodec/
|
||||
# avutil/swresample). Still GPU-free — the audio path is pure CPU.
|
||||
target_link_libraries(sae_tests PRIVATE
|
||||
Catch2::Catch2WithMain
|
||||
nlohmann_json::nlohmann_json
|
||||
ffmpeg_libs
|
||||
${OpenCV_LIBS}
|
||||
${HDF5_CXX_LIBRARIES})
|
||||
target_include_directories(sae_tests PRIVATE ${HDF5_INCLUDE_DIRS})
|
||||
|
||||
Reference in New Issue
Block a user