Skip to content

✅ Phase-5 – Automated regression, spectrogram diff & cross-browser stress tests #978

Description

@ford442

Parent: #972
Blocked by: #977 (Phase-4)

Why

We need objective, repeatable proof that the new high-fid engines are both more authentic and safe to use across browsers.

Scope

  1. Spectrogram Diff / Audio Quality Tests

    • Load Phase-0 reference + renders from stock, highfid-cpu, and gpu-highfid.
    • Compute error metrics (MSE per band, peak, etc.) and assert against the thresholds defined in the authenticity-gaps doc.
  2. Performance Benchmarks

    • Script that measures wall-clock time for representative offline renders at different oversample factors.
    • Record GPU dispatch + readback latency.
    • CI can later enforce soft budgets.
  3. Cross-Browser Matrix (Playwright)

    • Chrome / Edge (WebGPU expected) – GPU path active.
    • Firefox / Safari (or equivalent) – fallback to CPU path, no crashes, correct UI badge / toast.
  4. Stress / Memory

    • Concurrent offline renders via the worker pool; assert no OOM / crashes under reasonable load.
  5. Manual A/B Checklist

    • Extend or create docs/audio-engine/303-A-B-checklist.md in the same style as the existing voice docs.

Acceptance Criteria

  • Spectrogram / quality tests pass with the defined error thresholds.
  • Benchmark numbers are captured (as CI artifacts or logged).
  • Playwright (or equivalent) matrix covers the main browsers and shows correct engine selection / fallback.
  • Stress test completes without OOM or unhandled rejections.
  • Documentation and badges updated to reflect high-fid status.

Sub-tasks

  • Add / reuse FFT utilities for spectrogram diff tests.
  • Write the diff + quality test harness using Phase-0 reference material.
  • Implement benchmark script.
  • Extend Playwright config / add fallback-specific e2e tests.
  • Stress test using the offline worker API.
  • Update CI if needed to run the new jobs and upload artifacts.

Labels / Milestone

Intended milestone: M5 – Test Suite, Performance & Cross-Browser

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions