Skip to content

fix(golden): refresh knob_step reference #296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
j0sh wants to merge 2 commits into
base: main
Choose a base branch
from
Open

fix(golden): refresh knob_step reference #296

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0625449
fix(golden): refresh knob_step reference
j0sh Jun 25, 2026
ac9c72d
docs(golden): clarify refs env semantics
j0sh Jun 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions tests/golden/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,13 @@ identical suite at the refactored build.
| `test_golden.py` | Audio matches the canonical reference |
| `test_latency.py` | Coarse latency ceilings + per-build report artifact |

The manifest's per-scenario `env` block is provenance only. The golden
contract reads the bundle location/hashes and calibrated thresholds; it
does not gate pass/fail on `env`, nor does it attempt to recreate a run
from those fields. This matters for queued production-pod captures,
where metadata like `server_info`, `gpu`, or the exact signed pod URL
may be unavailable or intentionally normalized.

## Running

On a local GPU box the suite is self-contained: it spawns the server
Expand Down
33 changes: 11 additions & 22 deletions tests/golden/refs.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,31 +28,20 @@
},
"knob_step": {
"file": "knob_step.tar.gz",
"sha256": "5289d21984edbf0b0a52f7716802da5944ef4d250dd78e755bfea6a4b3283a6c",
"canonical_sha256": "0b2eac2a326cea51f36c277771adf0de333401a1af026ccce8ab39d81c63e238",
"sha256": "21db75608bf7e2d6d9212108eab6334c72fda79359b28e8f44ad50c9f6d9e1ec",
"canonical_sha256": "95e92b12dd2e6a95052395fc9aeae7c013c4110cfb1442fdac6f283b4338d808",
"thresholds": {
"_note": "Calibrated 2026-06-09. knob_step carries a stable, benign cross-build mel offset vs the 06-04 canonical (mel_l2 floor 0.0953, bit-stable across runs; win_cos_min 0.9999, rms 0 dB), distinct from peers because the knob legitimately alters generation after it fires. mel_l2 relaxed above that floor with margin; the cosine/level gates keep their strict defaults so a real regression still trips them.",
"mel_l2": 0.13,
"rms_db_diff": 0.5,
"win_cos_min": 0.995
"_note": "Refreshed 2026-06-25 from DEMON main after the 06-04 canonical drifted. Same-pod repeat-3 noise stayed tiny (mel_l2 0.00165), while fresh-pod verification runs against the new canonical landed at mel_l2 0.00143 and 0.051. mel_l2 is tightened to cover that observed cross-pod spread with margin; cosine and level gates are reset near their strict defaults.",
"mel_l2": 0.08,
"rms_db_diff": 0.1,
"win_cos_min": 0.999
},
"env": {
"harness_git_sha": "96c57bb",
"pod_url": "ws://127.0.0.1:49676",
"gpu": "NVIDIA GeForce RTX 5090",
"server_info": {
"no_backend": false,
"kiosk": false,
"default_mode": "graph",
"warmup": null,
"server_side_fixtures": [
"inside_confusion_loop_60s_gsm.wav",
"low_fi_Gm_loop_60s_gnm.wav",
"prog_rock_loop_60s_enm.wav",
"thrash_metal_loop_60s_enm.wav"
]
},
"captured_at": "2026-06-04T13:18:37.202829+00:00"
"harness_git_sha": "778caa4",
"pod_url": "wss://pod-a40a4cd3d6fc.daydream.live/",
"gpu": "unknown",
"server_info": {},
"captured_at": "2026-06-25T17:11:53.497956+00:00"
}
},
"lora_enable": {
Expand Down