test: pin resolve_runtime_dtype and compute-bound ceiling fallbacks (PRPUNDIT-11, PRPUNDIT-15) - #1330
test: pin resolve_runtime_dtype and compute-bound ceiling fallbacks (PRPUNDIT-11, PRPUNDIT-15)#1330jiagaoxiang wants to merge 5 commits into
Conversation
Cover quantization, pre-quantized meta, --dtype, bf16-floored fallback, and that workload.precision does not drive weight dtype.
…IT-15) Directly assert the weight_bytes fallback stays nonzero and missing peak/dtype/weight degrade to 0.0.
Keep both roofline unit tests in one file so the pair can land without a conflict.
zoroyihan7
left a comment
There was a problem hiding this comment.
Thanks — the four dtype branches and the three degrade-to-0.0 exits are all reached, and asserting the ceiling == 0.0 directly (rather than via bound_kind == "memory", the way test_conc_sweep_ceiling.py::test_dense_fallback_no_moe_fields does) is exactly right and avoids that masking pattern.
I mutation-tested the production code against these tests, and three of the behaviors the ticket asked for aren't actually pinned — the tests pass, but they'd also pass if the code regressed.
1. achievable-over-vendor precedence isn't pinned. roofline_ceiling.py:1012 is:
peak_tflops = _resolve_achievable_tflops(...) or _resolve_peak_tflops(...)The test patches achievable → 100.0 and vendor → 0.0, so achievable or vendor and vendor or achievable both yield 100.0. Swapping the operands leaves 44/44 green. Patch vendor to a different positive value (e.g. 500.0) and assert the result derives from 100.0.
2. The vendor-dense fallback leg is never exercised. The only case with achievable == 0 also has vendor at 0.0, so it only reaches the degrade exit. A case with achievable == 0.0, vendor > 0 asserting a positive ceiling would cover it.
3. Dtype priority is pinned for 2 of 4 adjacent pairs. Branch 1-over-3 (the --quantization fp8 --dtype bfloat16 case) and 3-over-4 are pinned, but 1-vs-2 and 2-vs-3 have no case where both sources are present — hoisting quantization_config above the --quantization branch, or --dtype above quantization_config, both leave 44/44 green. Two cases close it: recognized --quantization fp8 with a pre-quantized meta (weight_dtype_bytes=0.5) asserting source == "server_args_quantization"; and a pre-quantized meta plus --dtype float32 asserting source == "quantization_config".
Three smaller escapes found by mutation, worth a line each if you're already in here:
- Widening
if 0 < meta_w_bytes < 2.0:toif 0 <= meta_w_bytes <= 2.0:stays green — neither the2.0upper edge nor themeta == 0(unknown dtype) lower edge is pinned. - Deleting the documented bf16 activation floor (
act_bytes = max(..., 2.0)) stays green — every case uses bfloat16/float32/absent, none below 2 bytes. compute_precision_tagis never asserted in any branch; replacing_compute_tag_for_bytes(wb)with a constant stays green, even though that tag is the key for the downstream TFLOPS lookup.
Nits, take or leave: gpu_type="unknown-gpu" in the degrade case is inert since both resolvers are patched for the whole body (the name implies a real table miss that isn't happening); and both tests are monolithic multi-scenario functions, so an early failure hides later scenarios — parametrize would localize them.
Happy to re-review once the precedence cases are in.
Summary
resolve_runtime_dtypeprovenance branches and that workloadprecisiondoes not drive weight dtype.compute_compute_bound_ceiling_tok_per_secweight_bytesfallback stays nonzero, and missing peak / dtype / weight degrade to0.0.Covers test gaps PRPUNDIT-11 and PRPUNDIT-15 (merged from #1326 into this PR).
Test plan
PYTHONPATH=src pytest src/hyperloom/inference_optimizer/tests/test_roofline_ceiling_perfmodel_units.py::test_resolve_runtime_dtype_priority_and_ignores_workload_precision src/hyperloom/inference_optimizer/tests/test_roofline_ceiling_perfmodel_units.py::test_compute_compute_bound_ceiling_fallback_and_degrade_to_zero