cute: remove redundant subprocess-autotune DSL recompiles - #3148
Draft
calebmkim wants to merge 1 commit into
Draft
cute: remove redundant subprocess-autotune DSL recompiles#3148calebmkim wants to merge 1 commit into
calebmkim wants to merge 1 commit into
Conversation
calebmkim
force-pushed
the
calebmkim/stack/32
branch
from
July 24, 2026 17:09
dfd5530 to
917acec
Compare
This was referenced Jul 24, 2026
The killable-worker autotune path cold-compiled the CuTe DSL kernel ~twice per config (separate bench + accuracy jobs) and the worker disk cache was disabled (cache_key=None), so a fixed budget searched far fewer distinct configs under subprocess than in-process. A cold cutlass-DSL compile is ~3.5s vs a ~0.05s warm disk reload, so the wasted recompiles dominated. Fixes: (1) BenchmarkAndAccuracyJob compiles the source once for both bench and accuracy (behind HELION_FUSED_ACCURACY_CHECK, default off); (2) _reattach_cute_source_hash restores the worker disk-cache key so repeats are warm reloads. Effect (single B200, effort=full, seed=2000, subprocess=ON, fp8 4096^3): distinct configs in a 300 s budget: 44 -> 83 (+89%; 1 -> 2 generations). Inert on Triton and in-process. (The 44 "before" arm is the parent commit d51ece6d8^, which predates both fixes; "after" is this commit + HELION_FUSED_ACCURACY_CHECK=1.) Compile-COST only: does not change the benchmark timer or selection. Verified no selection regression from this change: on a >=2-generation autotune run, the final winner ([256,256,128] cm2 ab6) was independently cold-L2 cudagraph-timed against the heuristic seed and was within 30% of it (winner/seed = 1.00x, PASS -- the winner is the seed config) -- i.e. the fix does not cause the tuner to crown a config markedly worse than a known-good one. Gating: the two fixes have DIFFERENT blast radii. (1) the fused job is behind HELION_FUSED_ACCURACY_CHECK, default off. (2) _reattach_cute_source_hash is UNCONDITIONAL and takes effect on every worker load, i.e. on the default-ON subprocess-benchmark path. That is deliberate: the parent already stamps this hash and already uses the on-disk DSL cache, so the worker's cache_key=None was a defect, not a policy -- and a wrong stamp can only MISS the cache (cold recompile), never produce a false hit, because the key also carries the full input specialization, launch shape, compile options, CUTE_DSL_* env, and cutlass version. Its correctness does rest on the timer absorbing the first call: both do_bench (benchmarking.py, fn() + sync before capture) and do_bench_generic (fn() + sync before the estimate loop) already do, so the compile-or-reload cost never enters the timed samples. The NOTE added to do_bench_generic marks that ordering as load-bearing for future editors -- if warmup ever moved after the timed loop, the reattach would turn a uniform first-call penalty into a non-uniform one (cold for first-seen configs, warm for repeats), which would bias selection where a uniform penalty largely cancels. Timeout accounting: the fused job runs bench AND the accuracy check under a SINGLE autotune_benchmark_timeout window, where the legacy two-job path gave each step its own. The extra work inside that window is one kernel launch plus an assert_close -- not a second compile, which is the point of the change -- so the narrowing is small next to a ~3.5 s cold compile, but it is not zero: a config near the timeout boundary has slightly less headroom than before. Not compensated for deliberately (doubling the window would also double what a genuinely hung config gets to hang for, weakening the killable-worker hang protection). Adaptation: zero-arg synchronize_device() (PR #3026). Refactor: 3 copies of subprocess job-error handling -> _handle_subprocess_job_error. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> stack-info: PR: #3148, branch: calebmkim/stack/32
calebmkim
force-pushed
the
calebmkim/stack/32
branch
from
July 27, 2026 22:13
917acec to
b11428b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked PRs:
cute: remove redundant subprocess-autotune DSL recompiles
The killable-worker autotune path cold-compiled the CuTe DSL kernel ~twice per
config (separate bench + accuracy jobs) and the worker disk cache was disabled
(cache_key=None), so a fixed budget searched far fewer distinct configs under
subprocess than in-process. A cold cutlass-DSL compile is ~3.5s vs a ~0.05s warm
disk reload, so the wasted recompiles dominated.
Fixes: (1) BenchmarkAndAccuracyJob compiles the source once for both bench and
accuracy (behind HELION_FUSED_ACCURACY_CHECK, default off); (2)
_reattach_cute_source_hash restores the worker disk-cache key so repeats are warm
reloads.
Effect (single B200, effort=full, seed=2000, subprocess=ON, fp8 4096^3):
distinct configs in a 300 s budget: 44 -> 83 (+89%; 1 -> 2 generations). Inert on
Triton and in-process. (The 44 "before" arm is the parent commit d51ece6d8^, which
predates both fixes; "after" is this commit + HELION_FUSED_ACCURACY_CHECK=1.)
Compile-COST only: does not change the benchmark timer or selection. Verified no
selection regression from this change: on a >=2-generation autotune run, the final
winner ([256,256,128] cm2 ab6) was independently cold-L2 cudagraph-timed against the
heuristic seed and was within 30% of it (winner/seed = 1.00x, PASS -- the winner is
the seed config) -- i.e. the fix does not cause the tuner to crown a config markedly
worse than a known-good one.
Gating: the two fixes have DIFFERENT blast radii. (1) the fused job is behind
HELION_FUSED_ACCURACY_CHECK, default off. (2) reattach_cute_source_hash is
UNCONDITIONAL and takes effect on every worker load, i.e. on the default-ON
subprocess-benchmark path. That is deliberate: the parent already stamps this hash
and already uses the on-disk DSL cache, so the worker's cache_key=None was a defect,
not a policy -- and a wrong stamp can only MISS the cache (cold recompile), never
produce a false hit, because the key also carries the full input specialization,
launch shape, compile options, CUTE_DSL* env, and cutlass version. Its correctness
does rest on the timer absorbing the first call: both do_bench (benchmarking.py, fn()
already do, so the compile-or-reload cost never enters the timed samples. The NOTE
added to do_bench_generic marks that ordering as load-bearing for future editors --
if warmup ever moved after the timed loop, the reattach would turn a uniform
first-call penalty into a non-uniform one (cold for first-seen configs, warm for
repeats), which would bias selection where a uniform penalty largely cancels.
Timeout accounting: the fused job runs bench AND the accuracy check under a SINGLE
autotune_benchmark_timeout window, where the legacy two-job path gave each step its
own. The extra work inside that window is one kernel launch plus an assert_close --
not a second compile, which is the point of the change -- so the narrowing is small
next to a ~3.5 s cold compile, but it is not zero: a config near the timeout boundary
has slightly less headroom than before. Not compensated for deliberately (doubling
the window would also double what a genuinely hung config gets to hang for, weakening
the killable-worker hang protection).
Adaptation: zero-arg synchronize_device() (PR #3026). Refactor: 3 copies of
subprocess job-error handling -> _handle_subprocess_job_error.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com