Skip to content

cute: remove redundant subprocess-autotune DSL recompiles - #3148

Draft
calebmkim wants to merge 1 commit into
mainfrom
calebmkim/stack/32
Draft

cute: remove redundant subprocess-autotune DSL recompiles#3148
calebmkim wants to merge 1 commit into
mainfrom
calebmkim/stack/32

Conversation

@calebmkim

@calebmkim calebmkim commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Stacked PRs:


cute: remove redundant subprocess-autotune DSL recompiles

The killable-worker autotune path cold-compiled the CuTe DSL kernel ~twice per
config (separate bench + accuracy jobs) and the worker disk cache was disabled
(cache_key=None), so a fixed budget searched far fewer distinct configs under
subprocess than in-process. A cold cutlass-DSL compile is ~3.5s vs a ~0.05s warm
disk reload, so the wasted recompiles dominated.

Fixes: (1) BenchmarkAndAccuracyJob compiles the source once for both bench and
accuracy (behind HELION_FUSED_ACCURACY_CHECK, default off); (2)
_reattach_cute_source_hash restores the worker disk-cache key so repeats are warm
reloads.

Effect (single B200, effort=full, seed=2000, subprocess=ON, fp8 4096^3):
distinct configs in a 300 s budget: 44 -> 83 (+89%; 1 -> 2 generations). Inert on
Triton and in-process. (The 44 "before" arm is the parent commit d51ece6d8^, which
predates both fixes; "after" is this commit + HELION_FUSED_ACCURACY_CHECK=1.)

Compile-COST only: does not change the benchmark timer or selection. Verified no
selection regression from this change: on a >=2-generation autotune run, the final
winner ([256,256,128] cm2 ab6) was independently cold-L2 cudagraph-timed against the
heuristic seed and was within 30% of it (winner/seed = 1.00x, PASS -- the winner is
the seed config) -- i.e. the fix does not cause the tuner to crown a config markedly
worse than a known-good one.

Gating: the two fixes have DIFFERENT blast radii. (1) the fused job is behind
HELION_FUSED_ACCURACY_CHECK, default off. (2) reattach_cute_source_hash is
UNCONDITIONAL and takes effect on every worker load, i.e. on the default-ON
subprocess-benchmark path. That is deliberate: the parent already stamps this hash
and already uses the on-disk DSL cache, so the worker's cache_key=None was a defect,
not a policy -- and a wrong stamp can only MISS the cache (cold recompile), never
produce a false hit, because the key also carries the full input specialization,
launch shape, compile options, CUTE_DSL
* env, and cutlass version. Its correctness
does rest on the timer absorbing the first call: both do_bench (benchmarking.py, fn()

  • sync before capture) and do_bench_generic (fn() + sync before the estimate loop)
    already do, so the compile-or-reload cost never enters the timed samples. The NOTE
    added to do_bench_generic marks that ordering as load-bearing for future editors --
    if warmup ever moved after the timed loop, the reattach would turn a uniform
    first-call penalty into a non-uniform one (cold for first-seen configs, warm for
    repeats), which would bias selection where a uniform penalty largely cancels.

Timeout accounting: the fused job runs bench AND the accuracy check under a SINGLE
autotune_benchmark_timeout window, where the legacy two-job path gave each step its
own. The extra work inside that window is one kernel launch plus an assert_close --
not a second compile, which is the point of the change -- so the narrowing is small
next to a ~3.5 s cold compile, but it is not zero: a config near the timeout boundary
has slightly less headroom than before. Not compensated for deliberately (doubling
the window would also double what a genuinely hung config gets to hang for, weakening
the killable-worker hang protection).

Adaptation: zero-arg synchronize_device() (PR #3026). Refactor: 3 copies of
subprocess job-error handling -> _handle_subprocess_job_error.

Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com

The killable-worker autotune path cold-compiled the CuTe DSL kernel ~twice per
config (separate bench + accuracy jobs) and the worker disk cache was disabled
(cache_key=None), so a fixed budget searched far fewer distinct configs under
subprocess than in-process. A cold cutlass-DSL compile is ~3.5s vs a ~0.05s warm
disk reload, so the wasted recompiles dominated.

Fixes: (1) BenchmarkAndAccuracyJob compiles the source once for both bench and
accuracy (behind HELION_FUSED_ACCURACY_CHECK, default off); (2)
_reattach_cute_source_hash restores the worker disk-cache key so repeats are warm
reloads.

Effect (single B200, effort=full, seed=2000, subprocess=ON, fp8 4096^3):
distinct configs in a 300 s budget: 44 -> 83 (+89%; 1 -> 2 generations). Inert on
Triton and in-process. (The 44 "before" arm is the parent commit d51ece6d8^, which
predates both fixes; "after" is this commit + HELION_FUSED_ACCURACY_CHECK=1.)

Compile-COST only: does not change the benchmark timer or selection. Verified no
selection regression from this change: on a >=2-generation autotune run, the final
winner ([256,256,128] cm2 ab6) was independently cold-L2 cudagraph-timed against the
heuristic seed and was within 30% of it (winner/seed = 1.00x, PASS -- the winner is
the seed config) -- i.e. the fix does not cause the tuner to crown a config markedly
worse than a known-good one.

Gating: the two fixes have DIFFERENT blast radii. (1) the fused job is behind
HELION_FUSED_ACCURACY_CHECK, default off. (2) _reattach_cute_source_hash is
UNCONDITIONAL and takes effect on every worker load, i.e. on the default-ON
subprocess-benchmark path. That is deliberate: the parent already stamps this hash
and already uses the on-disk DSL cache, so the worker's cache_key=None was a defect,
not a policy -- and a wrong stamp can only MISS the cache (cold recompile), never
produce a false hit, because the key also carries the full input specialization,
launch shape, compile options, CUTE_DSL_* env, and cutlass version. Its correctness
does rest on the timer absorbing the first call: both do_bench (benchmarking.py, fn()
+ sync before capture) and do_bench_generic (fn() + sync before the estimate loop)
already do, so the compile-or-reload cost never enters the timed samples. The NOTE
added to do_bench_generic marks that ordering as load-bearing for future editors --
if warmup ever moved after the timed loop, the reattach would turn a uniform
first-call penalty into a non-uniform one (cold for first-seen configs, warm for
repeats), which would bias selection where a uniform penalty largely cancels.

Timeout accounting: the fused job runs bench AND the accuracy check under a SINGLE
autotune_benchmark_timeout window, where the legacy two-job path gave each step its
own. The extra work inside that window is one kernel launch plus an assert_close --
not a second compile, which is the point of the change -- so the narrowing is small
next to a ~3.5 s cold compile, but it is not zero: a config near the timeout boundary
has slightly less headroom than before. Not compensated for deliberately (doubling
the window would also double what a genuinely hung config gets to hang for, weakening
the killable-worker hang protection).

Adaptation: zero-arg synchronize_device() (PR #3026). Refactor: 3 copies of
subprocess job-error handling -> _handle_subprocess_job_error.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

stack-info: PR: #3148, branch: calebmkim/stack/32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant