Embed in-kernel Pallas runtime helpers into generated code - #3187
Closed
AmesingFlank wants to merge 1 commit into
Closed
Embed in-kernel Pallas runtime helpers into generated code#3187AmesingFlank wants to merge 1 commit into
AmesingFlank wants to merge 1 commit into
Conversation
AmesingFlank
force-pushed
the
AmesingFlank/stack/101
branch
from
July 28, 2026 16:02
b1bdb37 to
f8af008
Compare
AmesingFlank
force-pushed
the
AmesingFlank/stack/100
branch
from
July 28, 2026 16:02
201c9f5 to
5d5e92a
Compare
This was referenced Jul 28, 2026
AmesingFlank
force-pushed
the
AmesingFlank/stack/101
branch
2 times, most recently
from
July 28, 2026 17:39
1c9cc90 to
cdeadbd
Compare
AmesingFlank
force-pushed
the
AmesingFlank/stack/101
branch
from
July 28, 2026 20:09
cdeadbd to
8dbbe62
Compare
AmesingFlank
force-pushed
the
AmesingFlank/stack/101
branch
from
July 28, 2026 21:14
8dbbe62 to
d2bed12
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 28, 2026
The generated Pallas kernel calls two helion-defined, pure-`jax` helpers -- `divide_filter_topk` (aten.topk lowering, via `_helion_divide_filter_topk`) and `flatten_worklist` (compact-worklist builder) -- which were pulled in with `from helion... import ...`, leaving a helion dependency that blocks precompiling to a helion-free standalone. Add a `Backend.embedded_helper_source(body)` hook (default `""`) that `BoundKernel.to_code` injects between the imports and the kernel body. The Pallas backend overrides it to inline the source of whichever helper `body` references -- module docstring and `from __future__` stripped by `_embed_source` (located via `ast`, so a docstring containing a triple-quote can't corrupt the output) -- plus a `_helion_divide_filter_topk = divide_filter_topk` alias. The topk `library_imports` entry and the inline `flatten_worklist` import in `render_build_worklist` are dropped. Both helper modules import only `jax`/stdlib (and now carry a note to stay free of any `helion` import, which would trip the precompiler's substring guard), so the embedded module stays dependency-free. This runs for regular kernels too, keeping a single code path. Tests: the two builder tests that `exec` the rendered `_build_worklist` now supply `flatten_worklist` (no longer imported inline); `test_matching_kernel_generates_compact` and `test_topk_divide_and_filter_lowering` assert the embedded def + alias are present, the old helion import is gone, and the module still `ast.parse`s (guarding against a corrupt embed). Also repairs the window-guard test's stale call to the 2-arg `_get_vmem_limit_bytes`. Verified on TPU: topk (3 passed), compact guard/render/builder (10 passed); full `test_pallas_compact_worklist.py` shows only the 6 pre-existing `scratch_types` JAX-skew failures (identical to base), 66 passed. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected (`embedded_helper_source` is a no-op there). stack-info: PR: #3187, branch: AmesingFlank/stack/101
AmesingFlank
force-pushed
the
AmesingFlank/stack/101
branch
from
July 28, 2026 22:27
d2bed12 to
1efda11
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 29, 2026
The generated Pallas kernel calls two helion-defined, pure-`jax` helpers -- `divide_filter_topk` (aten.topk lowering, via `_helion_divide_filter_topk`) and `flatten_worklist` (compact-worklist builder) -- which were pulled in with `from helion... import ...`, leaving a helion dependency that blocks precompiling to a helion-free standalone. Add a `Backend.embedded_helper_source(body)` hook (default `""`) that `BoundKernel.to_code` injects between the imports and the kernel body. The Pallas backend overrides it to inline the source of whichever helper `body` references -- module docstring and `from __future__` stripped by `_embed_source` (located via `ast`, so a docstring containing a triple-quote can't corrupt the output) -- plus a `_helion_divide_filter_topk = divide_filter_topk` alias. The topk `library_imports` entry and the inline `flatten_worklist` import in `render_build_worklist` are dropped. Both helper modules import only `jax`/stdlib (and now carry a note to stay free of any `helion` import, which would trip the precompiler's substring guard), so the embedded module stays dependency-free. This runs for regular kernels too, keeping a single code path. Tests: the two builder tests that `exec` the rendered `_build_worklist` now supply `flatten_worklist` (no longer imported inline); `test_matching_kernel_generates_compact` and `test_topk_divide_and_filter_lowering` assert the embedded def + alias are present, the old helion import is gone, and the module still `ast.parse`s (guarding against a corrupt embed). Also repairs the window-guard test's stale call to the 2-arg `_get_vmem_limit_bytes`. Verified on TPU: topk (3 passed), compact guard/render/builder (10 passed); full `test_pallas_compact_worklist.py` shows only the 6 pre-existing `scratch_types` JAX-skew failures (identical to base), 66 passed. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected (`embedded_helper_source` is a no-op there). stack-info: PR: #3187, branch: AmesingFlank/stack/101
AmesingFlank
added a commit
that referenced
this pull request
Jul 29, 2026
The generated Pallas kernel calls two helion-defined, pure-`jax` helpers -- `divide_filter_topk` (aten.topk lowering, via `_helion_divide_filter_topk`) and `flatten_worklist` (compact-worklist builder) -- which were pulled in with `from helion... import ...`, leaving a helion dependency that blocks precompiling to a helion-free standalone. Add a `Backend.embedded_helper_source(body)` hook (default `""`) that `BoundKernel.to_code` injects between the imports and the kernel body. The Pallas backend overrides it to inline the source of whichever helper `body` references -- module docstring and `from __future__` stripped by `_embed_source` (located via `ast`, so a docstring containing a triple-quote can't corrupt the output) -- plus a `_helion_divide_filter_topk = divide_filter_topk` alias. The topk `library_imports` entry and the inline `flatten_worklist` import in `render_build_worklist` are dropped. Both helper modules import only `jax`/stdlib (and now carry a note to stay free of any `helion` import, which would trip the precompiler's substring guard), so the embedded module stays dependency-free. This runs for regular kernels too, keeping a single code path. Tests: the two builder tests that `exec` the rendered `_build_worklist` now supply `flatten_worklist` (no longer imported inline); `test_matching_kernel_generates_compact` and `test_topk_divide_and_filter_lowering` assert the embedded def + alias are present, the old helion import is gone, and the module still `ast.parse`s (guarding against a corrupt embed). Also repairs the window-guard test's stale call to the 2-arg `_get_vmem_limit_bytes`. Verified on TPU: topk (3 passed), compact guard/render/builder (10 passed); full `test_pallas_compact_worklist.py` shows only the 6 pre-existing `scratch_types` JAX-skew failures (identical to base), 66 passed. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected (`embedded_helper_source` is a no-op there). stack-info: PR: #3187, branch: AmesingFlank/stack/101
AmesingFlank
force-pushed
the
AmesingFlank/stack/101
branch
from
July 29, 2026 09:04
9b5944d to
e8b5964
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 29, 2026
The generated Pallas kernel calls two helion-defined, pure-`jax` helpers -- `divide_filter_topk` (aten.topk lowering, via `_helion_divide_filter_topk`) and `flatten_worklist` (compact-worklist builder) -- which were pulled in with `from helion... import ...`, leaving a helion dependency that blocks precompiling to a helion-free standalone. Add a `Backend.embedded_helper_source(body)` hook (default `""`) that `BoundKernel.to_code` injects between the imports and the kernel body. The Pallas backend overrides it to inline the source of whichever helper `body` references -- module docstring and `from __future__` stripped by `_embed_source` (located via `ast`, so a docstring containing a triple-quote can't corrupt the output) -- plus a `_helion_divide_filter_topk = divide_filter_topk` alias. The topk `library_imports` entry and the inline `flatten_worklist` import in `render_build_worklist` are dropped. Both helper modules import only `jax`/stdlib (and now carry a note to stay free of any `helion` import, which would trip the precompiler's substring guard), so the embedded module stays dependency-free. This runs for regular kernels too, keeping a single code path. Tests: the two builder tests that `exec` the rendered `_build_worklist` now supply `flatten_worklist` (no longer imported inline); `test_matching_kernel_generates_compact` and `test_topk_divide_and_filter_lowering` assert the embedded def + alias are present, the old helion import is gone, and the module still `ast.parse`s (guarding against a corrupt embed). Also repairs the window-guard test's stale call to the 2-arg `_get_vmem_limit_bytes`. Verified on TPU: topk (3 passed), compact guard/render/builder (10 passed); full `test_pallas_compact_worklist.py` shows only the 6 pre-existing `scratch_types` JAX-skew failures (identical to base), 66 passed. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected (`embedded_helper_source` is a no-op there). stack-info: PR: #3187, branch: AmesingFlank/stack/101
AmesingFlank
force-pushed
the
AmesingFlank/stack/101
branch
from
July 29, 2026 20:37
e8b5964 to
da14cb6
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
The generated Pallas kernel calls two helion-defined, pure-`jax` helpers -- `divide_filter_topk` (aten.topk lowering, via `_helion_divide_filter_topk`) and `flatten_worklist` (compact-worklist builder) -- which were pulled in with `from helion... import ...`, leaving a helion dependency that blocks precompiling to a helion-free standalone. Add a `Backend.embedded_helper_source(body)` hook (default `""`) that `BoundKernel.to_code` injects between the imports and the kernel body. The Pallas backend overrides it to inline the source of whichever helper `body` references -- module docstring and `from __future__` stripped by `_embed_source` (located via `ast`, so a docstring containing a triple-quote can't corrupt the output) -- plus a `_helion_divide_filter_topk = divide_filter_topk` alias. The topk `library_imports` entry and the inline `flatten_worklist` import in `render_build_worklist` are dropped. Both helper modules import only `jax`/stdlib (and now carry a note to stay free of any `helion` import, which would trip the precompiler's substring guard), so the embedded module stays dependency-free. This runs for regular kernels too, keeping a single code path. Tests: the two builder tests that `exec` the rendered `_build_worklist` now supply `flatten_worklist` (no longer imported inline); `test_matching_kernel_generates_compact` and `test_topk_divide_and_filter_lowering` assert the embedded def + alias are present, the old helion import is gone, and the module still `ast.parse`s (guarding against a corrupt embed). Also repairs the window-guard test's stale call to the 2-arg `_get_vmem_limit_bytes`. Verified on TPU: topk (3 passed), compact guard/render/builder (10 passed); full `test_pallas_compact_worklist.py` shows only the 6 pre-existing `scratch_types` JAX-skew failures (identical to base), 66 passed. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected (`embedded_helper_source` is a no-op there). stack-info: PR: #3187, branch: AmesingFlank/stack/101
AmesingFlank
force-pushed
the
AmesingFlank/stack/101
branch
from
July 30, 2026 01:56
da14cb6 to
8a33994
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
The generated Pallas kernel calls two helion-defined, pure-`jax` helpers -- `divide_filter_topk` (aten.topk lowering, via `_helion_divide_filter_topk`) and `flatten_worklist` (compact-worklist builder) -- which were pulled in with `from helion... import ...`, leaving a helion dependency that blocks precompiling to a helion-free standalone. Add a `Backend.embedded_helper_source(body)` hook (default `""`) that `BoundKernel.to_code` injects between the imports and the kernel body. The Pallas backend overrides it to inline the source of whichever helper `body` references -- module docstring and `from __future__` stripped by `_embed_source` (located via `ast`, so a docstring containing a triple-quote can't corrupt the output) -- plus a `_helion_divide_filter_topk = divide_filter_topk` alias. The topk `library_imports` entry and the inline `flatten_worklist` import in `render_build_worklist` are dropped. Both helper modules import only `jax`/stdlib (and now carry a note to stay free of any `helion` import, which would trip the precompiler's substring guard), so the embedded module stays dependency-free. This runs for regular kernels too, keeping a single code path. Tests: the two builder tests that `exec` the rendered `_build_worklist` now supply `flatten_worklist` (no longer imported inline); `test_matching_kernel_generates_compact` and `test_topk_divide_and_filter_lowering` assert the embedded def + alias are present, the old helion import is gone, and the module still `ast.parse`s (guarding against a corrupt embed). Also repairs the window-guard test's stale call to the 2-arg `_get_vmem_limit_bytes`. Verified on TPU: topk (3 passed), compact guard/render/builder (10 passed); full `test_pallas_compact_worklist.py` shows only the 6 pre-existing `scratch_types` JAX-skew failures (identical to base), 66 passed. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected (`embedded_helper_source` is a no-op there). stack-info: PR: #3187, branch: AmesingFlank/stack/101
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
The generated Pallas kernel calls two helion-defined, pure-`jax` helpers -- `divide_filter_topk` (aten.topk lowering, via `_helion_divide_filter_topk`) and `flatten_worklist` (compact-worklist builder) -- which were pulled in with `from helion... import ...`, leaving a helion dependency that blocks precompiling to a helion-free standalone. Add a `Backend.embedded_helper_source(body)` hook (default `""`) that `BoundKernel.to_code` injects between the imports and the kernel body. The Pallas backend overrides it to inline the source of whichever helper `body` references -- module docstring and `from __future__` stripped by `_embed_source` (located via `ast`, so a docstring containing a triple-quote can't corrupt the output) -- plus a `_helion_divide_filter_topk = divide_filter_topk` alias. The topk `library_imports` entry and the inline `flatten_worklist` import in `render_build_worklist` are dropped. Both helper modules import only `jax`/stdlib (and now carry a note to stay free of any `helion` import, which would trip the precompiler's substring guard), so the embedded module stays dependency-free. This runs for regular kernels too, keeping a single code path. Tests: the two builder tests that `exec` the rendered `_build_worklist` now supply `flatten_worklist` (no longer imported inline); `test_matching_kernel_generates_compact` and `test_topk_divide_and_filter_lowering` assert the embedded def + alias are present, the old helion import is gone, and the module still `ast.parse`s (guarding against a corrupt embed). Also repairs the window-guard test's stale call to the 2-arg `_get_vmem_limit_bytes`. Verified on TPU: topk (3 passed), compact guard/render/builder (10 passed); full `test_pallas_compact_worklist.py` shows only the 6 pre-existing `scratch_types` JAX-skew failures (identical to base), 66 passed. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected (`embedded_helper_source` is a no-op there). stack-info: PR: #3187, branch: AmesingFlank/stack/101
AmesingFlank
force-pushed
the
AmesingFlank/stack/101
branch
from
July 30, 2026 02:43
8a33994 to
cd771fb
Compare
AmesingFlank
force-pushed
the
AmesingFlank/stack/101
branch
from
July 30, 2026 03:17
cd771fb to
3517194
Compare
AmesingFlank
force-pushed
the
AmesingFlank/stack/101
branch
from
July 30, 2026 03:29
3517194 to
0763855
Compare
The generated Pallas kernel calls two helion-defined, pure-`jax` helpers -- `divide_filter_topk` (aten.topk lowering, via `_helion_divide_filter_topk`) and `flatten_worklist` (compact-worklist builder) -- which were pulled in with `from helion... import ...`, leaving a helion dependency that blocks precompiling to a helion-free standalone. Add a `Backend.embedded_helper_source(body)` hook (default `""`) that `BoundKernel.to_code` injects between the imports and the kernel body. The Pallas backend overrides it to inline the source of whichever helper `body` references -- module docstring and `from __future__` stripped by `_embed_source` (located via `ast`, so a docstring containing a triple-quote can't corrupt the output) -- plus a `_helion_divide_filter_topk = divide_filter_topk` alias. The topk `library_imports` entry and the inline `flatten_worklist` import in `render_build_worklist` are dropped. Both helper modules import only `jax`/stdlib (and now carry a note to stay free of any `helion` import, which would trip the precompiler's substring guard), so the embedded module stays dependency-free. This runs for regular kernels too, keeping a single code path. Tests: the two builder tests that `exec` the rendered `_build_worklist` now supply `flatten_worklist` (no longer imported inline); `test_matching_kernel_generates_compact` and `test_topk_divide_and_filter_lowering` assert the embedded def + alias are present, the old helion import is gone, and the module still `ast.parse`s (guarding against a corrupt embed). Also repairs the window-guard test's stale call to the 2-arg `_get_vmem_limit_bytes`. Verified on TPU: topk (3 passed), compact guard/render/builder (10 passed); full `test_pallas_compact_worklist.py` shows only the 6 pre-existing `scratch_types` JAX-skew failures (identical to base), 66 passed. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected (`embedded_helper_source` is a no-op there). stack-info: PR: #3187, branch: AmesingFlank/stack/101
AmesingFlank
force-pushed
the
AmesingFlank/stack/101
branch
from
July 30, 2026 03:47
0763855 to
70f46f9
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
The generated Pallas kernel calls two helion-defined, pure-`jax` helpers -- `divide_filter_topk` (aten.topk lowering, via `_helion_divide_filter_topk`) and `flatten_worklist` (compact-worklist builder) -- which were pulled in with `from helion... import ...`, leaving a helion dependency that blocks precompiling to a helion-free standalone. Add a `Backend.embedded_helper_source(body)` hook (default `""`) that `BoundKernel.to_code` injects between the imports and the kernel body. The Pallas backend overrides it to inline the source of whichever helper `body` references -- module docstring and `from __future__` stripped by `_embed_source` (located via `ast`, so a docstring containing a triple-quote can't corrupt the output) -- plus a `_helion_divide_filter_topk = divide_filter_topk` alias. The topk `library_imports` entry and the inline `flatten_worklist` import in `render_build_worklist` are dropped. Both helper modules import only `jax`/stdlib (and now carry a note to stay free of any `helion` import, which would trip the precompiler's substring guard), so the embedded module stays dependency-free. This runs for regular kernels too, keeping a single code path. Tests: the two builder tests that `exec` the rendered `_build_worklist` now supply `flatten_worklist` (no longer imported inline); `test_matching_kernel_generates_compact` and `test_topk_divide_and_filter_lowering` assert the embedded def + alias are present, the old helion import is gone, and the module still `ast.parse`s (guarding against a corrupt embed). Also repairs the window-guard test's stale call to the 2-arg `_get_vmem_limit_bytes`. Verified on TPU: topk (3 passed), compact guard/render/builder (10 passed); full `test_pallas_compact_worklist.py` shows only the 6 pre-existing `scratch_types` JAX-skew failures (identical to base), 66 passed. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected (`embedded_helper_source` is a no-op there). stack-info: PR: #3187, branch: AmesingFlank/stack/101
Contributor
Author
|
Folded into #3184. In-kernel Pallas helper embedding ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked PRs:
OutputCodeOptions(jax_fn=True)for Pallas (JAX-array entrypoints) #3186OutputCodeOptions(allow_helion_deps=False)for the Pallas backend (TorchTPU) #3184OutputCodeOptions(allow_helion_deps=False)toBoundKernel.to_codefor standalone Triton kernels #3181Embed in-kernel Pallas runtime helpers into generated code
The generated Pallas kernel calls two helion-defined, pure-
jaxhelpers --divide_filter_topk(aten.topk lowering, via_helion_divide_filter_topk) andflatten_worklist(compact-worklist builder) -- which were pulled in withfrom helion... import ..., leaving a helion dependency that blocksprecompiling to a helion-free standalone.
Add a
Backend.embedded_helper_source(body)hook (default"") thatBoundKernel.to_codeinjects between the imports and the kernel body. ThePallas backend overrides it to inline the source of whichever helper
bodyreferences -- module docstring and
from __future__stripped by_embed_source(located via
ast, so a docstring containing a triple-quote can't corrupt theoutput) -- plus a
_helion_divide_filter_topk = divide_filter_topkalias. Thetopk
library_importsentry and the inlineflatten_worklistimport inrender_build_worklistare dropped. Both helper modules import onlyjax/stdlib (and now carry a note to stay free of anyhelionimport, whichwould trip the precompiler's substring guard), so the embedded module stays
dependency-free. This runs for regular kernels too, keeping a single code path.
Tests: the two builder tests that
execthe rendered_build_worklistnowsupply
flatten_worklist(no longer imported inline);test_matching_kernel_generates_compactandtest_topk_divide_and_filter_loweringassert the embedded def + alias are present, the old helion import is gone, and
the module still
ast.parses (guarding against a corrupt embed). Also repairsthe window-guard test's stale call to the 2-arg
_get_vmem_limit_bytes.Verified on TPU: topk (3 passed), compact guard/render/builder (10 passed); full
test_pallas_compact_worklist.pyshows only the 6 pre-existingscratch_typesJAX-skew failures (identical to base), 66 passed.
ruff/pyreflyclean (83baseline); H100 Triton unaffected (
embedded_helper_sourceis a no-op there).