Support OutputCodeOptions(jax_fn=True) for Pallas (JAX-array entrypoints) - #3186
Merged
Conversation
AmesingFlank
force-pushed
the
AmesingFlank/stack/100
branch
from
July 28, 2026 16:02
201c9f5 to
5d5e92a
Compare
AmesingFlank
force-pushed
the
AmesingFlank/stack/99
branch
from
July 28, 2026 16:02
0f726c4 to
676d880
Compare
This was referenced Jul 28, 2026
AmesingFlank
force-pushed
the
AmesingFlank/stack/100
branch
2 times, most recently
from
July 28, 2026 17:39
3a4917c to
9281c35
Compare
AmesingFlank
force-pushed
the
AmesingFlank/stack/100
branch
from
July 28, 2026 20:09
9281c35 to
d059231
Compare
AmesingFlank
force-pushed
the
AmesingFlank/stack/100
branch
from
July 28, 2026 21:14
d059231 to
fc39ad1
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 28, 2026
`helion.precompile(PrecompilationInput(..., jax_fn=True))` emits a standalone whose entrypoint takes `jax.Array`s and runs the kernel via the real `pl.kernel` compile core -- the SAME launch path the jax_fn runtime uses -- with **only jax** at runtime (no torch, no helion). `_precompile_pallas_jax` runs the compiled host wrapper once with a capturing launcher to record the launch metadata (grid, per-tensor block specs, output shape/dtype, input/output arg positions), then: - inlines the JAX-only slice of `helion/runtime/pallas/launcher.py` -- the transitive code closure of `_pallas_jax_call` (compile core + block specs + `pl.kernel`), with the torch launcher / JaxCallable / conversions dropped; - inlines the generated (jax-native) device kernel; and - emits a jax-native wrapper that allocates `jnp.empty` output placeholders and calls the inlined `_pallas_jax_call`, so the standalone reuses the exact runtime launch logic rather than a hand-rolled `pl.pallas_call`. Scope: plain block specs (including multi-dim tile loops that lower to a flat grid). Kernels using advanced Pallas features -- scratch/VMEM, HBM pass-through, SMEM, dynamic-shape padding, in-place aliasing, compact-worklist, matmul dot_general -- or int64/uint64/float64 tensors raise `NotImplementedError` (detected from the captured launcher kwargs / dtypes). The Triton and Pallas/TorchTPU precompile paths are untouched. Verified on the TPU pod (`test_precompile.py` Pallas: 5 passed) -- add, a dtype-casting kernel (device body uses `lax`), a two-output kernel, and a matmul kernel that must raise; each jax_fn standalone has no `import torch`/`import helion`, uses `pl.kernel` via the inlined `_pallas_jax_call`, and runs correctly on JAX arrays. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected. stack-info: PR: #3186, branch: AmesingFlank/stack/100
AmesingFlank
force-pushed
the
AmesingFlank/stack/100
branch
from
July 28, 2026 22:27
fc39ad1 to
32926ab
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 29, 2026
`helion.precompile(PrecompilationInput(..., jax_fn=True))` emits a standalone whose entrypoint takes `jax.Array`s and runs the kernel via the real `pl.kernel` compile core -- the SAME launch path the jax_fn runtime uses -- with **only jax** at runtime (no torch, no helion). `_precompile_pallas_jax` runs the compiled host wrapper once with a capturing launcher to record the launch metadata (grid, per-tensor block specs, output shape/dtype, input/output arg positions), then: - inlines the JAX-only slice of `helion/runtime/pallas/launcher.py` -- the transitive code closure of `_pallas_jax_call` (compile core + block specs + `pl.kernel`), with the torch launcher / JaxCallable / conversions dropped; - inlines the generated (jax-native) device kernel; and - emits a jax-native wrapper that allocates `jnp.empty` output placeholders and calls the inlined `_pallas_jax_call`, so the standalone reuses the exact runtime launch logic rather than a hand-rolled `pl.pallas_call`. Scope: plain block specs (including multi-dim tile loops that lower to a flat grid). Kernels using advanced Pallas features -- scratch/VMEM, HBM pass-through, SMEM, dynamic-shape padding, in-place aliasing, compact-worklist, matmul dot_general -- or int64/uint64/float64 tensors raise `NotImplementedError` (detected from the captured launcher kwargs / dtypes). The Triton and Pallas/TorchTPU precompile paths are untouched. Verified on the TPU pod (`test_precompile.py` Pallas: 5 passed) -- add, a dtype-casting kernel (device body uses `lax`), a two-output kernel, and a matmul kernel that must raise; each jax_fn standalone has no `import torch`/`import helion`, uses `pl.kernel` via the inlined `_pallas_jax_call`, and runs correctly on JAX arrays. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected. stack-info: PR: #3186, branch: AmesingFlank/stack/100
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
`helion.precompile(PrecompilationInput(..., jax_fn=True))` emits a standalone whose entrypoint takes `jax.Array`s and runs the kernel via the real `pl.kernel` compile core -- the SAME launch path the jax_fn runtime uses -- with **only jax** at runtime (no torch, no helion). `_precompile_pallas_jax` runs the compiled host wrapper once with a capturing launcher to record the launch metadata (grid, per-tensor block specs, output shape/dtype, input/output arg positions), then: - inlines the JAX-only slice of `helion/runtime/pallas/launcher.py` -- the transitive code closure of `_pallas_jax_call` (compile core + block specs + `pl.kernel`), with the torch launcher / JaxCallable / conversions dropped; - inlines the generated (jax-native) device kernel; and - emits a jax-native wrapper that allocates `jnp.empty` output placeholders and calls the inlined `_pallas_jax_call`, so the standalone reuses the exact runtime launch logic rather than a hand-rolled `pl.pallas_call`. Scope: plain block specs (including multi-dim tile loops that lower to a flat grid). Kernels using advanced Pallas features -- scratch/VMEM, HBM pass-through, SMEM, dynamic-shape padding, in-place aliasing, compact-worklist, matmul dot_general -- or int64/uint64/float64 tensors raise `NotImplementedError` (detected from the captured launcher kwargs / dtypes). The Triton and Pallas/TorchTPU precompile paths are untouched. Verified on the TPU pod (`test_precompile.py` Pallas: 5 passed) -- add, a dtype-casting kernel (device body uses `lax`), a two-output kernel, and a matmul kernel that must raise; each jax_fn standalone has no `import torch`/`import helion`, uses `pl.kernel` via the inlined `_pallas_jax_call`, and runs correctly on JAX arrays. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected. stack-info: PR: #3186, branch: AmesingFlank/stack/100
AmesingFlank
force-pushed
the
AmesingFlank/stack/100
branch
from
July 30, 2026 03:17
fa15f88 to
7c11e8a
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
`helion.precompile(PrecompilationInput(..., jax_fn=True))` emits a standalone whose entrypoint takes `jax.Array`s and runs the kernel via the real `pl.kernel` compile core -- the SAME launch path the jax_fn runtime uses -- with **only jax** at runtime (no torch, no helion). `_precompile_pallas_jax` runs the compiled host wrapper once with a capturing launcher to record the launch metadata (grid, per-tensor block specs, output shape/dtype, input/output arg positions), then: - inlines the JAX-only slice of `helion/runtime/pallas/launcher.py` -- the transitive code closure of `_pallas_jax_call` (compile core + block specs + `pl.kernel`), with the torch launcher / JaxCallable / conversions dropped; - inlines the generated (jax-native) device kernel; and - emits a jax-native wrapper that allocates `jnp.empty` output placeholders and calls the inlined `_pallas_jax_call`, so the standalone reuses the exact runtime launch logic rather than a hand-rolled `pl.pallas_call`. Scope: plain block specs (including multi-dim tile loops that lower to a flat grid). Kernels using advanced Pallas features -- scratch/VMEM, HBM pass-through, SMEM, dynamic-shape padding, in-place aliasing, compact-worklist, matmul dot_general -- or int64/uint64/float64 tensors raise `NotImplementedError` (detected from the captured launcher kwargs / dtypes). The Triton and Pallas/TorchTPU precompile paths are untouched. Verified on the TPU pod (`test_precompile.py` Pallas: 5 passed) -- add, a dtype-casting kernel (device body uses `lax`), a two-output kernel, and a matmul kernel that must raise; each jax_fn standalone has no `import torch`/`import helion`, uses `pl.kernel` via the inlined `_pallas_jax_call`, and runs correctly on JAX arrays. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected. stack-info: PR: #3186, branch: AmesingFlank/stack/100
AmesingFlank
force-pushed
the
AmesingFlank/stack/100
branch
from
July 30, 2026 03:29
7c11e8a to
e9995c8
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
`helion.precompile(PrecompilationInput(..., jax_fn=True))` emits a standalone whose entrypoint takes `jax.Array`s and runs the kernel via the real `pl.kernel` compile core -- the SAME launch path the jax_fn runtime uses -- with **only jax** at runtime (no torch, no helion). `_precompile_pallas_jax` runs the compiled host wrapper once with a capturing launcher to record the launch metadata (grid, per-tensor block specs, output shape/dtype, input/output arg positions), then: - inlines the JAX-only slice of `helion/runtime/pallas/launcher.py` -- the transitive code closure of `_pallas_jax_call` (compile core + block specs + `pl.kernel`), with the torch launcher / JaxCallable / conversions dropped; - inlines the generated (jax-native) device kernel; and - emits a jax-native wrapper that allocates `jnp.empty` output placeholders and calls the inlined `_pallas_jax_call`, so the standalone reuses the exact runtime launch logic rather than a hand-rolled `pl.pallas_call`. Scope: plain block specs (including multi-dim tile loops that lower to a flat grid). Kernels using advanced Pallas features -- scratch/VMEM, HBM pass-through, SMEM, dynamic-shape padding, in-place aliasing, compact-worklist, matmul dot_general -- or int64/uint64/float64 tensors raise `NotImplementedError` (detected from the captured launcher kwargs / dtypes). The Triton and Pallas/TorchTPU precompile paths are untouched. Verified on the TPU pod (`test_precompile.py` Pallas: 5 passed) -- add, a dtype-casting kernel (device body uses `lax`), a two-output kernel, and a matmul kernel that must raise; each jax_fn standalone has no `import torch`/`import helion`, uses `pl.kernel` via the inlined `_pallas_jax_call`, and runs correctly on JAX arrays. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected. stack-info: PR: #3186, branch: AmesingFlank/stack/100
AmesingFlank
force-pushed
the
AmesingFlank/stack/100
branch
from
July 30, 2026 03:47
e9995c8 to
7d604d7
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
`helion.precompile(PrecompilationInput(..., jax_fn=True))` emits a standalone whose entrypoint takes `jax.Array`s and runs the kernel via the real `pl.kernel` compile core -- the SAME launch path the jax_fn runtime uses -- with **only jax** at runtime (no torch, no helion). `_precompile_pallas_jax` runs the compiled host wrapper once with a capturing launcher to record the launch metadata (grid, per-tensor block specs, output shape/dtype, input/output arg positions), then: - inlines the JAX-only slice of `helion/runtime/pallas/launcher.py` -- the transitive code closure of `_pallas_jax_call` (compile core + block specs + `pl.kernel`), with the torch launcher / JaxCallable / conversions dropped; - inlines the generated (jax-native) device kernel; and - emits a jax-native wrapper that allocates `jnp.empty` output placeholders and calls the inlined `_pallas_jax_call`, so the standalone reuses the exact runtime launch logic rather than a hand-rolled `pl.pallas_call`. Scope: plain block specs (including multi-dim tile loops that lower to a flat grid). Kernels using advanced Pallas features -- scratch/VMEM, HBM pass-through, SMEM, dynamic-shape padding, in-place aliasing, compact-worklist, matmul dot_general -- or int64/uint64/float64 tensors raise `NotImplementedError` (detected from the captured launcher kwargs / dtypes). The Triton and Pallas/TorchTPU precompile paths are untouched. Verified on the TPU pod (`test_precompile.py` Pallas: 5 passed) -- add, a dtype-casting kernel (device body uses `lax`), a two-output kernel, and a matmul kernel that must raise; each jax_fn standalone has no `import torch`/`import helion`, uses `pl.kernel` via the inlined `_pallas_jax_call`, and runs correctly on JAX arrays. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected. stack-info: PR: #3186, branch: AmesingFlank/stack/100
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
Add `OutputCodeOptions.jax_fn` (Pallas only): `to_code(jax_fn=True)` emits a module whose entrypoint operates on `jax.Array` inputs instead of TorchTPU tensors, driving the kernel through the real `pl.kernel` compile core (`_pallas_jax_call`) -- the same launch path the jax_fn runtime uses. `jax_fn` is orthogonal to `allow_helion_deps`: with `allow_helion_deps=False` the launch core is inlined for a pure-JAX module (`jax` the only dep) and any in-kernel helpers (topk) are embedded; with `allow_helion_deps=True` both the launch core and the helper imports come from helion (`jax` + `helion`). Either way the entrypoint takes jax arrays. The launch metadata (grid, block specs, output shape/dtype, arg positions) is captured by running the compiled host wrapper with a capturing launcher on real tensors materialized from the bound kernel's fake args, then a jax-native wrapper is emitted. Dynamic shapes work via a two-probe derivation: the symbolic input dims are recorded *before* the capture run (which specializes them), then a second probe at scaled shapes reveals which grid/output/scalar launch values track an input dim (derived as `inputs[i].shape[d]`) versus which are genuine constants (baked). Advanced Pallas features or int64/uint64/float64 args raise a clear `NotImplementedError`. `build_jax_fn_code` lives on the backend classes; `PallasBackend` implements it, other backends raise. Runs outside the fake-tensor env (it compiles + launches). stack-info: PR: #3186, branch: AmesingFlank/stack/100
AmesingFlank
force-pushed
the
AmesingFlank/stack/100
branch
from
July 30, 2026 08:15
7d604d7 to
4140b61
Compare
to_code(jax_fn=True) for Pallas (JAX-array entrypoints)
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
…oints) Add `OutputCodeOptions.jax_fn` (Pallas only): `to_code(jax_fn=True)` emits a module whose entrypoint operates on `jax.Array` inputs instead of TorchTPU tensors, driving the kernel through the real `pl.kernel` compile core (`_pallas_jax_call`) -- the same launch path the jax_fn runtime uses. `jax_fn` is orthogonal to `allow_helion_deps`: with `allow_helion_deps=False` the launch core is inlined for a pure-JAX module (`jax` the only dep) and any in-kernel helpers (topk) are embedded; with `allow_helion_deps=True` both the launch core and the helper imports come from helion (`jax` + `helion`). Either way the entrypoint takes jax arrays. The launch metadata (grid, block specs, output shape/dtype, arg positions) is captured by running the compiled host wrapper with a capturing launcher on real tensors materialized from the bound kernel's fake args, then a jax-native wrapper is emitted. Dynamic shapes work via a two-probe derivation: the symbolic input dims are recorded *before* the capture run (which specializes them), then a second probe at scaled shapes reveals which grid/output/scalar launch values track an input dim (derived as `inputs[i].shape[d]`) versus which are genuine constants (baked). Advanced Pallas features or int64/uint64/float64 args raise a clear `NotImplementedError`. `build_jax_fn_code` lives on the backend classes; `PallasBackend` implements it, other backends raise. Runs outside the fake-tensor env (it compiles + launches). stack-info: PR: #3186, branch: AmesingFlank/stack/100
AmesingFlank
force-pushed
the
AmesingFlank/stack/100
branch
from
July 30, 2026 15:31
4140b61 to
994e8b4
Compare
to_code(jax_fn=True) for Pallas (JAX-array entrypoints)OutputCodeOptions(jax_fn=True) for Pallas (JAX-array entrypoints)
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
…oints) Add `OutputCodeOptions.jax_fn` (Pallas only): `to_code(jax_fn=True)` emits a module whose entrypoint operates on `jax.Array` inputs instead of TorchTPU tensors, driving the kernel through the real `pl.kernel` compile core (`_pallas_jax_call`) -- the same launch path the jax_fn runtime uses. `jax_fn` is orthogonal to `allow_helion_deps`: with `allow_helion_deps=False` the launch core is inlined for a pure-JAX module (`jax` the only dep) and any in-kernel helpers (topk) are embedded; with `allow_helion_deps=True` both the launch core and the helper imports come from helion (`jax` + `helion`). Either way the entrypoint takes jax arrays. The launch metadata (grid, block specs, output shape/dtype, arg positions) is captured by running the compiled host wrapper with a capturing launcher on real tensors materialized from the bound kernel's fake args, then a jax-native wrapper is emitted. Dynamic shapes work via a two-probe derivation: the symbolic input dims are recorded *before* the capture run (which specializes them), then a second probe at scaled shapes reveals which grid/output/scalar launch values track an input dim (derived as `inputs[i].shape[d]`) versus which are genuine constants (baked). Advanced Pallas features or int64/uint64/float64 args raise a clear `NotImplementedError`. `build_jax_fn_code` lives on the backend classes; `PallasBackend` implements it, other backends raise. Runs outside the fake-tensor env (it compiles + launches). stack-info: PR: #3186, branch: AmesingFlank/stack/100
AmesingFlank
force-pushed
the
AmesingFlank/stack/100
branch
from
July 30, 2026 17:37
994e8b4 to
c547f98
Compare
jansel
approved these changes
Jul 31, 2026
…oints)
Add `OutputCodeOptions.jax_fn` (Pallas only): `to_code(jax_fn=True)` emits a
module whose entrypoint operates on `jax.Array` inputs instead of TorchTPU
tensors, driving the kernel through the real `pl.kernel` compile core
(`_pallas_jax_call`) -- the same launch path the jax_fn runtime uses.
`jax_fn` is orthogonal to `allow_helion_deps`: with `allow_helion_deps=False`
the launch core is inlined for a pure-JAX module (`jax` the only dep) and any
in-kernel helpers (topk) are embedded; with `allow_helion_deps=True` both the
launch core and the helper imports come from helion (`jax` + `helion`). Either
way the entrypoint takes jax arrays.
Example (kernel bound with backend="pallas"):
# pure JAX (dep: jax only):
code = bound.to_code(cfg, options=OutputCodeOptions(allow_helion_deps=False, jax_fn=True))
# or import the launch core from helion (deps: jax + helion):
code = bound.to_code(cfg, options=OutputCodeOptions(jax_fn=True))
the pure-JAX module:
import jax, jax.numpy as jnp
from jax.experimental import pallas as pl
def _pallas_jax_call(...): ... # inlined launch-core slice
# (deps mode instead: from helion.runtime.pallas.launcher import _pallas_jax_call)
def _helion_add(x, y, out): ...
_BLOCK_SPEC_INFO = [...]; _OUTPUT_INDICES = [2]; _N_ARGS = 3
def add(*inputs): # jax.Array in -> jax.Array out
... # grid + output shapes derived from inputs[i].shape[d]
The launch metadata (grid, block specs, output shape/dtype, arg positions) is
captured by running the compiled host wrapper with a capturing launcher on real
tensors materialized from the bound kernel's fake args, then a jax-native
wrapper is emitted. Dynamic shapes work via a two-probe derivation: the symbolic
input dims are recorded *before* the capture run (which specializes them), then a
second probe at scaled shapes reveals which grid/output/scalar launch values
track an input dim (derived as `inputs[i].shape[d]`) versus which are genuine
constants (baked). Advanced Pallas features or int64/uint64/float64 args raise a
clear `NotImplementedError`.
The whole rewrite is AST-level: `build_jax_fn_module` takes the generated
`body_root` AST and returns the jax module AST (device kernel nodes + inlined
launcher slice + metadata + entrypoint), leaving `to_code`'s single `unparse` to
render it. The one non-AST step, `capture_jax_launch_metadata`, runs before the
rewrite and outside the fake-tensor env (it compiles + launches). Backends
without a JAX launch path raise; `PallasBackend` implements both.
stack-info: PR: #3186, branch: AmesingFlank/stack/100
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
OutputCodeOptions.jax_fn(Pallas only):to_code(jax_fn=True)emits amodule whose entrypoint operates on
jax.Arrayinputs instead of TorchTPUtensors, driving the kernel through the real
pl.kernelcompile core(
_pallas_jax_call) -- the same launch path the jax_fn runtime uses.jax_fnis orthogonal toallow_helion_deps: withallow_helion_deps=Falsethe launch core is inlined for a pure-JAX module (
jaxthe only dep) and anyin-kernel helpers (topk) are embedded; with
allow_helion_deps=Trueboth thelaunch core and the helper imports come from helion (
jax+helion). Eitherway the entrypoint takes jax arrays.
Example (kernel bound with backend="pallas"):
the pure-JAX module:
The launch metadata (grid, block specs, output shape/dtype, arg positions) is
captured by running the compiled host wrapper with a capturing launcher on real
tensors materialized from the bound kernel's fake args, then a jax-native
wrapper is emitted. Dynamic shapes work via a two-probe derivation: the symbolic
input dims are recorded before the capture run (which specializes them), then a
second probe at scaled shapes reveals which grid/output/scalar launch values
track an input dim (derived as
inputs[i].shape[d]) versus which are genuineconstants (baked). Advanced Pallas features or int64/uint64/float64 args raise a
clear
NotImplementedError.The whole rewrite is AST-level:
build_jax_fn_moduletakes the generatedbody_rootAST and returns the jax module AST (device kernel nodes + inlinedlauncher slice + metadata + entrypoint), leaving
to_code's singleunparsetorender it. The one non-AST step,
capture_jax_launch_metadata, runs before therewrite and outside the fake-tensor env (it compiles + launches). Backends
without a JAX launch path raise;
PallasBackendimplements both.