Skip to content

Support OutputCodeOptions(jax_fn=True) for Pallas (JAX-array entrypoints) - #3186

Merged
AmesingFlank merged 1 commit into
mainfrom
AmesingFlank/stack/100
Jul 31, 2026
Merged

Support OutputCodeOptions(jax_fn=True) for Pallas (JAX-array entrypoints)#3186
AmesingFlank merged 1 commit into
mainfrom
AmesingFlank/stack/100

Conversation

@AmesingFlank

@AmesingFlank AmesingFlank commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Add OutputCodeOptions.jax_fn (Pallas only): to_code(jax_fn=True) emits a
module whose entrypoint operates on jax.Array inputs instead of TorchTPU
tensors, driving the kernel through the real pl.kernel compile core
(_pallas_jax_call) -- the same launch path the jax_fn runtime uses.

jax_fn is orthogonal to allow_helion_deps: with allow_helion_deps=False
the launch core is inlined for a pure-JAX module (jax the only dep) and any
in-kernel helpers (topk) are embedded; with allow_helion_deps=True both the
launch core and the helper imports come from helion (jax + helion). Either
way the entrypoint takes jax arrays.

Example (kernel bound with backend="pallas"):

# pure JAX (dep: jax only):
code = bound.to_code(cfg, options=OutputCodeOptions(allow_helion_deps=False, jax_fn=True))
# or import the launch core from helion (deps: jax + helion):
code = bound.to_code(cfg, options=OutputCodeOptions(jax_fn=True))

the pure-JAX module:

import jax, jax.numpy as jnp
from jax.experimental import pallas as pl
def _pallas_jax_call(...): ...   # inlined launch-core slice
    # (deps mode instead: from helion.runtime.pallas.launcher import _pallas_jax_call)
def _helion_add(x, y, out): ...
_BLOCK_SPEC_INFO = [...]; _OUTPUT_INDICES = [2]; _N_ARGS = 3
def add(*inputs):                # jax.Array in -> jax.Array out
    ...                          # grid + output shapes derived from inputs[i].shape[d]

The launch metadata (grid, block specs, output shape/dtype, arg positions) is
captured by running the compiled host wrapper with a capturing launcher on real
tensors materialized from the bound kernel's fake args, then a jax-native
wrapper is emitted. Dynamic shapes work via a two-probe derivation: the symbolic
input dims are recorded before the capture run (which specializes them), then a
second probe at scaled shapes reveals which grid/output/scalar launch values
track an input dim (derived as inputs[i].shape[d]) versus which are genuine
constants (baked). Advanced Pallas features or int64/uint64/float64 args raise a
clear NotImplementedError.

The whole rewrite is AST-level: build_jax_fn_module takes the generated
body_root AST and returns the jax module AST (device kernel nodes + inlined
launcher slice + metadata + entrypoint), leaving to_code's single unparse to
render it. The one non-AST step, capture_jax_launch_metadata, runs before the
rewrite and outside the fake-tensor env (it compiles + launches). Backends
without a JAX launch path raise; PallasBackend implements both.

@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/100 branch from 201c9f5 to 5d5e92a Compare July 28, 2026 16:02
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/99 branch from 0f726c4 to 676d880 Compare July 28, 2026 16:02
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jul 28, 2026
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/99 to main July 28, 2026 17:39
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/100 branch 2 times, most recently from 3a4917c to 9281c35 Compare July 28, 2026 17:39
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/104 July 28, 2026 17:39
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/104 to main July 28, 2026 20:09
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/100 branch from 9281c35 to d059231 Compare July 28, 2026 20:09
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/104 July 28, 2026 20:10
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/104 to main July 28, 2026 21:14
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/100 branch from d059231 to fc39ad1 Compare July 28, 2026 21:14
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/104 July 28, 2026 21:15
AmesingFlank added a commit that referenced this pull request Jul 28, 2026
`helion.precompile(PrecompilationInput(..., jax_fn=True))` emits a standalone
whose entrypoint takes `jax.Array`s and runs the kernel via the real `pl.kernel`
compile core -- the SAME launch path the jax_fn runtime uses -- with **only jax**
at runtime (no torch, no helion).

`_precompile_pallas_jax` runs the compiled host wrapper once with a capturing
launcher to record the launch metadata (grid, per-tensor block specs, output
shape/dtype, input/output arg positions), then:
- inlines the JAX-only slice of `helion/runtime/pallas/launcher.py` -- the
  transitive code closure of `_pallas_jax_call` (compile core + block specs +
  `pl.kernel`), with the torch launcher / JaxCallable / conversions dropped;
- inlines the generated (jax-native) device kernel; and
- emits a jax-native wrapper that allocates `jnp.empty` output placeholders and
  calls the inlined `_pallas_jax_call`, so the standalone reuses the exact
  runtime launch logic rather than a hand-rolled `pl.pallas_call`.

Scope: plain block specs (including multi-dim tile loops that lower to a flat
grid). Kernels using advanced Pallas features -- scratch/VMEM, HBM pass-through,
SMEM, dynamic-shape padding, in-place aliasing, compact-worklist, matmul
dot_general -- or int64/uint64/float64 tensors raise `NotImplementedError`
(detected from the captured launcher kwargs / dtypes). The Triton and
Pallas/TorchTPU precompile paths are untouched.

Verified on the TPU pod (`test_precompile.py` Pallas: 5 passed) -- add, a
dtype-casting kernel (device body uses `lax`), a two-output kernel, and a matmul
kernel that must raise; each jax_fn standalone has no `import torch`/`import
helion`, uses `pl.kernel` via the inlined `_pallas_jax_call`, and runs correctly
on JAX arrays. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected.

stack-info: PR: #3186, branch: AmesingFlank/stack/100
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/104 to main July 28, 2026 22:27
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/100 branch from fc39ad1 to 32926ab Compare July 28, 2026 22:27
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/104 July 28, 2026 22:27
AmesingFlank added a commit that referenced this pull request Jul 29, 2026
`helion.precompile(PrecompilationInput(..., jax_fn=True))` emits a standalone
whose entrypoint takes `jax.Array`s and runs the kernel via the real `pl.kernel`
compile core -- the SAME launch path the jax_fn runtime uses -- with **only jax**
at runtime (no torch, no helion).

`_precompile_pallas_jax` runs the compiled host wrapper once with a capturing
launcher to record the launch metadata (grid, per-tensor block specs, output
shape/dtype, input/output arg positions), then:
- inlines the JAX-only slice of `helion/runtime/pallas/launcher.py` -- the
  transitive code closure of `_pallas_jax_call` (compile core + block specs +
  `pl.kernel`), with the torch launcher / JaxCallable / conversions dropped;
- inlines the generated (jax-native) device kernel; and
- emits a jax-native wrapper that allocates `jnp.empty` output placeholders and
  calls the inlined `_pallas_jax_call`, so the standalone reuses the exact
  runtime launch logic rather than a hand-rolled `pl.pallas_call`.

Scope: plain block specs (including multi-dim tile loops that lower to a flat
grid). Kernels using advanced Pallas features -- scratch/VMEM, HBM pass-through,
SMEM, dynamic-shape padding, in-place aliasing, compact-worklist, matmul
dot_general -- or int64/uint64/float64 tensors raise `NotImplementedError`
(detected from the captured launcher kwargs / dtypes). The Triton and
Pallas/TorchTPU precompile paths are untouched.

Verified on the TPU pod (`test_precompile.py` Pallas: 5 passed) -- add, a
dtype-casting kernel (device body uses `lax`), a two-output kernel, and a matmul
kernel that must raise; each jax_fn standalone has no `import torch`/`import
helion`, uses `pl.kernel` via the inlined `_pallas_jax_call`, and runs correctly
on JAX arrays. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected.

stack-info: PR: #3186, branch: AmesingFlank/stack/100
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/98 to main July 30, 2026 03:17
AmesingFlank added a commit that referenced this pull request Jul 30, 2026
`helion.precompile(PrecompilationInput(..., jax_fn=True))` emits a standalone
whose entrypoint takes `jax.Array`s and runs the kernel via the real `pl.kernel`
compile core -- the SAME launch path the jax_fn runtime uses -- with **only jax**
at runtime (no torch, no helion).

`_precompile_pallas_jax` runs the compiled host wrapper once with a capturing
launcher to record the launch metadata (grid, per-tensor block specs, output
shape/dtype, input/output arg positions), then:
- inlines the JAX-only slice of `helion/runtime/pallas/launcher.py` -- the
  transitive code closure of `_pallas_jax_call` (compile core + block specs +
  `pl.kernel`), with the torch launcher / JaxCallable / conversions dropped;
- inlines the generated (jax-native) device kernel; and
- emits a jax-native wrapper that allocates `jnp.empty` output placeholders and
  calls the inlined `_pallas_jax_call`, so the standalone reuses the exact
  runtime launch logic rather than a hand-rolled `pl.pallas_call`.

Scope: plain block specs (including multi-dim tile loops that lower to a flat
grid). Kernels using advanced Pallas features -- scratch/VMEM, HBM pass-through,
SMEM, dynamic-shape padding, in-place aliasing, compact-worklist, matmul
dot_general -- or int64/uint64/float64 tensors raise `NotImplementedError`
(detected from the captured launcher kwargs / dtypes). The Triton and
Pallas/TorchTPU precompile paths are untouched.

Verified on the TPU pod (`test_precompile.py` Pallas: 5 passed) -- add, a
dtype-casting kernel (device body uses `lax`), a two-output kernel, and a matmul
kernel that must raise; each jax_fn standalone has no `import torch`/`import
helion`, uses `pl.kernel` via the inlined `_pallas_jax_call`, and runs correctly
on JAX arrays. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected.

stack-info: PR: #3186, branch: AmesingFlank/stack/100
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/100 branch from fa15f88 to 7c11e8a Compare July 30, 2026 03:17
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/98 July 30, 2026 03:17
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/98 to main July 30, 2026 03:28
AmesingFlank added a commit that referenced this pull request Jul 30, 2026
`helion.precompile(PrecompilationInput(..., jax_fn=True))` emits a standalone
whose entrypoint takes `jax.Array`s and runs the kernel via the real `pl.kernel`
compile core -- the SAME launch path the jax_fn runtime uses -- with **only jax**
at runtime (no torch, no helion).

`_precompile_pallas_jax` runs the compiled host wrapper once with a capturing
launcher to record the launch metadata (grid, per-tensor block specs, output
shape/dtype, input/output arg positions), then:
- inlines the JAX-only slice of `helion/runtime/pallas/launcher.py` -- the
  transitive code closure of `_pallas_jax_call` (compile core + block specs +
  `pl.kernel`), with the torch launcher / JaxCallable / conversions dropped;
- inlines the generated (jax-native) device kernel; and
- emits a jax-native wrapper that allocates `jnp.empty` output placeholders and
  calls the inlined `_pallas_jax_call`, so the standalone reuses the exact
  runtime launch logic rather than a hand-rolled `pl.pallas_call`.

Scope: plain block specs (including multi-dim tile loops that lower to a flat
grid). Kernels using advanced Pallas features -- scratch/VMEM, HBM pass-through,
SMEM, dynamic-shape padding, in-place aliasing, compact-worklist, matmul
dot_general -- or int64/uint64/float64 tensors raise `NotImplementedError`
(detected from the captured launcher kwargs / dtypes). The Triton and
Pallas/TorchTPU precompile paths are untouched.

Verified on the TPU pod (`test_precompile.py` Pallas: 5 passed) -- add, a
dtype-casting kernel (device body uses `lax`), a two-output kernel, and a matmul
kernel that must raise; each jax_fn standalone has no `import torch`/`import
helion`, uses `pl.kernel` via the inlined `_pallas_jax_call`, and runs correctly
on JAX arrays. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected.

stack-info: PR: #3186, branch: AmesingFlank/stack/100
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/100 branch from 7c11e8a to e9995c8 Compare July 30, 2026 03:29
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/98 July 30, 2026 03:29
AmesingFlank added a commit that referenced this pull request Jul 30, 2026
`helion.precompile(PrecompilationInput(..., jax_fn=True))` emits a standalone
whose entrypoint takes `jax.Array`s and runs the kernel via the real `pl.kernel`
compile core -- the SAME launch path the jax_fn runtime uses -- with **only jax**
at runtime (no torch, no helion).

`_precompile_pallas_jax` runs the compiled host wrapper once with a capturing
launcher to record the launch metadata (grid, per-tensor block specs, output
shape/dtype, input/output arg positions), then:
- inlines the JAX-only slice of `helion/runtime/pallas/launcher.py` -- the
  transitive code closure of `_pallas_jax_call` (compile core + block specs +
  `pl.kernel`), with the torch launcher / JaxCallable / conversions dropped;
- inlines the generated (jax-native) device kernel; and
- emits a jax-native wrapper that allocates `jnp.empty` output placeholders and
  calls the inlined `_pallas_jax_call`, so the standalone reuses the exact
  runtime launch logic rather than a hand-rolled `pl.pallas_call`.

Scope: plain block specs (including multi-dim tile loops that lower to a flat
grid). Kernels using advanced Pallas features -- scratch/VMEM, HBM pass-through,
SMEM, dynamic-shape padding, in-place aliasing, compact-worklist, matmul
dot_general -- or int64/uint64/float64 tensors raise `NotImplementedError`
(detected from the captured launcher kwargs / dtypes). The Triton and
Pallas/TorchTPU precompile paths are untouched.

Verified on the TPU pod (`test_precompile.py` Pallas: 5 passed) -- add, a
dtype-casting kernel (device body uses `lax`), a two-output kernel, and a matmul
kernel that must raise; each jax_fn standalone has no `import torch`/`import
helion`, uses `pl.kernel` via the inlined `_pallas_jax_call`, and runs correctly
on JAX arrays. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected.

stack-info: PR: #3186, branch: AmesingFlank/stack/100
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/98 to main July 30, 2026 03:47
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/100 branch from e9995c8 to 7d604d7 Compare July 30, 2026 03:47
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/98 July 30, 2026 03:48
AmesingFlank added a commit that referenced this pull request Jul 30, 2026
`helion.precompile(PrecompilationInput(..., jax_fn=True))` emits a standalone
whose entrypoint takes `jax.Array`s and runs the kernel via the real `pl.kernel`
compile core -- the SAME launch path the jax_fn runtime uses -- with **only jax**
at runtime (no torch, no helion).

`_precompile_pallas_jax` runs the compiled host wrapper once with a capturing
launcher to record the launch metadata (grid, per-tensor block specs, output
shape/dtype, input/output arg positions), then:
- inlines the JAX-only slice of `helion/runtime/pallas/launcher.py` -- the
  transitive code closure of `_pallas_jax_call` (compile core + block specs +
  `pl.kernel`), with the torch launcher / JaxCallable / conversions dropped;
- inlines the generated (jax-native) device kernel; and
- emits a jax-native wrapper that allocates `jnp.empty` output placeholders and
  calls the inlined `_pallas_jax_call`, so the standalone reuses the exact
  runtime launch logic rather than a hand-rolled `pl.pallas_call`.

Scope: plain block specs (including multi-dim tile loops that lower to a flat
grid). Kernels using advanced Pallas features -- scratch/VMEM, HBM pass-through,
SMEM, dynamic-shape padding, in-place aliasing, compact-worklist, matmul
dot_general -- or int64/uint64/float64 tensors raise `NotImplementedError`
(detected from the captured launcher kwargs / dtypes). The Triton and
Pallas/TorchTPU precompile paths are untouched.

Verified on the TPU pod (`test_precompile.py` Pallas: 5 passed) -- add, a
dtype-casting kernel (device body uses `lax`), a two-output kernel, and a matmul
kernel that must raise; each jax_fn standalone has no `import torch`/`import
helion`, uses `pl.kernel` via the inlined `_pallas_jax_call`, and runs correctly
on JAX arrays. `ruff`/`pyrefly` clean (83 baseline); H100 Triton unaffected.

stack-info: PR: #3186, branch: AmesingFlank/stack/100
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/98 to main July 30, 2026 08:14
AmesingFlank added a commit that referenced this pull request Jul 30, 2026
Add `OutputCodeOptions.jax_fn` (Pallas only): `to_code(jax_fn=True)` emits a
module whose entrypoint operates on `jax.Array` inputs instead of TorchTPU
tensors, driving the kernel through the real `pl.kernel` compile core
(`_pallas_jax_call`) -- the same launch path the jax_fn runtime uses.

`jax_fn` is orthogonal to `allow_helion_deps`: with `allow_helion_deps=False`
the launch core is inlined for a pure-JAX module (`jax` the only dep) and any
in-kernel helpers (topk) are embedded; with `allow_helion_deps=True` both the
launch core and the helper imports come from helion (`jax` + `helion`). Either
way the entrypoint takes jax arrays.

The launch metadata (grid, block specs, output shape/dtype, arg positions) is
captured by running the compiled host wrapper with a capturing launcher on real
tensors materialized from the bound kernel's fake args, then a jax-native
wrapper is emitted. Dynamic shapes work via a two-probe derivation: the symbolic
input dims are recorded *before* the capture run (which specializes them), then a
second probe at scaled shapes reveals which grid/output/scalar launch values
track an input dim (derived as `inputs[i].shape[d]`) versus which are genuine
constants (baked). Advanced Pallas features or int64/uint64/float64 args raise a
clear `NotImplementedError`.

`build_jax_fn_code` lives on the backend classes; `PallasBackend` implements it,
other backends raise. Runs outside the fake-tensor env (it compiles + launches).

stack-info: PR: #3186, branch: AmesingFlank/stack/100
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/100 branch from 7d604d7 to 4140b61 Compare July 30, 2026 08:15
@AmesingFlank AmesingFlank changed the title Make helion.precompile support Pallas jax_fn=True (pure JAX) Support to_code(jax_fn=True) for Pallas (JAX-array entrypoints) Jul 30, 2026
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/98 July 30, 2026 08:15
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/98 to main July 30, 2026 15:31
AmesingFlank added a commit that referenced this pull request Jul 30, 2026
…oints)

Add `OutputCodeOptions.jax_fn` (Pallas only): `to_code(jax_fn=True)` emits a
module whose entrypoint operates on `jax.Array` inputs instead of TorchTPU
tensors, driving the kernel through the real `pl.kernel` compile core
(`_pallas_jax_call`) -- the same launch path the jax_fn runtime uses.

`jax_fn` is orthogonal to `allow_helion_deps`: with `allow_helion_deps=False`
the launch core is inlined for a pure-JAX module (`jax` the only dep) and any
in-kernel helpers (topk) are embedded; with `allow_helion_deps=True` both the
launch core and the helper imports come from helion (`jax` + `helion`). Either
way the entrypoint takes jax arrays.

The launch metadata (grid, block specs, output shape/dtype, arg positions) is
captured by running the compiled host wrapper with a capturing launcher on real
tensors materialized from the bound kernel's fake args, then a jax-native
wrapper is emitted. Dynamic shapes work via a two-probe derivation: the symbolic
input dims are recorded *before* the capture run (which specializes them), then a
second probe at scaled shapes reveals which grid/output/scalar launch values
track an input dim (derived as `inputs[i].shape[d]`) versus which are genuine
constants (baked). Advanced Pallas features or int64/uint64/float64 args raise a
clear `NotImplementedError`.

`build_jax_fn_code` lives on the backend classes; `PallasBackend` implements it,
other backends raise. Runs outside the fake-tensor env (it compiles + launches).

stack-info: PR: #3186, branch: AmesingFlank/stack/100
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/100 branch from 4140b61 to 994e8b4 Compare July 30, 2026 15:31
@AmesingFlank AmesingFlank changed the title Support to_code(jax_fn=True) for Pallas (JAX-array entrypoints) Support OutputCodeOptions(jax_fn=True) for Pallas (JAX-array entrypoints) Jul 30, 2026
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/98 July 30, 2026 15:31
AmesingFlank added a commit that referenced this pull request Jul 30, 2026
…oints)

Add `OutputCodeOptions.jax_fn` (Pallas only): `to_code(jax_fn=True)` emits a
module whose entrypoint operates on `jax.Array` inputs instead of TorchTPU
tensors, driving the kernel through the real `pl.kernel` compile core
(`_pallas_jax_call`) -- the same launch path the jax_fn runtime uses.

`jax_fn` is orthogonal to `allow_helion_deps`: with `allow_helion_deps=False`
the launch core is inlined for a pure-JAX module (`jax` the only dep) and any
in-kernel helpers (topk) are embedded; with `allow_helion_deps=True` both the
launch core and the helper imports come from helion (`jax` + `helion`). Either
way the entrypoint takes jax arrays.

The launch metadata (grid, block specs, output shape/dtype, arg positions) is
captured by running the compiled host wrapper with a capturing launcher on real
tensors materialized from the bound kernel's fake args, then a jax-native
wrapper is emitted. Dynamic shapes work via a two-probe derivation: the symbolic
input dims are recorded *before* the capture run (which specializes them), then a
second probe at scaled shapes reveals which grid/output/scalar launch values
track an input dim (derived as `inputs[i].shape[d]`) versus which are genuine
constants (baked). Advanced Pallas features or int64/uint64/float64 args raise a
clear `NotImplementedError`.

`build_jax_fn_code` lives on the backend classes; `PallasBackend` implements it,
other backends raise. Runs outside the fake-tensor env (it compiles + launches).

stack-info: PR: #3186, branch: AmesingFlank/stack/100
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/98 to main July 30, 2026 17:37
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/100 branch from 994e8b4 to c547f98 Compare July 30, 2026 17:37
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/98 July 30, 2026 17:37
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/98 to main July 30, 2026 18:06
…oints)

Add `OutputCodeOptions.jax_fn` (Pallas only): `to_code(jax_fn=True)` emits a
module whose entrypoint operates on `jax.Array` inputs instead of TorchTPU
tensors, driving the kernel through the real `pl.kernel` compile core
(`_pallas_jax_call`) -- the same launch path the jax_fn runtime uses.

`jax_fn` is orthogonal to `allow_helion_deps`: with `allow_helion_deps=False`
the launch core is inlined for a pure-JAX module (`jax` the only dep) and any
in-kernel helpers (topk) are embedded; with `allow_helion_deps=True` both the
launch core and the helper imports come from helion (`jax` + `helion`). Either
way the entrypoint takes jax arrays.

Example (kernel bound with backend="pallas"):

    # pure JAX (dep: jax only):
    code = bound.to_code(cfg, options=OutputCodeOptions(allow_helion_deps=False, jax_fn=True))
    # or import the launch core from helion (deps: jax + helion):
    code = bound.to_code(cfg, options=OutputCodeOptions(jax_fn=True))

the pure-JAX module:

    import jax, jax.numpy as jnp
    from jax.experimental import pallas as pl
    def _pallas_jax_call(...): ...   # inlined launch-core slice
        # (deps mode instead: from helion.runtime.pallas.launcher import _pallas_jax_call)
    def _helion_add(x, y, out): ...
    _BLOCK_SPEC_INFO = [...]; _OUTPUT_INDICES = [2]; _N_ARGS = 3
    def add(*inputs):                # jax.Array in -> jax.Array out
        ...                          # grid + output shapes derived from inputs[i].shape[d]

The launch metadata (grid, block specs, output shape/dtype, arg positions) is
captured by running the compiled host wrapper with a capturing launcher on real
tensors materialized from the bound kernel's fake args, then a jax-native
wrapper is emitted. Dynamic shapes work via a two-probe derivation: the symbolic
input dims are recorded *before* the capture run (which specializes them), then a
second probe at scaled shapes reveals which grid/output/scalar launch values
track an input dim (derived as `inputs[i].shape[d]`) versus which are genuine
constants (baked). Advanced Pallas features or int64/uint64/float64 args raise a
clear `NotImplementedError`.

The whole rewrite is AST-level: `build_jax_fn_module` takes the generated
`body_root` AST and returns the jax module AST (device kernel nodes + inlined
launcher slice + metadata + entrypoint), leaving `to_code`'s single `unparse` to
render it. The one non-AST step, `capture_jax_launch_metadata`, runs before the
rewrite and outside the fake-tensor env (it compiles + launches). Backends
without a JAX launch path raise; `PallasBackend` implements both.

stack-info: PR: #3186, branch: AmesingFlank/stack/100
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants