Refactor the Pallas launcher to make it easier to operate on Jax Arrays as well as TorchTPU tensors - #3190
Open
AmesingFlank wants to merge 1 commit into
Open
Refactor the Pallas launcher to make it easier to operate on Jax Arrays as well as TorchTPU tensors#3190AmesingFlank wants to merge 1 commit into
AmesingFlank wants to merge 1 commit into
Conversation
AmesingFlank
force-pushed
the
AmesingFlank/stack/104
branch
from
July 28, 2026 17:39
ed61d45 to
1da69c2
Compare
This was referenced Jul 28, 2026
AmesingFlank
marked this pull request as draft
July 28, 2026 17:44
AmesingFlank
force-pushed
the
AmesingFlank/stack/104
branch
from
July 28, 2026 20:09
1da69c2 to
174c7fb
Compare
AmesingFlank
force-pushed
the
AmesingFlank/stack/104
branch
from
July 28, 2026 21:14
174c7fb to
b84ac40
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 28, 2026
Prepare the Pallas backend so every launch mode -- torch jit, torch precompiled, jax_fn jit, jax_fn precompiled -- shares one compile core (`pl.kernel`) instead of duplicating launch logic. No behavior change for existing modes. - `_pallas_compile_jit_fn` (and the compact variant) plus the block-spec / VMEM / reorder helpers now accept any array-like arg -- a `torch.Tensor`, the `_JaxExportTensor` adapter, or a raw `jax.Array` -- via a duck-typed `_is_array_arg` / `_ArrayLike` `TypeGuard`. The output placeholder is parameterized (`_pallas_torch_placeholder` for the torch launcher, `_pallas_jax_placeholder` -- the torch-free default -- for the jax paths) and `_bytes_per_element` no longer needs torch, so the core is torch-free. - New `_pallas_jax_call`: the shared raw-`jax.Array` launch path (compile core + `pl.kernel` jit_fn + output collection), used by both the jax_fn runtime launcher and (next commit) the jax_fn precompiled standalone. - `default_pallas_jax_launcher` unwraps its adapters to raw jax arrays and delegates to `_pallas_jax_call`, dropping its duplicated compile/run/collect block; the `_JaxExportTensor` adapter is now confined to the torch-flavored host wrapper's shape ops, not the compile core. Verified on TPU: jax_fn runtime (`test_pallas.py -k jax_fn`: 5 passed) and the torch launcher (compact-worklist 66 passed / 6 pre-existing JAX-skew fails; topk 3 passed) are unchanged. `ruff`/`pyrefly` clean (83 baseline). stack-info: PR: #3190, branch: AmesingFlank/stack/104
AmesingFlank
force-pushed
the
AmesingFlank/stack/104
branch
from
July 28, 2026 22:27
b84ac40 to
b650107
Compare
AmesingFlank
force-pushed
the
AmesingFlank/stack/104
branch
from
July 29, 2026 05:05
b650107 to
28e67c8
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
Prepare the Pallas backend so every launch mode -- torch jit, torch precompiled, jax_fn jit, jax_fn precompiled -- shares one compile core (`pl.kernel`) instead of duplicating launch logic. No behavior change for existing modes. - `_pallas_compile_jit_fn` (and the compact variant) plus the block-spec / VMEM / reorder helpers now accept any array-like arg -- a `torch.Tensor`, the `_JaxExportTensor` adapter, or a raw `jax.Array` -- via a duck-typed `_is_array_arg` / `_ArrayLike` `TypeGuard`. The output placeholder is parameterized (`_pallas_torch_placeholder` for the torch launcher, `_pallas_jax_placeholder` -- the torch-free default -- for the jax paths) and `_bytes_per_element` no longer needs torch, so the core is torch-free. - New `_pallas_jax_call`: the shared raw-`jax.Array` launch path (compile core + `pl.kernel` jit_fn + output collection), used by both the jax_fn runtime launcher and (next commit) the jax_fn precompiled standalone. - `default_pallas_jax_launcher` unwraps its adapters to raw jax arrays and delegates to `_pallas_jax_call`, dropping its duplicated compile/run/collect block; the `_JaxExportTensor` adapter is now confined to the torch-flavored host wrapper's shape ops, not the compile core. Verified on TPU: jax_fn runtime (`test_pallas.py -k jax_fn`: 5 passed) and the torch launcher (compact-worklist 66 passed / 6 pre-existing JAX-skew fails; topk 3 passed) are unchanged. `ruff`/`pyrefly` clean (83 baseline). stack-info: PR: #3190, branch: AmesingFlank/stack/104
AmesingFlank
force-pushed
the
AmesingFlank/stack/104
branch
from
July 30, 2026 03:47
68281ce to
22e226f
Compare
AmesingFlank
added a commit
that referenced
this pull request
Jul 30, 2026
Prepare the Pallas backend so every launch mode -- torch jit, torch precompiled, jax_fn jit, jax_fn precompiled -- shares one compile core (`pl.kernel`) instead of duplicating launch logic. No behavior change for existing modes. - `_pallas_compile_jit_fn` (and the compact variant) plus the block-spec / VMEM / reorder helpers now accept any array-like arg -- a `torch.Tensor`, the `_JaxExportTensor` adapter, or a raw `jax.Array` -- via a duck-typed `_is_array_arg` / `_ArrayLike` `TypeGuard`. The output placeholder is parameterized (`_pallas_torch_placeholder` for the torch launcher, `_pallas_jax_placeholder` -- the torch-free default -- for the jax paths) and `_bytes_per_element` no longer needs torch, so the core is torch-free. - New `_pallas_jax_call`: the shared raw-`jax.Array` launch path (compile core + `pl.kernel` jit_fn + output collection), used by both the jax_fn runtime launcher and (next commit) the jax_fn precompiled standalone. - `default_pallas_jax_launcher` unwraps its adapters to raw jax arrays and delegates to `_pallas_jax_call`, dropping its duplicated compile/run/collect block; the `_JaxExportTensor` adapter is now confined to the torch-flavored host wrapper's shape ops, not the compile core. Verified on TPU: jax_fn runtime (`test_pallas.py -k jax_fn`: 5 passed) and the torch launcher (compact-worklist 66 passed / 6 pre-existing JAX-skew fails; topk 3 passed) are unchanged. `ruff`/`pyrefly` clean (83 baseline). stack-info: PR: #3190, branch: AmesingFlank/stack/104
AmesingFlank
force-pushed
the
AmesingFlank/stack/104
branch
from
July 30, 2026 08:15
22e226f to
26d0651
Compare
AmesingFlank
force-pushed
the
AmesingFlank/stack/104
branch
from
July 30, 2026 15:31
26d0651 to
5c640db
Compare
AmesingFlank
marked this pull request as ready for review
July 30, 2026 15:54
AmesingFlank
marked this pull request as draft
July 30, 2026 17:36
AmesingFlank
marked this pull request as ready for review
July 30, 2026 17:37
AmesingFlank
marked this pull request as draft
July 30, 2026 18:06
AmesingFlank
marked this pull request as ready for review
July 30, 2026 18:07
AmesingFlank
marked this pull request as draft
July 30, 2026 19:10
AmesingFlank
force-pushed
the
AmesingFlank/stack/104
branch
from
July 30, 2026 19:10
5c640db to
d48e92c
Compare
AmesingFlank
marked this pull request as ready for review
July 30, 2026 19:10
AmesingFlank
marked this pull request as draft
July 31, 2026 15:47
AmesingFlank
force-pushed
the
AmesingFlank/stack/104
branch
from
July 31, 2026 15:47
d48e92c to
0a37a98
Compare
AmesingFlank
marked this pull request as ready for review
July 31, 2026 15:47
…ys as well as TorchTPU tensors Make the Pallas launch/compile core type-agnostic so a kernel can be driven from either a `torch.Tensor` (TorchTPU) or a raw `jax.Array` (jax_fn) through the same code, rather than the core assuming torch tensors and each launch mode duplicating its own compile/run/collect logic. This lets all four modes -- torch jit, torch precompiled, jax_fn jit, jax_fn precompiled -- share one `pl.kernel` compile core; it is preparation for the jax_fn precompiled standalone (next commit) and is a no-op for existing modes. How the changes get there: - Operate on any array-like, not just torch tensors: `_pallas_compile_jit_fn` (and the compact variant) plus the block-spec / VMEM / reorder helpers now accept a `torch.Tensor`, the `_JaxExportTensor` adapter, or a raw `jax.Array`, classified by a duck-typed `_is_array_arg` / `_ArrayLike` `TypeGuard`. The core touches only the attributes all three share (shape, dtype, ...), so one code path serves both element types. - Drop torch from the core so jax arrays need no torch round-trip: the output placeholder is parameterized (`_pallas_torch_placeholder` for the torch launcher, `_pallas_jax_placeholder` -- the torch-free default -- for the jax paths) and `_bytes_per_element` no longer imports torch, leaving the compile core torch-free. - Add `_pallas_jax_call`, the shared raw-`jax.Array` launch path (compile core + `pl.kernel` jit_fn + output collection), used by both the jax_fn runtime launcher and (next commit) the jax_fn precompiled standalone. - Route the jax launcher through it: `default_pallas_jax_launcher` unwraps its adapters to raw jax arrays and delegates to `_pallas_jax_call`, dropping its duplicated compile/run/collect block. The `_JaxExportTensor` adapter is now confined to the torch-flavored host wrapper's shape ops rather than the core. Verified on TPU: jax_fn runtime (`test_pallas.py -k jax_fn`: 5 passed) and the torch launcher (compact-worklist 66 passed / 6 pre-existing JAX-skew fails; topk 3 passed) are unchanged. `ruff`/`pyrefly` clean (83 baseline). stack-info: PR: #3190, branch: AmesingFlank/stack/104
AmesingFlank
marked this pull request as draft
July 31, 2026 17:07
AmesingFlank
force-pushed
the
AmesingFlank/stack/104
branch
from
July 31, 2026 17:07
0a37a98 to
1d76ea8
Compare
AmesingFlank
marked this pull request as ready for review
July 31, 2026 17:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked PRs:
OutputCodeOptions(jax_fn=True)for Pallas (JAX-array entrypoints) #3186OutputCodeOptions(allow_helion_deps=False)for the Pallas backend (TorchTPU) #3184Refactor the Pallas launcher to make it easier to operate on Jax Arrays as well as TorchTPU tensors
Make the Pallas launch/compile core type-agnostic so a kernel can be driven from
either a
torch.Tensor(TorchTPU) or a rawjax.Array(jax_fn) through the samecode, rather than the core assuming torch tensors and each launch mode duplicating
its own compile/run/collect logic. This lets all four modes -- torch jit, torch
precompiled, jax_fn jit, jax_fn precompiled -- share one
pl.kernelcompile core;it is preparation for the jax_fn precompiled standalone (next commit) and is a
no-op for existing modes.
How the changes get there:
_pallas_compile_jit_fn(andthe compact variant) plus the block-spec / VMEM / reorder helpers now accept a
torch.Tensor, the_JaxExportTensoradapter, or a rawjax.Array, classifiedby a duck-typed
_is_array_arg/_ArrayLikeTypeGuard. The core touches onlythe attributes all three share (shape, dtype, ...), so one code path serves both
element types.
placeholder is parameterized (
_pallas_torch_placeholderfor the torch launcher,_pallas_jax_placeholder-- the torch-free default -- for the jax paths) and_bytes_per_elementno longer imports torch, leaving the compile core torch-free._pallas_jax_call, the shared raw-jax.Arraylaunch path (compile core +pl.kerneljit_fn + output collection), used by both the jax_fn runtime launcherand (next commit) the jax_fn precompiled standalone.
default_pallas_jax_launcherunwraps itsadapters to raw jax arrays and delegates to
_pallas_jax_call, dropping itsduplicated compile/run/collect block. The
_JaxExportTensoradapter is nowconfined to the torch-flavored host wrapper's shape ops rather than the core.
Verified on TPU: jax_fn runtime (
test_pallas.py -k jax_fn: 5 passed) and thetorch launcher (compact-worklist 66 passed / 6 pre-existing JAX-skew fails; topk
3 passed) are unchanged.
ruff/pyreflyclean (83 baseline).