Skip to content

Refactor the Pallas launcher to make it easier to operate on Jax Arrays as well as TorchTPU tensors - #3190

Open
AmesingFlank wants to merge 1 commit into
mainfrom
AmesingFlank/stack/104
Open

Refactor the Pallas launcher to make it easier to operate on Jax Arrays as well as TorchTPU tensors#3190
AmesingFlank wants to merge 1 commit into
mainfrom
AmesingFlank/stack/104

Conversation

@AmesingFlank

@AmesingFlank AmesingFlank commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Stacked PRs:


Refactor the Pallas launcher to make it easier to operate on Jax Arrays as well as TorchTPU tensors

Make the Pallas launch/compile core type-agnostic so a kernel can be driven from
either a torch.Tensor (TorchTPU) or a raw jax.Array (jax_fn) through the same
code, rather than the core assuming torch tensors and each launch mode duplicating
its own compile/run/collect logic. This lets all four modes -- torch jit, torch
precompiled, jax_fn jit, jax_fn precompiled -- share one pl.kernel compile core;
it is preparation for the jax_fn precompiled standalone (next commit) and is a
no-op for existing modes.

How the changes get there:

  • Operate on any array-like, not just torch tensors: _pallas_compile_jit_fn (and
    the compact variant) plus the block-spec / VMEM / reorder helpers now accept a
    torch.Tensor, the _JaxExportTensor adapter, or a raw jax.Array, classified
    by a duck-typed _is_array_arg / _ArrayLike TypeGuard. The core touches only
    the attributes all three share (shape, dtype, ...), so one code path serves both
    element types.
  • Drop torch from the core so jax arrays need no torch round-trip: the output
    placeholder is parameterized (_pallas_torch_placeholder for the torch launcher,
    _pallas_jax_placeholder -- the torch-free default -- for the jax paths) and
    _bytes_per_element no longer imports torch, leaving the compile core torch-free.
  • Add _pallas_jax_call, the shared raw-jax.Array launch path (compile core +
    pl.kernel jit_fn + output collection), used by both the jax_fn runtime launcher
    and (next commit) the jax_fn precompiled standalone.
  • Route the jax launcher through it: default_pallas_jax_launcher unwraps its
    adapters to raw jax arrays and delegates to _pallas_jax_call, dropping its
    duplicated compile/run/collect block. The _JaxExportTensor adapter is now
    confined to the torch-flavored host wrapper's shape ops rather than the core.

Verified on TPU: jax_fn runtime (test_pallas.py -k jax_fn: 5 passed) and the
torch launcher (compact-worklist 66 passed / 6 pre-existing JAX-skew fails; topk
3 passed) are unchanged. ruff/pyrefly clean (83 baseline).

@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/104 branch from ed61d45 to 1da69c2 Compare July 28, 2026 17:39
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jul 28, 2026
@AmesingFlank
AmesingFlank marked this pull request as draft July 28, 2026 17:44
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/99 to main July 28, 2026 20:09
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/104 branch from 1da69c2 to 174c7fb Compare July 28, 2026 20:09
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/99 July 28, 2026 20:10
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/99 to main July 28, 2026 21:14
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/104 branch from 174c7fb to b84ac40 Compare July 28, 2026 21:14
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/99 July 28, 2026 21:15
AmesingFlank added a commit that referenced this pull request Jul 28, 2026
Prepare the Pallas backend so every launch mode -- torch jit, torch precompiled,
jax_fn jit, jax_fn precompiled -- shares one compile core (`pl.kernel`) instead of
duplicating launch logic. No behavior change for existing modes.

- `_pallas_compile_jit_fn` (and the compact variant) plus the block-spec / VMEM /
  reorder helpers now accept any array-like arg -- a `torch.Tensor`, the
  `_JaxExportTensor` adapter, or a raw `jax.Array` -- via a duck-typed
  `_is_array_arg` / `_ArrayLike` `TypeGuard`. The output placeholder is
  parameterized (`_pallas_torch_placeholder` for the torch launcher,
  `_pallas_jax_placeholder` -- the torch-free default -- for the jax paths) and
  `_bytes_per_element` no longer needs torch, so the core is torch-free.
- New `_pallas_jax_call`: the shared raw-`jax.Array` launch path (compile core +
  `pl.kernel` jit_fn + output collection), used by both the jax_fn runtime
  launcher and (next commit) the jax_fn precompiled standalone.
- `default_pallas_jax_launcher` unwraps its adapters to raw jax arrays and
  delegates to `_pallas_jax_call`, dropping its duplicated compile/run/collect
  block; the `_JaxExportTensor` adapter is now confined to the torch-flavored
  host wrapper's shape ops, not the compile core.

Verified on TPU: jax_fn runtime (`test_pallas.py -k jax_fn`: 5 passed) and the
torch launcher (compact-worklist 66 passed / 6 pre-existing JAX-skew fails; topk
3 passed) are unchanged. `ruff`/`pyrefly` clean (83 baseline).

stack-info: PR: #3190, branch: AmesingFlank/stack/104
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/99 to main July 28, 2026 22:27
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/104 branch from b84ac40 to b650107 Compare July 28, 2026 22:27
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/99 July 28, 2026 22:27
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/99 to main July 29, 2026 00:03
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/99 July 29, 2026 00:04
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/99 to main July 29, 2026 05:05
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/104 branch from b650107 to 28e67c8 Compare July 29, 2026 05:05
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/99 July 29, 2026 05:06
AmesingFlank added a commit that referenced this pull request Jul 30, 2026
Prepare the Pallas backend so every launch mode -- torch jit, torch precompiled,
jax_fn jit, jax_fn precompiled -- shares one compile core (`pl.kernel`) instead of
duplicating launch logic. No behavior change for existing modes.

- `_pallas_compile_jit_fn` (and the compact variant) plus the block-spec / VMEM /
  reorder helpers now accept any array-like arg -- a `torch.Tensor`, the
  `_JaxExportTensor` adapter, or a raw `jax.Array` -- via a duck-typed
  `_is_array_arg` / `_ArrayLike` `TypeGuard`. The output placeholder is
  parameterized (`_pallas_torch_placeholder` for the torch launcher,
  `_pallas_jax_placeholder` -- the torch-free default -- for the jax paths) and
  `_bytes_per_element` no longer needs torch, so the core is torch-free.
- New `_pallas_jax_call`: the shared raw-`jax.Array` launch path (compile core +
  `pl.kernel` jit_fn + output collection), used by both the jax_fn runtime
  launcher and (next commit) the jax_fn precompiled standalone.
- `default_pallas_jax_launcher` unwraps its adapters to raw jax arrays and
  delegates to `_pallas_jax_call`, dropping its duplicated compile/run/collect
  block; the `_JaxExportTensor` adapter is now confined to the torch-flavored
  host wrapper's shape ops, not the compile core.

Verified on TPU: jax_fn runtime (`test_pallas.py -k jax_fn`: 5 passed) and the
torch launcher (compact-worklist 66 passed / 6 pre-existing JAX-skew fails; topk
3 passed) are unchanged. `ruff`/`pyrefly` clean (83 baseline).

stack-info: PR: #3190, branch: AmesingFlank/stack/104
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/97 to main July 30, 2026 03:47
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/104 branch from 68281ce to 22e226f Compare July 30, 2026 03:47
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/97 July 30, 2026 03:48
AmesingFlank added a commit that referenced this pull request Jul 30, 2026
Prepare the Pallas backend so every launch mode -- torch jit, torch precompiled,
jax_fn jit, jax_fn precompiled -- shares one compile core (`pl.kernel`) instead of
duplicating launch logic. No behavior change for existing modes.

- `_pallas_compile_jit_fn` (and the compact variant) plus the block-spec / VMEM /
  reorder helpers now accept any array-like arg -- a `torch.Tensor`, the
  `_JaxExportTensor` adapter, or a raw `jax.Array` -- via a duck-typed
  `_is_array_arg` / `_ArrayLike` `TypeGuard`. The output placeholder is
  parameterized (`_pallas_torch_placeholder` for the torch launcher,
  `_pallas_jax_placeholder` -- the torch-free default -- for the jax paths) and
  `_bytes_per_element` no longer needs torch, so the core is torch-free.
- New `_pallas_jax_call`: the shared raw-`jax.Array` launch path (compile core +
  `pl.kernel` jit_fn + output collection), used by both the jax_fn runtime
  launcher and (next commit) the jax_fn precompiled standalone.
- `default_pallas_jax_launcher` unwraps its adapters to raw jax arrays and
  delegates to `_pallas_jax_call`, dropping its duplicated compile/run/collect
  block; the `_JaxExportTensor` adapter is now confined to the torch-flavored
  host wrapper's shape ops, not the compile core.

Verified on TPU: jax_fn runtime (`test_pallas.py -k jax_fn`: 5 passed) and the
torch launcher (compact-worklist 66 passed / 6 pre-existing JAX-skew fails; topk
3 passed) are unchanged. `ruff`/`pyrefly` clean (83 baseline).

stack-info: PR: #3190, branch: AmesingFlank/stack/104
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/97 to main July 30, 2026 08:14
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/104 branch from 22e226f to 26d0651 Compare July 30, 2026 08:15
@AmesingFlank AmesingFlank changed the title Refactor the Pallas launch onto a shared, duck-typed compile core Refactor the Pallas launcher to make it easier to operate on Jax Arrays as well as TorchTPU tensors Jul 30, 2026
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/104 branch from 26d0651 to 5c640db Compare July 30, 2026 15:31
@AmesingFlank
AmesingFlank marked this pull request as ready for review July 30, 2026 15:54
@AmesingFlank
AmesingFlank requested review from ethche and norx1991 July 30, 2026 15:54
@AmesingFlank
AmesingFlank marked this pull request as draft July 30, 2026 17:36
@AmesingFlank
AmesingFlank marked this pull request as ready for review July 30, 2026 17:37
@AmesingFlank
AmesingFlank marked this pull request as draft July 30, 2026 18:06
@AmesingFlank
AmesingFlank marked this pull request as ready for review July 30, 2026 18:07
@AmesingFlank
AmesingFlank requested review from jansel and oulgen July 30, 2026 18:16
@AmesingFlank
AmesingFlank marked this pull request as draft July 30, 2026 19:10
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/104 branch from 5c640db to d48e92c Compare July 30, 2026 19:10
@AmesingFlank
AmesingFlank marked this pull request as ready for review July 30, 2026 19:10
@AmesingFlank
AmesingFlank marked this pull request as draft July 31, 2026 15:47
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/104 branch from d48e92c to 0a37a98 Compare July 31, 2026 15:47
@AmesingFlank
AmesingFlank changed the base branch from main to AmesingFlank/stack/95 July 31, 2026 15:47
@AmesingFlank
AmesingFlank marked this pull request as ready for review July 31, 2026 15:47
…ys as well as TorchTPU tensors

Make the Pallas launch/compile core type-agnostic so a kernel can be driven from
either a `torch.Tensor` (TorchTPU) or a raw `jax.Array` (jax_fn) through the same
code, rather than the core assuming torch tensors and each launch mode duplicating
its own compile/run/collect logic. This lets all four modes -- torch jit, torch
precompiled, jax_fn jit, jax_fn precompiled -- share one `pl.kernel` compile core;
it is preparation for the jax_fn precompiled standalone (next commit) and is a
no-op for existing modes.

How the changes get there:

- Operate on any array-like, not just torch tensors: `_pallas_compile_jit_fn` (and
  the compact variant) plus the block-spec / VMEM / reorder helpers now accept a
  `torch.Tensor`, the `_JaxExportTensor` adapter, or a raw `jax.Array`, classified
  by a duck-typed `_is_array_arg` / `_ArrayLike` `TypeGuard`. The core touches only
  the attributes all three share (shape, dtype, ...), so one code path serves both
  element types.
- Drop torch from the core so jax arrays need no torch round-trip: the output
  placeholder is parameterized (`_pallas_torch_placeholder` for the torch launcher,
  `_pallas_jax_placeholder` -- the torch-free default -- for the jax paths) and
  `_bytes_per_element` no longer imports torch, leaving the compile core torch-free.
- Add `_pallas_jax_call`, the shared raw-`jax.Array` launch path (compile core +
  `pl.kernel` jit_fn + output collection), used by both the jax_fn runtime launcher
  and (next commit) the jax_fn precompiled standalone.
- Route the jax launcher through it: `default_pallas_jax_launcher` unwraps its
  adapters to raw jax arrays and delegates to `_pallas_jax_call`, dropping its
  duplicated compile/run/collect block. The `_JaxExportTensor` adapter is now
  confined to the torch-flavored host wrapper's shape ops rather than the core.

Verified on TPU: jax_fn runtime (`test_pallas.py -k jax_fn`: 5 passed) and the
torch launcher (compact-worklist 66 passed / 6 pre-existing JAX-skew fails; topk
3 passed) are unchanged. `ruff`/`pyrefly` clean (83 baseline).

stack-info: PR: #3190, branch: AmesingFlank/stack/104
@AmesingFlank
AmesingFlank marked this pull request as draft July 31, 2026 17:07
@AmesingFlank
AmesingFlank changed the base branch from AmesingFlank/stack/95 to main July 31, 2026 17:07
@AmesingFlank
AmesingFlank force-pushed the AmesingFlank/stack/104 branch from 0a37a98 to 1d76ea8 Compare July 31, 2026 17:07
@AmesingFlank
AmesingFlank marked this pull request as ready for review July 31, 2026 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant