Skip to content

Support in-tile reshapes for symbolic block sizes - #3165

Open
yushangdi wants to merge 2 commits into
mainfrom
fix/issue-3163-tile-reshape-guards
Open

Support in-tile reshapes for symbolic block sizes#3165
yushangdi wants to merge 2 commits into
mainfrom
fix/issue-3163-tile-reshape-guards

Conversation

@yushangdi

Copy link
Copy Markdown
Contributor

Fixes #3163

Summary

  • intercept ATen reshape/view before FakeTensor shape validation
  • derive block-size divisibility guards from symbolic tile reshapes
  • reject invalid explicit configs and filter invalid autotuning candidates
  • add FP8 grouped-quantization regression coverage based on the issue repro

Testing

  • pytest test/test_indexing.py test/test_tensor_numel_constraints.py test/test_config_api.py -n4 -x -q (154 passed, 8 skipped)
  • Ruff format and lint checks on touched files
  • standalone issue reproducer on H100

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jul 27, 2026
@yushangdi
yushangdi force-pushed the fix/issue-3163-tile-reshape-guards branch 3 times, most recently from ad4aae5 to 03017e5 Compare July 27, 2026 21:40
@yushangdi

yushangdi commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

The issue is that Helion traces the kernel before selecting a concrete configuration. During tracing, block sizes
are symbolic, so Helion cannot determine whether an expression such as block_size // 128 produces a valid reshape.

The fix has three parts:

  1. During tracing, derive and record the reshape’s element-count constraint. In this example, it requires the
    block size to be divisible by 128.

  2. Find a temporary power-of-two block-size assignment that satisfies all recorded constraints, allowing
    FakeTensor tracing to complete. (TODO: currently the search is quite brute-force, maybe there is a better way to do the search, or maybe we can take hints from user provided configs)

  3. Validate generated and user-provided autotuning configurations against these constraints, so only
    configurations that produce valid reshapes are compiled and benchmarked.

@yushangdi
yushangdi force-pushed the fix/issue-3163-tile-reshape-guards branch from 03017e5 to d136cf8 Compare July 27, 2026 21:59
@yushangdi
yushangdi marked this pull request as ready for review July 27, 2026 22:04
@yushangdi
yushangdi requested review from jansel and oulgen July 27, 2026 22:12
Comment thread test/test_indexing.py
torch.testing.assert_close(out2, x2)

@skipIfCudaCapabilityLessThan((9, 0), reason="FP8 requires CUDA capability >= 9.0")
@skipIfCute("CuTe does not support scalar float32 to float8 conversion")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we fix this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, I can try to fix it in a separate PR.

@yushangdi yushangdi Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the conversion fix in this stack: #3169

Even with the conversion problem fixed, there is still some other issue with the cute backend. Haven't reviewed the PRs, yet. Do you want to land this PR only after the cute backend is also working?

Comment thread test/test_indexing.py Outdated
@yushangdi
yushangdi requested review from echen4096, ethche and oulgen and removed request for echen4096 July 28, 2026 17:47

@ethche ethche left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex mentions these as potential issues. Could you double check?

  1. Blocker: only reshape-numel guards are recorded (compile_environment.py:1064). Other symbolic guards introduced during tracing are ignored. I reproduced a config with BLOCK_N=256 that passed normalization but later failed Triton compilation because squeeze(1) had been traced assuming the group dimension was 1. The fix should capture all block-dependent ShapeEnv guards, or conservatively pin involved block sizes to the chosen assignment.

  2. Autotuning gap: retries were added only to random_population() (config_generation.py:618), while Pattern Search and Differential Evolution use random_population_flat(). Invalid constrained configs are therefore discarded without replenishment, potentially starving the population.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

can helion support in-tile reshape for quantization casting?

3 participants