Skip to content

feat(0831): Localize constant ReduceSum axes inside ONNX subfunctions - #1294

Open
vbaddi wants to merge 8 commits into
quic:mainfrom
vbaddi:fix/einsum
Open

feat(0831): Localize constant ReduceSum axes inside ONNX subfunctions#1294
vbaddi wants to merge 8 commits into
quic:mainfrom
vbaddi:fix/einsum

Conversation

@vbaddi

@vbaddi vbaddi commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Added LocalizeFunctionReduceSumAxesTransform for ONNX subfunction exports.
  • Replaced the previous torch.einsum(... reduction ...) workarounds back to equivalent .sum(...) forms across QEff modeling/MoE/blocking code.
  • Enabled the ReduceSum axes localization only when use_onnx_subfunctions=True.

We previously used einsum in several reduction patterns to avoid ONNX subfunction export promoting constant ReduceSum axes into FunctionProto inputs. That workaround avoided compiler failures, but it could lower into heavier performance dip's due to lowering of BMM instead of the intended elementwise multiply plus ReduceSum path.

The desired PyTorch source should be able to stay as:

(query * query).sum(dim=-1, keepdim=True)
and export as:
Mul -> ReduceSum <- Constant([-1]) -- inside the ONNX function body.

ONNX Transform Details

LocalizeFunctionReduceSumAxesTransform it rewrites a function input when:

  • a node inside an ONNX FunctionProto is ReduceSum
  • the ReduceSum axes input is one of the function formal inputs
  • every top-level call site passes a compile-time constant for that formal input
  • the value is a valid integer scalar or 1-D axes tensor

When eligible, the transform:

  • inserts a local Constant node inside the function body
  • rewires each matching ReduceSum to use that local constant
  • removes the axes formal input from the function signature

@vbaddi vbaddi self-assigned this Aug 31, 2026
@vbaddi vbaddi added enhancement New feature or request 1.23 Release 1.23 Features labels Aug 31, 2026
@vbaddi

vbaddi commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

CI-Ready

@vbaddi
vbaddi requested a review from quic-rishinr August 31, 2026 08:48

@quic-rishinr quic-rishinr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Thanks!!

@vbaddi

vbaddi commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

CI-Ready

vbaddi and others added 6 commits September 10, 2026 15:32
  Replace QEff einsum-based reduction workarounds with equivalent torch.sum()
  forms now that ONNX subfunction ReduceSum axes are localized as constants.

  Add a narrow ONNX transform that detects ReduceSum nodes inside FunctionProto
  bodies where the axes input was promoted to a function formal input. When every
  top-level call site passes the same compile-time integer constant for that
  formal input, the transform inserts a local Constant node inside the function,
  rewires ReduceSum to use it, removes the formal function input, and removes the
  matching actual argument from each call site.

  This keeps PyTorch code such as:

      (query * query).sum(dim=-1, keepdim=True)

  lowering to Mul + ReduceSum instead of requiring an einsum workaround, while
  still presenting ReduceSum axes to the compiler as a compile-time constant
  inside the ONNX FunctionProto.

Signed-off-by: vbaddi <vbaddi@qti.qualcomm.com>
Signed-off-by: vbaddi <vbaddi@qti.qualcomm.com>
Signed-off-by: Kushal Dulla <kdulla@qti.qualcomm.com>
(cherry picked from commit 66b459b)
Signed-off-by: ochougul <ochougul@qti.qualcomm.com>
Signed-off-by: ochougul <ochougul@qti.qualcomm.com>
(cherry picked from commit c827393)
Signed-off-by: ochougul <ochougul@qti.qualcomm.com>
Signed-off-by: ochougul <ochougul@qti.qualcomm.com>
Signed-off-by: vbaddi <vbaddi@qti.qualcomm.com>
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
@mohiso22

Copy link
Copy Markdown
Contributor

CI-Ready

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.23 Release 1.23 Features enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants