Add PyTorchAlpaka mini-batching support by EmanueleCoradin · Pull Request #50794 · cms-sw/cmssw

EmanueleCoradin · 2026-04-23T08:24:31Z

This PR results from a collaboration with @valsdav. Thanks also to @Electricks94 for his help.

PR description:

This PR introduces native support for splitting a single SoA into multiple logical batches. TensorCollection now supports (batch_size, total_size) construction and computes per-batch offsets internally.

Validation

Validation is performed via dedicated mini-batch inference producers:

SimpleNetMiniBatch
TinyResNetMiniBatch

The latter reproduces the same inference as the non-batched implementations using the new batching API.
InspectionSink has been extended to compare outputs between the standard and the batched inference.
A relative difference check is enforced: |ref - batched| / ref < 1e-5

It can be run by:

cd src/PhysicsTools/PyTorchAlpakaTest/test/
cmsRun runPyTorchAlpakaTest.py --compareBatch -e 3

Notes / limitations

Batch size must be chosen to satisfy the alignment constraint.

EmanueleCoradin · 2026-04-23T08:24:49Z

type ngt

cmsbuild · 2026-04-23T08:24:52Z

cms-bot internal usage

cmsbuild · 2026-04-23T08:26:50Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50794/49105

There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/PyTorchAlpaka/interface/TensorHandle.h modified in PR(s): Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpakaTest/plugins/InspectionSink.cc modified in PR(s): Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpakaTest/test/runPyTorchAlpakaTest.py modified in PR(s): Add Torch FP16 inference support #50790

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50794/49105/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50794/49105/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2026-04-23T14:27:26Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50794/49114

There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/PyTorchAlpaka/README.md modified in PR(s): Add automatic eval and freeze to PyTorch #50783, Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpaka/interface/TensorHandle.h modified in PR(s): Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpakaTest/plugins/InspectionSink.cc modified in PR(s): Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpakaTest/test/runPyTorchAlpakaTest.py modified in PR(s): Add Torch FP16 inference support #50790

cmsbuild · 2026-04-23T14:27:31Z

Pull request #50794 was updated.

cmsbuild · 2026-04-23T20:20:44Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50794/49118

There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/PyTorchAlpaka/README.md modified in PR(s): Add automatic eval and freeze to PyTorch #50783, Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpaka/interface/TensorHandle.h modified in PR(s): Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpakaTest/plugins/InspectionSink.cc modified in PR(s): Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpakaTest/test/runPyTorchAlpakaTest.py modified in PR(s): Add Torch FP16 inference support #50790

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50794/49118/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50794/49118/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2026-04-23T20:26:08Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50794/49119

There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/PyTorchAlpaka/README.md modified in PR(s): Add automatic eval and freeze to PyTorch #50783, Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpaka/interface/TensorHandle.h modified in PR(s): Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpakaTest/plugins/InspectionSink.cc modified in PR(s): Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpakaTest/test/runPyTorchAlpakaTest.py modified in PR(s): Add Torch FP16 inference support #50790

cmsbuild · 2026-04-28T21:35:08Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50794/49166

There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/PyTorchAlpaka/README.md modified in PR(s): Add automatic eval and freeze to PyTorch #50783, Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpaka/interface/TensorHandle.h modified in PR(s): Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpakaTest/plugins/InspectionSink.cc modified in PR(s): Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpakaTest/plugins/alpaka/DataSource.cc modified in PR(s): Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpakaTest/plugins/alpaka/SimpleNet.cc modified in PR(s): Add Torch FP16 inference support #50790
- File PhysicsTools/PyTorchAlpakaTest/test/runPyTorchAlpakaTest.py modified in PR(s): Add Torch FP16 inference support #50790

cmsbuild · 2026-04-28T21:35:33Z

Pull request #50794 was updated. @cmsbuild, @fwyzard, @hjkwon260, @makortel, @valsdav, @y19y19 can you please check and sign again.

EmanueleCoradin · 2026-04-28T21:44:47Z

@fwyzard , thanks for the review. I addressed the other comments, the only ones left are:

While unrelated to this PR, I would suggest changing all print() functions to emit the final message using the MessageLogger.
While (mostly) unrelated to this PR, I would suggest to use a single letter for the short options (e.g. `'-t' for the number of threads, etc).

Since PR #50790 is also modifying the InspectionSink, I’d prefer to address these points before merging whichever of #50794 or #50790 goes in last. Would that be OK?

fwyzard · 2026-04-28T22:08:05Z

It's fine by me if those two points are addressed in a separate PR.

fwyzard · 2026-04-28T22:08:15Z

please test

cmsbuild · 2026-04-29T00:10:49Z

+1

Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-927f37/52928/summary.html
COMMIT: 43bafa2
CMSSW: CMSSW_17_0_X_2026-04-28-1100/el8_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/50794/52928/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 1 lines to the logs
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 53
DQMHistoTests: Total histograms compared: 4186963
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 4186940
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 52 files compared)
Checked 227 log files, 197 edm output root files, 53 DQM output files
TriggerResults: no differences found

mmusich · 2026-05-07T15:18:32Z

+heterogeneous

@fwyzard do you mind signing again?

fwyzard · 2026-05-07T16:33:59Z

+heterogeneous

valsdav · 2026-05-07T18:42:15Z

+ml

cmsbuild · 2026-05-07T18:42:39Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @ftenchini, @sextonkennedy, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2026-05-08T18:17:04Z

+1

cmsbuild added this to the CMSSW_17_0_X milestone Apr 23, 2026

cmsbuild added pending-signatures tests-pending orp-pending code-checks-pending heterogeneous-pending ml-pending labels Apr 23, 2026

cmsbuild added the ngt label Apr 23, 2026

cmsbuild added code-checks-rejected and removed code-checks-pending labels Apr 23, 2026

cmsbuild mentioned this pull request Apr 23, 2026

Add Torch FP16 inference support #50790

Merged

cmsbuild added code-checks-pending and removed code-checks-rejected labels Apr 23, 2026

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 23, 2026

EmanueleCoradin force-pushed the torch-batch-support branch from 86423c6 to bfe41a0 Compare April 23, 2026 20:19

cmsbuild added code-checks-pending and removed code-checks-approved labels Apr 23, 2026

cmsbuild added code-checks-rejected and removed code-checks-pending labels Apr 23, 2026

EmanueleCoradin force-pushed the torch-batch-support branch from bfe41a0 to 6999803 Compare April 23, 2026 20:23

cmsbuild added code-checks-pending and removed code-checks-rejected labels Apr 23, 2026

Add PyTorchAlpaka mini-batching support

43bafa2

EmanueleCoradin force-pushed the torch-batch-support branch from 2aa7a7f to 43bafa2 Compare April 28, 2026 21:32

cmsbuild added code-checks-pending and removed code-checks-approved labels Apr 28, 2026

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 28, 2026

cmsbuild added tests-started and removed tests-pending labels Apr 28, 2026

cmsbuild added tests-approved and removed tests-started labels Apr 29, 2026

cmsbuild added heterogeneous-approved and removed heterogeneous-pending labels May 7, 2026

cmsbuild added fully-signed ml-approved and removed pending-signatures ml-pending labels May 7, 2026

cmsbuild added orp-approved and removed orp-pending labels May 8, 2026

cmsbuild merged commit 49c66cf into cms-sw:master May 8, 2026
10 checks passed

Uh oh!

Conversation

EmanueleCoradin commented Apr 23, 2026

PR description:

Validation

Notes / limitations

Uh oh!

EmanueleCoradin commented Apr 23, 2026

Uh oh!

cmsbuild commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Apr 23, 2026

Uh oh!

cmsbuild commented Apr 23, 2026

Uh oh!

cmsbuild commented Apr 23, 2026

Uh oh!

cmsbuild commented Apr 23, 2026

Uh oh!

cmsbuild commented Apr 23, 2026

Uh oh!

cmsbuild commented Apr 28, 2026

Uh oh!

cmsbuild commented Apr 28, 2026

Uh oh!

EmanueleCoradin commented Apr 28, 2026

Uh oh!

fwyzard commented Apr 28, 2026

Uh oh!

fwyzard commented Apr 28, 2026

Uh oh!

cmsbuild commented Apr 29, 2026

Comparison Summary

Uh oh!

mmusich commented May 7, 2026

Uh oh!

fwyzard commented May 7, 2026

Uh oh!

valsdav commented May 7, 2026

Uh oh!

cmsbuild commented May 7, 2026

Uh oh!

mandrenguyen commented May 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

cmsbuild commented Apr 23, 2026 •

edited

Loading