Integrate tensorflow 2.21.0. - #10724
Open
akritkbehera wants to merge 5 commits into
Open
Conversation
Contributor
Author
|
please test |
Contributor
|
A new Pull Request was created by @akritkbehera for branch IB/CMSSW_20_1_X/master. @akritkbehera, @iarspider, @raoatifshad, @smuzaffar can you please review it and eventually sign? Thanks. |
Contributor
|
cms-bot internal usage |
Contributor
|
-1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8b927c/54710/summary.html Failed External BuildI found compilation error when building: cwd: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/py3-onnx/1.21.0-c375c743f4e6702e893aaddc8ec3f988/cmsdist-pip-src/onnx-1.21.0
Building wheel for onnx (pyproject.toml): finished with status 'error'
ERROR: Failed building wheel for onnx
Failed to build onnx
ERROR: Failed to build one or more wheels
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.VX1Rsl (%build)
RPM build warnings:
Macro expanded in comment on line 554: %{pkginstroot}/bin/*
Macro expanded in comment on line 559: %{pkginstroot}/${PYTHON3_LIB_SITE_PACKAGES}
|
cmsbuild
added
tests-rejected
tests-pending
and removed
tests-started
tests-rejected
labels
Contributor
|
Pull request #10724 was updated. |
Contributor
Author
|
please test |
Contributor
Author
|
please test |
Contributor
|
Pull request #10724 was updated. |
fwyzard
reviewed
Jul 13, 2026
| @@ -1,13 +1,13 @@ | |||
| ### RPM external acts v44.0.1 | |||
| ### RPM external acts 46.8.1 | |||
Contributor
There was a problem hiding this comment.
Wait, this will break our ACTS/Traccc tests and possible the ongoing developments.
I'm in favour of updating ACTS, but we need to synchronise our work.
Contributor
Author
There was a problem hiding this comment.
We can create a separate IBs for this then and track it.
Contributor
Author
There was a problem hiding this comment.
We can't create separate IBs for this.
Contributor
|
-1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8b927c/54727/summary.html Failed External BuildI found compilation error when building: libcudacxx_DIR:PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/cuda/13.3.1-723beba4702fd72a837a16c836f8224b/lib64/cmake/libcudacxx
nlohmann_json_DIR:PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/json/3.12.0-8ae359b0f8d167e8240a776436a49d72/share/cmake/nlohmann_json
pybind11_DIR:PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/py3-pybind11/3.0.1-9a0007668b2b31843b6b1eaf989187f9/share/cmake/pybind11
rocprim_DIR:PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/rocprim/7.2.4-2fba8176b61ff820348920a3cb5aaf98/lib/cmake/rocprim
rocthrust_DIR:PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/rocthrust/7.2.4-bec48eec9fdf9e0e6f1771602f9b21e5/lib/cmake/rocthrust
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.d2wdaU (%build)
RPM build warnings:
Macro expanded in comment on line 527: %{pkginstroot}/python
|
Contributor
|
please test |
Contributor
|
-1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8b927c/54735/summary.html Failed External BuildI found compilation error when building: libcudacxx_DIR:PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/cuda/13.3.1-723beba4702fd72a837a16c836f8224b/lib64/cmake/libcudacxx
nlohmann_json_DIR:PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/json/3.12.0-8ae359b0f8d167e8240a776436a49d72/share/cmake/nlohmann_json
pybind11_DIR:PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/py3-pybind11/3.0.1-9a0007668b2b31843b6b1eaf989187f9/share/cmake/pybind11
rocprim_DIR:PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/rocprim/7.2.4-2fba8176b61ff820348920a3cb5aaf98/lib/cmake/rocprim
rocthrust_DIR:PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/external/rocthrust/7.2.4-bec48eec9fdf9e0e6f1771602f9b21e5/lib/cmake/rocthrust
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.iEZduE (%build)
RPM build warnings:
Macro expanded in comment on line 539: %{pkginstroot}/python
|
Align the core C++ dependencies with the versions required by TensorFlow 2.21.0: - Refresh the abseil-cpp libraries and add abseil-cpp as an explicit dependency of tensorflow-xla-runtime and protobuf. - Update the protobuf, grpc, eigen, flatbuffers and java-env specs, and drop patches no longer needed with the new versions (grpc-mno-outline-atomics, protobuf-non-virtual-dtor, protobuf_text_format). - Adjust the tensorflow-xla-runtime spec and its abseil patch.
Contributor
|
Pull request #10724 was updated. |
1 similar comment
Contributor
|
Pull request #10724 was updated. |
Contributor
Author
|
please test with cms-sw/cmssw#51507 for el9_aarch64_gcc13 |
Contributor
|
-1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8b927c/54894/summary.html Failed External BuildI found compilation error when building: FATAL: malformed spec found while quering it. Command:
source /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/rpm-env.sh ; rpm -q --specfile /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/tmpspec-bazel --info --define "cmsdist_directory /data/cmsbld/jenkins/workspace/ib-run-pr-tests/cmsdist" --define "compilerv 1340" --define "cmscompilerv 13" --define "cmsos el9_aarch64" --define "__bootstrap ~bootstrap" --define "almalinux_ver 9" --define "almalinux 9" --define "centos_ver 9" --define "centos 9" --define "rhel 9" --define "dist %{!?distprefix0:%{?distprefix}}%{expand:%{lua:for i=0,9999 do print("%{?distprefix" .. i .."}") end}}.el9%{?with_bootstrap:~bootstrap}" --define "el9 1" --define "dist_vendor AlmaLinux OS Foundation" --define "dist_name AlmaLinux" --define "dist_home_url https://almalinux.org/" --define "dist_bug_report_url https://bugs.almalinux.org/" --define "package_vectorization %{nil}" --define "cmsswdata_version_link 1" --define "archfirst yes" --define "cmsBuild_bootstrap 1" --define 'buildroot /foo'
Resulted in:
warning: line 36: It's not recommended to have unversioned Obsoletes: Obsoletes: external+bazel+7.7.0-5f0a6f38d977157fbcf8813a3e764519
error: line 546: Unclosed %if
error: query of specfile /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/tmpspec-bazel failed, can't parse
Traceback (most recent call last):
File "/data/cmsbld/jenkins/workspace/ib-run-pr-tests/./pkgtools/cmsBuild", line 5312, in
build(opts, args[1:], PKGFactory)
File "/data/cmsbld/jenkins/workspace/ib-run-pr-tests/./pkgtools/cmsBuild", line 4479, in build
|
Update packages that failed in CI after the abseil-cpp and TensorFlow bumps: - Update the acts (with an ACTS tag bump), fasthadd and triton-inference-client specs. - Update pip packages onnx, torch and requirements, and the torch-extension-build helper. - Add lwtnn-assert-fix.patch and patches/pip/pyg-lib.patch. Address changes Fix Bazel for ARM
Contributor
Author
|
please test with cms-sw/cmssw#51507 for el9_aarch64_gcc13 |
Contributor
|
Pull request #10724 was updated. |
Contributor
|
-1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8b927c/54896/summary.html Failed External BuildI found compilation error when building: + bazel --batch --output_user_root ../build --host_jvm_args=--add-opens=java.base/java.nio=ALL-UNNAMED --host_jvm_args=--add-opens=java.base/java.lang=ALL-UNNAMED build -s --verbose_failures --copt=-march=armv8-a --copt=-mno-outline-atomics --config=opt --cxxopt=-std=c++20 --host_cxxopt=-std=c++20 --jobs=-j 20 --config=nogcp --config=nonccl --define=with_numa_support=true --config=clang_local '--per_file_copt=external/onednn/.*[.]cpp@-std=c++17' '--per_file_copt=external/jpegxl/.*enc_fast_lossless.*@-O1' --host_action_env=LD_LIBRARY_PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-keras-applications/1.0.8-46572922f77acd7aaa7eac6e31be4377/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-termcolor/3.3.0-ce58411ee7b9d2dd53d75e0d8899c356/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-absl-py/2.5.0-05c999ddb9a849bbe7acfa518e394329/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/grpc/1.82.0-945336eb83feb5e3f2d5d71e106e0c6a/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-keras-preprocessing/1.1.2-3ede73f5837d0036bf9a586f2a717ab7/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-mock/5.2.0-c8f699fa8b72c0a4a034ee616be6d287/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-portpicker/1.6.0-ed282dd153d4182c042b16712dd0e70d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-wrapt/1.14.1-a9d30184c425384c2f378793bacaf171/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-tblib/3.2.2-511a589823e94d4167f1825dea331f35/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-lit/18.1.8-be994f3da9b5cd44deb9a8812565040e/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-gast/0.7.0-bbc4d7d14d898d817cdfee899ad445f2/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-future/1.0.0-12d9b274c23c6207d7404c1f868a5314/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-flatbuffers/25.12.19-5adef237bdb68e32e4a68cb5917f57ad/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-astor/0.8.1-923c0dc4d3b7cb72fb71a86371efe95f/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/flatbuffers/25.9.23-924cdd667c776fd73e00352b00b243ef/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-h5py/3.13.0-d80bbc94d1a601a1dd2ad3e2e75a063d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/llvm/21.1.4-37d7e3d9353a61edb860f8dbfad90370/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/llvm/21.1.4-37d7e3d9353a61edb860f8dbfad90370/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-psutil/7.2.2-a447d9a5bc90c90c347b682e03c9fa3a/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-funcsigs/1.0.2-55d0f9726efce9ba1f95410db2d17465/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/c-ares/1.15.0-0059b1cc7e0798fa1c703d9a2398138c/lib:/usr/lib/jvm/java-21/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pkgconfig/1.6.0-1a452b1f4c138e07ceddf790b6a92eb5/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-protobuf/6.31.1-e7f44abf3563a34f293a77228434ac3b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/re2/2021-06-01-38cbd03a394ae1ed1c84cfadefad3d5d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-poetry/2.4.1-fb88b362dccf19ca8eb2841ccfadbd80/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/protobuf/6.31.1-469e24c819cc571ffc01231df9d32ffa/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pyparsing/3.3.2-ef0d5f04085b9bf32acfce938d575c66/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pbr/7.0.3-11aa53e334bbb6eb43339e4d694263d9/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libunwind/1.8.1-master-013128a89f9e7cca86b791b279fda4b0/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-cachecontrol/0.14.4-a59a98d19449661fbb8edc1a02fc8edc/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-cleo/2.1.0-cec6976f4ef302044eb5ccd9cf2bddb0/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-virtualenv/21.6.0-756561e77f5b494ef3fe250adab08a8d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-requests-toolbelt/1.0.0-bc744615927425cd14e36f2369468492/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-keyring/25.7.0-9d7a1604c52a8b24197e9e57a4ccb545/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-jsonschema/4.26.0-a025210fb34ec265f293559a1a37f01a/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-dulwich/1.2.10-7c720e73f49587dbe4003b87891621db/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pybind11/3.0.4-9f50732a0ffa4f98fb57015a2b00b8ca/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-findpython/0.8.0-807cc65bcbcb34971c5c699cba1c4e87/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pbs-installer/2026.6.10-506456f3fff2e93e1b82a1a99e42af90/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-poetry-plugin-export/1.10.0-88afc452ce5651486823c15dfe27f714/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pexpect/4.9.0-3f4db019c5ca9b2070575c9f70536c6e/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-shellingham/1.5.4-be82153ad9c665b8e7aa768966851893/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pkginfo/1.12.1.2-f348041ef5f2407c0700773014e1d20b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-fastjsonschema/2.21.2-d98ef09235efc921f305e1c81f829bd5/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-cachy/0.3.0-9cef4c8b7cd4decca5652b74856707bd/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/abseil-cpp/20250814.1-1b5d8455f00af5d4a81ea02cdde96054/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-rapidfuzz/3.14.5-8b578e8aec03f2dac786b6ca3ecfaaea/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-jsonschema-specifications/2025.9.1-d02baf5aa858d5cbf46a3f1ed550373b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-secretstorage/3.5.0-d6cc2aeeb1abd47bf5010b156f3b402d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-python-discovery/1.4.4-224875656cc9ad34eb21e1930a486a8e/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-jaraco-context/6.1.2-5b8957ab26b9b518653fff8814ab4c6b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-jaraco-functools/4.5.0-26cd931c21bb4eaca537fb154a26309b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/hdf5/1.14.6-d31efede7b38841ef8b9f8a848f90a37/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-importlib-resources/7.1.0-87c86909c77d8a0052834fa7348503d1/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-clikit/0.6.2-65dddd4fd75178692b7f9dc16f22f99b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-tomlkit/0.15.0-d8344010107c5e2bfd8607bd2528b50c/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-ptyprocess/0.7.0-08a807acbdeaf8a0210e4016950d9804/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-msgpack/1.2.1-68a5de3e0e8c9768069ca8794d85ceb9/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-jaraco-classes/3.4.0-d83ca159d2c773c0d731e176250dc973/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-distlib/0.4.3-e0026ae536b36d15d4015f9e440a4d88/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-backports-entry-points-selectable/1.3.0-31feb9ecebccc1a8463dd4a992b28349/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/eigen/5.0.1-5fc0214f9d987734a3d0affcb38f7444/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/pcre/8.43-7f0edb13c32a66555b25b5433dd2ceed/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-requests/2.34.2-025c1adff47bf7bb9d48f88718fdfdd9/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-scikit-build/0.19.1-2670f713659d6d9c9e63428a64840dad/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-referencing/0.37.0-c480bb95cc46f915d9f731eca2dae9ee/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-cryptography/49.0.0-25724bd905d43a3841d4657209ea42d0/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-backports-tarfile/1.2.0-179121f51cbd58ba375498d5e1ee6453/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pdm-backend/2.4.9-3bd8ae433256d64da6a788e88c2a6eaa/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-singledispatch/4.1.2-f9cbf839124b741b78ab92a34fc32fe5/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pastel/0.2.1-138dda9efc60d33504bff40848573209/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-jeepney/0.9.0-5b392ee60b07ffb11f57303ad05804de/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-crashtest/0.4.1-70d0e1d7f2b824ccdfe6a8491f529112/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pylev/1.4.0-7eba4bedb91f262b751664788cab27e9/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-chardet/7.4.3-b33984c4aa2779177349390d2c8b85d8/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-rpds-py/2026.6.3-f2de2b88f77ce3e2875cf4148f4fbb3d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-attrs/26.1.0-7f33d9110f4bbe1dae6b23edf093b040/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-wheel/0.47.0-61ad5ea596cd6e7c436a9e4c1885fd0e/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-cffi/2.1.0-37e5970c640f05a8b20b49a73972c4f5/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pysocks/1.7.1-00ca2016ed21d4af0b9feca69b86e6ec/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-distro/1.9.0-ac99506fd88c9183c0b0b17df4061a2d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-charset-normalizer/3.4.9-f0e42712ba08249068978ed2cfcbb2f5/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-scikit-build-core/1.0.1-a2ad1408729418652f335e06cb85b933/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-urllib3/2.7.0-d78f89ef235f52f601fe47d043ad088c/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pycparser/3.0-e9b3062937bdaf16adf6616c4b0ab91f/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-appdirs/1.4.4-57220776ff0f261705c951d8674b9763/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-filelock/3.29.7-05c5dfa441e707e01c096ed7fe580e25/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-platformdirs/4.10.0-f2e8b2543d6d54fd26870a999afae833/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-poetry-core/2.4.1-bf09a4be4e28d8a7800f857ed9dfcc5a/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-idna/3.18-db476473886141170f86642f22f4fa06/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-certifi/2026.6.17-dd79a47d7752e1c66137bef4c8f5c8a7/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/giflib/5.2.1-a527ad3fd94ee0fd141f1aa1140ddd0a/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-setuptools-rust/1.13.0-b337d6ca45c4dc7659c800b3f22ef057/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/pcre2/10.36-dd63a742c2ac6aacbf973ba2fec8598c/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-hatch-fancy-pypi-readme/25.1.0-0f6e3ed10c8596b564c13b5d13d1f048/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-toml/0.10.2-3e227de483fce18774a63921a5020c12/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-semantic-version/2.10.0-5ee420943b96687ebf47f0c0ba04b189/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libjpeg-turbo/3.0.4-ec71687bc2f615d6247f8963dc6fbbcf/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/rust/1.94.0-a476c8da7dc7533c0dd99cfcfbefb520/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-exceptiongroup/1.3.1-13da487028bb7455a896c6d52f358d49/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/openmpi/5.0.10-f8592f367b787e1ef2a11b2d79809a06/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libpng/1.6.44-023e6fa0640b53f2ede9af9e3b636961/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libfabric/2.1.0-b856e90ad2789c9f3ca0b428116c32dd/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/ucx/1.21.0-58b64ba4219817d492ee1e7978d8eede/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/rdma-core/57.0-b845b90cbb5d46e1466ee4c4023d2acb/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/gdrcopy/2.6-57edefc538efc276cb205e11e89995b4/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/xpmem/v2.6.3-20220308-26e853843d2d89608ffd6fe77c75b648/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-numpy/1.26.4-63bc1a4c80b32cb4cae7108bf4004f2b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-hatch-vcs/0.5.0-0dcb4bac79335ffa1258711f3e41dda1/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/hwloc/2.12.2-a4de4cbfe6aa30eaa46a9523cac1d4e4/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/OpenBLAS/0.3.27-47a3c8350874a1ad9f41674752b30581/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-cython/3.2.8-6a715fcf6f16ccc224d92c6fd40560b8/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libpciaccess/0.16-0c1f273cbc749e81d2c6cd3781ac7ea1/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/zstd/1.5.7-3d33d105dce0061a2affabfbb156d655/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/numactl/2.0.19-66dfe30a339dca923efdd4243f4033c5/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-hatchling/1.31.0-4030ec549d1c82c2b0cef5f18fa96af0/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pluggy/1.6.0-0fd75bfa4c03f78ee372363f6fd02b15/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pathspec/1.1.1-7a8ee2568b5bdae408b5e721682083ec/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-editables/0.6-8c8b187f379733101bcfe8acf78a3362/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-trove-classifiers/2026.6.1.19-71c879924699fdbe88616334b44e6182/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-calver/2025.10.20-b6bb5414d02c3e057e0c0f068566f2cb/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libxml2/2.9.10-e186e339339cb9854d2d7babb5add21f/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/autotools/1.5-14425bfcc7b9e51ea6c54a1d0ad392a6/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-setuptools-scm/10.2.0-18cca40f517e8ff7976812e43a765580/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-vcs-versioning/2.2.2-cd8b12bcbf8a5fdc2460579346ac29ec/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-typing-extensions/4.16.0-6faee870a5ccab5616ed7dcddae256f9/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-tomli/2.4.1-abc94ceb43778947e8a8830a6a99975a/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-importlib-metadata/9.0.0-1bb6a2677ac58a148c40f5cc47d62531/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-zipp/4.1.0-e3c8e5135ec7b8cf6d7d7d079c9c22ed/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pathlib2/2.3.7.post1-1e83eab522d166fa2a27f5800d8cee12/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-more-itertools/11.1.0-75070bd54163ff0866f569652b6b80f9/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-packaging/26.2-86cf170aed27d06c76ed45043d25b827/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-scandir/1.10.0-3727af7576d330852ac1b0b25771bca7/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-contextlib2/21.6.0-75a8d40aec06f809337eb7d37ca4b111/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-six/1.17.0-c60623093adc241e93b7acdc987cb482/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-flit-core/3.12.0-f76fdd63ae0837ca65184fffa8c808b6/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/curl/8.13.0-04f63768868ff074c0f3aac68b6dda8d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pip/26.1.2-a8f68b7ccb754f1f41f5635902962f19/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pip/26.1.2-a8f68b7ccb754f1f41f5635902962f19/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-setuptools/83.0.0-54b74ad177edab04c2d329a45044cb35/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/python3/3.12.13-94ad91f6663a74a1d7116c37a04d3863/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/python3/3.12.13-94ad91f6663a74a1d7116c37a04d3863/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/sqlite/3.48.0-507c76bdccf2917e64f1c155b7196897/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libffi/3.4.6-b461d13020c9e14e7e60de314b0911cc/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/gdbm/1.24-8a40192222c4b0afe814f539b1117fa8/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/db6/6.2.32-a42f52927cfa2d15e53d2a4cd62c6196/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libuuid/2.40-d94705b84648fd294d63bf9de23bc03b/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/xz/5.8.3-3303bcf2cc1251389e9225885f971ea3/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/expat/2.7.1-fd861297a37b8f9a1d9e1c6e2937faaa/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/bz2lib/1.0.8-353e2ef1c2f1d06b28a8938defa197bc/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/zlib/1.3.2-53c061acc706fe8f28fd9477d9981c20/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/gcc/13.4.0-6908cfdf803923e783448096ca4f0923/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/gcc/13.4.0-6908cfdf803923e783448096ca4f0923/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/rpm/4.18.0-43e4fb3f0bab5d700caac5a5f643d644/lib:/.singularity.d/libs --action_env=LD_LIBRARY_PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-keras-applications/1.0.8-46572922f77acd7aaa7eac6e31be4377/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-termcolor/3.3.0-ce58411ee7b9d2dd53d75e0d8899c356/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-absl-py/2.5.0-05c999ddb9a849bbe7acfa518e394329/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/grpc/1.82.0-945336eb83feb5e3f2d5d71e106e0c6a/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-keras-preprocessing/1.1.2-3ede73f5837d0036bf9a586f2a717ab7/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-mock/5.2.0-c8f699fa8b72c0a4a034ee616be6d287/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-portpicker/1.6.0-ed282dd153d4182c042b16712dd0e70d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-wrapt/1.14.1-a9d30184c425384c2f378793bacaf171/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-tblib/3.2.2-511a589823e94d4167f1825dea331f35/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-lit/18.1.8-be994f3da9b5cd44deb9a8812565040e/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-gast/0.7.0-bbc4d7d14d898d817cdfee899ad445f2/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-future/1.0.0-12d9b274c23c6207d7404c1f868a5314/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-flatbuffers/25.12.19-5adef237bdb68e32e4a68cb5917f57ad/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-astor/0.8.1-923c0dc4d3b7cb72fb71a86371efe95f/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/flatbuffers/25.9.23-924cdd667c776fd73e00352b00b243ef/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-h5py/3.13.0-d80bbc94d1a601a1dd2ad3e2e75a063d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/llvm/21.1.4-37d7e3d9353a61edb860f8dbfad90370/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/llvm/21.1.4-37d7e3d9353a61edb860f8dbfad90370/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-psutil/7.2.2-a447d9a5bc90c90c347b682e03c9fa3a/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-funcsigs/1.0.2-55d0f9726efce9ba1f95410db2d17465/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/c-ares/1.15.0-0059b1cc7e0798fa1c703d9a2398138c/lib:/usr/lib/jvm/java-21/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pkgconfig/1.6.0-1a452b1f4c138e07ceddf790b6a92eb5/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-protobuf/6.31.1-e7f44abf3563a34f293a77228434ac3b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/re2/2021-06-01-38cbd03a394ae1ed1c84cfadefad3d5d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-poetry/2.4.1-fb88b362dccf19ca8eb2841ccfadbd80/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/protobuf/6.31.1-469e24c819cc571ffc01231df9d32ffa/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pyparsing/3.3.2-ef0d5f04085b9bf32acfce938d575c66/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pbr/7.0.3-11aa53e334bbb6eb43339e4d694263d9/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libunwind/1.8.1-master-013128a89f9e7cca86b791b279fda4b0/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-cachecontrol/0.14.4-a59a98d19449661fbb8edc1a02fc8edc/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-cleo/2.1.0-cec6976f4ef302044eb5ccd9cf2bddb0/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-virtualenv/21.6.0-756561e77f5b494ef3fe250adab08a8d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-requests-toolbelt/1.0.0-bc744615927425cd14e36f2369468492/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-keyring/25.7.0-9d7a1604c52a8b24197e9e57a4ccb545/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-jsonschema/4.26.0-a025210fb34ec265f293559a1a37f01a/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-dulwich/1.2.10-7c720e73f49587dbe4003b87891621db/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pybind11/3.0.4-9f50732a0ffa4f98fb57015a2b00b8ca/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-findpython/0.8.0-807cc65bcbcb34971c5c699cba1c4e87/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pbs-installer/2026.6.10-506456f3fff2e93e1b82a1a99e42af90/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-poetry-plugin-export/1.10.0-88afc452ce5651486823c15dfe27f714/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pexpect/4.9.0-3f4db019c5ca9b2070575c9f70536c6e/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-shellingham/1.5.4-be82153ad9c665b8e7aa768966851893/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pkginfo/1.12.1.2-f348041ef5f2407c0700773014e1d20b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-fastjsonschema/2.21.2-d98ef09235efc921f305e1c81f829bd5/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-cachy/0.3.0-9cef4c8b7cd4decca5652b74856707bd/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/abseil-cpp/20250814.1-1b5d8455f00af5d4a81ea02cdde96054/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-rapidfuzz/3.14.5-8b578e8aec03f2dac786b6ca3ecfaaea/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-jsonschema-specifications/2025.9.1-d02baf5aa858d5cbf46a3f1ed550373b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-secretstorage/3.5.0-d6cc2aeeb1abd47bf5010b156f3b402d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-python-discovery/1.4.4-224875656cc9ad34eb21e1930a486a8e/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-jaraco-context/6.1.2-5b8957ab26b9b518653fff8814ab4c6b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-jaraco-functools/4.5.0-26cd931c21bb4eaca537fb154a26309b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/hdf5/1.14.6-d31efede7b38841ef8b9f8a848f90a37/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-importlib-resources/7.1.0-87c86909c77d8a0052834fa7348503d1/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-clikit/0.6.2-65dddd4fd75178692b7f9dc16f22f99b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-tomlkit/0.15.0-d8344010107c5e2bfd8607bd2528b50c/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-ptyprocess/0.7.0-08a807acbdeaf8a0210e4016950d9804/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-msgpack/1.2.1-68a5de3e0e8c9768069ca8794d85ceb9/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-jaraco-classes/3.4.0-d83ca159d2c773c0d731e176250dc973/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-distlib/0.4.3-e0026ae536b36d15d4015f9e440a4d88/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-backports-entry-points-selectable/1.3.0-31feb9ecebccc1a8463dd4a992b28349/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/eigen/5.0.1-5fc0214f9d987734a3d0affcb38f7444/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/pcre/8.43-7f0edb13c32a66555b25b5433dd2ceed/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-requests/2.34.2-025c1adff47bf7bb9d48f88718fdfdd9/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-scikit-build/0.19.1-2670f713659d6d9c9e63428a64840dad/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-referencing/0.37.0-c480bb95cc46f915d9f731eca2dae9ee/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-cryptography/49.0.0-25724bd905d43a3841d4657209ea42d0/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-backports-tarfile/1.2.0-179121f51cbd58ba375498d5e1ee6453/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pdm-backend/2.4.9-3bd8ae433256d64da6a788e88c2a6eaa/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-singledispatch/4.1.2-f9cbf839124b741b78ab92a34fc32fe5/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pastel/0.2.1-138dda9efc60d33504bff40848573209/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-jeepney/0.9.0-5b392ee60b07ffb11f57303ad05804de/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-crashtest/0.4.1-70d0e1d7f2b824ccdfe6a8491f529112/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pylev/1.4.0-7eba4bedb91f262b751664788cab27e9/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-chardet/7.4.3-b33984c4aa2779177349390d2c8b85d8/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-rpds-py/2026.6.3-f2de2b88f77ce3e2875cf4148f4fbb3d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-attrs/26.1.0-7f33d9110f4bbe1dae6b23edf093b040/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-wheel/0.47.0-61ad5ea596cd6e7c436a9e4c1885fd0e/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-cffi/2.1.0-37e5970c640f05a8b20b49a73972c4f5/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pysocks/1.7.1-00ca2016ed21d4af0b9feca69b86e6ec/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-distro/1.9.0-ac99506fd88c9183c0b0b17df4061a2d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-charset-normalizer/3.4.9-f0e42712ba08249068978ed2cfcbb2f5/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-scikit-build-core/1.0.1-a2ad1408729418652f335e06cb85b933/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-urllib3/2.7.0-d78f89ef235f52f601fe47d043ad088c/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pycparser/3.0-e9b3062937bdaf16adf6616c4b0ab91f/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-appdirs/1.4.4-57220776ff0f261705c951d8674b9763/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-filelock/3.29.7-05c5dfa441e707e01c096ed7fe580e25/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-platformdirs/4.10.0-f2e8b2543d6d54fd26870a999afae833/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-poetry-core/2.4.1-bf09a4be4e28d8a7800f857ed9dfcc5a/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-idna/3.18-db476473886141170f86642f22f4fa06/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-certifi/2026.6.17-dd79a47d7752e1c66137bef4c8f5c8a7/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/giflib/5.2.1-a527ad3fd94ee0fd141f1aa1140ddd0a/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-setuptools-rust/1.13.0-b337d6ca45c4dc7659c800b3f22ef057/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/pcre2/10.36-dd63a742c2ac6aacbf973ba2fec8598c/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-hatch-fancy-pypi-readme/25.1.0-0f6e3ed10c8596b564c13b5d13d1f048/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-toml/0.10.2-3e227de483fce18774a63921a5020c12/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-semantic-version/2.10.0-5ee420943b96687ebf47f0c0ba04b189/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libjpeg-turbo/3.0.4-ec71687bc2f615d6247f8963dc6fbbcf/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/rust/1.94.0-a476c8da7dc7533c0dd99cfcfbefb520/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-exceptiongroup/1.3.1-13da487028bb7455a896c6d52f358d49/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/openmpi/5.0.10-f8592f367b787e1ef2a11b2d79809a06/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libpng/1.6.44-023e6fa0640b53f2ede9af9e3b636961/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libfabric/2.1.0-b856e90ad2789c9f3ca0b428116c32dd/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/ucx/1.21.0-58b64ba4219817d492ee1e7978d8eede/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/rdma-core/57.0-b845b90cbb5d46e1466ee4c4023d2acb/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/gdrcopy/2.6-57edefc538efc276cb205e11e89995b4/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/xpmem/v2.6.3-20220308-26e853843d2d89608ffd6fe77c75b648/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-numpy/1.26.4-63bc1a4c80b32cb4cae7108bf4004f2b/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-hatch-vcs/0.5.0-0dcb4bac79335ffa1258711f3e41dda1/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/hwloc/2.12.2-a4de4cbfe6aa30eaa46a9523cac1d4e4/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/OpenBLAS/0.3.27-47a3c8350874a1ad9f41674752b30581/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-cython/3.2.8-6a715fcf6f16ccc224d92c6fd40560b8/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libpciaccess/0.16-0c1f273cbc749e81d2c6cd3781ac7ea1/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/zstd/1.5.7-3d33d105dce0061a2affabfbb156d655/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/numactl/2.0.19-66dfe30a339dca923efdd4243f4033c5/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-hatchling/1.31.0-4030ec549d1c82c2b0cef5f18fa96af0/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pluggy/1.6.0-0fd75bfa4c03f78ee372363f6fd02b15/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pathspec/1.1.1-7a8ee2568b5bdae408b5e721682083ec/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-editables/0.6-8c8b187f379733101bcfe8acf78a3362/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-trove-classifiers/2026.6.1.19-71c879924699fdbe88616334b44e6182/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-calver/2025.10.20-b6bb5414d02c3e057e0c0f068566f2cb/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libxml2/2.9.10-e186e339339cb9854d2d7babb5add21f/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/autotools/1.5-14425bfcc7b9e51ea6c54a1d0ad392a6/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-setuptools-scm/10.2.0-18cca40f517e8ff7976812e43a765580/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-vcs-versioning/2.2.2-cd8b12bcbf8a5fdc2460579346ac29ec/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-typing-extensions/4.16.0-6faee870a5ccab5616ed7dcddae256f9/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-tomli/2.4.1-abc94ceb43778947e8a8830a6a99975a/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-importlib-metadata/9.0.0-1bb6a2677ac58a148c40f5cc47d62531/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-zipp/4.1.0-e3c8e5135ec7b8cf6d7d7d079c9c22ed/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pathlib2/2.3.7.post1-1e83eab522d166fa2a27f5800d8cee12/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-more-itertools/11.1.0-75070bd54163ff0866f569652b6b80f9/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-packaging/26.2-86cf170aed27d06c76ed45043d25b827/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-scandir/1.10.0-3727af7576d330852ac1b0b25771bca7/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-contextlib2/21.6.0-75a8d40aec06f809337eb7d37ca4b111/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-six/1.17.0-c60623093adc241e93b7acdc987cb482/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-flit-core/3.12.0-f76fdd63ae0837ca65184fffa8c808b6/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/curl/8.13.0-04f63768868ff074c0f3aac68b6dda8d/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pip/26.1.2-a8f68b7ccb754f1f41f5635902962f19/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-pip/26.1.2-a8f68b7ccb754f1f41f5635902962f19/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/py3-setuptools/83.0.0-54b74ad177edab04c2d329a45044cb35/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/python3/3.12.13-94ad91f6663a74a1d7116c37a04d3863/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/python3/3.12.13-94ad91f6663a74a1d7116c37a04d3863/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/sqlite/3.48.0-507c76bdccf2917e64f1c155b7196897/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libffi/3.4.6-b461d13020c9e14e7e60de314b0911cc/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/gdbm/1.24-8a40192222c4b0afe814f539b1117fa8/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/db6/6.2.32-a42f52927cfa2d15e53d2a4cd62c6196/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/libuuid/2.40-d94705b84648fd294d63bf9de23bc03b/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/xz/5.8.3-3303bcf2cc1251389e9225885f971ea3/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/expat/2.7.1-fd861297a37b8f9a1d9e1c6e2937faaa/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/bz2lib/1.0.8-353e2ef1c2f1d06b28a8938defa197bc/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/zlib/1.3.2-53c061acc706fe8f28fd9477d9981c20/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/gcc/13.4.0-6908cfdf803923e783448096ca4f0923/lib64:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/gcc/13.4.0-6908cfdf803923e783448096ca4f0923/lib:/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/rpm/4.18.0-43e4fb3f0bab5d700caac5a5f643d644/lib:/.singularity.d/libs --define=zendnn=true --define=build_with_mkl_dnn_only=false --define=build_with_mkl=false --define=enable_mkl=false --define=tensorflow_mkldnn_contraction_kernel=0 --define=build_with_mkl_opensource=false //third_party/py/numpy
$TEST_TMPDIR defined: output root default is '/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/tensorflow-sources/2.21.0-3005ac050ba8566774930969b2d5f9cb/build' and max_idle_secs default is '15'.
Extracting Bazel installation...
OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
ERROR: While parsing option --jobs=-j: Parameter '-j' does not follow correct syntax. This flag takes an integer, or a keyword ("auto", "HOST_CPUS", "HOST_RAM"), optionally followed by an operation ([-|*]) eg. "auto", "HOST_CPUS*.5".
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.rDu0nn (%build)
RPM build errors:
Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.rDu0nn (%build)
Building py3-torch-cuda(build). Log can be found in /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/py3-torch-cuda/2.13.0-b3431d24c733594755770abd2aeb1ed4/log.
|
Contributor
|
Pull request #10724 was updated. |
Contributor
Author
|
please test with cms-sw/cmssw#51507 for el9_aarch64_gcc13 |
Contributor
|
-1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8b927c/54913/summary.html Failed External BuildI found compilation error when building: + perl -p -i -e 's|DQMServices/Core/interface/||' ROOTFilePB.pb.cc fastHadd.cc ++ root-config --cflags --libs + g++ -O2 -o /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/BUILDROOT/162021b900d432bb72f8cde54cf0fb4f/opt/cmssw/el9_aarch64_gcc13/external/fasthadd/2.4-162021b900d432bb72f8cde54cf0fb4f/bin/fastHadd ROOTFilePB.pb.cc ./fastHadd.cc -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/protobuf/6.31.1-469e24c819cc571ffc01231df9d32ffa/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/abseil-cpp/20250814.1-1b5d8455f00af5d4a81ea02cdde96054/include -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/protobuf/6.31.1-469e24c819cc571ffc01231df9d32ffa/lib -lprotobuf -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/abseil-cpp/20250814.1-1b5d8455f00af5d4a81ea02cdde96054/lib -pthread -std=c++20 -fsigned-char -I/cvmfs/cms-ib.cern.ch/sw/aarch64/nweek-02951/el9_aarch64_gcc13/lcg/root/6.36.13-0f6910381655efa33dd9a915c6f173c9/include -L/cvmfs/cms-ib.cern.ch/sw/aarch64/nweek-02951/el9_aarch64_gcc13/lcg/root/6.36.13-0f6910381655efa33dd9a915c6f173c9/lib -lCore -lImt -lRIO -lNet -lHist -lGraf -lGraf3d -lGpad -lROOTVecOps -lTree -lTreePlayer -lRint -lPostscript -lMatrix -lPhysics -lMathCore -lThread -lROOTNTuple -lMultiProc -lROOTDataFrame -lROOTNTupleUtil -Wl,-rpath,/cvmfs/cms-ib.cern.ch/sw/aarch64/nweek-02951/el9_aarch64_gcc13/lcg/root/6.36.13-0f6910381655efa33dd9a915c6f173c9/lib -pthread -lm -ldl -rdynamic /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/gcc/13.4.0-6908cfdf803923e783448096ca4f0923/bin/../lib/gcc/aarch64-redhat-linux-gnu/13.4.0/../../../../aarch64-redhat-linux-gnu/bin/ld: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/fasthadd/2.4-162021b900d432bb72f8cde54cf0fb4f/cmsdist-tmp/ccOvEWzJ.o: undefined reference to symbol '_ZN4absl12lts_2025081412log_internal17MakeCheckOpStringIPKvS4_EEPKcT_T0_S6_' /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/gcc/13.4.0-6908cfdf803923e783448096ca4f0923/bin/../lib/gcc/aarch64-redhat-linux-gnu/13.4.0/../../../../aarch64-redhat-linux-gnu/bin/ld: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/abseil-cpp/20250814.1-1b5d8455f00af5d4a81ea02cdde96054/lib/libabsl_log_internal_check_op.so.2508.0.0: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.pOju9X (%build) RPM build errors: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.pOju9X (%build) |
Contributor
Author
|
please test with cms-sw/cmssw#51507 for el9_aarch64_gcc13 |
Contributor
|
Pull request #10724 was updated. |
Contributor
Author
|
please test with cms-sw/cmssw#51507 |
Contributor
|
-1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8b927c/54941/summary.html Failed External BuildI found compilation error when building: cwd: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/py3-torch-cuda/2.13.0-cddc61ad8079f8892486e0c7494ceff8/cmsdist-pip-src/torch-2.13.0
Building wheel for torch (pyproject.toml): finished with status 'error'
ERROR: Failed building wheel for torch
Failed to build torch
ERROR: Failed to build one or more wheels
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.zpwFXW (%build)
RPM build warnings:
Macro expanded in comment on line 674: %{pkginstroot}/bin/*
Macro expanded in comment on line 679: %{pkginstroot}/${PYTHON3_LIB_SITE_PACKAGES}
|
Contributor
|
-1 Failed Tests: Build The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
You can see more details here: CMS deprecated warnings: 18 CMS deprecated warnings found, see summary page for details. Failed BuildI found compilation error when building: Copying tmp/el9_amd64_gcc13/src/HeterogeneousTest/AlpakaKernel/test/testAlpakaDeviceAdditionKernelROCmAsync/libtestAlpakaDeviceAdditionKernelROCmAsync_rocm.a to productstore area: Copying tmp/el9_amd64_gcc13/src/HeterogeneousTest/AlpakaWrapper/src/alpaka/HeterogeneousTestAlpakaWrapperCudaAsync/libHeterogeneousTestAlpakaWrapperCudaAsync_nv.a to productstore area: cp: cannot stat 'tmp/el9_amd64_gcc13/src/HeterogeneousTest/AlpakaWrapper/src/alpaka/HeterogeneousTestAlpakaWrapperCudaAsync/libHeterogeneousTestAlpakaWrapperCudaAsync_nv.a': No such file or directory Copying tmp/el9_amd64_gcc13/src/HeterogeneousTest/AlpakaWrapper/src/alpaka/HeterogeneousTestAlpakaWrapperROCmAsync/libHeterogeneousTestAlpakaWrapperROCmAsync_rocm.a to productstore area: >> Deleted: tmp/el9_amd64_gcc13/src/HeterogeneousTest/AlpakaWrapper/src/alpaka/HeterogeneousTestAlpakaWrapperCudaAsync/libHeterogeneousTestAlpakaWrapperCudaAsync_nv.a gmake: *** [config/SCRAM/GMake/Makefile.rules:1920: tmp/el9_amd64_gcc13/src/HeterogeneousTest/AlpakaWrapper/src/alpaka/HeterogeneousTestAlpakaWrapperCudaAsync/libHeterogeneousTestAlpakaWrapperCudaAsync_nv.a] Error 1 Copying tmp/el9_amd64_gcc13/src/HeterogeneousTest/CUDADevice/src/HeterogeneousTestCUDADevice/libHeterogeneousTestCUDADevice_nv.a to productstore area: cp: cannot stat 'tmp/el9_amd64_gcc13/src/HeterogeneousTest/CUDADevice/src/HeterogeneousTestCUDADevice/libHeterogeneousTestCUDADevice_nv.a': No such file or directory >> Deleted: tmp/el9_amd64_gcc13/src/HeterogeneousTest/CUDADevice/src/HeterogeneousTestCUDADevice/libHeterogeneousTestCUDADevice_nv.a gmake: *** [config/SCRAM/GMake/Makefile.rules:1920: tmp/el9_amd64_gcc13/src/HeterogeneousTest/CUDADevice/src/HeterogeneousTestCUDADevice/libHeterogeneousTestCUDADevice_nv.a] Error 1 Copying tmp/el9_amd64_gcc13/src/HeterogeneousTest/CUDAKernel/src/HeterogeneousTestCUDAKernel/libHeterogeneousTestCUDAKernel_nv.a to productstore area: |
Contributor
Author
|
please test with cms-sw/cmssw#51507 |
Contributor
Author
|
please test with cms-sw/cmssw#51507 for el9_aarch64_gcc13 |
Contributor
|
-1 Failed Tests: RelVals-AMD_MI300X amd_w7900UnitTests Failed RelVals-AMD_MI300X
Comparison SummarySummary:
NVIDIA_H100 Comparison SummarySummary:
NVIDIA_L40S Comparison SummarySummary:
NVIDIA_T4 Comparison SummarySummary:
Max Memory Comparisons exceeding threshold@cms-sw/core-l2 , I found 21 workflow step(s) with memory usage exceeding the error threshold: Expand to see workflows ...
Max Memory Comparisons exceeding threshold NVIDIA_H100@cms-sw/core-l2 , I found 12 workflow step(s) with memory usage exceeding the error threshold: Expand to see workflows ...
Max Memory Comparisons exceeding threshold NVIDIA_L40S@cms-sw/core-l2 , I found 12 workflow step(s) with memory usage exceeding the error threshold: Expand to see workflows ...
Max Memory Comparisons exceeding threshold NVIDIA_T4@cms-sw/core-l2 , I found 12 workflow step(s) with memory usage exceeding the error threshold: Expand to see workflows ...
|
Contributor
Contributor
Author
|
The unit tests failure are also present in IBs are not cause by the changes. This looks good. |
Contributor
Author
|
The rest of tests looks fine. |
Contributor
Author
|
assign ml |
Contributor
|
New categories assigned: ml @hjkwon260,@valsdav,@y19y19 you have been requested to review this Pull request/Issue and eventually sign? Thanks |
Contributor
Author
|
hold |
Contributor
|
Pull request has been put on hold by @akritkbehera |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.