Add CMake options to build static hwloc from local sources or a local git mirror by bratpiorka · Pull Request #1610 · oneapi-src/unified-memory-framework

bratpiorka · 2026-09-30T18:47:17Z

UMF currently always clones hwloc from GitHub. This PR adds two ways to avoid that, which is useful for offline builds:

UMF_HWLOC_SOURCE_DIR - path to a local hwloc working tree., which is copied into the build directory (the original tree is unmodified)
UMF_HWLOC_REPO - a URL or a local mirror path (default is https://github.com/open-mpi/hwloc.git)

If UMF_HWLOC_SOURCE_DIR is set, it takes precedence over UMF_HWLOC_REPO and UMF_HWLOC_TAG.

Examples:

# local working tree
cmake -B build -DUMF_LINK_HWLOC_STATICALLY=ON -DUMF_HWLOC_SOURCE_DIR=/path/to/hwloc

# local git mirror (e.g. created with: git clone --mirror https://github.com/open-mpi/hwloc.git)
cmake -B build -DUMF_LINK_HWLOC_STATICALLY=ON -DUMF_HWLOC_REPO=/path/to/hwloc.git


+    - name: Clone hwloc sources
+      if: matrix.local_hwloc == 'ON'
+      run: git clone --depth 1 -b hwloc-2.3.0 https://github.com/open-mpi/hwloc.git ${{env.HWLOC_SRC_DIR}}


Copilot

Copilot review overview

🟡 Changes recommended

The Windows companion Debug build does not receive the local hwloc path and can unexpectedly fetch a different source.

Review effort: Balanced
Findings: 1

Open (1)

Forward UMF_HWLOC_SOURCE_DIR to nested build_umfd configure · New

What changed in this PR

Adds support for building statically linked hwloc from a local source directory.

Changes:

Adds and documents UMF_HWLOC_SOURCE_DIR.
Supports local sources on Windows and Unix builds.
Adds CI coverage using hwloc 2.3.0 and fixes test compatibility.

File	Description
`CMakeLists.txt`	Configures local hwloc source handling.
`README.md`	Documents the new option.
`.github/workflows/reusable_basic.yml`	Tests a local hwloc checkout.
`test/memspaces/memspace_highest_bandwidth.cpp`	Supports the older hwloc enum scope.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot

Copilot review overview

🟡 Changes recommended

The distinct Windows local-source build path lacks CI coverage.

Review effort: Balanced
Findings: 1

Open (1)

Add Windows local-source coverage for static hwloc · New

Resolved since last review (1)

Forward UMF_HWLOC_SOURCE_DIR to nested build_umfd configure

Copilot

Copilot review overview

🟢 Approval recommended

The local-source flow is validated across supported platforms and no blocking correctness issues were found.

Review effort: Balanced
Findings: None

Resolved since last review (1)

Add Windows local-source coverage for static hwloc

lukaszstolarczuk

LGTM

lukaszstolarczuk · 2026-10-01T14:29:22Z

+        if(UMF_HWLOC_SOURCE_DIR)
+            # Drop build artifacts copied over from an already built local tree
+            set(UMF_HWLOC_PATCH_CMD
+                "(test ! -f Makefile || make distclean) && (${UMF_HWLOC_PATCH_CMD})"


just a question, why we have to run this patch cmd here and right in the next step (ExternalProject_Add)?

We don't run anything here, we only add make distclean to the front of the single PATCH_COMMAND so that build artifacts copied from an already built local tree aren't reused and linked

bratpiorka · 2026-10-01T16:30:26Z

NOTE: This needs to be set as a draft, as I need to add a separate option to use a mirrored HWLOC repository path.
E.g.

cmake -B build -DUMF_LINK_HWLOC_STATICALLY=ON -DUMF_HWLOC_REPO=/home/user/hwloc.git
# CMake
git clone -b hwloc-2.13.0 /home/user/hwloc.git /home/user/hwloc-src

+
+    - name: Create hwloc git mirror
+      if: matrix.local_hwloc == 'mirror'
+      run: git clone --mirror https://github.com/open-mpi/hwloc.git ${{env.HWLOC_MIRROR_DIR}}


        -DUMF_BUILD_LIBUMF_POOL_JEMALLOC=ON
        -DUMF_TESTS_FAIL_ON_SKIP=ON
        -DUMF_LINK_HWLOC_STATICALLY=${{matrix.link_hwloc_statically}}
+        ${{ matrix.local_hwloc == 'source' && format('-DUMF_HWLOC_SOURCE_DIR={0}', env.HWLOC_SRC_DIR) || '' }}


        -DUMF_TESTS_FAIL_ON_SKIP=ON
        -DUMF_LINK_HWLOC_STATICALLY=${{matrix.link_hwloc_statically}}
+        ${{ matrix.local_hwloc == 'source' && format('-DUMF_HWLOC_SOURCE_DIR={0}', env.HWLOC_SRC_DIR) || '' }}
+        ${{ matrix.local_hwloc == 'mirror' && format('-DUMF_HWLOC_REPO={0}', env.HWLOC_MIRROR_DIR) || '' }}


Copilot

Copilot review overview

🟢 Approval recommended

The implementation is consistent, documented, and covered by targeted CI configurations.

Review effort: Balanced
Findings: None

ldorau

LGTM

github-advanced-security Bot found potential problems Sep 30, 2026

View reviewed changes

Comment thread .github/workflows/reusable_basic.yml

- name: Clone hwloc sources

if: matrix.local_hwloc == 'ON'

run: git clone --depth 1 -b hwloc-2.3.0 https://github.com/open-mpi/hwloc.git ${{env.HWLOC_SRC_DIR}}

Comment thread .github/workflows/reusable_basic.yml Fixed

bratpiorka force-pushed the rrudnick_hwloc_source_dir branch 2 times, most recently from 26b779f to fbd7fd6 Compare October 1, 2026 07:33

bratpiorka requested a balanced review from Copilot October 1, 2026 08:16

Copilot started reviewing on behalf of bratpiorka October 1, 2026 08:16 View session

Copilot AI reviewed Oct 1, 2026

View reviewed changes

Comment thread CMakeLists.txt

bratpiorka force-pushed the rrudnick_hwloc_source_dir branch from fbd7fd6 to 77af354 Compare October 1, 2026 08:32

bratpiorka requested a balanced review from Copilot October 1, 2026 08:38

Copilot started reviewing on behalf of bratpiorka October 1, 2026 08:39 View session

Copilot AI reviewed Oct 1, 2026

View reviewed changes

Comment thread .github/workflows/reusable_basic.yml

bratpiorka force-pushed the rrudnick_hwloc_source_dir branch from 77af354 to 274a17f Compare October 1, 2026 09:17

github-advanced-security Bot found potential problems Oct 1, 2026

View reviewed changes

Comment thread .github/workflows/reusable_basic.yml Fixed

Comment thread .github/workflows/reusable_basic.yml Fixed

bratpiorka requested a balanced review from Copilot October 1, 2026 09:18

Copilot started reviewing on behalf of bratpiorka October 1, 2026 09:19 View session

Copilot AI reviewed Oct 1, 2026

View reviewed changes

bratpiorka marked this pull request as ready for review October 1, 2026 09:33

bratpiorka requested a review from a team as a code owner October 1, 2026 09:33

bratpiorka requested a review from lukaszstolarczuk October 1, 2026 09:33

bratpiorka assigned ldorau and unassigned ldorau Oct 1, 2026

bratpiorka requested a review from ldorau October 1, 2026 09:33

lukaszstolarczuk approved these changes Oct 1, 2026

View reviewed changes

bratpiorka marked this pull request as draft October 1, 2026 16:24

ldorau requested changes Oct 2, 2026

View reviewed changes

Comment thread CMakeLists.txt

Comment thread CMakeLists.txt

Comment thread CMakeLists.txt

Comment thread CMakeLists.txt

Comment thread README.md

bratpiorka force-pushed the rrudnick_hwloc_source_dir branch from 274a17f to 6188113 Compare October 5, 2026 12:22

github-advanced-security Bot found potential problems Oct 5, 2026

View reviewed changes

bratpiorka changed the title ~~Add CMake opt to get HWLOC sources from path instead of github~~ Add CMake options to build static hwloc from local sources or a local git mirror Oct 5, 2026

bratpiorka force-pushed the rrudnick_hwloc_source_dir branch from 6188113 to f2c5035 Compare October 5, 2026 12:39

Add CMake opt to get HWLOC sources from path instead of github

567dd8c

Use scoped hwloc location type enum in bandwidth test

f2c5035

bratpiorka marked this pull request as ready for review October 5, 2026 15:04

bratpiorka requested review from ldorau and a balanced review from Copilot October 5, 2026 15:04

Copilot started reviewing on behalf of bratpiorka October 5, 2026 15:05 View session

Copilot AI reviewed Oct 5, 2026

View reviewed changes

ldorau approved these changes Oct 6, 2026

View reviewed changes

ldorau merged commit 841fcb3 into oneapi-src:main Oct 6, 2026
79 checks passed

Conversation

bratpiorka commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟢 Approval recommended

Uh oh!

lukaszstolarczuk left a comment

Choose a reason for hiding this comment

Uh oh!

lukaszstolarczuk Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

bratpiorka Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

bratpiorka commented Oct 1, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟢 Approval recommended

Uh oh!

ldorau left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

bratpiorka commented Sep 30, 2026 •

edited

Loading