Skip to content

Cherry-pick upstream Spack package updates - #82

Draft
mathomp4 wants to merge 33 commits into
JCSDA:spack-stack-devfrom
GMAO-SI-Team:feature/cherry-pick-upstream-packages
Draft

Cherry-pick upstream Spack package updates#82
mathomp4 wants to merge 33 commits into
JCSDA:spack-stack-devfrom
GMAO-SI-Team:feature/cherry-pick-upstream-packages

Conversation

@mathomp4

@mathomp4 mathomp4 commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

This PR cherry-picks upstream spack/spack-packages package updates, essential fork fixes, and new geospatial packages (pyresample stack) into spack-stack-dev:

Upstream PR Cherry-Picks & Updates:

New Packages (pyresample stack from PR #70):

  • py-pyresample: New package recipe for resampling geospatial image data
  • py-pykdtree: New package recipe for fast KD-tree nearest neighbor search
  • py-pytest-lazy-fixtures: New package recipe for pytest parametrized fixtures

Fork-Specific Fixes:

  • hdf5: Add NAG compiler patches for Linux threads and HL Fortran MPI module (nag_linux_threads.patch, nag_hl_fortran_mpi_module.patch).
  • py-scipy: Restrict Darwin+GCC use_stdc_no_threads.patch to @:1.8 since newer SciPy versions use Meson instead of setup.py.
  • py-torch: Fix INTEL_MKL_DIR path when using intel-oneapi-mkl.
  • python: Add spec membership checks for gettext and ncurses in configure_args before accessing prefix/libs.

Note: PRs spack#5627 (bufr), spack#5628 (ecflow), spack#5537 (py-xarray), and spack#5631 (uwtools) were already merged into spack-stack-dev in previous PRs.

Please merge (do not squash merge) so that individual commit histories are preserved.

mathomp4 and others added 16 commits August 18, 2026 11:39
* py-numba: add 0.66.0, py-llvmlite: add 0.48.0

* Add myself as maintainer
NAG's linker forwards GNU linker flags (like --version-script) to
GCC without the required -Wl, prefix, causing the link step to fail.
Add a patch to scipy's meson.build that excludes nagfor from the
version-script code path, matching the existing clang-cl exclusion.

Applies to @1.17: %nag.
* py-torch: Fix for Cmake and environment variables

* Add link

* Committed unneeded file. Oops

* Narrow patch

* Move to backport patch
Rust currently overwrites CARGO_HOME for dependent builds with a
package-stage directory. This prevents users from using a prepopulated
Cargo cache when CARGO_NET_OFFLINE=true, since Cargo instead searches
the empty stage-local cache.

Honor an explicitly configured CARGO_HOME only for offline builds.
Retain Spack's isolated per-package Cargo cache for the normal online
case.
Co-authored-by: Alec Scott <hi@alecbcs.com>
Co-authored-by: Matthew Mehrtens <12023414+mcmehrtens@users.noreply.github.com>
@mathomp4
mathomp4 marked this pull request as draft August 18, 2026 15:41
@mathomp4

Copy link
Copy Markdown
Collaborator Author

Whoops. Gemini made a PR. I was just thinking of pushing a branch. I've drafted this until I can test!

adamjstewart and others added 6 commits August 18, 2026 11:45
* py-urllib3: deprecate versions with CVEs

Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>

* ruff

Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>

* Bump spack commit to allow multiple setuptools-scm versions

---------

Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Alec Scott <hi@alecbcs.com>
Co-authored-by: Alec Scott <alec@llnl.gov>
@mathomp4

Copy link
Copy Markdown
Collaborator Author

@rickgrubin-noaa I've added your changes from #70 into here. I think.

Comment thread repos/spack_repo/builtin/packages/py_scipy/package.py Outdated
@climbfuji

Copy link
Copy Markdown
Collaborator

Question for PR #70 - the PR still says WIP. Are these changes tested and ready to go in? @rickgrubin-noaa

mathomp4 and others added 2 commits August 20, 2026 12:14
@mathomp4

Copy link
Copy Markdown
Collaborator Author

Question for PR #70 - the PR still says WIP. Are these changes tested and ready to go in? @rickgrubin-noaa

Okay. As a test I decided to try:

spack add py-pyresample
spack install

in one of my envs. And it failed on the install of py-pykdtree as the SHA in there seemed incorrect. I've updated that.

@rickgrubin-noaa can you test to make sure it works for you.

@rickgrubin-noaa

Copy link
Copy Markdown
Collaborator

And it failed on the install of py-pykdtree as the SHA in there seemed incorrect. I've updated that.

@rickgrubin-noaa can you test to make sure it works for you.

Wow, not sure how I screwed up that one. Works fine with the sha256 hash confirmed here.

@mathomp4

Copy link
Copy Markdown
Collaborator Author

And it failed on the install of py-pykdtree as the SHA in there seemed incorrect. I've updated that.
@rickgrubin-noaa can you test to make sure it works for you.

Wow, not sure how I screwed up that one. Works fine with the sha256 hash confirmed here.

Well if it's like the manual hashes I've done before, it's because I downloaded a zip file version or .xz or something.

That's always a fun day's worth of "Why did the hash change?!" 😄

@mathomp4

Copy link
Copy Markdown
Collaborator Author

@climbfuji et al. I'm doing my final sweep of this now. I'm currently adding FMS 2026.01.01 for my GEOS library builds. If that works, I think I might be done with the package updates.

of course, lots of the FMS 2026.01.01 stuff then goes into spack-stack in the yamls there. But I want to make sure it builds for me and I can use it with GEOS.

@mathomp4

Copy link
Copy Markdown
Collaborator Author

Uh oh. I'm having issues building 2026.01.01 and my fear is we might need a similar CMake patch for disabling the unit tests that @rem1776 had for 2025.04 and 2026.01.

@AlexanderHrabski-NOAA were you able to build 2026.01.01 in your testing?

@AlexanderHrabski-NOAA

Copy link
Copy Markdown
Collaborator

Uh oh. I'm having issues building 2026.01.01 and my fear is we might need a similar CMake patch for disabling the unit tests that @rem1776 had for 2025.04 and 2026.01.

@AlexanderHrabski-NOAA were you able to build 2026.01.01 in your testing?

@mathomp4 I recall building 2026.01.01 during my testing of the linking patch, but that testing occurred before the CMake patch for disabling cmake testing. During that session, I had my AI code assistant fix a couple different cmake issues in a couple different FMS versions so that I could test my patch with a single MRE. But I was so focused on verifying my own patch that I don't recall if anything was broken in 2026.01.01.

@mathomp4

Copy link
Copy Markdown
Collaborator Author

@mathomp4 I recall building 2026.01.01 during my testing of the linking patch, but that testing occurred before the CMake patch for disabling cmake testing. During that session, I had my AI code assistant fix a couple different cmake issues in a couple different FMS versions so that I could test my patch with a single MRE. But I was so focused on verifying my own patch that I don't recall if anything was broken in 2026.01.01.

@AlexanderHrabski-NOAA Okay. GPT whipped up a patch for 2026.01.01 pretty simply. I'm now working my way to the build. Of course, it's like package 190 out of 200 in this build so it'll be a bit. But if it works, I'll update this and make a PR to spack-packages.

Thankfully, I think 2026.02 when it's released has the fix in code. :)

@mathomp4

Copy link
Copy Markdown
Collaborator Author

Okay. I made a PR for spack-packages here:

spack#6235

and pushed the same change here.

I'm almost there! Testing found a couple more small issues I'm working out but I'm so close!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants