Fix Cube.rolling_window with lazy auxiliary coordinates (#6480)#7123
Fix Cube.rolling_window with lazy auxiliary coordinates (#6480)#7123gaoflow wants to merge 2 commits into
Conversation
81f6da8 to
aa05a2e
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7123 +/- ##
=======================================
Coverage 90.15% 90.15%
=======================================
Files 91 91
Lines 24985 24985
Branches 4685 4685
=======================================
Hits 22526 22526
Misses 1682 1682
Partials 777 777 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
aa05a2e to
e4ea7a2
Compare
|
Rebased onto current
Local verification:
I could not run the targeted pytest locally because this environment is missing the Iris runtime dependency |
| def test_lazy_aux_coord(self): | ||
| # A lazy aux-coord paralleling the rolled dimension must not cause a | ||
| # failure, and should give the same result as a real one, staying lazy | ||
| # (see #6480). |
There was a problem hiding this comment.
| # (see #6480). | |
| # (see #6480). | |
| window = 2 |
Not drastically important, but keeping related tests aligned helps with readability.
| # failure, and should give the same result as a real one, staying lazy | ||
| # (see #6480). | ||
| self.cube.add_aux_coord(AuxCoord(da.arange(6), long_name="lazy_extra"), 0) | ||
| res_cube = self.cube.rolling_window("val", iris.analysis.MEAN, 3) |
There was a problem hiding this comment.
| res_cube = self.cube.rolling_window("val", iris.analysis.MEAN, 3) | |
| res_cube = self.cube.rolling_window("val", iris.analysis.MEAN, window, mdtol=0) |
| # Index with a list rather than a tuple: a tuple is interpreted | ||
| # as a multidimensional index by Dask, so lazy coordinates would | ||
| # otherwise fail here (see #6480). |
There was a problem hiding this comment.
I don't think this comment is needed at all, but if you'd like to keep it I'd shorten it.
| # A lazy aux-coord paralleling the rolled dimension must not cause a | ||
| # failure, and should give the same result as a real one, staying lazy | ||
| # (see #6480). |
There was a problem hiding this comment.
Don't think this is needed, it's implicit in the test.
rolling_window built each window's coordinate bounds with new_bounds[:, (0, -1)]. A tuple index is treated by Dask as a multidimensional index, so a lazy coordinate along the windowed dimension raised a TypeError. Index with the list [0, -1] instead, which numpy and Dask both treat as selecting the first and last column, giving an identical (and still lazy) result. Fixes SciTools#6480.
e4ea7a2 to
bc6142c
Compare
|
Updated in Review follow-up:
Verification after the update: uv run --with pytest --with pytest-mock --with requests --with filelock --with distributed --with-editable . python -m pytest lib/iris/tests/unit/cube/test_Cube.py::Test_rolling_window::test_lazy_aux_coord -q
ruff check lib/iris/cube.py
ruff format --check lib/iris/cube.py lib/iris/tests/unit/cube/test_Cube.py
uv run --with towncrier towncrier build --draft --version 0.0
git diff --check origin/main...HEAD |
🚀 Pull Request
Description
Closes #6480 (and the duplicate #6639).
:meth:
iris.cube.Cube.rolling_windowraised aTypeErrorwhen the cube had a lazy auxiliary coordinate running along the windowed dimension:Cause
When building the new coordinate bounds,
rolling_windowselects the first and last point of each window with:A tuple index is interpreted by Dask as a multidimensional index (and so it tries to use
0and-1as indices into successive axes), which fails for a lazy array. NumPy happens to tolerate the tuple here, so the bug only surfaces for lazy coordinates.Fix
Index with the list
[0, -1]instead. NumPy and Dask both treat a list as selecting the first and last column, so the result is identical for real arrays and now also works (and stays lazy) for lazy ones. This is the fix hinted at by @pp-mo on the issue. Both the numeric and string-coordinate branches are updated.Verification
Test_rolling_window::test_lazy_aux_coord— asserts a lazy aux-coord no longer fails, stays lazy (points and bounds), and equals the real-array result. It fails onmainand passes here.Test_rolling_windowandtests/unit/util/test_rolling_window.pysuites pass (25 tests).ruff check/ruff formatclean.