Skip to content

fix: executor shutdown race, dropped kwargs, chunk revalidation, unknown-class cache - #1699

Draft
ariostas wants to merge 1 commit into
scikit-hep:mainfrom
ariostas:fix-source-hygiene
Draft

fix: executor shutdown race, dropped kwargs, chunk revalidation, unknown-class cache#1699
ariostas wants to merge 1 commit into
scikit-hep:mainfrom
ariostas:fix-source-hygiene

Conversation

@ariostas

@ariostas ariostas commented Aug 5, 2026

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

Four independent, low-risk correctness fixes from #1688 (findings 1–3 of "PR 3" and finding 3 of "PR 5"). Grouped because each is a few lines with no API surface; happy to split if you'd rather review them separately.

Note this deliberately does not include finding 4 of "PR 3" (with uproot.open(...) not shutting down attached executors). That one changes what __exit__ closes and would shut down a user-supplied executor reused across files, so it needs its own discussion.

1. Submit racing with shutdown was orphaned

ThreadPoolExecutor.shutdown queues one sentinel per worker and then joins them; ResourceThreadPoolExecutor only set _closed after that returned. A submit in that window passed the guard, landed behind a sentinel, and was never run — its Future blocked forever. Reproduced by occupying the single worker so join() blocks:

ex.closed during shutdown: False
late submit ACCEPTED
late future ORPHANED (timed out, would hang forever)

Sets the flag before queuing the sentinels. ThreadPoolExecutor now gets the same guarantee, which it previously lacked entirely.

2. submit accepted keyword arguments and discarded them

ThreadPoolExecutor.submit(task, /, *args, **kwargs) built Future(task, args), so submit(task, 1, x=3) silently ran task(1). Now carried on the Future.

3. A failed chunk length check only failed once

Chunk.wait assigned self._raw_data before validating its length, so a chunk declared as 5 bytes but backed by 3 raised OSError on first access and returned the truncated buffer on every access after that. The data is now published only once it has passed.

4. Unknown model classes were cached by classname alone

uproot.unknown_classes was keyed on the bare classname, so version 2 of a class got back the class built for version 1 (Unknown_MyClass_v1), and a versionless UnknownClass and a versioned UnknownClassVersion for the same classname evicted each other. Keyed on the encoded classname instead, which already carries both distinctions. The dict stays str -> class.

Tests

tests/test_1688_source_hygiene.py, 9 tests; 6 fail on main.

Full suite passes locally (1032 passed, 90 skipped).

…own-class cache

Four independent correctness fixes in the source and model layers.

ThreadPoolExecutor.shutdown queued one sentinel per worker and only then joined
them, while ResourceThreadPoolExecutor set its closed flag after that. A submit
racing with shutdown passed the guard, landed behind a sentinel, and was never
run, so its Future blocked forever. Set the flag before queuing the sentinels
and reject submits from that point on; ThreadPoolExecutor now has the same
guarantee, which it previously lacked entirely.

ThreadPoolExecutor.submit accepted **kwargs and dropped them on the floor:
submit(task, 1, x=3) silently ran task(1). Carry them on the Future.

Chunk.wait assigned self._raw_data before checking its length, so a chunk that
failed validation raised on first access and handed back the short buffer on
every access after that. Publish the data only once it has passed.

Unknown model classes were cached in uproot.unknown_classes under the bare
classname, so version 2 of a class got the class built for version 1, and a
versionless UnknownClass and a versioned UnknownClassVersion evicted each other.
Key on the encoded classname, which already carries both distinctions.

Assisted-by: claude-code:claude-opus-5[1m]
@codecov

codecov Bot commented Aug 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.94%. Comparing base (e598b61) to head (5c9785e).

Additional details and impacted files
Files with missing lines Coverage Δ
src/uproot/model.py 81.69% <100.00%> (+1.16%) ⬆️
src/uproot/reading.py 80.95% <100.00%> (+0.04%) ⬆️
src/uproot/source/chunk.py 84.00% <100.00%> (+1.24%) ⬆️
src/uproot/source/futures.py 89.52% <100.00%> (+0.45%) ⬆️

... and 1 file with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant