gh-153062: Fix a crash iterating itertools.tee on the free-threaded build#153063
Open
tonghuaroot wants to merge 3 commits into
Open
gh-153062: Fix a crash iterating itertools.tee on the free-threaded build#153063tonghuaroot wants to merge 3 commits into
tonghuaroot wants to merge 3 commits into
Conversation
…aded build itertools.tee branches share a linked list of teedataobject cells. On the free-threaded build, iterating one branch from multiple threads, or iterating sibling branches concurrently, raced on the shared cells and each branch's position, corrupting refcounts and crashing. Lock each teedataobject while reading, extending, or clearing it, and snapshot each branch's position under the tee object's own lock, revalidating before advancing, so the two locks are never nested. Concurrent iteration of one tee is undefined and may raise RuntimeError as documented, but no longer crashes.
Drop scratch files accidentally staged in the previous commit.
corona10
reviewed
Jul 5, 2026
The free-threading snapshot and revalidation add per-element overhead on the default build, where the GIL already serializes access. Guard that path under Py_GIL_DISABLED so the default build keeps the original iteration and the free-threaded path is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Iterating an
itertools.teeiterator concurrently crashes the free-threaded build. tee iterators are documented as not thread-safe, so concurrent use returning undefined results or raisingRuntimeErroris by design and is preserved here; this fixes only the memory-unsafety (corrupted reference counts and a segfault), which the free-threaded build must not permit. The otheritertoolsiterators already lock their__next__;teewas the remaining one.The shared
teedataobjectcells are locked while read, extended, or cleared. Each branch's position (dataobjandindex) is snapshotted under the tee object's own critical section and revalidated before it is advanced, so the tee lock and the data-object lock are never nested: a nested critical section can be suspended under contention, which would break the atomicity the position fetch depends on (this is whyPy_BEGIN_CRITICAL_SECTION2on the two objects does not work here).Tests in
Lib/test/test_free_threading/cover both a single branch consumed by many threads and sibling branches each consumed by their own thread;run_concurrentlyis barrier-synchronized so the threads actually contend. The re-entrancy guard'sRuntimeErroris tolerated, since it is a documented outcome. Single-threaded behavior is unchanged andtest_itertoolsstill passes.Backport labels left to a maintainer.