CI probe: concurrent aclose_forcefully() on UDP sockets and Process - #1
Open
graingert-coef wants to merge 8 commits into
Open
CI probe: concurrent aclose_forcefully() on UDP sockets and Process#1graingert-coef wants to merge 8 commits into
graingert-coef wants to merge 8 commits into
Conversation
for more information, see https://pre-commit.ci
Extends the TCP regression test from agronholm#1277 to UDPSocket, ConnectedUDPSocket and Process. The plain variants pass on master too, so they are guards rather than regression tests. The pending-send variants target Windows specifically: _ProactorDatagramTransport.sendto() always buffers and starts an overlapped operation, so transport.close() declines to schedule connection_lost while _write_fut is pending, deferring it to an IOCP completion on a later loop iteration. That should expose UDPSocket.aclose()'s unshielded wait on closed_event, which is invisible on selector loops. Pushed to run the Windows CI matrix; not intended for upstream as-is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts: # src/anyio/_backends/_asyncio.py
The previous versions queued sends and then entered a task group, but start_soon() only schedules — the children first run when the host yields at __aexit__, and one event loop iteration is all the proactor needs to reap the overlapped send. close() then took the no-pending-write path and the tests passed on Windows for the wrong reason. anyio puts send()'s checkpoint before transport.sendto(), so the operation is only still in flight if nothing yields afterwards. Drop the pointless 10x send loop (each iteration yielded, so the buffer drained every time) and close directly after a single send, as PR agronholm#1246's own reproducer does. Adds a concurrent variant gated on an Event so the second closer only runs once the first has yielded inside aclose(), which is where its unshielded wait on closed_event is exposed. Neither PR covers that caller. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Not for upstream — pushed to run the Windows CI matrix on a hypothesis.
Question
anyio's
UDPSocket.aclose()/ConnectedUDPSocket.aclose()awaitself._protocol.closed_event.wait()unshielded. Underaclose_forcefully()that wait is cancelled — instrumenting it showsaclose()returning withclosed_eventnever set — but on selector loops the fd is already released by then, because_call_connection_lostis queued one callback ahead of the cancelled task's resumption.So the defect is invisible on Linux/macOS. Windows should differ:
_ProactorDatagramTransport.sendto()always buffers and starts an overlapped op, setting_write_fut(proactor_events.py:501-506)._ProactorBasePipeTransport.close()only schedules_call_connection_lostif not self._buffer and self._write_fut is None(:108)._loop_writing(:524-525) — an IOCP completion callback on a later loop iteration.That should remove the margin, so both closers see a live fd.
Expected results
TestTCPStream::test_concurrent_aclose_forcefully_waits_for_socket_closeTest{UDPSocket,ConnectedUDPSocket}::..._waits_for_socket_closeTest{UDPSocket,ConnectedUDPSocket}::..._with_pending_sendtest_concurrent_aclose_forcefully_waits_for_process_exitWindows 3.10 is also of interest:
_force_close's_called_connection_lostguard (:140) looks like a 3.11 addition, which may matter for whether agronholm#1277'sabort()is effective for the second concurrent caller.If the pending-send variants pass on Windows too, the UDP wait is unshielded but unobservable, and the tests are guards rather than regression tests.