Skip to content

Document multi-producer/consumer channel pattern (#330) - #1141

Open
jbbqqf wants to merge 1 commit into
agronholm:masterfrom
jbbqqf:docs/330-mpmc-channel-clone
Open

Document multi-producer/consumer channel pattern (#330)#1141
jbbqqf wants to merge 1 commit into
agronholm:masterfrom
jbbqqf:docs/330-mpmc-channel-clone

Conversation

@jbbqqf

@jbbqqf jbbqqf commented May 9, 2026

Copy link
Copy Markdown

Changes

Closes #330.

The clone() method on memory object streams is documented at API level and mentioned in passing in docs/streams.rst, but the canonical multi-producer/multi-consumer shutdown pattern (give every task its own clone, close the originals) is not. Users coming from trio.MemoryChannel look for it specifically because Trio's guide has had a dedicated section since 2019.

This PR adds a Multiple producers and/or multiple consumers subsection under Memory object streams that:

  • shows the canonical pattern with two producers, two consumers and a single shared channel pair, using clone() and async with for tear-down
  • explicitly flags the two well-known footguns:
    • sharing a stream end across tasks without cloning -> ClosedResourceError
    • cloning for every task without closing the originals -> consumers hang forever

Inspired by Trio's Managing multiple producers and/or multiple consumers section, but the prose and example are written from scratch in anyio terms (create_memory_object_stream, MemoryObjectSendStream, create_task_group) so there is no copyright concern.

The example was run as-is against current master; output is the expected six lines plus a clean exit (the order varies because tasks race, but the program terminates without exceptions).

Checklist

  • You've added tests (in tests/) which would fail without your patch
  • You've updated the documentation (in docs/), in case of behavior changes or new features
  • You've added a new changelog entry (in docs/versionhistory.rst).

This is a documentation-only change so no test was added, but the example was executed end-to-end. Sphinx strict build (sphinx-build -W --keep-going -b html . _build/html) was clean.

Reproduce BEFORE/AFTER yourself (copy-paste)

git clone https://github.com/agronholm/anyio.git /tmp/anyio-330 && cd /tmp/anyio-330
python -m venv .venv && . .venv/bin/activate
pip install -e . sphinx sphinx_rtd_theme sphinx-autodoc-typehints sphinx-tabs

# --- BEFORE (master): no MPMC subsection in streams.rst ---
git checkout origin/master
grep -nE "Multiple producers|clone\(\)" docs/streams.rst | head -5
# Expected: only the brief line "Memory object streams can be cloned by calling
# the clone() method..." -- no dedicated subsection.

# --- AFTER (this PR): subsection is present and example runs ---
git fetch https://github.com/jbbqqf/anyio.git docs/330-mpmc-channel-clone
git checkout FETCH_HEAD
grep -nE "Multiple producers" docs/streams.rst
# Expected: "Multiple producers and/or multiple consumers" heading

# Build docs in strict mode to confirm no Sphinx warnings
sphinx-build -W --keep-going -b html docs docs/_build/html >/dev/null && echo OK
# Expected: OK

# Run the documented example end-to-end
sed -n '/from anyio import create_memory_object_stream/,/run(main)/p' docs/streams.rst \
    | head -33 > /tmp/mpmc.py
python /tmp/mpmc.py
# Expected: 6 lines of "consumer X|Y got 'i from producer A|B'", clean exit.

Risk / blast radius

Documentation-only.


PR drafted with assistance from Claude (Anthropic). The example was executed end-to-end and the docs build was checked in strict mode.

The clone() method on memory object streams is mentioned in passing
in the streams guide, but the user needs to read the API doc and
infer the multi-producer / multi-consumer shutdown pattern from it.
Add a dedicated subsection that walks through the canonical pattern
(give every task its own clone, close the originals) and calls out
the two well-known footguns:

  - sharing a stream end across tasks without cloning -> ClosedResourceError
  - cloning for every task without closing the originals -> consumers hang

Inspired by Trio's "Managing multiple producers and/or multiple
consumers" section, but written from scratch in anyio terms with a
runnable example.

Closes agronholm#330.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

document memory object stream clone with a "Managing multiple producers and/or multiple consumers" section

1 participant