Skip to content

test: remove unused historical snapshot references - #321

Open
virtuald wants to merge 1 commit into
pybind:mainfrom
virtuald:prune-historical-snapshots
Open

virtuald wants to merge 1 commit into
pybind:mainfrom
virtuald:prune-historical-snapshots

Conversation

@virtuald

@virtuald virtuald commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

... I think the clanker removed too much, let's see if CI passes.

Heh, it did pass. Here's the 🤖 explanation of what was removed:


The large count comes from removing five complete historical snapshot trees, each containing 30 generated .pyi files—not 150 distinct tests or features.

All deletions are under tests/stubs/:

Category Removed Why unused
Python 3.8 leftovers 2 files A partial pybind11 2.13 snapshot containing only _bindings/__init__.pyi and hidden_builtins.pyi. Python 3.8 is no longer in the matrix.
Older pybind11 versions on Python 3.11 90 files Three annotated-mode trees for pybind11 2.9, 2.11, and 2.12. The current matrix tests those versions only on Python 3.13.
Type-variable mode on Python 3.11 60 files Two trees for pybind11 2.13 and 3.0. The current matrix tests this NumPy rendering mode only on Python 3.13.
Python 3.10 aliases to removed trees 3 symlinks Links for pybind11 2.11, 2.12, and 3.0/type-variable mode. Removing them avoids dangling links.

Total: 152 files + 3 symlinks, approximately 130 KiB.

What remains

  • All 13 configured compatibility cases.
  • Python 3.10–3.13 annotated-mode coverage for pybind11 2.13 and 3.0.
  • Older pybind11 coverage and both type-variable cases on Python 3.13, using retained Python 3.12 reference directories.
  • Every error snapshot, all fixture source code, test scripts, and configuration.

Here, “unused” means not selected by the existing tox/CI matrix, not necessarily incorrect or duplicate. Historical manual combinations lose their stored references; no currently configured test case was removed.


The history strongly suggests an intentional sparse matrix—not accidentally dropped jobs. Older pybind11 versions are still tested, just not against every Python version.

Evidence:

  • 91adf2e / f5b044d (2023): pybind11 2.9 and 2.11 coverage was introduced only on Python 3.12, while the main pybind11 version ran across multiple Python versions.
  • 60728d3 (2023): type-variable NumPy mode likewise started with just one Python version.
  • 433c899 (2025): those extra cases were explicitly moved from Python 3.12 to 3.13, and pybind11 2.12 was added using the same pattern.
  • 53a0b13 (2026): “Simplify test matrix” explicitly says it does not add or remove tests.
  • c16ac20 (2026): removing Python below 3.10 was explicitly intentional: “All EOL. Less silver files.”

My interpretation is that maintainers chose to test:

  1. Python-version differences with recent pybind11.
  2. Older pybind11 differences on one recent Python.
  3. Alternate rendering mode on one recent Python.

That plausibly limits CI cost, although I haven’t found an explicit statement explaining that motivation.

The leftover snapshots are less clear: they could be incomplete cleanup or intentionally retained for manual runs. Their existence alone doesn’t demonstrate forgotten CI coverage.

So I’m confident the matrix shape was deliberate. But our pruning only proves those references were unused by configured tests, not that the omitted combinations can never reveal bugs. Expanding that coverage would be a separate policy decision.

@virtuald
virtuald marked this pull request as ready for review September 25, 2026 15:55
@virtuald
virtuald requested a review from ax3l September 25, 2026 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant