Skip to content

Honor --lfnf=none when no selected test previously failed - #15417

Closed
cleissonom wants to merge 1 commit into
pytest-dev:mainfrom
cleissonom:fix/lfnf-unselected-13614
Closed

cleissonom wants to merge 1 commit into
pytest-dev:mainfrom
cleissonom:fix/lfnf-unselected-13614

Conversation

@cleissonom

@cleissonom cleissonom commented Nov 19, 2026 •

Copy link
Copy Markdown

Closes #13614.

Scope under discussion

Maintainer feedback distinguishes two cases: stale cache entries left by deleted or renamed tests, and real failures that still exist outside the current selection. The review identifies the latter's existing fallback as intentional and asks for a decision on #13614 before extending --lfnf=none to it.

The current patch at 3eafe99bb treats both cases the same. It does not remove stale entries or establish that an unselected test no longer exists. That distinction remains unresolved in this version.

Change and regression coverage

In the original reproduction, removing a failing test leaves its node ID in the last-failed cache. Because the cache is nonempty, pytest --lf --lfnf=none enters the branch for known failures outside the selection and runs the remaining passing test.

This patch adds a check for --lf and --lfnf=none when none of the selected items match a cached failure. It reports the remaining items through pytest_deselected and clears the items list. The fallback for plain --lf is unchanged.

The added assertions extend test_lastfailed_with_all_filtered. After the failing test is removed, --lf --lfnf=none reports one deselected test and exits with ExitCode.NO_TESTS_COLLECTED (5). The existing --lf --co assertions check that the remaining test is still collected under the default behavior.

The added assertions cover the deleted-test example. They do not add separate coverage for failures excluded with -k or --pyargs, and they do not validate cache cleanup.

Validation and AI assistance

OpenAI GPT-6.2 Astra in OpenCode prepared the patch, regression test and initial PR text. The agent reported reproducing the behavior on pytest 9.2.0 and main.

At 3eafe99bb, it reported running all 68 cacheprovider tests and tox -e linting,py313 on macOS with CPython 3.13.15. Linting passed. The full suite reported 4,753 passed, 52 skipped, 15 xfailed, 5 xpassed and 2 warnings.

Cleisson reviewed the patch and approved it for submission.

A nonempty lastfailed cache can contain only removed or unselected tests. In that case --lf fell back to running every selected item even when the caller explicitly requested none.

Co-authored-by: OpenAI GPT-6.2 Astra <noreply@openai.com>
@pyba-chronographer pyba-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Nov 19, 2026

Copy link
Copy Markdown
Member

🤖 Written by Claude Opus 5.5 via Claude Code for the pytest maintainers; I prompted it, it did the work, I read it.

This change targets the wrong layer and leaves a dissonance in place.

--lfnf was added in #3139 / #3295 for exactly one case: the lastfailed cache is empty. The branch this PR changes ("N known failures not in selected tests") is a different state. It has deliberately run the selection since #5333, because lastfailed remembers failures outside the current selection (#2621).

#13614 lands in that branch for two different reasons:

  1. Real failures outside the selection (the -k / --pyargs report in the issue comments). The current behaviour is intended. Making --lfnf=none apply here changes what the option means, from "cache is empty" to "nothing in my selection failed". That is a design decision, and it would need its own discussion, docs and changelog.
  2. Stale entries (the original report: the failing test was deleted or renamed). The cache only looks non-empty because entries for tests that no longer exist are never removed. That is the actual bug.

With this PR the dissonance stays and gets worse:

  • The status line still says "1 known failures not in selected tests" while everything is deselected, which contradicts the existing "no previously failed tests, deselecting all items." path that --lfnf=none documents.
  • The stale entry is never cleared. If the failing test was renamed and still fails, --lf --lfnf=none deselects it, and every later --lf --lfnf=none run also runs nothing until the user runs without --lf.

Pruning stale entries is the more direct direction, but it is easy to get wrong too. A test that is missing from one run is not necessarily gone:

  • --ignore, collect_ignore and norecursedirs
  • tests defined only on some platforms or Python versions
  • parametrize ids that depend on CLI options or the environment
  • plugins that create items dynamically
  • xdist, where the controller doesn't see the workers' collect reports

Each of these would silently forget a real failure, which is worse than the current symptom. Any pruning would have to be limited to cases where absence is certain, and it would need tests for each of these situations.

So I don't think this can go in as is. The two cases should be separated: the semantics question for case 1 needs a decision on #13614 first, and a fix for case 2 needs a careful definition of when an entry is actually stale.


Generated by Claude Code

@cleissonom

Copy link
Copy Markdown
Author

Thanks for the detailed review. I checked the rename case: with this patch, a renamed test that still fails is deselected on every --lf --lfnf=none run, and its stale entry is never cleared. pytest 9.2.0 still catches it. The patch makes that case worse, so I'm closing this PR.

I agree the two cases need to be handled separately. The --lfnf semantics question can stay on #13614. If you'd like stale entries pruned only where absence is certain, I'm happy to work on that once we agree on the definition.

@cleissonom cleissonom closed this Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

All tests are run when only last failed tests are requested

2 participants