feat: improve CodeRabbit review guardrails and verification#5122
Merged
rnetser merged 7 commits intoJun 8, 2026
Conversation
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
Contributor
|
Warning Ignoring CodeRabbit configuration file changes. For security, only the configuration from the base branch is applied for open source repositories. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (3)
📜 Recent review details🧰 Additional context used🧠 Learnings (49)📓 Common learnings📚 Learning: 2026-01-05T10:33:55.037ZApplied to files:
📚 Learning: 2026-06-01T06:43:11.108ZApplied to files:
📚 Learning: 2026-05-29T11:02:22.804ZApplied to files:
📚 Learning: 2026-04-30T13:42:43.892ZApplied to files:
📚 Learning: 2026-05-18T06:31:35.221ZApplied to files:
📚 Learning: 2026-05-17T14:29:44.166ZApplied to files:
📚 Learning: 2026-05-31T17:55:25.719ZApplied to files:
📚 Learning: 2026-05-18T06:30:28.071ZApplied to files:
📚 Learning: 2026-05-04T13:45:41.527ZApplied to files:
📚 Learning: 2026-05-14T14:01:37.353ZApplied to files:
📚 Learning: 2025-06-10T09:41:58.035ZApplied to files:
📚 Learning: 2026-05-20T11:18:46.206ZApplied to files:
📚 Learning: 2026-05-10T08:15:30.936ZApplied to files:
📚 Learning: 2026-05-19T10:17:44.423ZApplied to files:
📚 Learning: 2026-05-18T15:22:26.757ZApplied to files:
📚 Learning: 2026-05-18T06:31:18.023ZApplied to files:
📚 Learning: 2026-05-18T06:31:20.848ZApplied to files:
📚 Learning: 2025-12-16T14:00:59.076ZApplied to files:
📚 Learning: 2026-03-17T01:32:13.674ZApplied to files:
📚 Learning: 2026-03-17T01:32:13.674ZApplied to files:
📚 Learning: 2025-12-22T15:56:00.157ZApplied to files:
📚 Learning: 2026-05-18T15:22:26.757ZApplied to files:
📚 Learning: 2026-05-18T15:22:26.757ZApplied to files:
📚 Learning: 2026-06-04T10:17:16.006ZApplied to files:
📚 Learning: 2026-05-19T17:16:58.838ZApplied to files:
📚 Learning: 2026-04-14T16:15:36.741ZApplied to files:
📚 Learning: 2026-05-04T13:45:35.675ZApplied to files:
📚 Learning: 2026-05-18T06:31:00.848ZApplied to files:
📚 Learning: 2026-06-04T10:17:16.006ZApplied to files:
📚 Learning: 2026-05-18T15:22:26.757ZApplied to files:
📚 Learning: 2026-06-04T10:17:16.006ZApplied to files:
📚 Learning: 2026-03-31T08:35:26.070ZApplied to files:
📚 Learning: 2026-05-17T13:28:29.374ZApplied to files:
📚 Learning: 2026-04-24T18:35:11.039ZApplied to files:
📚 Learning: 2026-04-27T15:30:20.252ZApplied to files:
📚 Learning: 2026-05-07T11:11:59.120ZApplied to files:
📚 Learning: 2026-05-18T11:17:18.196ZApplied to files:
📚 Learning: 2026-05-20T09:43:31.397ZApplied to files:
📚 Learning: 2025-12-16T14:06:22.391ZApplied to files:
📚 Learning: 2026-06-02T21:17:09.602ZApplied to files:
📚 Learning: 2026-06-03T15:08:46.871ZApplied to files:
📚 Learning: 2026-06-03T09:46:34.574ZApplied to files:
📚 Learning: 2026-05-17T14:29:54.117ZApplied to files:
📚 Learning: 2026-06-04T10:17:10.615ZApplied to files:
📚 Learning: 2026-05-19T10:17:37.060ZApplied to files:
📚 Learning: 2026-05-20T11:18:31.677ZApplied to files:
📚 Learning: 2025-11-25T01:56:54.902ZApplied to files:
📚 Learning: 2026-05-27T09:38:55.486ZApplied to files:
🪛 zizmor (1.25.2).github/workflows/request-coderabbit-test-instructions.yml[error] 5-9: use of fundamentally insecure workflow trigger (dangerous-triggers): pull_request_target is almost always used insecurely (dangerous-triggers) [warning] 5-9: insufficient job-level concurrency limits (concurrency-limits): workflow is missing concurrency setting (concurrency-limits) [error] 41-41: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy) (unpinned-uses) [info] 41-41: action functionality is already included by the runner (superfluous-actions): use (superfluous-actions) .github/workflows/unresolve-coderabbit-threads.yml[error] 21-21: overly broad permissions (excessive-permissions): pull-requests: write is overly broad at the workflow level (excessive-permissions) [error] 11-13: use of fundamentally insecure workflow trigger (dangerous-triggers): pull_request_target is almost always used insecurely (dangerous-triggers) [warning] 21-21: permissions without explanatory comments (undocumented-permissions): needs an explanatory comment (undocumented-permissions) [info] 25-25: workflow or action definition without a name (anonymous-definition): this job (anonymous-definition) 🔀 Multi-repo contextLinked repositories findingsRedHatQE/openshift-virtualization-tests-design-docs
Summary assessment:
🔇 Additional comments (11)
📝 WalkthroughWalkthroughThis PR relaxes CodeRabbit rules for utilities unit tests, clarifies an Approval Policy for test-plan replies, tightens the test-instruction comment template (mandatory pytest commands and single non-blocking inline comment), and adds a workflow that unresolves review threads lacking substantive human responses. ChangesTest Requirements and Approval Policy
Automatic Thread Resolution Cleanup
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
✨ Finishing Touches🧪 Generate unit tests (beta)
|
Contributor
|
Report bugs in Issues Welcome! 🎉This pull request will be automatically processed with the following features: 🔄 Automatic Actions
📋 Available CommandsPR Status Management
Review & Approval
Testing & Validation
Container Operations
Cherry-pick Operations
Label Management
✅ Merge RequirementsThis PR will be automatically approved when the following conditions are met:
📊 Review ProcessApprovers and ReviewersApprovers:
Reviewers:
Available Labels
AI Features
💡 Tips
For more information, please refer to the project documentation or contact the maintainers. |
Contributor
|
Caution Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted. Error details |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.coderabbit.yaml:
- Line 49: Update the approval-gate wording in the policy string that currently
reads "All your review comments have been addressed (resolved or fixed), OR you
had no review comments." to remove counting "resolved" as sufficient; require a
substantive fix or author response instead (e.g., change to "All your review
comments have been addressed (fixed or responded to), OR you had no review
comments") so the rule enforces concrete fixes or replies rather than thread
state; locate and update the exact policy entry containing that sentence in
.coderabbit.yaml.
In @.github/workflows/unresolve-coderabbit-threads.yml:
- Around line 43-49: The GraphQL comments fetch uses comments(first: 20) but the
code later selects the last comment via .comments.nodes[-1], causing false
negatives on threads >20 comments; change the query to fetch the most recent
comments (e.g., replace comments(first: 20) with comments(last: 5)) so
.comments.nodes[-1] truly reflects the latest comment, and verify the jq filter
.comments.nodes[-1] remains the selector used by the workflow.
- Around line 11-23: Add concurrency control to the workflow to prevent
concurrent runs from racing when mutating PR threads: add a concurrency stanza
(at workflow level) using a unique group key that includes the pull request
number (e.g. "unresolve-threads-${{ github.event.pull_request.number }}") and
set cancel-in-progress: true so new runs cancel previous in-flight runs; update
the workflow containing the unresolve-threads job (job name "unresolve-threads")
to include this concurrency group to ensure only one run operates on a PR at a
time.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: df668759-6834-4848-869c-997ec880a353
📒 Files selected for processing (3)
.coderabbit.yaml.github/workflows/request-coderabbit-test-instructions.yml.github/workflows/unresolve-coderabbit-threads.yml
📜 Review details
🧰 Additional context used
🧠 Learnings (49)
📓 Common learnings
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-04-30T13:42:43.892Z
Learning: In the openshift-virtualization-tests repository, when processing test execution plan requests for PRs that introduce NEW test classes or test files (not just modifications to existing tests), always verify STD-first workflow compliance as part of the analysis: (1) check if the new test class has actual implementation code (assertions, test logic) — if so, it needs a prior STD PR, (2) check if the test file has a module docstring with STP or Jira/RFE link, (3) check if `What this PR does / why we need it:` in the PR description has meaningful content. Flag any violation even when the bot's primary request is only for a test execution plan. These checks should be appended to the test execution plan inline comment.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-29T11:02:22.804Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, version-bump PRs that update the main branch to reference a new CNV release (e.g., "Update project `main` to reference cnv 5.0") do NOT add new test files or test modules. These PRs only update metadata (pyproject.toml, tox.ini, .coderabbit.yaml) and clean up resolved Jira fixtures/markers in existing files. The "Stp Link Required" pre-merge check may incorrectly flag hundreds of "newly added" test files as a false positive due to the base-branch comparison shifting after branching — this should be ignored for such PRs.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3228
File: .coderabbit.yaml:30-41
Timestamp: 2026-01-05T10:33:55.037Z
Learning: In the openshift-virtualization-tests repository, user rnetser prefers minimal pre-merge checks in CodeRabbit configuration: only docstrings enforcement (80% threshold) is needed, not title or description checks.
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:13.674Z
Learning: In RedHatQE/openshift-virtualization-tests, maintainer dshchedr prefers CodeRabbit to post targeted inline comments on the Files Changed tab at each applicable location rather than aggregating multiple issues into a single PR discussion thread reply.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-16T14:00:59.076Z
Learning: In the openshift-virtualization-tests repository, when responding to test execution plan requests from openshift-virtualization-qe-bot-3, CodeRabbit must post ONLY an inline review comment on the Files Changed tab and then stop immediately without generating any follow-up comments in the PR discussion thread. No acknowledgment messages, no confirmation of posting, no explanation - silence after posting the inline review equals success. Additional comments create empty/meaningless reviews that clutter the PR.
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:13.674Z
Learning: In RedHatQE/openshift-virtualization-tests, maintainer dshchedr prefers CodeRabbit to post targeted inline comments on the Files changed tab for each applicable location rather than aggregating multiple issues into a single discussion thread.
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-22T15:56:00.157Z
Learning: In the openshift-virtualization-tests repository, when responding to test execution plan requests from openshift-virtualization-qe-bot-3, do NOT use "REQUEST_CHANGES" review type if the PR author has already marked the PR as verified (e.g., with `/verified` command). Test execution plans are informational guides, not blocking requirements. Use COMMENT event for informational test plans, or only REQUEST_CHANGES if there are actual code issues that need to be addressed before merging.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 2469
File: utilities/sanity.py:139-142
Timestamp: 2025-11-08T07:36:57.616Z
Learning: In the openshift-virtualization-tests repository, user rnetser prefers to keep refactoring PRs (like PR `#2469`) strictly focused on moving/organizing code into more granular modules without adding new functionality, error handling, or behavioral changes. Such improvements should be handled in separate PRs.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-14T14:01:37.353Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when a PR has the `needs-rebase` label and the pre-merge check reports a large number of "new test_*.py files" (e.g., 300+) that do not match the actual PR diff, this is a false positive caused by the check comparing the entire diverged branch file tree against main rather than just the PR diff. Do not raise STP Link Required or Docstring Coverage violations based on such check results — only flag these if the PR diff itself actually adds new test files or functions lacking docstrings.
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4867
File: tests/network/libs/stuntime.py:1-1
Timestamp: 2026-05-20T11:18:46.206Z
Learning: In RedHatQE/openshift-virtualization-tests, the PR template validation heuristic has produced repeated false positives on PRs where the description actually contains all required sections (e.g., PR `#4867` by Anatw). When an author explicitly rebuts a missing-template-section flag and confirms all sections are present with meaningful content, treat the original flag as a false positive and do not re-raise it. Be more conservative about firing this HIGH-severity flag — only raise it when a required section header (e.g., "##### What this PR does / why we need it:") is clearly absent or its content is verifiably empty/placeholder-only.
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-05-18T15:22:26.757Z
Learning: Applies to tests/**/test_*.py : New feature tests must follow the STD-first workflow: (1) STP (Software Test Plan) reviewed and approved, (2) STD (Software Test Description) placeholder tests with docstrings and `__test__ = False` reviewed, (3) Implementation only after STD review. Never submit test implementation without prior STD review.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 1168
File: tests/storage/storage_migration/utils.py:92-99
Timestamp: 2025-06-10T09:41:58.035Z
Learning: When CodeRabbit flags technical issues in code reviews, do not resolve the comments without first addressing them and getting a response from the user. Proper procedure is to discuss the technical issues and proposed solutions before marking comments as resolved.
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 5111
File: tests/network/localnet/upgrade/test_default_bridge_connectivity.py:1-11
Timestamp: 2026-06-03T09:46:29.920Z
Learning: In RedHatQE/openshift-virtualization-tests, new test files (test_*.py) written as bug automation — i.e., tests that verify a fix for an OCP bug (OCPBUGS-*) or a CNV Jira bug (CNV-*) — do NOT require the STD-first workflow (separate STD placeholder PR before implementation) and do NOT require an STP link. The bug tracker URL (e.g., https://redhat.atlassian.net/browse/OCPBUGS-66994) in the module docstring is sufficient traceability. The STD may be included in the same implementation PR. Do not flag missing STP links or missing dedicated STD PRs for OCPBUGS/CNV bug-automation tests.
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 5101
File: .github/workflows/request-coderabbit-test-instructions.yml:17-17
Timestamp: 2026-06-02T11:53:37.264Z
Learning: In RedHatQE/openshift-virtualization-tests, the `request-coderabbit-test-instructions.yml` workflow intentionally always creates a new comment (never updates/deduplicates) when either `generate_test_report` or `verified` labels trigger it. This is by design: the PR diff can change between the two triggers (new commits pushed), so each trigger produces an independent TIA snapshot of a different diff state. Deduplicating with `find-comment + edit-mode: replace` would be harmful because it would overwrite the earlier report, removing the ability to compare the two analyses. The historical record of both reports is a desired feature.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 1244
File: utilities/os_utils.py:248-250
Timestamp: 2025-07-08T05:51:06.314Z
Learning: User rnetser prefers fail-fast code design where functions should raise exceptions immediately when encountering unexpected or malformed input, rather than implementing defensive error handling with fallbacks. They value making problems visible immediately during development over graceful degradation.
📚 Learning: 2026-01-05T10:33:55.037Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3228
File: .coderabbit.yaml:30-41
Timestamp: 2026-01-05T10:33:55.037Z
Learning: In the openshift-virtualization-tests repository, user rnetser prefers minimal pre-merge checks in CodeRabbit configuration: only docstrings enforcement (80% threshold) is needed, not title or description checks.
Applied to files:
.coderabbit.yaml.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-05-29T11:02:22.804Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-29T11:02:22.804Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, version-bump PRs that update the main branch to reference a new CNV release (e.g., "Update project `main` to reference cnv 5.0") do NOT add new test files or test modules. These PRs only update metadata (pyproject.toml, tox.ini, .coderabbit.yaml) and clean up resolved Jira fixtures/markers in existing files. The "Stp Link Required" pre-merge check may incorrectly flag hundreds of "newly added" test files as a false positive due to the base-branch comparison shifting after branching — this should be ignored for such PRs.
Applied to files:
.coderabbit.yaml.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-05-17T14:29:44.166Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4884
File: .coderabbit.yaml:77-81
Timestamp: 2026-05-17T14:29:44.166Z
Learning: For RedHatQE/openshift-virtualization-tests’ `.coderabbit.yaml` custom check (“STP link required”), ensure the STP/RFE/Jira link appears in at least one of these docstring locations: the module-level docstring, an enclosing class docstring, or the individual test function docstring. This satisfies the requirement for all tests covered by that scope. The check must be satisfied for (A) newly added test files named `test_*.py` and (B) newly added `def test_*` functions inside existing Python files. Do not rely on `pytest.mark.polarion` (decorators) as a substitute for the STP docstring link.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-04-30T13:42:43.892Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-04-30T13:42:43.892Z
Learning: In the openshift-virtualization-tests repository, when processing test execution plan requests for PRs that introduce NEW test classes or test files (not just modifications to existing tests), always verify STD-first workflow compliance as part of the analysis: (1) check if the new test class has actual implementation code (assertions, test logic) — if so, it needs a prior STD PR, (2) check if the test file has a module docstring with STP or Jira/RFE link, (3) check if `What this PR does / why we need it:` in the PR description has meaningful content. Flag any violation even when the bot's primary request is only for a test execution plan. These checks should be appended to the test execution plan inline comment.
Applied to files:
.coderabbit.yaml.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-05-18T06:30:28.071Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/architecture.py:25-25
Timestamp: 2026-05-18T06:30:28.071Z
Learning: In RedHatQE/openshift-virtualization-tests, when a PR enforces a new Ruff rule (e.g., PLC0415) and explicitly documents in the PR description that pre-existing violations are suppressed with `# noqa: <rule>` as a temporary measure for follow-up cleanup PRs, do not raise review findings against those suppressions. The suppression policy is intentional and the violations are tracked for future resolution. Only flag new suppressions introduced without such documented justification.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-18T06:31:35.221Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_sanity.py:22-22
Timestamp: 2026-05-18T06:31:35.221Z
Learning: In RedHatQE/openshift-virtualization-tests, when the PLC0415 (import-outside-top-level) Ruff rule is enabled, pre-existing violations in utilities/unittests/ test files are intentionally suppressed with per-line `# noqa: PLC0415` comments as a temporary measure. These suppressions are documented in the PR description and are candidates for dedicated follow-up cleanup PRs. Do not flag these suppressions as violations; they are an accepted part of the incremental rule-enablement strategy. Additionally, imports inside unit test methods in utilities/unittests/ may be intentionally placed there so they execute after patch decorators activate, which matters when modules have import-time side effects.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-18T06:31:18.023Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_pytest_utils.py:270-270
Timestamp: 2026-05-18T06:31:18.023Z
Learning: In RedHatQE/openshift-virtualization-tests, when a Ruff rule enforcement PR (e.g., enabling PLC0415 import-outside-top-level) is introduced, pre-existing violations are intentionally annotated with per-line `# noqa: PLC0415` comments as a temporary suppression to keep CI green. These suppressions are NOT code-quality issues to flag during review — they are an agreed-upon, documented mechanism (stated in the PR's "Special notes for reviewer" section) for incremental cleanup. Do not suggest removing or consolidating these suppressions in the same PR; they are addressed in dedicated follow-up PRs.
Applied to files:
.coderabbit.yaml
📚 Learning: 2025-12-16T14:00:59.076Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-16T14:00:59.076Z
Learning: In the openshift-virtualization-tests repository, when responding to test execution plan requests from openshift-virtualization-qe-bot-3, CodeRabbit must post ONLY an inline review comment on the Files Changed tab and then stop immediately without generating any follow-up comments in the PR discussion thread. No acknowledgment messages, no confirmation of posting, no explanation - silence after posting the inline review equals success. Additional comments create empty/meaningless reviews that clutter the PR.
Applied to files:
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-05-14T14:01:37.353Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-14T14:01:37.353Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when a PR has the `needs-rebase` label and the pre-merge check reports a large number of "new test_*.py files" (e.g., 300+) that do not match the actual PR diff, this is a false positive caused by the check comparing the entire diverged branch file tree against main rather than just the PR diff. Do not raise STP Link Required or Docstring Coverage violations based on such check results — only flag these if the PR diff itself actually adds new test files or functions lacking docstrings.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-06-01T06:43:11.108Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4908
File: utilities/unittests/test_pytest_utils.py:374-374
Timestamp: 2026-06-01T06:43:11.108Z
Learning: In RedHatQE/openshift-virtualization-tests, unit test files located under `utilities/unittests/` (and its subdirectories) do NOT require STP/RFE/Jira traceability links in test function docstrings. The STP/RFE/Jira traceability requirement applies only to integration/functional tests under the `tests/` directory. Do not flag missing STP/RFE/Jira links in any `test_*.py` file under `utilities/unittests/`.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-18T15:22:26.757Z
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-05-18T15:22:26.757Z
Learning: Applies to tests/**/test_*.py : New feature tests must follow the STD-first workflow: (1) STP (Software Test Plan) reviewed and approved, (2) STD (Software Test Description) placeholder tests with docstrings and `__test__ = False` reviewed, (3) Implementation only after STD review. Never submit test implementation without prior STD review.
Applied to files:
.coderabbit.yaml.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-05-18T15:22:26.757Z
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-05-18T15:22:26.757Z
Learning: Applies to tests/**/test_*.py : STD (Software Test Description) docstrings must follow the format in `docs/SOFTWARE_TEST_DESCRIPTION.md`. Required sections: `Preconditions:`, `Steps:`, `Expected:`. Never use alternative section names. Each test verifies ONE thing with ONE `Expected:` assertion. Do NOT include implementation details, fixture names, code references, or variable names. Include STP link at the module, class, or test level. Markers can be at any level. Use `[Markers: ...]` syntax for parametrized markers. Name resources by function (e.g., 'client VM') not labels (e.g., 'VM-A'). Class/module docstring holds shared `Preconditions:`, individual tests add only their own. Include `[NEGATIVE]` indicator for failure scenario tests.
Applied to files:
.coderabbit.yaml.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-05-31T17:55:25.719Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-31T17:55:25.719Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, the STP/RFE/Jira link requirement applies ONLY to test files under the `tests/` directory (feature/STD tests). It does NOT apply to unit tests under `utilities/unittests/`, which cover internal utility functions and bug fixes.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-19T10:17:44.423Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4833
File: tests/network/localnet/migration_stuntime/libstuntime.py:13-13
Timestamp: 2026-05-19T10:17:44.423Z
Learning: In RedHatQE/openshift-virtualization-tests, the PR template validation coding guideline (using the `**` path glob) can fire as a false positive when the PR description already contains all required sections with meaningful content. Do not re-post the PR template violation comment if the PR objectives or description summary shows the required sections (`##### What this PR does / why we need it:`, `##### Which issue(s) this PR fixes:`, `##### Special notes for reviewer:`, `##### jira-ticket:`) are already present and populated. The `**` glob triggers the check per-file in every diff context, so verify the actual PR description content before flagging it as a violation.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-19T17:16:58.838Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4833
File: tests/network/localnet/migration_stuntime/test_migration_stuntime.py:33-38
Timestamp: 2026-05-19T17:16:58.838Z
Learning: In RedHatQE/openshift-virtualization-tests, standalone triple-quoted string blocks (e.g., `"""Preconditions:..."""`) placed at module level after `pytestmark` in test files under `tests/` are the established STD (Standard Test Definition) documentation convention for shared preconditions. Do not flag these as dead code or suggest moving them into module/class docstrings. This pattern is intentional and used consistently across STD test modules.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-06-02T13:45:51.700Z
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: docs/AI_CONTRIBUTION_POLICY.md:0-0
Timestamp: 2026-06-02T13:45:51.700Z
Learning: Applies to docs/**/test_*.py : Do not submit AI-generated Software Test Description (STD) docstring placeholders that skip the required docstring format; ensure complete and properly formatted docstrings
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-20T11:18:46.206Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4867
File: tests/network/libs/stuntime.py:1-1
Timestamp: 2026-05-20T11:18:46.206Z
Learning: In RedHatQE/openshift-virtualization-tests, the PR template validation heuristic has produced repeated false positives on PRs where the description actually contains all required sections (e.g., PR `#4867` by Anatw). When an author explicitly rebuts a missing-template-section flag and confirms all sections are present with meaningful content, treat the original flag as a false positive and do not re-raise it. Be more conservative about firing this HIGH-severity flag — only raise it when a required section header (e.g., "##### What this PR does / why we need it:") is clearly absent or its content is verifiably empty/placeholder-only.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-18T15:22:26.757Z
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-05-18T15:22:26.757Z
Learning: PR discipline: Keep PRs focused on ONE topic. Out-of-scope improvements go in separate PRs. PR title must reflect the actual change, not a side effect. PR description must include motivation (WHY), not just what changed. Mark PR as draft when there are unresolved blockers, failing CI, or open design questions. Never merge a PR with known unresolved issues. DCO (Signed-off-by) REQUIRED on all commits. PR template sections REQUIRED: preserve all sections from `.github/pull_request_template.md`, including 'What this PR does / why we need it:' with meaningful content (not blank/placeholder), 'Which issue(s) this PR fixes:', 'Special notes for reviewer:', and 'jira-ticket:'.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-04-14T16:15:36.741Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4328
File: tests/network/flat_overlay/test_multi_network_policy.py:32-32
Timestamp: 2026-04-14T16:15:36.741Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when reviewing any PR with "Quarantine" in the title or a `quarantine` label, always verify compliance with docs/QUARANTINE_GUIDELINES.md:
- Category 1 (Product Bug): must use `pytest.mark.jira("CNV-XXXXX", run=False)` — the `pytest_jira` plugin conditionally skips the test when the Jira issue is open. Do NOT suggest replacing this with `xfail`.
- Category 2 (Automation Issue): must use `pytest.mark.xfail(run=False, reason=...)` — pytest itself handles the skip.
Flag any quarantine PR that uses the wrong category marker, is missing a Jira ticket reference, or uses `run=False` in the wrong context. Raising these compliance questions is always appropriate for quarantine PRs, even if the final resolution confirms the marker is correct.
Applied to files:
.coderabbit.yaml.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-05-04T13:45:35.675Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-04T13:45:35.675Z
Learning: In the `openshift-virtualization-tests` repository, the PR description template is fulfilled on GitHub's rendered view. Do not flag missing PR template sections based on the raw diff alone.
Applied to files:
.coderabbit.yaml.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-05-10T08:15:30.936Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-10T08:15:30.936Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, the AI Contribution Policy (docs/AI_CONTRIBUTION_POLICY.md) requires that AI-assisted commits use the `Assisted-by:` trailer (e.g., `Assisted-by: Claude Sonnet 4.6 <noreplyanthropic.com>`), NOT `Co-Authored-By:`. Using `Co-Authored-By` for AI tools is a policy violation because it implies human co-authorship. All commits must also have a `Signed-off-by:` trailer for DCO compliance. During PR reviews, flag any commit that uses `Co-Authored-By` with an AI tool name instead of `Assisted-by`.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-04T13:45:41.527Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4725
File: tests/virt/cluster/common_templates/centos/test_centos_os_support.py:78-83
Timestamp: 2026-05-04T13:45:41.527Z
Learning: In RedHatQE/openshift-virtualization-tests, when reviewing lint-cleanup or formatting-only PRs (e.g., removing `# noqa` comments, UTF-8 headers), do not flag pre-existing code patterns such as `.is_connective(tcp_timeout=120)` that were present before the PR. Only raise issues when the PR itself introduces or modifies the problematic construct. This applies to all Python files in the repository.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-18T06:31:00.848Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_bitwarden.py:207-207
Timestamp: 2026-05-18T06:31:00.848Z
Learning: In RedHatQE/openshift-virtualization-tests, when a PR enables a new Ruff/lint rule (e.g., PLC0415 — import-outside-top-level), it is explicitly acceptable to annotate pre-existing violations with per-line `# noqa: PLC0415` as a temporary suppression to keep CI green. These suppressions are intentional and documented in the PR description as candidates for follow-up cleanup PRs. Do NOT flag such `# noqa: PLC0415` comments as policy violations in the context of rule-enablement PRs. Blanket `# noqa` and `per-file-ignores` remain disallowed; only targeted single-rule suppressions are permitted as a transitional measure.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-06-02T13:45:57.660Z
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-06-02T13:45:57.660Z
Learning: Applies to test_*.py : Newly added test files (test_*.py) MUST contain at least one of the following in the module, class, or test function docstring: a line with 'STP:' followed by a URL, a line with 'RFE:' followed by an issue tracker URL, or a line with 'Jira:' followed by an issue tracker URL. If a module-level or class-level STP link already exists, newly added test functions in that same file are considered covered. The pytest.mark.polarion decorator alone does NOT satisfy this requirement.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-06-02T13:45:57.660Z
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-06-02T13:45:57.660Z
Learning: Applies to test_*.py : Newly added test functions (def test_*) in existing test files MUST have at least one of the following in the function or parent class/module docstring: a line with 'STP:' followed by a URL, a line with 'RFE:' followed by an issue tracker URL, or a line with 'Jira:' followed by an issue tracker URL. Module-level or class-level STP links in the same file satisfy this requirement for all test functions. The pytest.mark.polarion decorator alone does NOT satisfy this requirement.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-18T15:22:26.757Z
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-05-18T15:22:26.757Z
Learning: Applies to tests/**/test_*.py : Every new feature test file must include an STP link in the module, class, or test docstring. If there is no STP, the module, class, or test docstring must include a link to the RFE or Jira epic (not support cases) for coverage tracking.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-03-31T08:35:26.070Z
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 4318
File: tests/network/bandwidth/test_bandwidth.py:1-3
Timestamp: 2026-03-31T08:35:26.070Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, the requirement to include an STP link in the module docstring applies only to STD tests for new features. Tests that are support exceptions (linked to SUPPORTEX-* tickets, e.g., SUPPORTEX-29574) do NOT have an associated STP and therefore do NOT require an STP link in the module docstring. Do not flag missing STP links for support exception test files.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-17T13:28:29.374Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4884
File: .coderabbit.yaml:77-81
Timestamp: 2026-05-17T13:28:29.374Z
Learning: In RedHatQE/openshift-virtualization-tests, STP links in test file docstrings use two label variants: "STP:" (followed by URL) or "STP Reference:" (followed by URL). The URL always contains "openshift-virtualization-tests-design-docs/blob/main/stps/". The pytest.mark.polarion("CNV-XXXX") decorator is a separate per-test Polarion identifier and does NOT substitute for the STP docstring link. New test files must have the STP link at module, class, or test docstring level, with RFE: or Jira: links accepted as alternatives when no STP exists.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-04-24T18:35:11.039Z
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 4542
File: tests/network/libs/bgp.py:148-177
Timestamp: 2026-04-24T18:35:11.039Z
Learning: In tests/network/libs/bgp.py, do not flag or recommend changing the `create_evpn_frr_configuration` default `label=None`. This is intentional: EVPN RouteAdvertisements use a narrow `frr_configuration_selector` that targets only EVPN-labeled FRRConfigurations, while BGP RouteAdvertisements use an empty (match-all) selector. Because BGP and EVPN FRRConfigurations apply to different routers with disjoint prefix sets, there is no conflict, so `label` does not need to be required.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-04-27T15:30:20.252Z
Learnt from: SamAlber
Repo: RedHatQE/openshift-virtualization-tests PR: 3748
File: tests/storage/cdi_import/test_import_http.py:506-508
Timestamp: 2026-04-27T15:30:20.252Z
Learning: In tests/storage/cdi_import/test_import_http.py, when using pytest parametrize dicts for image paths, prefer referencing explicit image constants (e.g., Images.Windows.WIN2022_IMG) directly rather than deriving the value via lookups like LATEST_WINDOWS_OS_DICT.get("image_path"). This keeps the test inputs immediately visible for readability and failure triage. Do not request changes that replace these explicit constant references with dynamic lookups from LATEST_WINDOWS_OS_DICT.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-07T11:11:59.120Z
Learnt from: OhadRevah
Repo: RedHatQE/openshift-virtualization-tests PR: 4162
File: tests/install_upgrade_operators/crypto_policy/constants.py:4-4
Timestamp: 2026-05-07T11:11:59.120Z
Learning: In tests/install_upgrade_operators/crypto_policy/constants.py, treat `MANAGED_CRS_LIST` and `MANAGED_CRS_LIST_WITH_AAQ` as intentionally different scopes. `MANAGED_CRS_LIST` is meant for existing crypto-policy tests and intentionally excludes `AAQ` (it includes only KubeVirt, CDI, NetworkAddonsConfig, SSP). The separate `MANAGED_CRS_LIST_WITH_AAQ` explicitly extends it with `AAQ` (e.g., `[*MANAGED_CRS_LIST, AAQ]`) for AAQ-inclusive tests such as fixtures in conftest.py (e.g., `modern_tls_profile_applied`). Do not raise a review issue claiming `AAQ` is missing from `MANAGED_CRS_LIST`; verify tests use the correct list for their intended scope instead.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-18T11:17:18.196Z
Learnt from: RoniKishner
Repo: RedHatQE/openshift-virtualization-tests PR: 4898
File: tests/infrastructure/instance_types/conftest.py:158-162
Timestamp: 2026-05-18T11:17:18.196Z
Learning: In tests/infrastructure/instance_types/conftest.py, do not treat the windows_test_images_namespace_role_binding fixture as an overly permissive RoleBinding. It is intentionally designed to grant the view verb to system:authenticated on the windows-test-images namespace so shared Windows test images are universally readable across all namespaces/tests in the cluster (matching the OpenShift golden-image namespace pattern).
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-20T09:43:31.397Z
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 4932
File: tests/network/l2_bridge/nad_ref_change/lib_helpers.py:78-91
Timestamp: 2026-05-20T09:43:31.397Z
Learning: In tests/network/l2_bridge/nad_ref_change/lib_helpers.py, inside the bridge_vm() helper, do not raise guidance to use zip(..., strict=True) or to add explicit length-mismatch validation around zip(actual_iface_names, nad_names). The nad_names, ip_addresses, and iface_names inputs are intentionally controlled by each callsite and guaranteed to be aligned by design for this internal test fixture builder.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-06-02T11:53:37.264Z
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 5101
File: .github/workflows/request-coderabbit-test-instructions.yml:17-17
Timestamp: 2026-06-02T11:53:37.264Z
Learning: In RedHatQE/openshift-virtualization-tests, the `request-coderabbit-test-instructions.yml` workflow intentionally always creates a new comment (never updates/deduplicates) when either `generate_test_report` or `verified` labels trigger it. This is by design: the PR diff can change between the two triggers (new commits pushed), so each trigger produces an independent TIA snapshot of a different diff state. Deduplicating with `find-comment + edit-mode: replace` would be harmful because it would overwrite the earlier report, removing the ability to compare the two analyses. The historical record of both reports is a desired feature.
Applied to files:
.github/workflows/unresolve-coderabbit-threads.yml.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2025-11-25T01:56:54.902Z
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 2838
File: .github/workflows/net-utils-builder-staging.yml:37-37
Timestamp: 2025-11-25T01:56:54.902Z
Learning: In the openshift-virtualization-tests repository, when renaming container images that are built and used by GitHub Actions workflows, the changes must be done sequentially: first merge the workflow files (.github/workflows/) that update the image name in the CI/CD pipelines, then update the code references (like constants.py and manifest files) in a follow-up PR. This prevents the old workflow from running with mismatched image names during the transition.
Applied to files:
.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2025-06-10T09:41:58.035Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 1168
File: tests/storage/storage_migration/utils.py:92-99
Timestamp: 2025-06-10T09:41:58.035Z
Learning: When CodeRabbit flags technical issues in code reviews, do not resolve the comments without first addressing them and getting a response from the user. Proper procedure is to discuss the technical issues and proposed solutions before marking comments as resolved.
Applied to files:
.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2026-05-19T10:17:37.060Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4833
File: tests/network/localnet/migration_stuntime/libstuntime.py:13-13
Timestamp: 2026-05-19T10:17:37.060Z
Learning: In RedHatQE/openshift-virtualization-tests, when the PR template validation rule is triggered, avoid posting the PR template violation comment if the PR description already contains the required sections with meaningful content:
- `##### What this PR does / why we need it:`
- `##### Which issue(s) this PR fixes:`
- `##### Special notes for reviewer:`
- `##### jira-ticket:`
Because the current implementation can use a broad `**` path glob and re-run the check per diff context/file, reviewers/automation should verify the actual PR description content before flagging it as a violation or duplicating the comment.
Applied to files:
.github/workflows/unresolve-coderabbit-threads.yml.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-05-20T11:18:31.677Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4867
File: tests/network/libs/stuntime.py:1-1
Timestamp: 2026-05-20T11:18:31.677Z
Learning: In RedHatQE/openshift-virtualization-tests, make the PR template validation (HIGH-severity) heuristic more conservative: if the author explicitly rebuts the “missing template section” finding and confirms the required sections are present with meaningful content, treat the original finding as a false positive and do not re-raise it. Only raise the HIGH-severity flag when a required PR section header (e.g., "##### What this PR does / why we need it:") is clearly absent or its content can be verified as empty/placeholder-only.
Applied to files:
.github/workflows/unresolve-coderabbit-threads.yml.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2025-12-22T15:56:00.157Z
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-22T15:56:00.157Z
Learning: In the openshift-virtualization-tests repository, when responding to test execution plan requests from openshift-virtualization-qe-bot-3, do NOT use "REQUEST_CHANGES" review type if the PR author has already marked the PR as verified (e.g., with `/verified` command). Test execution plans are informational guides, not blocking requirements. Use COMMENT event for informational test plans, or only REQUEST_CHANGES if there are actual code issues that need to be addressed before merging.
Applied to files:
.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2025-12-16T14:06:22.391Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-16T14:06:22.391Z
Learning: In the openshift-virtualization-tests repository, when posting test execution plan inline review comments using GitHub API, the full test execution plan content must go in the `comments[].body` field (which appears on Files Changed tab), NOT in the top-level `body` field (which appears in PR discussion thread). The top-level `body` field should be omitted or left empty to avoid posting redundant comments in the PR discussion thread.
Applied to files:
.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-03-17T01:32:13.674Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:13.674Z
Learning: In RedHatQE/openshift-virtualization-tests, maintainer dshchedr prefers CodeRabbit to post targeted inline comments on the Files changed tab for each applicable location rather than aggregating multiple issues into a single discussion thread.
Applied to files:
.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-03-17T01:32:13.674Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:13.674Z
Learning: In RedHatQE/openshift-virtualization-tests, maintainer dshchedr prefers CodeRabbit to post targeted inline comments on the Files Changed tab at each applicable location rather than aggregating multiple issues into a single PR discussion thread reply.
Applied to files:
.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2025-12-16T10:28:54.212Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-16T10:28:54.212Z
Learning: In the openshift-virtualization-tests repository, when determining smoke test impact for test execution plans, only set "Run smoke tests: True" if there is a verified, traceable dependency path from smoke tests to the changed code. Session-scoped fixtures or infrastructure-sounding changes do NOT automatically imply smoke test impact - the dependency chain must be explicitly verified using shell scripts before recommending smoke test execution.
Applied to files:
.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-06-02T13:45:51.700Z
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: docs/AI_CONTRIBUTION_POLICY.md:0-0
Timestamp: 2026-06-02T13:45:51.700Z
Learning: Applies to docs/**/test_*.py : Do not submit AI-generated tests without verifying marker completeness and proper fixture scoping per project test standards
Applied to files:
.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-05-18T15:22:26.757Z
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-05-18T15:22:26.757Z
Learning: Applies to tests/**/test_*.py : When reviewing quarantine PRs, verify the quarantine mechanism matches the failure category: Product bugs use `pytest.mark.jira('CNV-XXXXX', run=False)`. Automation issues or unidentified failures use `pytest.mark.xfail(reason=f'{QUARANTINED}: ...', run=False)`. `pytest.mark.jira` is conditional (auto-re-enables when Jira resolved). `pytest.mark.xfail` quarantine is unconditional (requires manual de-quarantine PR).
Applied to files:
.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-05-18T15:22:26.757Z
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-05-18T15:22:26.757Z
Learning: Applies to tests/**/test_*.py : Tests directory organization: Each feature must have its own subdirectory under component (e.g., `tests/network/ipv6/`). Tests belong under the feature they test—do NOT create standalone directories for cross-cutting concerns. Test file naming REQUIRED: always use `test_<functionality>.py` format. Place local helper utils in `<feature_dir>/utils.py`. Place local fixtures in `<feature_dir>/conftest.py`. Move to shared location (`utilities/` or `tests/conftest.py`) only when used by different team directories.
Applied to files:
.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2026-01-07T09:52:22.771Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3358
File: tests/network/sriov/test_sriov.py:21-21
Timestamp: 2026-01-07T09:52:22.771Z
Learning: For PRs that only remove or modify pytest markers (like removing pytest.mark.post_upgrade) without changing test logic in openshift-virtualization-tests, collection verification using `pytest --collect-only` is sufficient. Full test execution is not required to verify marker-only changes.
Applied to files:
.github/workflows/request-coderabbit-test-instructions.yml
📚 Learning: 2025-12-22T16:27:44.327Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3196
File: tests/network/upgrade/test_upgrade_network.py:4-4
Timestamp: 2025-12-22T16:27:44.327Z
Learning: For PRs that remove test cases (especially redundant test cleanup PRs in openshift-virtualization-tests), test collection verification (pytest --collect-only showing selected/deselected counts) is sufficient to confirm the removal was clean and the test module remains functional. Full test execution is not required for test deletion PRs.
Applied to files:
.github/workflows/request-coderabbit-test-instructions.yml
🪛 zizmor (1.25.2)
.github/workflows/unresolve-coderabbit-threads.yml
[error] 16-16: overly broad permissions (excessive-permissions): pull-requests: write is overly broad at the workflow level
(excessive-permissions)
[error] 11-13: use of fundamentally insecure workflow trigger (dangerous-triggers): pull_request_target is almost always used insecurely
(dangerous-triggers)
[warning] 16-16: permissions without explanatory comments (undocumented-permissions): needs an explanatory comment
(undocumented-permissions)
[info] 20-20: workflow or action definition without a name (anonymous-definition): this job
(anonymous-definition)
[warning] 11-13: insufficient job-level concurrency limits (concurrency-limits): workflow is missing concurrency setting
(concurrency-limits)
🔇 Additional comments (4)
.coderabbit.yaml (1)
21-31: LGTM!Also applies to: 103-109
.github/workflows/request-coderabbit-test-instructions.yml (1)
109-114: LGTM!Also applies to: 117-142
.github/workflows/unresolve-coderabbit-threads.yml (2)
19-23: LGTM!
77-102: LGTM!
Collaborator
Author
what??? |
Contributor
🧩 Analysis chain🏁 Script executed: #!/bin/bash
cat .github/workflows/request-coderabbit-test-instructions.yml
echo "---"
cat .github/workflows/unresolve-coderabbit-threads.ymlLength of output: 15923
Nothing is broken in your workflow files — the error is on my side. I can retry the summary generation now: 🧠 Learnings used✅ Action performedSummary regeneration triggered. |
RoniKishner
reviewed
Jun 3, 2026
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
.github/workflows/unresolve-coderabbit-threads.yml (2)
44-57:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftHIGH: Unpaginated
reviewThreads(first: 100)can miss threads on noisy PRs.In
.github/workflows/unresolve-coderabbit-threads.yml, the GraphQL query fetches only the first 100reviewThreadsand never paginates, even though the workflow comment says it’s “Fetch all resolved review threads.” Any qualifying resolved threads beyond that first page are never reconsidered/unresolved, weakening the policy where it matters most.Use
pageInfo { hasNextPage endCursor }and paginate withafter(loop in the bash step). Example query shape:query($owner: String!, $repo: String!, $pr: Int!, $after: String) { repository(owner: $owner, name: $repo) { pullRequest(number: $pr) { reviewThreads(first: 100, after: $after) { nodes { id isResolved comments(last: 5) { nodes { author { login } body } } } pageInfo { hasNextPage endCursor } } } } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/unresolve-coderabbit-threads.yml around lines 44 - 57, The GraphQL query uses reviewThreads(first: 100) without pagination so it can miss threads on noisy PRs; update the query used in the workflow to accept an optional cursor variable (e.g., $after) and include pageInfo { hasNextPage endCursor } alongside reviewThreads(first: 100, after: $after), then implement a loop in the bash step that repeatedly calls the query with after set to endCursor until hasNextPage is false, collecting/processing each page of reviewThreads (use the same variable names like $owner, $repo, $pr, $after and the field names reviewThreads, pageInfo, endCursor, hasNextPage to locate and change the query and the calling logic).
48-80:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMEDIUM: Fix thread opener detection and add pagination in unresolve workflow
The workflow’s detector intends to unresolve threads where a CodeRabbit-resolved conversation lacked a substantive response, but it can miss valid threads:
- Correctness:
comments(last: 5)is used, while jq checks.comments.nodes[0]forcoderabbitai[bot]—nodes[0]is only the oldest comment within that last-5 window, not necessarily the thread opener. If the opener falls outside the slice, the thread is skipped (.github/workflows/unresolve-coderabbit-threads.ymllines ~48 and ~67).- Reliability:
reviewThreads(first: 100)is capped with no pagination, so PRs with >100 review threads won’t all be processed (line ~44).Minimal fix (separate opener vs recent comments)
- comments(last: 5) { + opening_comment: comments(first: 1) { + nodes { + author { + login + } + } + } + recent_comments: comments(last: 5) { nodes { author { login @@ - | select(.comments.nodes[0].author.login == "coderabbitai[bot]") + | select(.opening_comment.nodes[0].author.login == "coderabbitai[bot]") | select( @@ - (.comments.nodes[-1].author.login == "coderabbitai[bot]") + (.recent_comments.nodes[-1].author.login == "coderabbitai[bot]") or @@ - (.comments.nodes[-1].author.login != "coderabbitai[bot]") + (.recent_comments.nodes[-1].author.login != "coderabbitai[bot]") and - ((.comments.nodes[-1].body | length) < 15) + ((.recent_comments.nodes[-1].body | length) < 15) ) ) | .idImplement pagination for
reviewThreads(and optionallycommentsif needed) to avoid partial coverage.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/unresolve-coderabbit-threads.yml around lines 48 - 80, The detector misses thread openers and can skip threads when there are >100 threads; fix by adding GraphQL pagination for reviewThreads (iterate using pageInfo.hasNextPage and endCursor instead of relying on reviewThreads(first: 100) only) and fetch a larger, ordered comment window so opener detection is correct: change comments(last: 5) to comments(first: 100, orderBy: {field: CREATED_AT, direction: ASC}) (or paginate comments similarly) so .comments.nodes[0] reliably represents the opener and .comments.nodes[-1] the most recent comment; update the jq selectors accordingly to continue checking opener (.comments.nodes[0].author.login == "coderabbitai[bot]") and last comment length checks.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.github/workflows/unresolve-coderabbit-threads.yml:
- Around line 44-57: The GraphQL query uses reviewThreads(first: 100) without
pagination so it can miss threads on noisy PRs; update the query used in the
workflow to accept an optional cursor variable (e.g., $after) and include
pageInfo { hasNextPage endCursor } alongside reviewThreads(first: 100, after:
$after), then implement a loop in the bash step that repeatedly calls the query
with after set to endCursor until hasNextPage is false, collecting/processing
each page of reviewThreads (use the same variable names like $owner, $repo, $pr,
$after and the field names reviewThreads, pageInfo, endCursor, hasNextPage to
locate and change the query and the calling logic).
- Around line 48-80: The detector misses thread openers and can skip threads
when there are >100 threads; fix by adding GraphQL pagination for reviewThreads
(iterate using pageInfo.hasNextPage and endCursor instead of relying on
reviewThreads(first: 100) only) and fetch a larger, ordered comment window so
opener detection is correct: change comments(last: 5) to comments(first: 100,
orderBy: {field: CREATED_AT, direction: ASC}) (or paginate comments similarly)
so .comments.nodes[0] reliably represents the opener and .comments.nodes[-1] the
most recent comment; update the jq selectors accordingly to continue checking
opener (.comments.nodes[0].author.login == "coderabbitai[bot]") and last comment
length checks.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 642a2eee-e23f-4c27-810a-65f2d18c2be7
📒 Files selected for processing (2)
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: verify-bugs-are-open
- GitHub Check: tox
- GitHub Check: build-container
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
🧰 Additional context used
🧠 Learnings (32)
📓 Common learnings
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:13.674Z
Learning: In RedHatQE/openshift-virtualization-tests, maintainer dshchedr prefers CodeRabbit to post targeted inline comments on the Files Changed tab at each applicable location rather than aggregating multiple issues into a single PR discussion thread reply.
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:13.674Z
Learning: In RedHatQE/openshift-virtualization-tests, maintainer dshchedr prefers CodeRabbit to post targeted inline comments on the Files changed tab for each applicable location rather than aggregating multiple issues into a single discussion thread.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-16T14:00:59.076Z
Learning: In the openshift-virtualization-tests repository, when responding to test execution plan requests from openshift-virtualization-qe-bot-3, CodeRabbit must post ONLY an inline review comment on the Files Changed tab and then stop immediately without generating any follow-up comments in the PR discussion thread. No acknowledgment messages, no confirmation of posting, no explanation - silence after posting the inline review equals success. Additional comments create empty/meaningless reviews that clutter the PR.
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-04-30T13:42:43.892Z
Learning: In the openshift-virtualization-tests repository, when processing test execution plan requests for PRs that introduce NEW test classes or test files (not just modifications to existing tests), always verify STD-first workflow compliance as part of the analysis: (1) check if the new test class has actual implementation code (assertions, test logic) — if so, it needs a prior STD PR, (2) check if the test file has a module docstring with STP or Jira/RFE link, (3) check if `What this PR does / why we need it:` in the PR description has meaningful content. Flag any violation even when the bot's primary request is only for a test execution plan. These checks should be appended to the test execution plan inline comment.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3228
File: .coderabbit.yaml:30-41
Timestamp: 2026-01-05T10:33:55.037Z
Learning: In the openshift-virtualization-tests repository, user rnetser prefers minimal pre-merge checks in CodeRabbit configuration: only docstrings enforcement (80% threshold) is needed, not title or description checks.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-29T11:02:22.804Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, version-bump PRs that update the main branch to reference a new CNV release (e.g., "Update project `main` to reference cnv 5.0") do NOT add new test files or test modules. These PRs only update metadata (pyproject.toml, tox.ini, .coderabbit.yaml) and clean up resolved Jira fixtures/markers in existing files. The "Stp Link Required" pre-merge check may incorrectly flag hundreds of "newly added" test files as a false positive due to the base-branch comparison shifting after branching — this should be ignored for such PRs.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 2469
File: utilities/sanity.py:139-142
Timestamp: 2025-11-08T07:36:57.616Z
Learning: In the openshift-virtualization-tests repository, user rnetser prefers to keep refactoring PRs (like PR `#2469`) strictly focused on moving/organizing code into more granular modules without adding new functionality, error handling, or behavioral changes. Such improvements should be handled in separate PRs.
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-22T15:56:00.157Z
Learning: In the openshift-virtualization-tests repository, when responding to test execution plan requests from openshift-virtualization-qe-bot-3, do NOT use "REQUEST_CHANGES" review type if the PR author has already marked the PR as verified (e.g., with `/verified` command). Test execution plans are informational guides, not blocking requirements. Use COMMENT event for informational test plans, or only REQUEST_CHANGES if there are actual code issues that need to be addressed before merging.
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 5101
File: .github/workflows/request-coderabbit-test-instructions.yml:17-17
Timestamp: 2026-06-02T11:53:37.264Z
Learning: In RedHatQE/openshift-virtualization-tests, the `request-coderabbit-test-instructions.yml` workflow intentionally always creates a new comment (never updates/deduplicates) when either `generate_test_report` or `verified` labels trigger it. This is by design: the PR diff can change between the two triggers (new commits pushed), so each trigger produces an independent TIA snapshot of a different diff state. Deduplicating with `find-comment + edit-mode: replace` would be harmful because it would overwrite the earlier report, removing the ability to compare the two analyses. The historical record of both reports is a desired feature.
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4867
File: tests/network/libs/stuntime.py:1-1
Timestamp: 2026-05-20T11:18:46.206Z
Learning: In RedHatQE/openshift-virtualization-tests, the PR template validation heuristic has produced repeated false positives on PRs where the description actually contains all required sections (e.g., PR `#4867` by Anatw). When an author explicitly rebuts a missing-template-section flag and confirms all sections are present with meaningful content, treat the original flag as a false positive and do not re-raise it. Be more conservative about firing this HIGH-severity flag — only raise it when a required section header (e.g., "##### What this PR does / why we need it:") is clearly absent or its content is verifiably empty/placeholder-only.
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-05-18T15:22:26.757Z
Learning: Applies to tests/**/test_*.py : New feature tests must follow the STD-first workflow: (1) STP (Software Test Plan) reviewed and approved, (2) STD (Software Test Description) placeholder tests with docstrings and `__test__ = False` reviewed, (3) Implementation only after STD review. Never submit test implementation without prior STD review.
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/architecture.py:25-25
Timestamp: 2026-05-18T06:30:28.071Z
Learning: In RedHatQE/openshift-virtualization-tests, when a PR enforces a new Ruff rule (e.g., PLC0415) and explicitly documents in the PR description that pre-existing violations are suppressed with `# noqa: <rule>` as a temporary measure for follow-up cleanup PRs, do not raise review findings against those suppressions. The suppression policy is intentional and the violations are tracked for future resolution. Only flag new suppressions introduced without such documented justification.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-03T15:38:39.607Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, lint cleanup PRs (e.g. pre-commit hook bumps with ruff/mypy) should NOT be asked to fix pre-existing code patterns that are outside the diff. Issues like bare `except Exception`, missing `raise ... from`, timezone-naive datetime comparisons, implicit `return None`, TRY003 long error messages in exceptions, and boolean accumulator logic in utility functions are pre-existing and should only be addressed in dedicated follow-up PRs, not in automated lint-cleanup PRs.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4701
File: tests/virt/node/general/test_windows_vtpm_bitlocker.py:50-52
Timestamp: 2026-05-03T15:38:12.880Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, do not flag pre-existing code patterns (e.g., nested `if` statements, unnecessary list comprehensions) that are not introduced by the current PR when the PR is a lint cleanup/tooling bump. Such improvements should be deferred to dedicated follow-up PRs. This applies to all Python files under tests/ and utilities/.
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_sanity.py:22-22
Timestamp: 2026-05-18T06:31:35.221Z
Learning: In RedHatQE/openshift-virtualization-tests, when the PLC0415 (import-outside-top-level) Ruff rule is enabled, pre-existing violations in utilities/unittests/ test files are intentionally suppressed with per-line `# noqa: PLC0415` comments as a temporary measure. These suppressions are documented in the PR description and are candidates for dedicated follow-up cleanup PRs. Do not flag these suppressions as violations; they are an accepted part of the incremental rule-enablement strategy. Additionally, imports inside unit test methods in utilities/unittests/ may be intentionally placed there so they execute after patch decorators activate, which matters when modules have import-time side effects.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 1244
File: utilities/os_utils.py:248-250
Timestamp: 2025-07-08T05:51:06.314Z
Learning: User rnetser prefers fail-fast code design where functions should raise exceptions immediately when encountering unexpected or malformed input, rather than implementing defensive error handling with fallbacks. They value making problems visible immediately during development over graceful degradation.
📚 Learning: 2026-01-05T10:33:55.037Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3228
File: .coderabbit.yaml:30-41
Timestamp: 2026-01-05T10:33:55.037Z
Learning: In the openshift-virtualization-tests repository, user rnetser prefers minimal pre-merge checks in CodeRabbit configuration: only docstrings enforcement (80% threshold) is needed, not title or description checks.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-04-30T13:42:43.892Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-04-30T13:42:43.892Z
Learning: In the openshift-virtualization-tests repository, when processing test execution plan requests for PRs that introduce NEW test classes or test files (not just modifications to existing tests), always verify STD-first workflow compliance as part of the analysis: (1) check if the new test class has actual implementation code (assertions, test logic) — if so, it needs a prior STD PR, (2) check if the test file has a module docstring with STP or Jira/RFE link, (3) check if `What this PR does / why we need it:` in the PR description has meaningful content. Flag any violation even when the bot's primary request is only for a test execution plan. These checks should be appended to the test execution plan inline comment.
Applied to files:
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2026-06-01T06:43:11.108Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4908
File: utilities/unittests/test_pytest_utils.py:374-374
Timestamp: 2026-06-01T06:43:11.108Z
Learning: In RedHatQE/openshift-virtualization-tests, unit test files located under `utilities/unittests/` (and its subdirectories) do NOT require STP/RFE/Jira traceability links in test function docstrings. The STP/RFE/Jira traceability requirement applies only to integration/functional tests under the `tests/` directory. Do not flag missing STP/RFE/Jira links in any `test_*.py` file under `utilities/unittests/`.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-17T14:29:44.166Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4884
File: .coderabbit.yaml:77-81
Timestamp: 2026-05-17T14:29:44.166Z
Learning: For RedHatQE/openshift-virtualization-tests’ `.coderabbit.yaml` custom check (“STP link required”), ensure the STP/RFE/Jira link appears in at least one of these docstring locations: the module-level docstring, an enclosing class docstring, or the individual test function docstring. This satisfies the requirement for all tests covered by that scope. The check must be satisfied for (A) newly added test files named `test_*.py` and (B) newly added `def test_*` functions inside existing Python files. Do not rely on `pytest.mark.polarion` (decorators) as a substitute for the STP docstring link.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-29T11:02:22.804Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-29T11:02:22.804Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, version-bump PRs that update the main branch to reference a new CNV release (e.g., "Update project `main` to reference cnv 5.0") do NOT add new test files or test modules. These PRs only update metadata (pyproject.toml, tox.ini, .coderabbit.yaml) and clean up resolved Jira fixtures/markers in existing files. The "Stp Link Required" pre-merge check may incorrectly flag hundreds of "newly added" test files as a false positive due to the base-branch comparison shifting after branching — this should be ignored for such PRs.
Applied to files:
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2026-05-14T14:01:37.353Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-14T14:01:37.353Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when a PR has the `needs-rebase` label and the pre-merge check reports a large number of "new test_*.py files" (e.g., 300+) that do not match the actual PR diff, this is a false positive caused by the check comparing the entire diverged branch file tree against main rather than just the PR diff. Do not raise STP Link Required or Docstring Coverage violations based on such check results — only flag these if the PR diff itself actually adds new test files or functions lacking docstrings.
Applied to files:
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2026-05-31T17:55:25.719Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-31T17:55:25.719Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, the STP/RFE/Jira link requirement applies ONLY to test files under the `tests/` directory (feature/STD tests). It does NOT apply to unit tests under `utilities/unittests/`, which cover internal utility functions and bug fixes.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-18T06:30:28.071Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/architecture.py:25-25
Timestamp: 2026-05-18T06:30:28.071Z
Learning: In RedHatQE/openshift-virtualization-tests, when a PR enforces a new Ruff rule (e.g., PLC0415) and explicitly documents in the PR description that pre-existing violations are suppressed with `# noqa: <rule>` as a temporary measure for follow-up cleanup PRs, do not raise review findings against those suppressions. The suppression policy is intentional and the violations are tracked for future resolution. Only flag new suppressions introduced without such documented justification.
Applied to files:
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2026-05-18T06:31:35.221Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_sanity.py:22-22
Timestamp: 2026-05-18T06:31:35.221Z
Learning: In RedHatQE/openshift-virtualization-tests, when the PLC0415 (import-outside-top-level) Ruff rule is enabled, pre-existing violations in utilities/unittests/ test files are intentionally suppressed with per-line `# noqa: PLC0415` comments as a temporary measure. These suppressions are documented in the PR description and are candidates for dedicated follow-up cleanup PRs. Do not flag these suppressions as violations; they are an accepted part of the incremental rule-enablement strategy. Additionally, imports inside unit test methods in utilities/unittests/ may be intentionally placed there so they execute after patch decorators activate, which matters when modules have import-time side effects.
Applied to files:
.coderabbit.yaml
📚 Learning: 2025-12-16T14:00:59.076Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-16T14:00:59.076Z
Learning: In the openshift-virtualization-tests repository, when responding to test execution plan requests from openshift-virtualization-qe-bot-3, CodeRabbit must post ONLY an inline review comment on the Files Changed tab and then stop immediately without generating any follow-up comments in the PR discussion thread. No acknowledgment messages, no confirmation of posting, no explanation - silence after posting the inline review equals success. Additional comments create empty/meaningless reviews that clutter the PR.
Applied to files:
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2025-06-10T09:41:58.035Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 1168
File: tests/storage/storage_migration/utils.py:92-99
Timestamp: 2025-06-10T09:41:58.035Z
Learning: When CodeRabbit flags technical issues in code reviews, do not resolve the comments without first addressing them and getting a response from the user. Proper procedure is to discuss the technical issues and proposed solutions before marking comments as resolved.
Applied to files:
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2026-05-20T11:18:46.206Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4867
File: tests/network/libs/stuntime.py:1-1
Timestamp: 2026-05-20T11:18:46.206Z
Learning: In RedHatQE/openshift-virtualization-tests, the PR template validation heuristic has produced repeated false positives on PRs where the description actually contains all required sections (e.g., PR `#4867` by Anatw). When an author explicitly rebuts a missing-template-section flag and confirms all sections are present with meaningful content, treat the original flag as a false positive and do not re-raise it. Be more conservative about firing this HIGH-severity flag — only raise it when a required section header (e.g., "##### What this PR does / why we need it:") is clearly absent or its content is verifiably empty/placeholder-only.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-10T08:15:30.936Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2026-05-10T08:15:30.936Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, the AI Contribution Policy (docs/AI_CONTRIBUTION_POLICY.md) requires that AI-assisted commits use the `Assisted-by:` trailer (e.g., `Assisted-by: Claude Sonnet 4.6 <noreplyanthropic.com>`), NOT `Co-Authored-By:`. Using `Co-Authored-By` for AI tools is a policy violation because it implies human co-authorship. All commits must also have a `Signed-off-by:` trailer for DCO compliance. During PR reviews, flag any commit that uses `Co-Authored-By` with an AI tool name instead of `Assisted-by`.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-19T10:17:44.423Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4833
File: tests/network/localnet/migration_stuntime/libstuntime.py:13-13
Timestamp: 2026-05-19T10:17:44.423Z
Learning: In RedHatQE/openshift-virtualization-tests, the PR template validation coding guideline (using the `**` path glob) can fire as a false positive when the PR description already contains all required sections with meaningful content. Do not re-post the PR template violation comment if the PR objectives or description summary shows the required sections (`##### What this PR does / why we need it:`, `##### Which issue(s) this PR fixes:`, `##### Special notes for reviewer:`, `##### jira-ticket:`) are already present and populated. The `**` glob triggers the check per-file in every diff context, so verify the actual PR description content before flagging it as a violation.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-18T15:22:26.757Z
Learnt from: CR
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-05-18T15:22:26.757Z
Learning: PR discipline: Keep PRs focused on ONE topic. Out-of-scope improvements go in separate PRs. PR title must reflect the actual change, not a side effect. PR description must include motivation (WHY), not just what changed. Mark PR as draft when there are unresolved blockers, failing CI, or open design questions. Never merge a PR with known unresolved issues. DCO (Signed-off-by) REQUIRED on all commits. PR template sections REQUIRED: preserve all sections from `.github/pull_request_template.md`, including 'What this PR does / why we need it:' with meaningful content (not blank/placeholder), 'Which issue(s) this PR fixes:', 'Special notes for reviewer:', and 'jira-ticket:'.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-18T06:31:18.023Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_pytest_utils.py:270-270
Timestamp: 2026-05-18T06:31:18.023Z
Learning: In RedHatQE/openshift-virtualization-tests, when a Ruff rule enforcement PR (e.g., enabling PLC0415 import-outside-top-level) is introduced, pre-existing violations are intentionally annotated with per-line `# noqa: PLC0415` comments as a temporary suppression to keep CI green. These suppressions are NOT code-quality issues to flag during review — they are an agreed-upon, documented mechanism (stated in the PR's "Special notes for reviewer" section) for incremental cleanup. Do not suggest removing or consolidating these suppressions in the same PR; they are addressed in dedicated follow-up PRs.
Applied to files:
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2026-05-18T06:31:20.848Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_hco.py:501-501
Timestamp: 2026-05-18T06:31:20.848Z
Learning: In RedHatQE/openshift-virtualization-tests, when a PR enables a new Ruff/lint rule (e.g., PLC0415 import-outside-top-level) and annotates pre-existing violations with `# noqa: <RULE>` as a temporary measure to keep CI green (with follow-up PRs planned for cleanup), do NOT flag those suppressions as issues. The suppressions are intentional, documented in the PR description under "Special notes for reviewer", and should be treated as deferred technical debt — not immediate violations. Only flag `# noqa: PLC0415` suppressions that are newly introduced *without* such documented intent.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-03-17T01:32:13.674Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:13.674Z
Learning: In RedHatQE/openshift-virtualization-tests, maintainer dshchedr prefers CodeRabbit to post targeted inline comments on the Files Changed tab at each applicable location rather than aggregating multiple issues into a single PR discussion thread reply.
Applied to files:
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2026-03-17T01:32:13.674Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:13.674Z
Learning: In RedHatQE/openshift-virtualization-tests, maintainer dshchedr prefers CodeRabbit to post targeted inline comments on the Files changed tab for each applicable location rather than aggregating multiple issues into a single discussion thread.
Applied to files:
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2025-12-22T15:56:00.157Z
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-22T15:56:00.157Z
Learning: In the openshift-virtualization-tests repository, when responding to test execution plan requests from openshift-virtualization-qe-bot-3, do NOT use "REQUEST_CHANGES" review type if the PR author has already marked the PR as verified (e.g., with `/verified` command). Test execution plans are informational guides, not blocking requirements. Use COMMENT event for informational test plans, or only REQUEST_CHANGES if there are actual code issues that need to be addressed before merging.
Applied to files:
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2026-06-02T11:53:37.264Z
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 5101
File: .github/workflows/request-coderabbit-test-instructions.yml:17-17
Timestamp: 2026-06-02T11:53:37.264Z
Learning: In RedHatQE/openshift-virtualization-tests, the `request-coderabbit-test-instructions.yml` workflow intentionally always creates a new comment (never updates/deduplicates) when either `generate_test_report` or `verified` labels trigger it. This is by design: the PR diff can change between the two triggers (new commits pushed), so each trigger produces an independent TIA snapshot of a different diff state. Deduplicating with `find-comment + edit-mode: replace` would be harmful because it would overwrite the earlier report, removing the ability to compare the two analyses. The historical record of both reports is a desired feature.
Applied to files:
.coderabbit.yaml.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2026-04-14T16:15:36.741Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4328
File: tests/network/flat_overlay/test_multi_network_policy.py:32-32
Timestamp: 2026-04-14T16:15:36.741Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when reviewing any PR with "Quarantine" in the title or a `quarantine` label, always verify compliance with docs/QUARANTINE_GUIDELINES.md:
- Category 1 (Product Bug): must use `pytest.mark.jira("CNV-XXXXX", run=False)` — the `pytest_jira` plugin conditionally skips the test when the Jira issue is open. Do NOT suggest replacing this with `xfail`.
- Category 2 (Automation Issue): must use `pytest.mark.xfail(run=False, reason=...)` — pytest itself handles the skip.
Flag any quarantine PR that uses the wrong category marker, is missing a Jira ticket reference, or uses `run=False` in the wrong context. Raising these compliance questions is always appropriate for quarantine PRs, even if the final resolution confirms the marker is correct.
Applied to files:
.coderabbit.yaml
📚 Learning: 2025-12-16T14:06:22.391Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-16T14:06:22.391Z
Learning: In the openshift-virtualization-tests repository, when posting test execution plan inline review comments using GitHub API, the full test execution plan content must go in the `comments[].body` field (which appears on Files Changed tab), NOT in the top-level `body` field (which appears in PR discussion thread). The top-level `body` field should be omitted or left empty to avoid posting redundant comments in the PR discussion thread.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-04-24T18:35:11.039Z
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 4542
File: tests/network/libs/bgp.py:148-177
Timestamp: 2026-04-24T18:35:11.039Z
Learning: In tests/network/libs/bgp.py, do not flag or recommend changing the `create_evpn_frr_configuration` default `label=None`. This is intentional: EVPN RouteAdvertisements use a narrow `frr_configuration_selector` that targets only EVPN-labeled FRRConfigurations, while BGP RouteAdvertisements use an empty (match-all) selector. Because BGP and EVPN FRRConfigurations apply to different routers with disjoint prefix sets, there is no conflict, so `label` does not need to be required.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-04-27T15:30:20.252Z
Learnt from: SamAlber
Repo: RedHatQE/openshift-virtualization-tests PR: 3748
File: tests/storage/cdi_import/test_import_http.py:506-508
Timestamp: 2026-04-27T15:30:20.252Z
Learning: In tests/storage/cdi_import/test_import_http.py, when using pytest parametrize dicts for image paths, prefer referencing explicit image constants (e.g., Images.Windows.WIN2022_IMG) directly rather than deriving the value via lookups like LATEST_WINDOWS_OS_DICT.get("image_path"). This keeps the test inputs immediately visible for readability and failure triage. Do not request changes that replace these explicit constant references with dynamic lookups from LATEST_WINDOWS_OS_DICT.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-07T11:11:59.120Z
Learnt from: OhadRevah
Repo: RedHatQE/openshift-virtualization-tests PR: 4162
File: tests/install_upgrade_operators/crypto_policy/constants.py:4-4
Timestamp: 2026-05-07T11:11:59.120Z
Learning: In tests/install_upgrade_operators/crypto_policy/constants.py, treat `MANAGED_CRS_LIST` and `MANAGED_CRS_LIST_WITH_AAQ` as intentionally different scopes. `MANAGED_CRS_LIST` is meant for existing crypto-policy tests and intentionally excludes `AAQ` (it includes only KubeVirt, CDI, NetworkAddonsConfig, SSP). The separate `MANAGED_CRS_LIST_WITH_AAQ` explicitly extends it with `AAQ` (e.g., `[*MANAGED_CRS_LIST, AAQ]`) for AAQ-inclusive tests such as fixtures in conftest.py (e.g., `modern_tls_profile_applied`). Do not raise a review issue claiming `AAQ` is missing from `MANAGED_CRS_LIST`; verify tests use the correct list for their intended scope instead.
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-18T11:17:18.196Z
Learnt from: RoniKishner
Repo: RedHatQE/openshift-virtualization-tests PR: 4898
File: tests/infrastructure/instance_types/conftest.py:158-162
Timestamp: 2026-05-18T11:17:18.196Z
Learning: In tests/infrastructure/instance_types/conftest.py, do not treat the windows_test_images_namespace_role_binding fixture as an overly permissive RoleBinding. It is intentionally designed to grant the view verb to system:authenticated on the windows-test-images namespace so shared Windows test images are universally readable across all namespaces/tests in the cluster (matching the OpenShift golden-image namespace pattern).
Applied to files:
.coderabbit.yaml
📚 Learning: 2026-05-20T09:43:31.397Z
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 4932
File: tests/network/l2_bridge/nad_ref_change/lib_helpers.py:78-91
Timestamp: 2026-05-20T09:43:31.397Z
Learning: In tests/network/l2_bridge/nad_ref_change/lib_helpers.py, inside the bridge_vm() helper, do not raise guidance to use zip(..., strict=True) or to add explicit length-mismatch validation around zip(actual_iface_names, nad_names). The nad_names, ip_addresses, and iface_names inputs are intentionally controlled by each callsite and guaranteed to be aligned by design for this internal test fixture builder.
Applied to files:
.coderabbit.yaml
📚 Learning: 2025-11-25T01:56:54.902Z
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 2838
File: .github/workflows/net-utils-builder-staging.yml:37-37
Timestamp: 2025-11-25T01:56:54.902Z
Learning: In the openshift-virtualization-tests repository, when renaming container images that are built and used by GitHub Actions workflows, the changes must be done sequentially: first merge the workflow files (.github/workflows/) that update the image name in the CI/CD pipelines, then update the code references (like constants.py and manifest files) in a follow-up PR. This prevents the old workflow from running with mismatched image names during the transition.
Applied to files:
.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2026-05-20T11:18:31.677Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4867
File: tests/network/libs/stuntime.py:1-1
Timestamp: 2026-05-20T11:18:31.677Z
Learning: In RedHatQE/openshift-virtualization-tests, make the PR template validation (HIGH-severity) heuristic more conservative: if the author explicitly rebuts the “missing template section” finding and confirms the required sections are present with meaningful content, treat the original finding as a false positive and do not re-raise it. Only raise the HIGH-severity flag when a required PR section header (e.g., "##### What this PR does / why we need it:") is clearly absent or its content can be verified as empty/placeholder-only.
Applied to files:
.github/workflows/unresolve-coderabbit-threads.yml
📚 Learning: 2026-05-19T10:17:37.060Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4833
File: tests/network/localnet/migration_stuntime/libstuntime.py:13-13
Timestamp: 2026-05-19T10:17:37.060Z
Learning: In RedHatQE/openshift-virtualization-tests, when the PR template validation rule is triggered, avoid posting the PR template violation comment if the PR description already contains the required sections with meaningful content:
- `##### What this PR does / why we need it:`
- `##### Which issue(s) this PR fixes:`
- `##### Special notes for reviewer:`
- `##### jira-ticket:`
Because the current implementation can use a broad `**` path glob and re-run the check per diff context/file, reviewers/automation should verify the actual PR description content before flagging it as a violation or duplicating the comment.
Applied to files:
.github/workflows/unresolve-coderabbit-threads.yml
🔀 Multi-repo context
Findings
- Repository contains a CodeRabbit config file: .coderabbit.yaml is present at the repo root. [.::RedHatQE/openshift-virtualization-tests-design-docs::]
- Repo contains STP and testing guidance docs that the PR’s changes reference/affect:
- docs/stp-guide.md (Software Test Plan guidance) — includes STP expectations and templates referenced by CodeRabbit checks. [.::RedHatQE/openshift-virtualization-tests-design-docs::]
- docs/testing-tiers.md (Unit/Tier1/Tier2 guidance) — defines when real-cluster pytest commands are required vs. unit tests. [.::RedHatQE/openshift-virtualization-tests-design-docs::]
- Other CI/automation config present that may interact with review workflows:
- .github/ (workflows and config exist; labeler and in-solidarity shown). [.::RedHatQE/openshift-virtualization-tests-design-docs::]
Evaluation
- Presence of .coderabbit.yaml and STP/testing docs confirms this repository is a direct consumer/context for the PR’s CodeRabbit policy and the stricter test-plan requirements (real pytest commands, checkboxes, STP enforcement). No additional repositories were provided to search.
🔇 Additional comments (1)
.coderabbit.yaml (1)
49-53: LGTM!
openshift-virtualization-qe-bot-5
added
commented-coderabbitai[bot]
commented-rnetser
and removed
commented-coderabbitai[bot]
labels
Collaborator
Author
|
@coderabbitai[bot] The following review comments were reviewed and a decision was made:
|
- Check for substantive PR-author reply (not just any non-bot reply) - Only post warning comment if unresolveReviewThread mutation succeeded Signed-off-by: Ron Netser <rnetser@redhat.com> Assisted-by: Claude <noreply@anthropic.com>
rnetser
force-pushed
the
feat/coderabbit-review-improvements
branch
from
0601bea to
1ba9d1b
Compare
openshift-virtualization-qe-bot
added
tox:verify-tc-requirement-polarion:running
RoniKishner
approved these changes
Jun 8, 2026
vsibirsk
approved these changes
Jun 8, 2026
Collaborator
Author
|
/verified merging and will monitor the behavior |
Contributor
Test execution plan request detailsCRITICAL: You MUST post an inline review comment on the first changed line of the first file. As an expert software testing engineer, analyze all modified files in this PR and create a targeted test execution plan. Analysis Requirements:
Output rules: Your deliverable: Test Execution Plan
Guidelines:
Hardware-Related Checks (SR-IOV, GPU, DPDK): When PR modifies fixtures for hardware-specific resources:
CRITICAL WORKFLOW COMPLETION RULES: When responding to this test execution plan request, you MUST follow these rules EXACTLY:
ABSOLUTE PROHIBITIONS (violating these creates empty/meaningless reviews):
Remember: The pull request review is visible to users. Additional comments are redundant noise. CRITICAL — Verification rules for smoke and gating decisions: |
| @@ -18,6 +18,18 @@ issue_enrichment: {} | |||
| reviews: | |||
| # Explicit instructions applied to every review, including PR description validation | |||
Contributor
There was a problem hiding this comment.
Test Execution Plan
- Run smoke tests: False — No Python code, fixtures, or test utilities were modified. All changes are GitHub Actions workflows and CodeRabbit config (YAML only).
- Run gating tests: False — Same reason; no traced dependency path to any gating-marked test.
- Affected tests to run: None — Changes are confined to
.coderabbit.yaml,.github/workflows/request-coderabbit-test-instructions.yml, and.github/workflows/unresolve-coderabbit-threads.yml. Noutilities/,libs/, ortests/Python code was touched.
Contributor
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Get list of changed files and first changed line of first file
git diff --name-only HEAD~1 HEAD 2>/dev/null || git diff --name-only origin/main...HEAD 2>/dev/null
echo "---"
git diff HEAD~1 HEAD -- .coderabbit.yaml 2>/dev/null | head -20
echo "---COMMITS---"
git log --oneline -5Length of output: 716 🏁 Script executed: #!/bin/bash
# Get all changed files vs main
git diff --name-only origin/main...HEAD 2>/dev/null
echo "---"
# Get first changed line of first file (.coderabbit.yaml)
git diff origin/main...HEAD -- .coderabbit.yaml 2>/dev/null | head -30
echo "---WORKFLOW1---"
git diff origin/main...HEAD -- .github/workflows/request-coderabbit-test-instructions.yml 2>/dev/null | head -10
echo "---WORKFLOW2---"
git diff origin/main...HEAD -- .github/workflows/unresolve-coderabbit-threads.yml 2>/dev/null | head -10Length of output: 3206 🏁 Script executed: #!/bin/bash
# Get the latest commit SHA for the PR
gh api repos/RedHatQE/openshift-virtualization-tests/pulls/5122 --jq '.head.sha'Length of output: 217 🏁 Script executed: #!/bin/bash
# Post inline review comment on .coderabbit.yaml line 19
gh api repos/RedHatQE/openshift-virtualization-tests/pulls/5122/reviews \
--method POST \
--field commit_id='1ba9d1b7edf90c4bd69773de3f610980ffd31832' \
--field event='COMMENT' \
--field body='' \
--field 'comments[][path]=.coderabbit.yaml' \
--field 'comments[][line]=19' \
--field 'comments[][side]=RIGHT' \
--field 'comments[][body]=**Test Execution Plan**
- **Run smoke tests: False** — No Python code, fixtures, or test utilities were modified. All changes are GitHub Actions workflows and CodeRabbit config (YAML only).
- **Run gating tests: False** — Same reason; no traced dependency path to any gating-marked test.
- **Affected tests to run:** None — Changes are confined to `.coderabbit.yaml`, `.github/workflows/request-coderabbit-test-instructions.yml`, and `.github/workflows/unresolve-coderabbit-threads.yml`. No `utilities/`, `libs/`, or `tests/` Python code was touched.'Length of output: 2759 🧠 Learnings used |
|
New container for quay.io/openshift-cnv/openshift-virtualization-tests:latest published |
acinko-rh
pushed a commit
to acinko-rh/openshift-virtualization-tests
that referenced
this pull request
Jun 9, 2026
…#5122) ##### What this PR does / why we need it: Improves CodeRabbit review workflows to ensure higher quality reviews and PR verification: 1. **Skip STD/STP checks for unit tests** — `utilities/unittests/` files no longer trigger false STP link warnings 2. **Test plan uses checkboxes** — every listed test is a GitHub checkbox the PR author checks off after running 3. **Real cluster test commands** — when changes affect session/runtime code, the test plan includes concrete pytest commands (error path + happy path), not just unit test paths 4. **Auto-unresolve prematurely resolved threads** — detects when CodeRabbit threads are resolved without any reply or with a trivial (<15 char) response, unresolves them with a warning 5. **Approval policy** — CodeRabbit accepts reasonable test plan responses from PR authors without blocking ##### Which issue(s) this PR fixes: ##### Special notes for reviewer: The unresolve workflow triggers on `synchronize` (new push) and checks all resolved CodeRabbit threads. Two cases caught: - Thread resolved with no reply at all (just clicked "Resolve conversation") - Thread resolved with trivial reply (<15 chars) ##### jira-ticket: Assisted-by: Claude <noreply@anthropic.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Relaxed review/enforcement for internal unit-test areas to reduce docstring/link/coverage strictness. * Clarified PR approval policy: prior review comments must be addressed via changes or substantive author explanations; “resolved” alone is insufficient. * Tightened test-execution-plan guidance: require concrete test commands when runtime/session code is affected and a single non-blocking inline test-plan comment. * Added automation to reopen review threads lacking substantive author replies and to post a follow-up warning. * **Tests** * STP/RFE/Jira link enforcement limited to newly added test files/functions only. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: rnetser <rnetser@redhat.com> Signed-off-by: Ron Netser <rnetser@redhat.com> Signed-off-by: Adam Cinko <acinko@redhat.com>
OhadRevah
pushed a commit
to OhadRevah/openshift-virtualization-tests
that referenced
this pull request
Jun 10, 2026
…#5122) ##### What this PR does / why we need it: Improves CodeRabbit review workflows to ensure higher quality reviews and PR verification: 1. **Skip STD/STP checks for unit tests** — `utilities/unittests/` files no longer trigger false STP link warnings 2. **Test plan uses checkboxes** — every listed test is a GitHub checkbox the PR author checks off after running 3. **Real cluster test commands** — when changes affect session/runtime code, the test plan includes concrete pytest commands (error path + happy path), not just unit test paths 4. **Auto-unresolve prematurely resolved threads** — detects when CodeRabbit threads are resolved without any reply or with a trivial (<15 char) response, unresolves them with a warning 5. **Approval policy** — CodeRabbit accepts reasonable test plan responses from PR authors without blocking ##### Which issue(s) this PR fixes: ##### Special notes for reviewer: The unresolve workflow triggers on `synchronize` (new push) and checks all resolved CodeRabbit threads. Two cases caught: - Thread resolved with no reply at all (just clicked "Resolve conversation") - Thread resolved with trivial reply (<15 chars) ##### jira-ticket: Assisted-by: Claude <noreply@anthropic.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Relaxed review/enforcement for internal unit-test areas to reduce docstring/link/coverage strictness. * Clarified PR approval policy: prior review comments must be addressed via changes or substantive author explanations; “resolved” alone is insufficient. * Tightened test-execution-plan guidance: require concrete test commands when runtime/session code is affected and a single non-blocking inline test-plan comment. * Added automation to reopen review threads lacking substantive author replies and to post a follow-up warning. * **Tests** * STP/RFE/Jira link enforcement limited to newly added test files/functions only. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: rnetser <rnetser@redhat.com> Signed-off-by: Ron Netser <rnetser@redhat.com>
rnetser
added a commit
to rnetser/openshift-virtualization-tests
that referenced
this pull request
Jun 13, 2026
…#5122) ##### What this PR does / why we need it: Improves CodeRabbit review workflows to ensure higher quality reviews and PR verification: 1. **Skip STD/STP checks for unit tests** — `utilities/unittests/` files no longer trigger false STP link warnings 2. **Test plan uses checkboxes** — every listed test is a GitHub checkbox the PR author checks off after running 3. **Real cluster test commands** — when changes affect session/runtime code, the test plan includes concrete pytest commands (error path + happy path), not just unit test paths 4. **Auto-unresolve prematurely resolved threads** — detects when CodeRabbit threads are resolved without any reply or with a trivial (<15 char) response, unresolves them with a warning 5. **Approval policy** — CodeRabbit accepts reasonable test plan responses from PR authors without blocking ##### Which issue(s) this PR fixes: ##### Special notes for reviewer: The unresolve workflow triggers on `synchronize` (new push) and checks all resolved CodeRabbit threads. Two cases caught: - Thread resolved with no reply at all (just clicked "Resolve conversation") - Thread resolved with trivial reply (<15 chars) ##### jira-ticket: Assisted-by: Claude <noreply@anthropic.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Relaxed review/enforcement for internal unit-test areas to reduce docstring/link/coverage strictness. * Clarified PR approval policy: prior review comments must be addressed via changes or substantive author explanations; “resolved” alone is insufficient. * Tightened test-execution-plan guidance: require concrete test commands when runtime/session code is affected and a single non-blocking inline test-plan comment. * Added automation to reopen review threads lacking substantive author replies and to post a follow-up warning. * **Tests** * STP/RFE/Jira link enforcement limited to newly added test files/functions only. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: rnetser <rnetser@redhat.com> Signed-off-by: Ron Netser <rnetser@redhat.com>
This was referenced Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Improves CodeRabbit review workflows to ensure higher quality reviews and PR verification:
utilities/unittests/files no longer trigger false STP link warningsWhich issue(s) this PR fixes:
Special notes for reviewer:
The unresolve workflow triggers on
synchronize(new push) and checks all resolved CodeRabbit threads. Two cases caught:jira-ticket:
Assisted-by: Claude noreply@anthropic.com
Summary by CodeRabbit