Skip to content

fix(ci): use GraphQL __typename to identify CodeRabbit bot in unresolve workflow#5291

Merged
rnetser merged 2 commits into
RedHatQE:mainfrom
rnetser:fix/issue-5290-graphql-login-mismatch
Jun 21, 2026
Merged

fix(ci): use GraphQL __typename to identify CodeRabbit bot in unresolve workflow#5291
rnetser merged 2 commits into
RedHatQE:mainfrom
rnetser:fix/issue-5290-graphql-login-mismatch

Conversation

@rnetser

@rnetser rnetser commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator
What this PR does / why we need it:

The unresolve-coderabbit-threads.yml workflow checked for "coderabbitai[bot]" but the GitHub GraphQL API returns bot logins without the [bot] suffix — just "coderabbitai". This caused:

  1. Opener check to always skip CodeRabbit threads on the pull_request_target path
  2. CodeRabbit verification check to never detect fix confirmations

Result: 19 out of 20 auto-unresolves in the past 14 days were false positives, including ping-pong loops of up to 7 cycles per thread (PR #5145).

Fix: Add __typename to GraphQL author queries and check both login=="coderabbitai" AND __typename=="Bot". This correctly identifies the bot and is immune to username impersonation (human users have __typename=="User").

Generated-by: Claude noreply@anthropic.com

Which issue(s) this PR fixes:

Fixes #5290

Special notes for reviewer:

The SENDER check for pull_request_review_thread events (== "coderabbitai[bot]") is left unchanged because it uses the REST API event context format which correctly includes [bot].

jira-ticket:

Summary by CodeRabbit

  • Chores
    • Improved automated thread management in workflows to be more resilient when pull request context is missing.
    • Refined verification logic so threads are only automatically handled when matching bot-origin verification comments are confirmed.
    • Enhanced GraphQL data checks to better distinguish bot authors and verification messages.

@coderabbitai

coderabbitai Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@rnetser, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 2 minutes and 31 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: b7f1086d-1dd5-4068-83e9-260c6ada647e

📥 Commits

Reviewing files that changed from the base of the PR and between c519e63 and 1eda57d.

📒 Files selected for processing (1)
  • .github/workflows/unresolve-coderabbit-threads.yml
📝 Walkthrough

Walkthrough

The unresolve-coderabbit-threads.yml workflow is updated to identify CodeRabbit bot accounts using both author.login and author.__typename from the GraphQL API, replacing checks that relied on the login string "coderabbitai[bot]" which the GraphQL API never returns. The job guard becomes null-safe for PR context, both single-thread and paginated GraphQL queries now fetch __typename, and the verification logic gates on both fields.

Changes

Bot Identity Detection Fix

Layer / File(s) Summary
Job-level null-safety guard
.github/workflows/unresolve-coderabbit-threads.yml
The job if: condition (line 35–39) now checks github.event.pull_request != null before applying the bot-skip rule, preventing null-reference failures when webhook event lacks PR context.
GraphQL queries extended with __typename
.github/workflows/unresolve-coderabbit-threads.yml
Both the single-thread (line 141–149) and paginated (line 183–191) GraphQL queries now request __typename alongside login for opening_comment.author, supplying the type data required by the updated identity checks.
Bot identity verification logic
.github/workflows/unresolve-coderabbit-threads.yml
check_and_unresolve_thread now extracts opener_type from __typename and requires both opener_login == "coderabbitai" and opener_type == "Bot" before unresolving (line 56–85). The verified-fix clause applies the same dual condition before treating a comment body as CodeRabbit verification.
SENDER webhook payload format documentation
.github/workflows/unresolve-coderabbit-threads.yml
Clarifying comment at line 125 documents that SENDER comes from the webhook event context and uses REST API format with [bot] suffix, distinct from the GraphQL login format.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

Suggested reviewers

  • dshchedr
  • vsibirsk
  • RoniKishner
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: fixing the workflow to use GraphQL __typename field to identify the CodeRabbit bot, and is 80 characters (under 120 limit).
Description check ✅ Passed The description includes all required sections with complete information: What/Why explains the login format mismatch and false positives, Issues fixes #5290, and reviewer notes explain the SENDER check approach.
Linked Issues check ✅ Passed The PR fully addresses all coding objectives from #5290: adds __typename to GraphQL author queries, fixes opener check with login + type validation, fixes CodeRabbit verification check with login + type validation, and keeps SENDER check unchanged.
Out of Scope Changes check ✅ Passed All changes are scoped to fixing the identified issues in the unresolve workflow; no unrelated modifications to other workflows, tests, or unrelated infrastructure code are present.
Stp Link Required ✅ Passed PR modifies only .github/workflows/unresolve-coderabbit-threads.yml (+22/-7); no new test files or test functions under tests/ directory are added, so STP link requirement does not apply.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

Review ran into problems

🔥 Problems

Linked repositories: Your configuration references 1 linked repositories, but your current plan allows 0. Analyzed ``, skipped RedHatQE/openshift-virtualization-tests-design-docs.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-virtualization-qe-bot-4

Copy link
Copy Markdown

Report bugs in Issues

Welcome! 🎉

This pull request will be automatically processed with the following features:

🔄 Automatic Actions

  • Reviewer Assignment: Reviewers are automatically assigned based on the OWNERS file in the repository root
  • Size Labeling: PR size labels (XS, S, M, L, XL, XXL) are automatically applied based on changes
  • Issue Creation: A tracking issue is created for this PR and will be closed when the PR is merged or closed
  • Branch Labeling: Branch-specific labels are applied to track the target branch
  • Auto-verification: Auto-verified users have their PRs automatically marked as verified
  • Labels: Enabled categories: branch, can-be-merged, cherry-pick, has-conflicts, hold, needs-rebase, size, verified, wip

📋 Available Commands

PR Status Management

  • /wip - Mark PR as work in progress (adds WIP: prefix to title)
  • /wip cancel - Remove work in progress status
  • /hold - Block PR merging (approvers only)
  • /hold cancel - Unblock PR merging
  • /verified - Mark PR as verified
  • /verified cancel - Remove verification status
  • /reprocess - Trigger complete PR workflow reprocessing (useful if webhook failed or configuration changed)
  • /regenerate-welcome - Regenerate this welcome message
  • /security-override - Set security check runs to pass (maintainers only)
  • /security-override cancel - Re-run security checks

Review & Approval

  • /lgtm - Approve changes (looks good to me)
  • /approve - Approve PR (approvers only)
  • /assign-reviewers - Assign reviewers based on OWNERS file
  • /assign-reviewer @username - Assign specific reviewer
  • /check-can-merge - Check if PR meets merge requirements

Testing & Validation

  • /retest tox - Run Python test suite with tox
  • /retest build-container - Rebuild and test container image
  • /retest verify-bugs-are-open - verify-bugs-are-open
  • /retest all - Run all available tests

Container Operations

  • /build-and-push-container - Build and push container image (tagged with PR number)
    • Supports additional build arguments: /build-and-push-container --build-arg KEY=value

Cherry-pick Operations

  • /cherry-pick <branch> - Schedule cherry-pick to target branch when PR is merged
    • Multiple branches: /cherry-pick branch1 branch2 branch3
  • /cherry-pick-retry <branch> - Retry a failed cherry-pick (merged PRs only)

Branch Management

  • /rebase - Rebase this PR branch onto its base branch

Label Management

  • /<label-name> - Add a label to the PR
  • /<label-name> cancel - Remove a label from the PR

✅ Merge Requirements

This PR will be automatically approved when the following conditions are met:

  1. Approval: /approve from at least one approver
  2. LGTM Count: Minimum 2 /lgtm from reviewers
  3. Status Checks: All required status checks must pass
  4. No Blockers: No wip, hold, has-conflicts labels and PR must be mergeable (no conflicts)
  5. Verified: PR must be marked as verified

📊 Review Process

Approvers and Reviewers

Approvers:

  • dshchedr
  • myakove
  • rnetser
  • vsibirsk

Reviewers:

  • RoniKishner
  • dshchedr
  • rnetser
  • vsibirsk
Available Labels
  • hold
  • verified
  • wip
  • lgtm
  • approve
AI Features
  • Cherry-Pick Conflict Resolution: Enabled (claude/claude-opus-4-6[1m])
Security Checks
  • Suspicious Path Detection: Monitors paths: .claude/, .vscode/, .cursor/, .devcontainer/, .pi/, .github/workflows/, .github/actions/
  • Committer Identity Check: Verifies last committer matches PR author
  • Mandatory: Security checks block merge (use /security-override to bypass — maintainers only)

💡 Tips

  • WIP Status: Use /wip when your PR is not ready for review
  • Verification: The verified label is removed on new commits unless the push is detected as a clean rebase
  • Cherry-picking: Cherry-pick labels are processed when the PR is merged
  • Container Builds: Container images are automatically tagged with the PR number
  • Permission Levels: Some commands require approver permissions
  • Auto-verified Users: Certain users have automatic verification and merge privileges

For more information, please refer to the project documentation or contact the maintainers.

@rnetser

rnetser commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator Author

/security-override

@openshift-virtualization-qe-bot-6

Copy link
Copy Markdown

Security checks overridden by @rnetser. Security check runs set to pass.

@rnetser

rnetser commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator Author

/approvr

/verified (after merge)

@openshift-virtualization-qe-bot-3

Copy link
Copy Markdown
Contributor

@coderabbitai

Test execution plan request details

CRITICAL: You MUST post an inline review comment on the first changed line of the first file.
The inline comment should contain the full Test Execution Plan (smoke decision, gating decision, and specific affected tests).
Do NOT submit a blocking review event (REQUEST_CHANGES/APPROVE).
Post a single inline PR comment on Files Changed (non-blocking COMMENT flow).

As an expert software testing engineer, analyze all modified files in this PR and create a targeted test execution plan.
You will post an inline review comment with the test execution plan on the first changed file.
If you fail to run or post a comment, retry.

Analysis Requirements:

  1. Examine code changes in each modified file

  2. Identify affected code paths, functions, and classes

  3. Analyze pytest-specific elements: fixtures (scope, dependencies), parametrization, markers, conftest changes

  4. Trace test dependencies through imports, shared utilities, fixture inheritance, fixture teardown, and yield from cleanup in conftest

  5. Detect new tests introduced in the PR

  6. Utilities and libs impact (when utilities/ or libs/ changes):
    You MUST use shell scripts (rg, git diff) to trace the full impact.
    Follow these sub-steps in order:

    6a. Identify modified symbols: For each changed file under utilities/ or libs/,
    list every modified function or method.
    Example: git diff HEAD~1 --unified=0 -- utilities/hco.py | grep '^[+-]def '

    6b. Find direct callers: Search tests and conftest for each symbol from 6a.
    Example: rg -l 'get_hco_version' tests/

    6c. Trace fixture teardown and cleanup: Find fixtures that reach
    the modified symbol through yield from or context-manager wrappers.
    Example: rg -l 'yield from.*enable_common_boot|def.*enable_common_boot' tests/

    6d. Trace same-file callers: In each changed file, find other functions
    whose body calls a modified symbol (including code after yield
    in @contextmanager helpers).
    Example: rg 'get_hco_version|enable_common_boot' utilities/hco.py

    6e. Expand transitively: If function A calls modified B, then
    tests/fixtures that call A are affected — even when the test body
    never imports B directly.

    Do NOT limit impact to tests that import the modified symbol only.

  7. Smoke test impact: Intersect the affected set from step 6 with smoke-marked tests.
    Run: rg -l '@pytest.mark.smoke' tests/
    VERIFY the above command returned actual file paths before concluding False.
    Set True if either condition is met:

    • a smoke-marked file appears in the affected set from 6b-6e, OR
    • any conftest.py in the smoke test's parent-directory hierarchy (up to repo root)
      imports or calls a modified utilities/libs symbol — including autouse fixtures
      that depend on modified functions. ALL tests in that directory and below are affected.
      Example check: for each smoke_file, scan dirname(smoke_file)/conftest.py,
      dirname(dirname(smoke_file))/conftest.py, etc. for modified symbol imports
      and autouse fixtures that depend on modified symbols.
  8. Gating test impact: Intersect the affected set from step 6 with gating-marked tests.
    Run: rg -l '@pytest.mark.gating' tests/
    Set True if a gating-marked file also appears in the affected set from 6b-6e.
    Utilities/libs changes often affect gating tests without affecting smoke tests.
    Do NOT stop analysis after concluding Run smoke tests: False.

Output rules:
Do NOT include analysis step numbers (1-8) in your visible output.

Your deliverable:
Your inline informational comment will be based on the following requirements:

Test Execution Plan

  • Run smoke tests: True / False — If True, state the dependency path (test → fixture → changed symbol). True ONLY with a verified path.
  • Run gating tests: True / False — If True, state the dependency path. True if any gating-marked test is in the affected set.
  • Affected tests to run (required when utilities/, libs/, or shared conftest changes — list concrete paths even when smoke is False)

Use these formats:

  • path/to/test_file.py - When the entire test file needs verification
  • path/to/test_file.py::TestClass::test_method - When specific test(s) needed
  • path/to/test_file.py::test_function - When specific test(s) needed
  • -m marker - When a marker covers multiple affected tests (e.g. -m gating only if ALL gating tests in scope need run)
  • Tag each listed test or group with its marker when not obvious, e.g. (gating) or (smoke)

Real test commands (MANDATORY when changes affect session/runtime code):

When the affected code runs at session/collection time (conftest fixtures, pytest plugins,
config hooks, session-scoped setup) or modifies runtime behavior that unit tests mock away,
you MUST include concrete pytest commands the PR author must run on a real cluster
to verify the change works end-to-end. Include:

  • A command for the error/fix path (the scenario the PR fixes)
  • A command for the happy path (regression: the normal case still works)
  • Use lightweight tests (e.g., --collect-only for startup failures,
    a single small test for runtime behavior)
    If the PR only changes test logic (not utilities/libs/conftest), the affected test
    paths themselves serve as the real test commands — no separate section needed.

Example output for a session-startup fix:

**Real tests (cluster required)**
Error path (the fix):
`pytest tests/storage/.../test_foo.py --storage-class-matrix=nonexistent-sc --collect-only`
Expected: ValueError with clear message, not IndexError

Happy path (regression):
`pytest tests/storage/.../test_foo.py --storage-class-matrix=<valid-sc> -k test_bar`
Expected: session starts normally

Guidelines:

  • Include tests affected directly OR via fixture setup/teardown, yield from cleanup, or transitive utility call chains (caller calls modified helper)
  • Use a full file path only if ALL tests in that file require verification
  • Use file path + test name when only specific tests use an affected fixture or utility wrapper (preferred for partial file impact)
  • If a test marker can cover multiple files/tests, provide the marker
  • Balance coverage vs over-testing - Keep descriptions minimal
  • Example: if leaf helper foo() changes, include tests whose fixture teardown calls wrapper bar() where bar() calls foo(), even when the test body only imports an unrelated symbol from the same utilities module

Hardware-Related Checks (SR-IOV, GPU, DPDK):

When PR modifies fixtures for hardware-specific resources:

  • Collection Safety: Fixtures MUST have existence checks (return None when hardware unavailable)
  • Test Plan: MUST verify both WITH and WITHOUT hardware:
    • Run affected tests on cluster WITH hardware
    • Verify collection succeeds on cluster WITHOUT hardware

CRITICAL WORKFLOW COMPLETION RULES:

When responding to this test execution plan request, you MUST follow these rules EXACTLY:

  1. YOUR ONLY DELIVERABLE: Post one non-blocking inline comment containing the test execution plan on the first changed line
  2. THEN STOP IMMEDIATELY - Do NOT generate any additional response
  3. FALLBACK ONLY: If inline comment API calls fail after retrying, post as a regular PR comment
  4. SILENCE = SUCCESS: After successfully submitting the review, your task is complete. No confirmation needed.

ABSOLUTE PROHIBITIONS (violating these creates empty/meaningless reviews):

  • ❌ Do NOT post acknowledgment messages like "Test execution plan posted", "Review posted successfully", "I've successfully posted"
  • ❌ Do NOT mention review IDs, URLs, or confirmation of posting in the PR thread
  • ❌ Do NOT add any follow-up comments after submitting the review
  • ❌ Do NOT reply to confirm task completion
  • ❌ Do NOT explain what you did - just do it and stop

Remember: The pull request review is visible to users. Additional comments are redundant noise.

CRITICAL — Verification rules for smoke and gating decisions:
- For each True decision, you must have traced a concrete path: test file → fixture → utility function → modified symbol.
- Do NOT assume impact based on scope, file proximity, or infrastructure-sounding names.
Only set True when you trace a concrete path through steps 6a-6e, then 7 or 8.
WRONG: "utilities/hco.py changed → run smoke" (no traced dependency path)
WRONG: "Run smoke tests: False" with no affected-test list when gating tests use the changed utility
RIGHT: "Run smoke tests: False, Run gating tests: True" plus tests/.../test_foo.py::TestBar::test_baz (gating)
RIGHT: "Smoke test X uses fixture Y → wrapper Z → changed function W in utilities/hco.py"

Comment thread .github/workflows/unresolve-coderabbit-threads.yml
@coderabbitai

coderabbitai Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Get the PR diff to find the first changed line in the first file
gh pr view 5291 --repo RedHatQE/openshift-virtualization-tests --json files --jq '.files[0]'
gh api repos/RedHatQE/openshift-virtualization-tests/pulls/5291/files --jq '.[0] | {filename, patch}'

Length of output: 4983


🏁 Script executed:

#!/bin/bash
# Get the latest commit SHA for the PR
COMMIT_SHA=$(gh api repos/RedHatQE/openshift-virtualization-tests/pulls/5291 --jq '.head.sha')
echo "Commit SHA: $COMMIT_SHA"

# Post a non-blocking inline review comment on the first changed line (diff position 5)
gh api repos/RedHatQE/openshift-virtualization-tests/pulls/5291/reviews \
  --method POST \
  --field commit_id="$COMMIT_SHA" \
  --field body="" \
  --field event="COMMENT" \
  --field "comments[][path]=.github/workflows/unresolve-coderabbit-threads.yml" \
  --field "comments[][position]=5" \
  --field "comments[][body]=**Test Execution Plan**

- **Run smoke tests: False** — This PR modifies only \`.github/workflows/unresolve-coderabbit-threads.yml\` (a GitHub Actions workflow). No pytest code, fixtures, utilities, or conftest files are touched. No traced path to any smoke-marked test exists.
- **Run gating tests: False** — Same reason; no gating-marked test has a dependency path through this change.
- **Affected tests to run:** None — the change is purely a CI/CD workflow fix (bot login format: GraphQL \`__typename\` check). No cluster testing is required."

Length of output: 2976



Prevent false positives where "unresolved" matches "resolved" and
"unverified" matches "verified" in the CodeRabbit verification check.

Generated-by: Claude <noreply@anthropic.com>
Signed-off-by: rnetser <rnetser@redhat.com>
@rnetser

rnetser commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@rnetser

rnetser commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator Author

/security-override

@openshift-virtualization-qe-bot-3

Copy link
Copy Markdown
Contributor

Security checks overridden by @rnetser. Security check runs set to pass.

@rnetser

rnetser commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator Author

/verified (removed unsupporte dflow)
/approve

@openshift-virtualization-qe-bot-3

Copy link
Copy Markdown
Contributor

@coderabbitai

Test execution plan request details

CRITICAL: You MUST post an inline review comment on the first changed line of the first file.
The inline comment should contain the full Test Execution Plan (smoke decision, gating decision, and specific affected tests).
Do NOT submit a blocking review event (REQUEST_CHANGES/APPROVE).
Post a single inline PR comment on Files Changed (non-blocking COMMENT flow).

As an expert software testing engineer, analyze all modified files in this PR and create a targeted test execution plan.
You will post an inline review comment with the test execution plan on the first changed file.
If you fail to run or post a comment, retry.

Analysis Requirements:

  1. Examine code changes in each modified file

  2. Identify affected code paths, functions, and classes

  3. Analyze pytest-specific elements: fixtures (scope, dependencies), parametrization, markers, conftest changes

  4. Trace test dependencies through imports, shared utilities, fixture inheritance, fixture teardown, and yield from cleanup in conftest

  5. Detect new tests introduced in the PR

  6. Utilities and libs impact (when utilities/ or libs/ changes):
    You MUST use shell scripts (rg, git diff) to trace the full impact.
    Follow these sub-steps in order:

    6a. Identify modified symbols: For each changed file under utilities/ or libs/,
    list every modified function or method.
    Example: git diff HEAD~1 --unified=0 -- utilities/hco.py | grep '^[+-]def '

    6b. Find direct callers: Search tests and conftest for each symbol from 6a.
    Example: rg -l 'get_hco_version' tests/

    6c. Trace fixture teardown and cleanup: Find fixtures that reach
    the modified symbol through yield from or context-manager wrappers.
    Example: rg -l 'yield from.*enable_common_boot|def.*enable_common_boot' tests/

    6d. Trace same-file callers: In each changed file, find other functions
    whose body calls a modified symbol (including code after yield
    in @contextmanager helpers).
    Example: rg 'get_hco_version|enable_common_boot' utilities/hco.py

    6e. Expand transitively: If function A calls modified B, then
    tests/fixtures that call A are affected — even when the test body
    never imports B directly.

    Do NOT limit impact to tests that import the modified symbol only.

  7. Smoke test impact: Intersect the affected set from step 6 with smoke-marked tests.
    Run: rg -l '@pytest.mark.smoke' tests/
    VERIFY the above command returned actual file paths before concluding False.
    Set True if either condition is met:

    • a smoke-marked file appears in the affected set from 6b-6e, OR
    • any conftest.py in the smoke test's parent-directory hierarchy (up to repo root)
      imports or calls a modified utilities/libs symbol — including autouse fixtures
      that depend on modified functions. ALL tests in that directory and below are affected.
      Example check: for each smoke_file, scan dirname(smoke_file)/conftest.py,
      dirname(dirname(smoke_file))/conftest.py, etc. for modified symbol imports
      and autouse fixtures that depend on modified symbols.
  8. Gating test impact: Intersect the affected set from step 6 with gating-marked tests.
    Run: rg -l '@pytest.mark.gating' tests/
    Set True if a gating-marked file also appears in the affected set from 6b-6e.
    Utilities/libs changes often affect gating tests without affecting smoke tests.
    Do NOT stop analysis after concluding Run smoke tests: False.

Output rules:
Do NOT include analysis step numbers (1-8) in your visible output.

Your deliverable:
Your inline informational comment will be based on the following requirements:

Test Execution Plan

  • Run smoke tests: True / False — If True, state the dependency path (test → fixture → changed symbol). True ONLY with a verified path.
  • Run gating tests: True / False — If True, state the dependency path. True if any gating-marked test is in the affected set.
  • Affected tests to run (required when utilities/, libs/, or shared conftest changes — list concrete paths even when smoke is False)

Use these formats:

  • path/to/test_file.py - When the entire test file needs verification
  • path/to/test_file.py::TestClass::test_method - When specific test(s) needed
  • path/to/test_file.py::test_function - When specific test(s) needed
  • -m marker - When a marker covers multiple affected tests (e.g. -m gating only if ALL gating tests in scope need run)
  • Tag each listed test or group with its marker when not obvious, e.g. (gating) or (smoke)

Real test commands (MANDATORY when changes affect session/runtime code):

When the affected code runs at session/collection time (conftest fixtures, pytest plugins,
config hooks, session-scoped setup) or modifies runtime behavior that unit tests mock away,
you MUST include concrete pytest commands the PR author must run on a real cluster
to verify the change works end-to-end. Include:

  • A command for the error/fix path (the scenario the PR fixes)
  • A command for the happy path (regression: the normal case still works)
  • Use lightweight tests (e.g., --collect-only for startup failures,
    a single small test for runtime behavior)
    If the PR only changes test logic (not utilities/libs/conftest), the affected test
    paths themselves serve as the real test commands — no separate section needed.

Example output for a session-startup fix:

**Real tests (cluster required)**
Error path (the fix):
`pytest tests/storage/.../test_foo.py --storage-class-matrix=nonexistent-sc --collect-only`
Expected: ValueError with clear message, not IndexError

Happy path (regression):
`pytest tests/storage/.../test_foo.py --storage-class-matrix=<valid-sc> -k test_bar`
Expected: session starts normally

Guidelines:

  • Include tests affected directly OR via fixture setup/teardown, yield from cleanup, or transitive utility call chains (caller calls modified helper)
  • Use a full file path only if ALL tests in that file require verification
  • Use file path + test name when only specific tests use an affected fixture or utility wrapper (preferred for partial file impact)
  • If a test marker can cover multiple files/tests, provide the marker
  • Balance coverage vs over-testing - Keep descriptions minimal
  • Example: if leaf helper foo() changes, include tests whose fixture teardown calls wrapper bar() where bar() calls foo(), even when the test body only imports an unrelated symbol from the same utilities module

Hardware-Related Checks (SR-IOV, GPU, DPDK):

When PR modifies fixtures for hardware-specific resources:

  • Collection Safety: Fixtures MUST have existence checks (return None when hardware unavailable)
  • Test Plan: MUST verify both WITH and WITHOUT hardware:
    • Run affected tests on cluster WITH hardware
    • Verify collection succeeds on cluster WITHOUT hardware

CRITICAL WORKFLOW COMPLETION RULES:

When responding to this test execution plan request, you MUST follow these rules EXACTLY:

  1. YOUR ONLY DELIVERABLE: Post one non-blocking inline comment containing the test execution plan on the first changed line
  2. THEN STOP IMMEDIATELY - Do NOT generate any additional response
  3. FALLBACK ONLY: If inline comment API calls fail after retrying, post as a regular PR comment
  4. SILENCE = SUCCESS: After successfully submitting the review, your task is complete. No confirmation needed.

ABSOLUTE PROHIBITIONS (violating these creates empty/meaningless reviews):

  • ❌ Do NOT post acknowledgment messages like "Test execution plan posted", "Review posted successfully", "I've successfully posted"
  • ❌ Do NOT mention review IDs, URLs, or confirmation of posting in the PR thread
  • ❌ Do NOT add any follow-up comments after submitting the review
  • ❌ Do NOT reply to confirm task completion
  • ❌ Do NOT explain what you did - just do it and stop

Remember: The pull request review is visible to users. Additional comments are redundant noise.

CRITICAL — Verification rules for smoke and gating decisions:
- For each True decision, you must have traced a concrete path: test file → fixture → utility function → modified symbol.
- Do NOT assume impact based on scope, file proximity, or infrastructure-sounding names.
Only set True when you trace a concrete path through steps 6a-6e, then 7 or 8.
WRONG: "utilities/hco.py changed → run smoke" (no traced dependency path)
WRONG: "Run smoke tests: False" with no affected-test list when gating tests use the changed utility
RIGHT: "Run smoke tests: False, Run gating tests: True" plus tests/.../test_foo.py::TestBar::test_baz (gating)
RIGHT: "Smoke test X uses fixture Y → wrapper Z → changed function W in utilities/hco.py"

Comment thread .github/workflows/unresolve-coderabbit-threads.yml
@coderabbitai

coderabbitai Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Get the latest commit SHA for the PR
COMMIT_SHA=$(gh api repos/RedHatQE/openshift-virtualization-tests/pulls/5291 --jq '.head.sha')
echo "Commit SHA: $COMMIT_SHA"

# Post a non-blocking inline review comment on the first changed line
gh api repos/RedHatQE/openshift-virtualization-tests/pulls/5291/reviews \
  --method POST \
  --field commit_id="$COMMIT_SHA" \
  --field body="" \
  --field event="COMMENT" \
  --field "comments[][path]=.github/workflows/unresolve-coderabbit-threads.yml" \
  --field "comments[][position]=5" \
  --field "comments[][body]=**Test Execution Plan**

- **Run smoke tests: False** — This PR modifies only \`.github/workflows/unresolve-coderabbit-threads.yml\` (a GitHub Actions workflow). No pytest code, fixtures, utilities, or conftest files are changed. No dependency path to any smoke-marked test exists.
- **Run gating tests: False** — Same reason; no gating-marked test has a dependency path through this change.
- **Affected tests to run:** None — the change is a CI/CD workflow fix correcting bot login format detection (GraphQL \`__typename\` + \`login\` check). No cluster testing required."

Length of output: 2984



@rnetser

rnetser commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@rnetser rnetser merged commit db6df4d into RedHatQE:main Jun 21, 2026
18 of 19 checks passed
@openshift-virtualization-qe-bot-5

Copy link
Copy Markdown

New container for quay.io/openshift-cnv/openshift-virtualization-tests:latest published

@rnetser

rnetser commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

ema-aka-young pushed a commit to ema-aka-young/openshift-virtualization-tests that referenced this pull request Jun 22, 2026
…ve workflow (RedHatQE#5291)

##### What this PR does / why we need it:

The `unresolve-coderabbit-threads.yml` workflow checked for
`"coderabbitai[bot]"` but the GitHub GraphQL API returns bot logins
without the `[bot]` suffix — just `"coderabbitai"`. This caused:

1. **Opener check** to always skip CodeRabbit threads on the
`pull_request_target` path
2. **CodeRabbit verification check** to never detect fix confirmations

**Result:** 19 out of 20 auto-unresolves in the past 14 days were false
positives, including ping-pong loops of up to 7 cycles per thread (PR
RedHatQE#5145).

**Fix:** Add `__typename` to GraphQL `author` queries and check both
`login=="coderabbitai"` AND `__typename=="Bot"`. This correctly
identifies the bot and is immune to username impersonation (human users
have `__typename=="User"`).

Generated-by: Claude <noreply@anthropic.com>

##### Which issue(s) this PR fixes:

Fixes RedHatQE#5290

##### Special notes for reviewer:

The `SENDER` check for `pull_request_review_thread` events (`==
"coderabbitai[bot]"`) is left unchanged because it uses the REST API
event context format which correctly includes `[bot]`.

##### jira-ticket:

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Improved automated thread management in workflows to be more resilient
when pull request context is missing.
* Refined verification logic so threads are only automatically handled
when matching bot-origin verification comments are confirmed.
* Enhanced GraphQL data checks to better distinguish bot authors and
verification messages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: rnetser <rnetser@redhat.com>
Signed-off-by: Emanuele Prella <eprella@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(ci): unresolve workflow never matches CodeRabbit threads due to GraphQL login format mismatch

7 participants