Add GitHub coverage reporting workflow#702
Conversation
Signed-off-by: Andrew Russell <arussell@nvidia.com>
📝 WalkthroughWalkthroughA new GitHub Actions Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
.github/workflows/coverage.yml (1)
27-30: 🔒 Security & Privacy | 🔵 TrivialDisable persisted checkout credentials.
coverage.ymldoesn’t use git after checkout, so addpersist-credentials: falseto avoid leaving the workflow token in local git config.Suggested fix
- name: Checkout code uses: actions/checkout@v6 with: fetch-depth: 0 + persist-credentials: false🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/coverage.yml around lines 27 - 30, Add persist-credentials: false to the actions/checkout step in coverage.yml so the workflow does not store the token in local git config. Update the existing Checkout code step only; the change belongs in the checkout configuration alongside fetch-depth, with no other workflow behavior altered.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/coverage.yml:
- Line 113: The coverage artifact upload is configured to only warn when files
are missing, which can hide broken coverage generation and still pass the
workflow. Update the artifact upload step in the coverage workflow to fail the
job when expected coverage files are absent by changing the existing
if-no-files-found setting on the upload action, keeping the rest of the coverage
job unchanged.
- Line 28: The workflow currently uses mutable GitHub Actions tags for
actions/checkout and actions/upload-artifact, so update both references to
immutable commit SHAs to lock the versions. Locate the uses entries for
actions/checkout and actions/upload-artifact in the coverage workflow and
replace the tag refs with the pinned SHA equivalents while keeping the same
action behavior.
---
Nitpick comments:
In @.github/workflows/coverage.yml:
- Around line 27-30: Add persist-credentials: false to the actions/checkout step
in coverage.yml so the workflow does not store the token in local git config.
Update the existing Checkout code step only; the change belongs in the checkout
configuration alongside fetch-depth, with no other workflow behavior altered.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: f7e45d5e-e9a8-4d97-aef2-b2660eb24239
📒 Files selected for processing (1)
.github/workflows/coverage.yml
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v6 |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the workflow with line numbers
sed -n '1,220p' .github/workflows/coverage.yml | cat -nRepository: NVIDIA/IsaacTeleop
Length of output: 4648
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '1,220p' .github/workflows/coverage.yml | cat -nRepository: NVIDIA/IsaacTeleop
Length of output: 4648
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '1,220p' .github/workflows/coverage.yml | cat -nRepository: NVIDIA/IsaacTeleop
Length of output: 4648
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '1,220p' .github/workflows/coverage.yml | cat -nRepository: NVIDIA/IsaacTeleop
Length of output: 4648
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '1,220p' .github/workflows/coverage.yml | cat -nRepository: NVIDIA/IsaacTeleop
Length of output: 4648
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '1,220p' .github/workflows/coverage.yml | cat -nRepository: NVIDIA/IsaacTeleop
Length of output: 4648
Pin the GitHub Actions refs to commit SHAs. actions/checkout@v6 and actions/upload-artifact@v6 are mutable tag refs; lock both to immutable SHAs to reduce supply-chain risk. (Lines 28, 105)
🧰 Tools
🪛 zizmor (1.26.1)
[error] 28-28: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/coverage.yml at line 28, The workflow currently uses
mutable GitHub Actions tags for actions/checkout and actions/upload-artifact, so
update both references to immutable commit SHAs to lock the versions. Locate the
uses entries for actions/checkout and actions/upload-artifact in the coverage
workflow and replace the tag refs with the pinned SHA equivalents while keeping
the same action behavior.
Source: Linters/SAST tools
There was a problem hiding this comment.
Fixed in 2dc9c2e: pinned actions/checkout and actions/upload-artifact to the resolved v6 commit SHAs, and added persist-credentials: false to checkout since this workflow does not push back to the repo.
| coverage/totals.txt | ||
| coverage/coverage.xml | ||
| coverage/html | ||
| if-no-files-found: warn |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Fail the job when expected coverage artifacts are missing.
Line 113 uses warn, which can mask a broken coverage output path and still report a successful run.
Suggested fix
- if-no-files-found: warn
+ if-no-files-found: errorBased on learnings: “if the expected output files are not produced, fail the run instead of silently succeeding.”
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if-no-files-found: warn | |
| if-no-files-found: error |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/coverage.yml at line 113, The coverage artifact upload is
configured to only warn when files are missing, which can hide broken coverage
generation and still pass the workflow. Update the artifact upload step in the
coverage workflow to fail the job when expected coverage files are absent by
changing the existing if-no-files-found setting on the upload action, keeping
the rest of the coverage job unchanged.
Source: Learnings
There was a problem hiding this comment.
Fixed in 2dc9c2e: changed if-no-files-found to error, and renamed the XML artifact to coverage/cobertura.xml so a report generation/path issue fails loudly.
|
@aristarkhovNV can you help take a look? Thanks! |
| with: | ||
| name: isaacteleop-coverage | ||
| path: | | ||
| coverage/summary.txt |
There was a problem hiding this comment.
just to confirm: is this the same coverage report format that NVIDIA's internal tooling expects?
There was a problem hiding this comment.
Confirmed: the workflow generates Cobertura XML with gcovr --xml-pretty and now publishes it explicitly as coverage/cobertura.xml, alongside the HTML and text summaries in the isaacteleop-coverage artifact. If NVIDIA internal tooling expects a different filename/path in addition to Cobertura XML, I can add that too.
Signed-off-by: Andrew Russell <arussell@nvidia.com>
qingsi-at-nv
left a comment
qingsi-at-nv
left a comment
There was a problem hiding this comment.
PR and the generated coverage report looks reasonable to me: https://github.com/NVIDIA/IsaacTeleop/actions/runs/28116596678/job/83257560203?pr=702. Defer to @aristarkhovNV and @jiwenc-nv for any second opinion before merging.
|
|
||
| - name: Upload coverage artifact | ||
| if: ${{ always() }} | ||
| # actions/upload-artifact@v6 |
There was a problem hiding this comment.
nit: could add a comment that this pins to the 6.0.3 release of actions/checkout.
qingsi-at-nv
left a comment
qingsi-at-nv
left a comment
There was a problem hiding this comment.
Please update the PR description as discussed.
|
We already run ctest as part of the existing Ubuntu build. I'd suggest just adding coverage report there instead of adding a whole another Build & test workflow which is 90% identical to the existing one. |
|
Addressed the workflow-duplication review feedback in
Expected coverage artifact: |
|
Added a follow-up commit for the OSS dependency reporting half of the CI healthiness baseline. What changed:
Local validation: the generator ran successfully and produced 148 dependency entries; |
|
Pushed a small follow-up after the first CI poll caught formatting-only failures:
Local validation after the fix: OSS inventory generation still reports 148 entries, |
|
@arussell-nvidia could you keep this PR scope to adding the coverage test? It lgtm after your merged with the existing workflow, but the additional code to add OSS scanning should be done separately. Also the coverage improvement proposal does not belong to the repo document. |
|
CI is now green on Re-review summary:
Ready for another review pass when you have time. |
Signed-off-by: Andrew Russell <arussell@nvidia.com>
arussell-nvidia
force-pushed
the
codex/cxr-3577-github-coverage
branch
from
7aaff4e to
ca50d47
Compare
|
Addressed in
CI is restarting on the updated branch. |
|
CI on ca50d47 is now green after the scope cleanup. This PR is coverage-only: coverage generation remains in the existing Build Ubuntu Debug / x64 / Python 3.11 CTest matrix entry and publishes isaacteleop-coverage-ubuntu-debug. The latest coverage artifact from run 28553138603 reports lines 18.0% (1510 / 8405) and branches 11.0% (1250 / 11413). No OSS/dependency reporting changes remain in this PR. Ready for re-review. |
4 participants
Description
Adds GitHub-native coverage reporting to the existing
Build Ubuntuworkflow. The Ubuntu Debug / x64 / Python 3.11 CTest matrix entry now builds with coverage flags, runs the existing CTest path, publishes totals to the GitHub Actions step summary, and uploads text, Cobertura XML, and HTML coverage artifacts.This PR is intentionally scoped to coverage reporting only.
Type of change
Testing
git diff --checkChecklist
SKIP=check-copyright-year pre-commit run --all-filesgit commit -s) per the DCO