Skip to content

Add Repo Radius GHCR state end-to-end test workflow - #12476

Open
brooke-hamilton wants to merge 14 commits into
radius-project:mainfrom
brooke-hamilton:brooke-hamilton-repo-radius-state-e2e
Open

Add Repo Radius GHCR state end-to-end test workflow#12476
brooke-hamilton wants to merge 14 commits into
radius-project:mainfrom
brooke-hamilton:brooke-hamilton-repo-radius-state-e2e

Conversation

@brooke-hamilton

@brooke-hamilton brooke-hamilton commented Jul 18, 2026

Copy link
Copy Markdown
Member

Summary

This PR adds a scheduled daily end-to-end workflow that tests the ability to run a control plane, serialize state to GHCR, then create a new control plane instance that rehydrates state from GHCR. If the workflow fails it adds an issue tagged with test-failure (the same as our other scheduled tests). There is already a functional test for GHCR state storage, and this adds an end-to-end test scenario that includes deployments to a local cluster.

  • add a scheduled and manually dispatchable Repo Radius GHCR state rehydration test
  • exercise state persistence across replacement control planes while preserving a workload on a separate target cluster
  • add timeout-safe remote state cleanup, diagnostics, and scheduled failure issue reporting
  • document package provisioning and the standalone functional-test lifecycle
  • add the Radius application model used by the graph tooling

Testing

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8a31277c-6702-45f2-876b-013c8a0ec466
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8a31277c-6702-45f2-876b-013c8a0ec466
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8a31277c-6702-45f2-876b-013c8a0ec466
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 29eb21e5-6540-4bbc-9e29-3ea5c1aeb994
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@codecov

codecov Bot commented Jul 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.19%. Comparing base (a25786d) to head (8638e9e).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #12476      +/-   ##
==========================================
- Coverage   54.20%   54.19%   -0.02%     
==========================================
  Files         770      770              
  Lines       51085    51085              
==========================================
- Hits        27690    27684       -6     
- Misses      20789    20792       +3     
- Partials     2606     2609       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a scheduled + manually-dispatchable GitHub Actions workflow that validates Radius OCI state persistence in GHCR by saving state, replacing the control plane, restoring state, and confirming an existing workload on a separate target cluster remains managed. This fits into the repo’s CI/validation story by providing an end-to-end durability signal for the GHCR-backed state archive that can’t run in per-PR CI due to packages: write + private package requirements.

Changes:

  • Adds repo-radius-state-e2e GitHub Actions workflow plus supporting shell scripts to provision/validate a private GHCR package, run the lifecycle, collect diagnostics, and ensure cleanup.
  • Adds a minimal Radius application model (Bicep) used by the workflow to deploy/update a workload that proves state rehydration.
  • Documents how to provision the GHCR package and how to run/troubleshoot the standalone scheduled workflow; links this into contributing + architecture docs.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
test/functional-portable/statestore/noncloud/testdata/repo-radius-state-app.bicep Adds the app/container Bicep model used by the E2E workflow to verify updates across restore.
docs/contributing/contributing-code/contributing-code-tests/repo-radius-state-e2e.md New contributor doc explaining provisioning, running, and troubleshooting the state E2E workflow.
docs/contributing/contributing-code/contributing-code-tests/README.md Links the new E2E workflow doc from the tests docs index.
docs/architecture/state-archive.md Documents the existence/purpose of the dedicated E2E test for the OCI state archive.
.github/workflows/repo-radius-state-e2e.yaml New scheduled + dispatchable workflow to run the state save/restore lifecycle against GHCR.
.github/scripts/test-repo-radius-state-e2e.sh Implements the E2E lifecycle phases, diagnostics collection, and cleanup behavior.
.github/scripts/precreate-repo-radius-state-package.sh Idempotent helper to create/verify the required private/internal, linked GHCR package bootstrap tag.

Comment thread .github/workflows/repo-radius-state-e2e.yaml Outdated
Comment thread .github/workflows/repo-radius-state-e2e.yaml
Comment thread .github/workflows/repo-radius-state-e2e.yaml
Comment thread docs/architecture/state-archive.md Outdated
@github-actions

github-actions Bot commented Jul 18, 2026

Copy link
Copy Markdown

Functional Tests - corerp-cloud

30 tests  ±0   29 ✅ ±0   20m 32s ⏱️ - 2m 34s
 2 suites ±0    1 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit 60a2052. ± Comparison against base commit 67c36c6.

♻️ This comment has been updated with latest results.

brooke-hamilton and others added 2 commits July 18, 2026 11:50
Use vars.RADIUS_REPOSITORY in the state-rehydration job gate to match
the report-failure job in the same workflow and the sibling
functional-test workflows, instead of hardcoding the repository name.

Reflow the new End-to-End Test section in state-archive.md to a single
line per paragraph, per the repo Markdown convention (MD013 disabled).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
Rename the workflow display name from "Repo Radius GHCR State E2E" to
"Repo Radius State E2E" so it derives directly from the filename
repo-radius-state-e2e.yaml, per the GitHub Workflows instructions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton
brooke-hamilton marked this pull request as ready for review July 18, 2026 19:01
@brooke-hamilton
brooke-hamilton requested review from a team as code owners July 18, 2026 19:01
@lakshmimsft
lakshmimsft requested a review from sk593 July 20, 2026 18:18
pull Bot pushed a commit to TheTechOddBug/radius that referenced this pull request Jul 20, 2026
…es (radius-project#12493)

## Summary

The OCI-backed Radius state archive authenticates to GHCR **only** via
the runner's Docker credential store (`credentials.NewStoreFromDocker`
in `pkg/statearchive/oci/oci.go` and `ghcr.go`). The Azure and AWS
deploy workflow templates never ran a `docker login` to GHCR before `rad
startup` (the `restore-state` action) or `rad shutdown` (the `teardown`
action), so opening a private `radius-state` package produced an
**anonymous** token request that GHCR rejected:

```
Error: failed to open state archive: failed to resolve OCI archive "radius-state":
HEAD ".../manifests/radius-state": GET ".../token?scope=repository%3A...%3Apull...":
response status code 401: unauthorized: authentication required
```

The registry credentials that already exist in the templates
(`github.actor` / `secrets.GITHUB_TOKEN`) were passed only to the later
`run-rad-commands` action, where they create a Kubernetes secret for the
containerImages recipe — they never logged the runner into GHCR.

## Change

Add a `docker/login-action` step (`ghcr.io`, `github.actor` /
`secrets.GITHUB_TOKEN`) early in the `deploy` job of both
`run-rad-commands-azure.yml` and `run-rad-commands-aws.yml`, before `Set
up control plane` / `Restore Radius state` / `Teardown`. `docker login`
writes `~/.docker/config.json`, which persists for the whole job, so
`rad startup` and `rad shutdown` both authenticate. `packages: write` is
already granted at the job level.

This mirrors the GHCR auth setup in the `repo-radius-state-e2e.yaml`
workflow (radius-project#12476), which logs into GHCR the same way before its state
save/restore steps.

## Note on package provisioning

The login is necessary but not always sufficient: the GHCR state package
must also be **private/internal and linked to the repository** for
`GITHUB_TOKEN` to authenticate, and Radius' visibility guard refuses
public packages for state. Provisioning the package with that
visibility/linkage is the responsibility of the per-environment deploy
tooling that sets `RADIUS_STATE_REGISTRY` (analogous to the
`precreate-repo-radius-state-package.sh` step in radius-project#12476), not these
templates.

## Testing

- YAML of both templates validated with `yaml.safe_load`.
- `docker/login-action` pinned to
`af1e73f918a031802d376d3c8bbc3fe56130a9b0` (v4.4.0), consistent with the
other pins under `.github/workflows/`.

Signed-off-by: sk593 <shruthikumar@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Comment thread test/functional-portable/statestore/noncloud/testdata/repo-radius-state-app.bicep Outdated
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 29eb21e5-6540-4bbc-9e29-3ea5c1aeb994
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 30, 2026 23:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Human review recommended

It introduces a new scheduled workflow with package write/delete automation and a large orchestration script, which warrants final human verification despite only minor review comments.

Review details

Comments suppressed due to low confidence (2)

.github/workflows/repo-radius-state-e2e.yaml:19

  • Workflow name should match the workflow file name (without extension) per repo workflow conventions. This makes it easier to correlate runs in the Actions UI.
name: Repo Radius State E2E

test/functional-portable/statestore/noncloud/testdata/repo-radius-state-app.bicep:32

  • Bicep symbolic resource names app and container are overly generic and make the file harder to read/maintain. Use descriptive lowerCamelCase resource symbols and update references accordingly (per Bicep conventions).
resource app 'Radius.Core/applications@2025-08-01-preview' = {
  name: 'repo-radius-state-e2e'
  location: 'global'
  properties: {
    environment: environment
  • Files reviewed: 9/9 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

@brooke-hamilton brooke-hamilton changed the title Add Repo Radius GHCR state end-to-end workflow Add Repo Radius GHCR state end-to-end test workflow Aug 3, 2026
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 29eb21e5-6540-4bbc-9e29-3ea5c1aeb994
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 3, 2026 15:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Not ready to approve

There are a couple of repo-convention issues to fix (notably the workflow name: not matching the file name) before this should be approved.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (2)

.github/workflows/repo-radius-state-e2e.yaml:19

  • Workflow name does not match the workflow file name (repo-radius-state-e2e.yaml), which makes it harder to correlate runs in the Actions UI. Per .github/instructions/github-workflows.instructions.md ("Workflow Names Must Match File Names"), set name: to the file base name.
name: Repo Radius State E2E

test/functional-portable/statestore/noncloud/testdata/repo-radius-state-app.bicep:22

  • Bicep symbolic resource names app and container are overly generic; this repo’s Bicep guidelines prefer descriptive symbolic names. Rename them to something specific (and update the application: reference) to keep the template easier to scan and refactor.
resource app 'Radius.Core/applications@2025-08-01-preview' = {
  name: 'repo-radius-state-e2e'
  location: 'global'
  properties: {
    environment: environment
  }
}

resource container 'Radius.Compute/containers@2025-08-01-preview' = {
  name: 'repo-radius-state-container'
  location: 'global'
  properties: {
    application: app.id
    environment: environment
  • Files reviewed: 9/9 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Copilot AI review requested due to automatic review settings August 3, 2026 15:25
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 29eb21e5-6540-4bbc-9e29-3ea5c1aeb994
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Not ready to approve

The new workflow’s name: does not match its filename exactly, which violates the repo’s workflow naming convention and should be corrected before merge.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (1)

.github/workflows/repo-radius-state-e2e.yaml:19

  • Workflow name: should match the workflow file name exactly (per .github/instructions/github-workflows.instructions.md “Workflow Names Must Match File Names”). This is currently Repo Radius State E2E but the file is repo-radius-state-e2e.yaml.
name: Repo Radius State E2E
  • Files reviewed: 9/9 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Copilot AI review requested due to automatic review settings August 3, 2026 15:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Not ready to approve

It has at least one repo-convention mismatch in the new workflow metadata and a documentation statement that overstates what the E2E test actually exercises.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (2)

.github/workflows/repo-radius-state-e2e.yaml:19

  • Workflow name should match the workflow file name exactly (per repo workflow guidelines) to make it easy to correlate entries in the Actions UI.
name: Repo Radius State E2E

docs/architecture/state-archive.md:219

  • This section says the E2E test validates the bootstrap-tag behavior for when the package does not yet exist, but the workflow requires a precreated private package and does not exercise package creation/bootstrap-tag creation. Reword to avoid overstating coverage.
The OCI state archive has a dedicated end-to-end test that exercises the full save/restore lifecycle against a real GHCR package. It deploys an application through one ephemeral Radius control plane to a separate persistent target cluster, saves state to a private GHCR package, replaces the control plane, restores the saved state, and confirms the replacement control plane still manages the existing workload. This validates the round-trip durability contract and the GHCR visibility guard end to end, including the bootstrap-tag behavior used when the package does not yet exist. It runs on a schedule rather than in the per-PR matrix because it needs `packages: write` and a precreated private package. See [Repo Radius GHCR state end-to-end test](../contributing/contributing-code/contributing-code-tests/repo-radius-state-e2e.md) for how to provision, run, and troubleshoot it.
  • Files reviewed: 9/9 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

brooke-hamilton and others added 2 commits August 14, 2026 16:52
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton

Copy link
Copy Markdown
Member Author

Evaluated Copilot's three "Not ready to approve" review summaries:

  • The workflow-name finding was valid. Commit 8638e9e changes the workflow name: to the exact filename stem, repo-radius-state-e2e, as required by .github/instructions/github-workflows.instructions.md.
  • The architecture-doc finding was valid. The same commit removes the claim that this workflow exercises package-absent bootstrap-tag creation; the workflow requires a precreated private package and only validates/uses its existing bootstrap tag.
  • The generic Bicep symbolic-name finding is stale at the current head. Both application models already use repoRadiusStateApp and repoRadiusStateContainer, including the updated application: reference, so no further change is needed.

The workflow-name code blocks in the reviews repeated the prior title-cased value, but the rule's examples require the lowercase filename stem exactly.

@radius-functional-tests

radius-functional-tests Bot commented Aug 14, 2026

Copy link
Copy Markdown

Radius functional test overview

🔍 Go to test action run

Click here to see the test run details
Name Value
Repository brooke-hamilton/radius
Commit ref 8638e9e
Unique ID funcb1d5a2d917
Image tag pr-funcb1d5a2d917
  • Dapr: 1.14.4
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcb1d5a2d917
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcb1d5a2d917
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-funcb1d5a2d917
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-funcb1d5a2d917
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcb1d5a2d917
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
⌛ Starting ucp-cloud functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
⌛ Starting corerp-cloud functional tests...
✅ corerp-cloud functional tests succeeded
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants