Skip to content

Track CrocoDash main: rebuild, gate, then publish - #8

Open
manishvenu wants to merge 1 commit into
regional-create-test-suitefrom
crocodash-main-tracker
Open

manishvenu wants to merge 1 commit into
regional-create-test-suitefrom
crocodash-main-tracker

Conversation

@manishvenu

Copy link
Copy Markdown
Member

Keeps the published container in step with CrocoDash's main branch, without letting a CrocoDash commit publish itself.

Stacked on #7 (base regional-create-test-suite), which is where the three test workflows this reuses are defined. Merge that first.

Why a rebuild alone isn't enough

The image bakes CrocoDash in — COPY CrocoDash/ /workspace/CrocoDash/ in the Dockerfile — so rebuilding without moving the submodule pointer produces a byte-identical image. Picking up new CrocoDash means moving the submodule to main's tip and rebuilding from that.

What crocodash-main.yml does

Daily at 3am UTC, and on demand:

  1. Compares the CrocoDash submodule pointer against CrocoDash main's tip. Stops if they match.
  2. Checks that tip out in the working tree only — deliberately not committed — and builds an amd64 image, pushed as crocodash-<sha>-amd64.
  3. Runs the smoke test, the 17-domain sweep, and the MOM6 runs against that tag.
  4. Only on a green suite: retags it to latest-amd64 and opens a PR bumping the submodule pointer.

The ordering is the point. A broken CrocoDash main leaves latest-amd64 untouched, and the pointer ends up recording the newest CrocoDash main that is known to work in the container rather than merely the newest that exists.

Changes to the existing workflows

container-test.yml, domain-sweep.yml and mom6-runs.yml gain an optional image input via workflow_call. All 9 previously hardcoded image references now read $IMAGE, which defaults to ghcr.io/crocodile-cesm/crocontainer:latest-amd64 — so their behaviour on a pull request is unchanged. Verified: the full 20-check suite passed on this refactor before the split.

Deliberate limits

  • amd64 only. Every CI consumer pulls latest-amd64, and arm64 under QEMU has taken upwards of four hours — recent build.yml runs are 26m, 4h39m, 31m, 40m, 43m, 3h50m. Far too much to spend daily. build.yml's weekly run stays multi-arch and keeps arm64 and the merged latest manifest current, which is also why promote retags only latest-amd64 and leaves latest alone: republishing the manifest here would pair a new amd64 with a stale arm64.
  • Polls rather than being pushed to. A repository_dispatch from CrocoDash would be immediate, but needs a PAT with write access to this repo stored as a secret over there. Daily is fine for something whose rebuild alone takes half an hour.
  • The pointer-bump PR shows no checks of its own, because PRs opened with GITHUB_TOKEN do not start workflow runs. The gate that matters already ran; the PR body links to that run.

Not yet exercised

workflow_dispatch resolves against the default branch, so this workflow cannot be triggered until it is on main — the same limitation mom6-runs.yml has. Its check logic was run by hand against the real repos (correctly reports changed=true today) and every shell block parses, but the workflow-level plumbing — the workflow_call fan-out, the retag, the PR creation — is untested by construction. Worth a force: true dispatch immediately after merge rather than discovering a problem at 3am.

Adds crocodash-main.yml, which keeps the published container in step with
CrocoDash's main branch.

The image bakes CrocoDash in -- the Dockerfile does COPY CrocoDash/ -- so a
plain rebuild produces a byte-identical image. Picking up new CrocoDash
means moving the submodule to main's tip and rebuilding from that. Each
daily run:

  1. compares the submodule pointer against CrocoDash main's tip, and stops
     if they match;
  2. checks that tip out in the working tree and builds an amd64-only image,
     pushed as crocodash-<sha>-amd64;
  3. runs the smoke test, the domain sweep and the MOM6 runs against that tag;
  4. only on a green suite, retags it to latest-amd64 and opens a PR bumping
     the pointer.

A CrocoDash commit therefore cannot publish itself. A broken main leaves
latest-amd64 untouched, and the pointer ends up recording the newest
CrocoDash main that is known to work in the container rather than merely the
newest that exists.

The three test workflows gain an optional `image` input via workflow_call to
make step 3 possible; every podman invocation now reads $IMAGE, which
defaults to the published latest-amd64, so their behaviour on a PR is
unchanged.

Two deliberate limits, both documented in the README:

- amd64 only. Every CI consumer pulls latest-amd64, and arm64 under QEMU has
  taken upwards of four hours -- recent build.yml runs are 26m, 4h39m, 31m,
  40m, 43m, 3h50m. build.yml's weekly run stays multi-arch and keeps arm64
  and the merged `latest` manifest current, which is also why promote/
  retags only latest-amd64 and leaves `latest` alone.
- The pointer-bump PR shows no checks of its own, because PRs opened with
  GITHUB_TOKEN do not start workflow runs. The gate that matters already ran
  in the tracker, and the PR body links to it.

Polls daily rather than being pushed to by CrocoDash: a repository_dispatch
would be faster but needs a PAT with write access to this repo stored as a
secret in CrocoDash, and daily is fine for something whose rebuild alone
takes half an hour.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant