diff --git a/core/skills/rh-qs-test-suite/SKILL.md b/core/skills/rh-qs-test-suite/SKILL.md index e29d982..034a466 100644 --- a/core/skills/rh-qs-test-suite/SKILL.md +++ b/core/skills/rh-qs-test-suite/SKILL.md @@ -11,6 +11,8 @@ description: | **Category:** `github/` **Reference implementation:** [it-self-service-agent](https://github.com/rh-ai-quickstart/it-self-service-agent) `.github/` +For image publishing, use the skill-local [build-and-push workflow reference](./references/build-and-push-workflow.yaml) as the canonical pattern. It replaces the external example for this workflow. + ## Trigger - Helm chart and Makefile deploy targets exist from **`rh-qs-deploy`** if not create it. @@ -21,6 +23,15 @@ description: | If the request is scoped to nightly testing specifically (e.g. "add nightly testing to ``", "update the nightly workflow", "refresh the nightly CI setup") rather than the full CI build-out, skip straight to it: resolve the slug/repo as usual, then invoke **`subagents/nightly-tests-prompt.md`** directly (`invocation_mode: standalone`) and report its result. Do not run the PR-checks/E2E/build-and-push phases for this request — they are unrelated to what was asked. This is the same subagent step 8 below delegates to; the only difference is you stop after it instead of continuing the rest of the workflow. +## Build-and-push-only fast path + +If the request is scoped to image publishing specifically (e.g. "create only the build-and-push workflow", "update image publishing", "fix the build-and-push workflow") rather than the full CI build-out, resolve the quickstart repository and then invoke **`subagents/build-and-push-prompt.md`** directly (`invocation_mode: standalone`). Do not run the PR-checks/E2E/nightly/promotion phases for this request. This is the same subagent step 7 below delegates to; the only difference is you stop after it instead of continuing the rest of the workflow. + +This is a hard scope gate: in standalone build-and-push mode, the only repository +file this path may create or update is `.github/workflows/build-and-push.yaml`. +Do not modify `Makefile`, `README.md`, Helm files, scripts, or any other +workflow merely to make the generated workflow work. + ## What it does 1. Splits CI into **focused workflows** (quality, unit tests, Kind E2E, evals, image build) instead of one monolithic job @@ -40,7 +51,7 @@ If the request is scoped to nightly testing specifically (e.g. "add nightly test - [ ] 4. Add composite actions (prepare-runner, kind, inspect) - [ ] 5. Add PR workflows (checks, e2e, branch policy — as needed) - [ ] 6. Configure repo secrets/vars (see references/secrets-and-variables.md) -- [ ] 7. Add build-and-push on push to main/dev (if publishing images) +- [ ] 7. When publishing images, invoke `subagents/build-and-push-prompt.md` to add the self-contained workflow for push to main/dev and manual dispatch - [ ] 8. When design requires nightly testing, invoke `subagents/nightly-tests-prompt.md`; add dev→main promotion workflow separately if needed - [ ] 9. Register required status checks in branch protection - [ ] 10. Open test PR and confirm all workflows green @@ -68,7 +79,7 @@ If the request is scoped to nightly testing specifically (e.g. "add nightly test | `pr-pip-install-build.yml` | `pull_request` | Validate alternate install path (`use_pip_install`) | | `pr-evaluation-check.yml` | `pull_request_target` | Known-bad conversation / regression eval gate | | `pr-branch-check.yml` | `pull_request` → `main` | Enforce `dev` or `dev-` source branch | -| `build-and-push.yaml` | `push` to `main`/`dev` | Build and push images to Quay (org-owned repos only) | +| `build-and-push.yaml` | `push` to `main`/`dev`, `workflow_dispatch` | Self-contained build and push using Makefile targets; validates OCI tags and supports `FORCE_BUILD_ALL=TRUE` on manual dispatch to override the optional image-needed target | | `test-.yml` | `workflow_call`, `workflow_dispatch` | Reusable deploy + test + teardown against a real OpenShift namespace, via actions copied from `quickstart-ci-testing` — generated by `subagents/nightly-tests-prompt.md` | | `nightly-main.yml` / `nightly-.yml` | `schedule`, `workflow_dispatch` | Thin scheduled callers of `test-.yml`, one per long-lived branch — generated by `subagents/nightly-tests-prompt.md` | | `create-dev-to-main-pr.yaml` | `workflow_dispatch` | Version-checked promotion PR with temp branch | @@ -125,6 +136,8 @@ Implement these targets (names can alias existing ones): Full list: [references/makefile-ci-contract.md](./references/makefile-ci-contract.md) +Build-and-push generation is delegated to [the dedicated prompt](./subagents/build-and-push-prompt.md). It resolves the version, build, and publish targets from the quickstart's Makefile and README, asks for missing or ambiguous mappings before writing anything, and never substitutes canonical target names for absent targets. + ## Secrets and variables Configure in GitHub **before** enabling `pull_request_target` E2E workflows: @@ -194,4 +207,6 @@ When CI workflows are green on a test PR → **`rh-qs-verify-deploy`**, then **` - [Secrets and variables](./references/secrets-and-variables.md) - [Makefile CI contract](./references/makefile-ci-contract.md) - [Workflow skeletons](./references/workflow-skeletons.md) — copy-paste YAML starters +- [Build-and-push workflow reference](./references/build-and-push-workflow.yaml) — canonical local workflow for automatic and manual image publishing +- [Build-and-push subagent](./subagents/build-and-push-prompt.md) — deterministic Make-target resolution and workflow generation - [Nightly actions catalog](./references/nightly-actions-catalog.md) — `quickstart-ci-testing` actions, reusable test workflow + nightly caller skeletons, provisioning script (used by `subagents/nightly-tests-prompt.md`) diff --git a/core/skills/rh-qs-test-suite/references/build-and-push-workflow.yaml b/core/skills/rh-qs-test-suite/references/build-and-push-workflow.yaml new file mode 100644 index 0000000..fbbfcf0 --- /dev/null +++ b/core/skills/rh-qs-test-suite/references/build-and-push-workflow.yaml @@ -0,0 +1,170 @@ +# Canonical local reference for rh-qs-test-suite image publishing. +# Adapt Make target names, variable names, REGISTRY, and toolchain setup to the +# quickstart's Makefile and README. Set IMAGE_REPOSITORIES to every image +# repository built by build-all-images, using fully qualified registry names. +# This workflow is self-contained; it is not a reusable workflow. +name: Build and push images + +on: + push: + branches: + - main + - dev + workflow_dispatch: + inputs: + target_branch: + description: Branch whose images should be built + required: true + default: main + type: choice + options: + - main + - dev + force_build_all: + description: Build and publish every image even when the optional Make target says none are needed + required: true + default: 'FALSE' + type: choice + options: + - 'FALSE' + - 'TRUE' + +concurrency: + group: ${{ github.workflow }}-${{ github.repository }}-${{ inputs.target_branch || github.ref_name }} + cancel-in-progress: false + queue: max + +permissions: + contents: read + +env: + REGISTRY: quay.io/rh-ai-quickstart + FORCE_BUILD_ALL: ${{ inputs.force_build_all || 'FALSE' }} + IMAGE_REPOSITORIES: "quay.io/rh-ai-quickstart/replace-with-image-name" # Space-separated; include every image built by the Makefile. + UV_VERSION: "0.8.9" # Replace with the quickstart's Makefile value when uv is used. + PYTHON_VERSION: "3.12" # Replace with the quickstart's required Python version. + +jobs: + build-and-push: + if: github.repository_owner == 'rh-ai-quickstart' && github.event.repository.fork == false + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v5 + with: + ref: ${{ inputs.target_branch || github.sha }} + + - name: Check whether image publishing is needed + id: image-check + run: | + MAKE_STATUS=0 + MAKE_DATABASE="$(make -qp 2>/dev/null)" || MAKE_STATUS=$? + if [ "${MAKE_STATUS}" -gt 1 ]; then + echo "Unable to inspect Makefile targets (make exited ${MAKE_STATUS})." >&2 + exit "${MAKE_STATUS}" + fi + + if grep -Eq '^build-push-images-needed::?([[:space:]]|$)' <<< "${MAKE_DATABASE}"; then + echo "Found build-push-images-needed; checking whether images need publishing." + TARGET_STATUS=0 + if make build-push-images-needed; then + TARGET_STATUS=0 + else + TARGET_STATUS=$? + fi + + if [ "${FORCE_BUILD_ALL}" = "TRUE" ]; then + echo "FORCE_BUILD_ALL=TRUE; building and publishing all images regardless of the optional target result." + echo "needed=true" >> "$GITHUB_OUTPUT" + elif [ "${TARGET_STATUS}" -eq 0 ]; then + echo "Image build and push are needed." + echo "needed=true" >> "$GITHUB_OUTPUT" + else + echo "Image build and push are not needed; skipping them." + echo "needed=false" >> "$GITHUB_OUTPUT" + fi + else + echo "No build-push-images-needed target found; proceeding with image publishing." + echo "needed=true" >> "$GITHUB_OUTPUT" + fi + + - name: Extract image tags + id: version + if: steps.image-check.outputs.needed == 'true' + env: + TARGET_BRANCH: ${{ inputs.target_branch || github.ref_name }} + run: | + BASE_VERSION=$(make version) + if [[ ! "${BASE_VERSION}" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]]; then + echo "make version must return a valid OCI tag (1-128 characters, starting with a letter, number, or underscore)." >&2 + exit 1 + fi + if [ "${TARGET_BRANCH}" = "main" ]; then + TAG_BASE="${BASE_VERSION}" + elif [ "${TARGET_BRANCH}" = "dev" ]; then + TAG_BASE="${BASE_VERSION}-dev" + else + echo "Unsupported target branch: ${TARGET_BRANCH}" >&2 + exit 1 + fi + COMMIT_SHA=$(git rev-parse HEAD) + VERSIONS="${TAG_BASE} ${TAG_BASE}-${COMMIT_SHA}" + if [ "${TARGET_BRANCH}" = "main" ]; then + VERSIONS="${VERSIONS} latest" + else + VERSIONS="${VERSIONS} dev" + fi + for TAG in ${VERSIONS}; do + if [[ ! "${TAG}" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]]; then + echo "Derived image tag is not a valid OCI tag: ${TAG}" >&2 + exit 1 + fi + done + echo "versions=${VERSIONS}" >> "$GITHUB_OUTPUT" + + - name: Set up Docker Buildx + if: steps.image-check.outputs.needed == 'true' + uses: docker/setup-buildx-action@v3 + + - name: Log in to Quay.io + if: steps.image-check.outputs.needed == 'true' + uses: docker/login-action@v3 + with: + registry: quay.io + username: ${{ secrets.QUAY_USERNAME }} + password: ${{ secrets.QUAY_PASSWORD }} + + # Include this step only if the quickstart uses uv, with versions from + # its Makefile and README. Add other required runtime setup as needed. + - name: Set up uv + if: steps.image-check.outputs.needed == 'true' + uses: astral-sh/setup-uv@v6 + with: + version: ${{ env.UV_VERSION }} + python-version: ${{ env.PYTHON_VERSION }} + + - name: Build all images + if: steps.image-check.outputs.needed == 'true' + run: | + IFS=' ' read -ra VERSIONS <<< "${{ steps.version.outputs.versions }}" + VERSION="${VERSIONS[0]}" make build-all-images + + - name: Push all images + if: steps.image-check.outputs.needed == 'true' + run: | + IFS=' ' read -ra VERSIONS <<< "${{ steps.version.outputs.versions }}" + VERSION="${VERSIONS[0]}" make push-all-images + + - name: Tag pushed images + if: steps.image-check.outputs.needed == 'true' + env: + IMAGE_TAGS: ${{ steps.version.outputs.versions }} + run: | + IFS=' ' read -ra VERSIONS <<< "${IMAGE_TAGS}" + PRIMARY_TAG="${VERSIONS[0]}" + for IMAGE in ${IMAGE_REPOSITORIES}; do + for TAG in "${VERSIONS[@]:1}"; do + echo "Tagging ${IMAGE}:${PRIMARY_TAG} as ${IMAGE}:${TAG}" + docker buildx imagetools create --tag "${IMAGE}:${TAG}" "${IMAGE}:${PRIMARY_TAG}" + done + done diff --git a/core/skills/rh-qs-test-suite/references/workflow-catalog.md b/core/skills/rh-qs-test-suite/references/workflow-catalog.md index 7b7eca8..88c077b 100644 --- a/core/skills/rh-qs-test-suite/references/workflow-catalog.md +++ b/core/skills/rh-qs-test-suite/references/workflow-catalog.md @@ -34,8 +34,7 @@ with: | File | Trigger | Purpose | |------|---------|---------| -| `build-and-push.yaml` | `push` to `main`, `dev` | `if: github.repository_owner == 'rh-ai-quickstart'`; version from `make version`; multi-tag push to `quay.io/rh-ai-quickstart` | -| `build-and-push-manual.yaml` | `workflow_dispatch` | Manual image build | +| `build-and-push.yaml` | `push` to `main`, `dev`; `workflow_dispatch` | Self-contained workflow from [the local build-and-push reference](./build-and-push-workflow.yaml); uses `make version`, `make build-all-images`, and `make push-all-images`; manual dispatch can set `FORCE_BUILD_ALL=TRUE` to override the optional image-needed target; excludes forks | | `create-dev-to-main-pr.yaml` | `workflow_dispatch` | GitHub App token; version sync Makefile ↔ Chart.yaml ↔ values image tags; temp branch `dev-` | | `create-dev-version-bump-pr.yml` | `workflow_dispatch` | Automated version bump on dev | diff --git a/core/skills/rh-qs-test-suite/references/workflow-skeletons.md b/core/skills/rh-qs-test-suite/references/workflow-skeletons.md index a49952b..0e243d0 100644 --- a/core/skills/rh-qs-test-suite/references/workflow-skeletons.md +++ b/core/skills/rh-qs-test-suite/references/workflow-skeletons.md @@ -173,48 +173,7 @@ jobs: ## build-and-push.yaml -```yaml -name: Build and push images - -on: - push: - branches: [main, dev] - -concurrency: - group: ${{ github.workflow }}-${{ github.sha }} - cancel-in-progress: true - -env: - REGISTRY: quay.io/rh-ai-quickstart - UV_VERSION: "0.8.9" - PYTHON_VERSION: "3.12" - -jobs: - build-and-push: - if: github.repository_owner == 'rh-ai-quickstart' - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - id: version - run: | - BASE_VERSION=$(make version) - # tag logic: main → latest, dev → dev tag, etc. - - uses: docker/setup-buildx-action@v3 - - uses: docker/login-action@v3 - with: - registry: quay.io - username: ${{ secrets.QUAY_USERNAME }} - password: ${{ secrets.QUAY_PASSWORD }} - - uses: astral-sh/setup-uv@v6 - with: - version: ${{ env.UV_VERSION }} - python-version: ${{ env.PYTHON_VERSION }} - - run: | - for version in ${{ steps.version.outputs.versions }}; do - VERSION="${version}" make build-all-images - VERSION="${version}" make push-all-images - done -``` +Use the skill-local [build-and-push workflow reference](./build-and-push-workflow.yaml). It combines push triggers for `main` and `dev` with manual dispatch, checks the repository owner and fork status, and calls the quickstart's Makefile build and push targets. Adapt the registry and toolchain steps to the quickstart. ## Evaluations artifact tarball (reusable snippet) diff --git a/core/skills/rh-qs-test-suite/subagents/README.md b/core/skills/rh-qs-test-suite/subagents/README.md index dbc3179..995d38f 100644 --- a/core/skills/rh-qs-test-suite/subagents/README.md +++ b/core/skills/rh-qs-test-suite/subagents/README.md @@ -2,4 +2,5 @@ | Name | Purpose | Input | Output | When used | Why subagent | |------|---------|-------|--------|-----------|---------------| +| `build-and-push-prompt.md` | Generate a self-contained build-and-push workflow from the quickstart's real Makefile target contract | `repo_root`, optional `target_mapping`, `invocation_mode` | `.github/workflows/build-and-push.yaml` plus a report of resolved targets, variables, and toolchain setup | Step 7 of the main `rh-qs-test-suite` workflow, or directly when only image publishing is requested | Target discovery and toolchain selection are self-contained and must be deterministic; this prompt forces explicit evidence or a user question before generating the workflow | | `nightly-tests-prompt.md` | Generate a reusable `workflow_call` test workflow plus scheduled `nightly-main`/`nightly-` callers, copying shared composite actions from `quickstart-ci-testing`, a namespace/ServiceAccount provisioning script, and reports on required secrets/variables and toolchain setup | `slug`, `repo_root`, `invocation_mode` (`standalone` or `full-pipeline`) | `.qs-factory/pipeline/nightly-tests-manifest.yaml` + a report (toolchain steps used, secrets/variables needed, cluster-admin warning if any, next steps) | Step 8 of the main `rh-qs-test-suite` workflow, or directly via the nightly-only fast path in `SKILL.md` when a quickstart already has CI and just needs nightly testing added or refreshed | Nightly generation is self-contained detection + file-generation logic (Makefile contract resolution, toolchain detection, secret discovery, cluster-admin scan, action copying) that the orchestrator shouldn't have to hold in its own context, and that also needs to be independently invokable without running the rest of the CI build-out | diff --git a/core/skills/rh-qs-test-suite/subagents/build-and-push-prompt.md b/core/skills/rh-qs-test-suite/subagents/build-and-push-prompt.md new file mode 100644 index 0000000..500eca6 --- /dev/null +++ b/core/skills/rh-qs-test-suite/subagents/build-and-push-prompt.md @@ -0,0 +1,97 @@ +--- +description: Generate a self-contained GitHub Actions workflow that builds and publishes a quickstart's images +--- + +# Build And Push Images + +## Your Role + +You generate or update a quickstart's self-contained `.github/workflows/build-and-push.yaml`. You are invoked by `rh-qs-test-suite` when image publishing is required, or directly when a quickstart already has CI and only needs this workflow. + +Your working directory is the root of the quickstart's own repository. Treat every path below as relative to that root. + +You never run `oc`, `kubectl`, `gh secret set`, or a container engine. You generate files only; humans perform operational actions. + +## Inputs + +- `repo_root`: absolute or relative path to the quickstart repository +- `target_mapping`: optional pre-confirmed mapping containing `version`, `build`, `publish`, variable names, and image repositories +- `invocation_mode`: `"standalone"` or `"full-pipeline"` — affects only final report framing + +## Required Target Contract + +Resolve the Make targets, variable contract, and image repositories before writing anything: + +1. A target that prints the image release version +2. A target that builds all images +3. A target that publishes all images +4. Every image repository built by the build target, including its fully qualified registry name +5. Whether the optional `build-push-images-needed` target exists + +Use this resolution order: + +1. Use `target_mapping` if all three targets, their variables, and the complete image repository list are already explicitly confirmed. +2. Otherwise, inspect `Makefile` and `README.md` together. +3. If all three canonical targets `version`, `build-all-images`, and `push-all-images` exist, use them directly. Treat `VERSION` and `REGISTRY` as their default variable contract unless `Makefile` or `README.md` clearly documents a different interface. +4. If any canonical target is absent, search both files for a clearly documented equivalent. Do not infer a target solely from its name. +5. Identify every image repository built by the build target from the Makefile and README. Use fully qualified registry names in `IMAGE_REPOSITORIES`. +6. If `build-push-images-needed` exists, run it before build and publish. Exit status 0 means proceed; any nonzero status means skip build, push, and retagging unless `FORCE_BUILD_ALL` is exactly `TRUE`, which forces all images to build and publish regardless of the target's exit status. If the target is absent, preserve the default behavior and publish images. +7. Record the evidence that proves each target's purpose and variable contract and confirms the complete image repository list, including whether the optional target exists. +8. Stop and ask the user when the image list or variable contract is missing or ambiguous. Canonical target names alone do not confirm which images they build. + +If `README.md` explicitly says the mapping is undocumented, ambiguous, or that +the quickstart owner must confirm it, treat the mapping as unresolved and ask. +Do not infer target roles from descriptive target names in that situation. + +If you must ask, ask one question that requests all three target names and their variable contract, for example: + +> Which Makefile target prints the release version, which builds all images, and which publishes them? What variables do the build and publish targets require, and what are the fully qualified repository names of every image built? + +Never substitute canonical target names for missing targets. Never modify `Makefile`, `README.md`, or any other existing file merely to make the workflow work. + +Create or update only `.github/workflows/build-and-push.yaml`. Do not create, +copy, or modify any other file. + +## Workflow Requirements + +Use `references/build-and-push-workflow.yaml` as the canonical template, then adapt the target names, variable names, registry, image repositories, and toolchain setup. + +The generated workflow must: + +- Run on every push to `main` and `dev` +- Support `workflow_dispatch` with a `target_branch` choice limited to `main` and `dev`, plus a `force_build_all` choice of `TRUE` or `FALSE` defaulting to `FALSE` +- Set the `FORCE_BUILD_ALL` environment variable from the manual input, defaulting to `FALSE` for push events +- Queue same-branch runs with `concurrency.queue: max` and `cancel-in-progress: false` +- Restrict publishing to `github.repository_owner == 'rh-ai-quickstart'` and non-fork repositories +- Read `QUAY_USERNAME` and `QUAY_PASSWORD` from GitHub secrets +- Validate the base version and every derived OCI tag before build or publish +- Check for the optional `build-push-images-needed` target; if present, run build, push, and retag steps when it exits 0 or when `FORCE_BUILD_ALL` is `TRUE`, and if absent, run those steps as usual +- Invoke the resolved version target, build all images once using the first derived tag, and publish that tag once using the existing Makefile contract +- Use `docker buildx imagetools create` to add each remaining tag to every image in `IMAGE_REPOSITORIES`, without rebuilding +- Populate `IMAGE_REPOSITORIES` with every fully qualified image repository built by the quickstart +- Pass the resolved variables, normally `VERSION` and `REGISTRY`, exactly as documented +- Include Python/uv setup only when the quickstart actually requires it; otherwise omit those steps and version variables entirely +- Preserve every file that existed before this workflow generation request + +## Verification + +Before reporting completion: + +1. Confirm `.github/workflows/build-and-push.yaml` exists. +2. Confirm it is the only file created, updated, or deleted by this request. +3. Parse the generated YAML. +4. Confirm every `make` invocation uses one of the three resolved target names. +5. Confirm every variable passed to those targets matches the resolved target contract. +6. Confirm `IMAGE_REPOSITORIES` lists every image built by the Makefile. +7. Confirm that a successful optional-target result enables build/push, a nonzero result skips them unless `FORCE_BUILD_ALL` is `TRUE`, and when the target is absent publishing proceeds. +8. If any check fails, report failure or the blocking question; do not report success. + +## Output + +Return a concise report containing: + +- The created or updated workflow path +- The resolved version, build, and publish target names +- The variable contract used +- Whether Python/uv setup was included and why +- Any user question that blocked generation diff --git a/core/skills/rh-qs-test-suite/subagents/nightly-tests-prompt.md b/core/skills/rh-qs-test-suite/subagents/nightly-tests-prompt.md index 4ee22ac..038aa1b 100644 --- a/core/skills/rh-qs-test-suite/subagents/nightly-tests-prompt.md +++ b/core/skills/rh-qs-test-suite/subagents/nightly-tests-prompt.md @@ -81,7 +81,7 @@ If nothing is found, record "No cluster-admin requirement detected." If somethin ### Step 5: Detect the branch model -Determine the default branch (normally `main`) and whether a second long-lived branch exists (`dev`, `development`, or `develop` — check for all three; use whichever exists). If no second branch exists, you will generate only `nightly-main.yml` in Step 9 and note in your report that the dev nightly was skipped. +Determine the default branch and whether a second long-lived branch exists (`dev`, `development`, or `develop` — check for all three; use whichever exists) from Git metadata when it is available. If no Git metadata is available, assume `main` is the only branch; do not ask the user in this case. If no second branch exists, generate only `nightly-main.yml` in Step 9, use `dev: null` in the manifest, and note in your report that Git metadata was unavailable and the main-only branch model was assumed. ### Step 6: Copy composite actions from quickstart-ci-testing @@ -172,6 +172,19 @@ Summarize what was created/updated/skipped, then always include: 3. **Required secrets/variables**, names and explanations only, split into Secrets and Variables, plus any "found but unconfirmed" entries — never invent or ask for actual values. 4. **Next steps**: run `scripts/ci/setup-nightly-namespace.sh` against the target cluster, set the printed `PROD_TOKEN`/`PROD_SERVER` plus every other listed secret/variable in GitHub, then trigger `test-.yml` via `workflow_dispatch` once to confirm it's green before relying on the schedule. +## Verification + +Before reporting completion: + +1. Confirm every file you claim to have created or updated actually exists at + its declared path under the quickstart repository root. +2. Confirm the reusable workflow, at least one nightly caller, the provisioning + script, and the nightly manifest are present. +3. Confirm no unrelated file was created, updated, or deleted. +4. Parse every generated YAML file and run `bash -n` on the provisioning script. +5. If any required artifact is missing or any check fails, report failure rather + than claiming success. + ## Output `.qs-factory/pipeline/nightly-tests-manifest.yaml` (schema above) plus the report described in Step 11, returned as your final message to the orchestrator. diff --git a/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/.github/workflows/build-and-push.yaml b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/.github/workflows/build-and-push.yaml new file mode 100644 index 0000000..3abadee --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/.github/workflows/build-and-push.yaml @@ -0,0 +1,159 @@ +# Canonical local reference for rh-qs-test-suite image publishing. +# Adapt Make target names, variable names, REGISTRY, and toolchain setup to the +# quickstart's Makefile and README. Set IMAGE_REPOSITORIES to every image +# repository built by build-all-images, using fully qualified registry names. +# This workflow is self-contained; it is not a reusable workflow. +name: Build and push images + +on: + push: + branches: + - main + - dev + workflow_dispatch: + inputs: + target_branch: + description: Branch whose images should be built + required: true + default: main + type: choice + options: + - main + - dev + force_build_all: + description: Build and publish every image even when the optional Make target says none are needed + required: true + default: 'FALSE' + type: choice + options: + - 'FALSE' + - 'TRUE' + +concurrency: + group: ${{ github.workflow }}-${{ github.repository }}-${{ inputs.target_branch || github.ref_name }} + cancel-in-progress: false + queue: max + +permissions: + contents: read + +env: + REGISTRY: quay.io/rh-ai-quickstart + FORCE_BUILD_ALL: ${{ inputs.force_build_all || 'FALSE' }} + IMAGE_REPOSITORIES: "quay.io/rh-ai-quickstart/qs-test-quickstart" # Space-separated; include every image built by the Makefile. + +jobs: + build-and-push: + if: github.repository_owner == 'rh-ai-quickstart' && github.event.repository.fork == false + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v5 + with: + ref: ${{ inputs.target_branch || github.sha }} + + - name: Check whether image publishing is needed + id: image-check + run: | + MAKE_STATUS=0 + MAKE_DATABASE="$(make -qp 2>/dev/null)" || MAKE_STATUS=$? + if [ "${MAKE_STATUS}" -gt 1 ]; then + echo "Unable to inspect Makefile targets (make exited ${MAKE_STATUS})." >&2 + exit "${MAKE_STATUS}" + fi + + if grep -Eq '^build-push-images-needed::?([[:space:]]|$)' <<< "${MAKE_DATABASE}"; then + echo "Found build-push-images-needed; checking whether images need publishing." + TARGET_STATUS=0 + if make build-push-images-needed; then + TARGET_STATUS=0 + else + TARGET_STATUS=$? + fi + + if [ "${FORCE_BUILD_ALL}" = "TRUE" ]; then + echo "FORCE_BUILD_ALL=TRUE; building and publishing all images regardless of the optional target result." + echo "needed=true" >> "$GITHUB_OUTPUT" + elif [ "${TARGET_STATUS}" -eq 0 ]; then + echo "Image build and push are needed." + echo "needed=true" >> "$GITHUB_OUTPUT" + else + echo "Image build and push are not needed; skipping them." + echo "needed=false" >> "$GITHUB_OUTPUT" + fi + else + echo "No build-push-images-needed target found; proceeding with image publishing." + echo "needed=true" >> "$GITHUB_OUTPUT" + fi + + - name: Extract image tags + id: version + if: steps.image-check.outputs.needed == 'true' + env: + TARGET_BRANCH: ${{ inputs.target_branch || github.ref_name }} + run: | + BASE_VERSION=$(make image-version) + if [[ ! "${BASE_VERSION}" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]]; then + echo "make version must return a valid OCI tag (1-128 characters, starting with a letter, number, or underscore)." >&2 + exit 1 + fi + if [ "${TARGET_BRANCH}" = "main" ]; then + TAG_BASE="${BASE_VERSION}" + elif [ "${TARGET_BRANCH}" = "dev" ]; then + TAG_BASE="${BASE_VERSION}-dev" + else + echo "Unsupported target branch: ${TARGET_BRANCH}" >&2 + exit 1 + fi + COMMIT_SHA=$(git rev-parse HEAD) + VERSIONS="${TAG_BASE} ${TAG_BASE}-${COMMIT_SHA}" + if [ "${TARGET_BRANCH}" = "main" ]; then + VERSIONS="${VERSIONS} latest" + else + VERSIONS="${VERSIONS} dev" + fi + for TAG in ${VERSIONS}; do + if [[ ! "${TAG}" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]]; then + echo "Derived image tag is not a valid OCI tag: ${TAG}" >&2 + exit 1 + fi + done + echo "versions=${VERSIONS}" >> "$GITHUB_OUTPUT" + + - name: Set up Docker Buildx + if: steps.image-check.outputs.needed == 'true' + uses: docker/setup-buildx-action@v3 + + - name: Log in to Quay.io + if: steps.image-check.outputs.needed == 'true' + uses: docker/login-action@v3 + with: + registry: quay.io + username: ${{ secrets.QUAY_USERNAME }} + password: ${{ secrets.QUAY_PASSWORD }} + + - name: Build all images + if: steps.image-check.outputs.needed == 'true' + run: | + IFS=' ' read -ra VERSIONS <<< "${{ steps.version.outputs.versions }}" + REGISTRY="${REGISTRY}" VERSION="${VERSIONS[0]}" make build-images + + - name: Push all images + if: steps.image-check.outputs.needed == 'true' + run: | + IFS=' ' read -ra VERSIONS <<< "${{ steps.version.outputs.versions }}" + REGISTRY="${REGISTRY}" VERSION="${VERSIONS[0]}" make publish-images + + - name: Tag pushed images + if: steps.image-check.outputs.needed == 'true' + env: + IMAGE_TAGS: ${{ steps.version.outputs.versions }} + run: | + IFS=' ' read -ra VERSIONS <<< "${IMAGE_TAGS}" + PRIMARY_TAG="${VERSIONS[0]}" + for IMAGE in ${IMAGE_REPOSITORIES}; do + for TAG in "${VERSIONS[@]:1}"; do + echo "Tagging ${IMAGE}:${PRIMARY_TAG} as ${IMAGE}:${TAG}" + docker buildx imagetools create --tag "${IMAGE}:${TAG}" "${IMAGE}:${PRIMARY_TAG}" + done + done diff --git a/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/.github/workflows/pr-checks.yml b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/.github/workflows/pr-checks.yml new file mode 100644 index 0000000..3f5e3bb --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/.github/workflows/pr-checks.yml @@ -0,0 +1,9 @@ +name: PR checks +on: + pull_request: +jobs: + checks: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - run: make test-all diff --git a/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/Makefile b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/Makefile new file mode 100644 index 0000000..25c2ff7 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/Makefile @@ -0,0 +1,17 @@ +REGISTRY ?= quay.io/rh-ai-quickstart +VERSION ?= +IMAGE_NAME := qs-test-quickstart + +.PHONY: image-version build-images publish-images test-all + +image-version: + @echo 2.4.1 + +build-images: + @echo "Build all quickstart images with VERSION=$(VERSION) and REGISTRY=$(REGISTRY)" + +publish-images: + @echo "Publish all quickstart images with VERSION=$(VERSION) and REGISTRY=$(REGISTRY)" + +test-all: + @echo "Tests passed" diff --git a/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/README.md b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/README.md new file mode 100644 index 0000000..bef5e84 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/README.md @@ -0,0 +1,14 @@ +# Alternate Make target quickstart + +This quickstart publishes all of its images to `quay.io/rh-ai-quickstart`. +Its image repository is `quay.io/rh-ai-quickstart/qs-test-quickstart`. + +- `make image-version` prints the release version used for image tags. +- `make build-images VERSION= REGISTRY=` builds every image. +- `make publish-images VERSION= REGISTRY=` publishes every image. +- The Makefile does not define `version`, `build-all-images`, or `push-all-images`. +- These targets use the hosted runner's container tooling directly and do not + require Python or `uv`. + +The hosted image-publishing workflow should use the existing targets and pass +the `VERSION` and `REGISTRY` variables as documented here. diff --git a/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/helm/Chart.yaml b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/helm/Chart.yaml new file mode 100644 index 0000000..5cfae8d --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/expected-output-repo/helm/Chart.yaml @@ -0,0 +1,4 @@ +apiVersion: v2 +name: qs-test-quickstart +version: 0.1.0 +description: Quickstart fixture for alternate image Make targets diff --git a/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/input-repo/.github/workflows/pr-checks.yml b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/input-repo/.github/workflows/pr-checks.yml new file mode 100644 index 0000000..3f5e3bb --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/input-repo/.github/workflows/pr-checks.yml @@ -0,0 +1,9 @@ +name: PR checks +on: + pull_request: +jobs: + checks: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - run: make test-all diff --git a/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/input-repo/Makefile b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/input-repo/Makefile new file mode 100644 index 0000000..25c2ff7 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/input-repo/Makefile @@ -0,0 +1,17 @@ +REGISTRY ?= quay.io/rh-ai-quickstart +VERSION ?= +IMAGE_NAME := qs-test-quickstart + +.PHONY: image-version build-images publish-images test-all + +image-version: + @echo 2.4.1 + +build-images: + @echo "Build all quickstart images with VERSION=$(VERSION) and REGISTRY=$(REGISTRY)" + +publish-images: + @echo "Publish all quickstart images with VERSION=$(VERSION) and REGISTRY=$(REGISTRY)" + +test-all: + @echo "Tests passed" diff --git a/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/input-repo/README.md b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/input-repo/README.md new file mode 100644 index 0000000..bef5e84 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/input-repo/README.md @@ -0,0 +1,14 @@ +# Alternate Make target quickstart + +This quickstart publishes all of its images to `quay.io/rh-ai-quickstart`. +Its image repository is `quay.io/rh-ai-quickstart/qs-test-quickstart`. + +- `make image-version` prints the release version used for image tags. +- `make build-images VERSION= REGISTRY=` builds every image. +- `make publish-images VERSION= REGISTRY=` publishes every image. +- The Makefile does not define `version`, `build-all-images`, or `push-all-images`. +- These targets use the hosted runner's container tooling directly and do not + require Python or `uv`. + +The hosted image-publishing workflow should use the existing targets and pass +the `VERSION` and `REGISTRY` variables as documented here. diff --git a/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/input-repo/helm/Chart.yaml b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/input-repo/helm/Chart.yaml new file mode 100644 index 0000000..5cfae8d --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/input-repo/helm/Chart.yaml @@ -0,0 +1,4 @@ +apiVersion: v2 +name: qs-test-quickstart +version: 0.1.0 +description: Quickstart fixture for alternate image Make targets diff --git a/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/test-config.yaml b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/test-config.yaml new file mode 100644 index 0000000..4868618 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-alternate-targets/test-config.yaml @@ -0,0 +1,12 @@ +run-skill-command: Create only the build-and-push images workflow for qs-test-quickstart. +validation-options: + mode: full-baseline + additional-instructions: | + Confirm the workflow uses the documented existing targets: make image-version, + make build-images, and make publish-images. Pass VERSION and REGISTRY as + documented. Do not call absent make version, build-all-images, or + push-all-images targets. Confirm it retains main/dev push triggers, manual + branch selection, OCI tag validation, owner/fork guard, Quay secrets, and + branch queue behavior, omits Python/uv setup, and preserves all input files. + Confirm the Makefile lacks build-push-images-needed and the workflow + proceeds with publishing by default when the optional target is absent. diff --git a/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/.github/workflows/build-and-push.yaml b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/.github/workflows/build-and-push.yaml new file mode 100644 index 0000000..ebf9cf6 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/.github/workflows/build-and-push.yaml @@ -0,0 +1,159 @@ +# Canonical local reference for rh-qs-test-suite image publishing. +# Adapt Make target names, variable names, REGISTRY, and toolchain setup to the +# quickstart's Makefile and README. Set IMAGE_REPOSITORIES to every image +# repository built by build-all-images, using fully qualified registry names. +# This workflow is self-contained; it is not a reusable workflow. +name: Build and push images + +on: + push: + branches: + - main + - dev + workflow_dispatch: + inputs: + target_branch: + description: Branch whose images should be built + required: true + default: main + type: choice + options: + - main + - dev + force_build_all: + description: Build and publish every image even when the optional Make target says none are needed + required: true + default: 'FALSE' + type: choice + options: + - 'FALSE' + - 'TRUE' + +concurrency: + group: ${{ github.workflow }}-${{ github.repository }}-${{ inputs.target_branch || github.ref_name }} + cancel-in-progress: false + queue: max + +permissions: + contents: read + +env: + REGISTRY: quay.io/rh-ai-quickstart + FORCE_BUILD_ALL: ${{ inputs.force_build_all || 'FALSE' }} + IMAGE_REPOSITORIES: "quay.io/rh-ai-quickstart/qs-test-quickstart" # Space-separated; include every image built by the Makefile. + +jobs: + build-and-push: + if: github.repository_owner == 'rh-ai-quickstart' && github.event.repository.fork == false + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v5 + with: + ref: ${{ inputs.target_branch || github.sha }} + + - name: Check whether image publishing is needed + id: image-check + run: | + MAKE_STATUS=0 + MAKE_DATABASE="$(make -qp 2>/dev/null)" || MAKE_STATUS=$? + if [ "${MAKE_STATUS}" -gt 1 ]; then + echo "Unable to inspect Makefile targets (make exited ${MAKE_STATUS})." >&2 + exit "${MAKE_STATUS}" + fi + + if grep -Eq '^build-push-images-needed::?([[:space:]]|$)' <<< "${MAKE_DATABASE}"; then + echo "Found build-push-images-needed; checking whether images need publishing." + TARGET_STATUS=0 + if make build-push-images-needed; then + TARGET_STATUS=0 + else + TARGET_STATUS=$? + fi + + if [ "${FORCE_BUILD_ALL}" = "TRUE" ]; then + echo "FORCE_BUILD_ALL=TRUE; building and publishing all images regardless of the optional target result." + echo "needed=true" >> "$GITHUB_OUTPUT" + elif [ "${TARGET_STATUS}" -eq 0 ]; then + echo "Image build and push are needed." + echo "needed=true" >> "$GITHUB_OUTPUT" + else + echo "Image build and push are not needed; skipping them." + echo "needed=false" >> "$GITHUB_OUTPUT" + fi + else + echo "No build-push-images-needed target found; proceeding with image publishing." + echo "needed=true" >> "$GITHUB_OUTPUT" + fi + + - name: Extract image tags + id: version + if: steps.image-check.outputs.needed == 'true' + env: + TARGET_BRANCH: ${{ inputs.target_branch || github.ref_name }} + run: | + BASE_VERSION=$(make release-tag-source) + if [[ ! "${BASE_VERSION}" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]]; then + echo "make version must return a valid OCI tag (1-128 characters, starting with a letter, number, or underscore)." >&2 + exit 1 + fi + if [ "${TARGET_BRANCH}" = "main" ]; then + TAG_BASE="${BASE_VERSION}" + elif [ "${TARGET_BRANCH}" = "dev" ]; then + TAG_BASE="${BASE_VERSION}-dev" + else + echo "Unsupported target branch: ${TARGET_BRANCH}" >&2 + exit 1 + fi + COMMIT_SHA=$(git rev-parse HEAD) + VERSIONS="${TAG_BASE} ${TAG_BASE}-${COMMIT_SHA}" + if [ "${TARGET_BRANCH}" = "main" ]; then + VERSIONS="${VERSIONS} latest" + else + VERSIONS="${VERSIONS} dev" + fi + for TAG in ${VERSIONS}; do + if [[ ! "${TAG}" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]]; then + echo "Derived image tag is not a valid OCI tag: ${TAG}" >&2 + exit 1 + fi + done + echo "versions=${VERSIONS}" >> "$GITHUB_OUTPUT" + + - name: Set up Docker Buildx + if: steps.image-check.outputs.needed == 'true' + uses: docker/setup-buildx-action@v3 + + - name: Log in to Quay.io + if: steps.image-check.outputs.needed == 'true' + uses: docker/login-action@v3 + with: + registry: quay.io + username: ${{ secrets.QUAY_USERNAME }} + password: ${{ secrets.QUAY_PASSWORD }} + + - name: Build all images + if: steps.image-check.outputs.needed == 'true' + run: | + IFS=' ' read -ra VERSIONS <<< "${{ steps.version.outputs.versions }}" + REGISTRY="${REGISTRY}" VERSION="${VERSIONS[0]}" make assemble-container-set + + - name: Push all images + if: steps.image-check.outputs.needed == 'true' + run: | + IFS=' ' read -ra VERSIONS <<< "${{ steps.version.outputs.versions }}" + REGISTRY="${REGISTRY}" VERSION="${VERSIONS[0]}" make replicate-container-set + + - name: Tag pushed images + if: steps.image-check.outputs.needed == 'true' + env: + IMAGE_TAGS: ${{ steps.version.outputs.versions }} + run: | + IFS=' ' read -ra VERSIONS <<< "${IMAGE_TAGS}" + PRIMARY_TAG="${VERSIONS[0]}" + for IMAGE in ${IMAGE_REPOSITORIES}; do + for TAG in "${VERSIONS[@]:1}"; do + echo "Tagging ${IMAGE}:${PRIMARY_TAG} as ${IMAGE}:${TAG}" + docker buildx imagetools create --tag "${IMAGE}:${TAG}" "${IMAGE}:${PRIMARY_TAG}" + done + done diff --git a/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/.github/workflows/pr-checks.yml b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/.github/workflows/pr-checks.yml new file mode 100644 index 0000000..3f5e3bb --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/.github/workflows/pr-checks.yml @@ -0,0 +1,9 @@ +name: PR checks +on: + pull_request: +jobs: + checks: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - run: make test-all diff --git a/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/Makefile b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/Makefile new file mode 100644 index 0000000..610fdc5 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/Makefile @@ -0,0 +1,16 @@ +REGISTRY ?= quay.io/rh-ai-quickstart +VERSION ?= + +.PHONY: release-tag-source assemble-container-set replicate-container-set test-all + +release-tag-source: + @echo 2.4.2 + +assemble-container-set: + @echo "Fixture target; VERSION=$(VERSION) REGISTRY=$(REGISTRY)" + +replicate-container-set: + @echo "Fixture target; VERSION=$(VERSION) REGISTRY=$(REGISTRY)" + +test-all: + @echo "Tests passed" diff --git a/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/README.md b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/README.md new file mode 100644 index 0000000..21f4ce2 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/README.md @@ -0,0 +1,11 @@ +# Quickstart with undocumented image publishing targets + +This quickstart publishes all images to `quay.io/rh-ai-quickstart` on pushes to +`main` and `dev`, and supports manual dispatch for either branch. It uses +`QUAY_USERNAME` and `QUAY_PASSWORD` repository secrets and does not require +Python or `uv` on the hosted runner. + +The Makefile contains three legacy image-release targets, but their roles and +variable contract are not documented here. Ask the quickstart owner to confirm +which target prints the image release version, which builds all images, and +which publishes them before using those targets in CI. diff --git a/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/helm/Chart.yaml b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/helm/Chart.yaml new file mode 100644 index 0000000..5b79c7b --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/expected-output-repo/helm/Chart.yaml @@ -0,0 +1,4 @@ +apiVersion: v2 +name: qs-test-quickstart +version: 0.1.0 +description: Quickstart fixture without image publishing targets diff --git a/test/skills/rh-qs-test-suite/build-and-push-missing-targets/input-repo/.github/workflows/pr-checks.yml b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/input-repo/.github/workflows/pr-checks.yml new file mode 100644 index 0000000..3f5e3bb --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/input-repo/.github/workflows/pr-checks.yml @@ -0,0 +1,9 @@ +name: PR checks +on: + pull_request: +jobs: + checks: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - run: make test-all diff --git a/test/skills/rh-qs-test-suite/build-and-push-missing-targets/input-repo/Makefile b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/input-repo/Makefile new file mode 100644 index 0000000..610fdc5 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/input-repo/Makefile @@ -0,0 +1,16 @@ +REGISTRY ?= quay.io/rh-ai-quickstart +VERSION ?= + +.PHONY: release-tag-source assemble-container-set replicate-container-set test-all + +release-tag-source: + @echo 2.4.2 + +assemble-container-set: + @echo "Fixture target; VERSION=$(VERSION) REGISTRY=$(REGISTRY)" + +replicate-container-set: + @echo "Fixture target; VERSION=$(VERSION) REGISTRY=$(REGISTRY)" + +test-all: + @echo "Tests passed" diff --git a/test/skills/rh-qs-test-suite/build-and-push-missing-targets/input-repo/README.md b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/input-repo/README.md new file mode 100644 index 0000000..21f4ce2 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/input-repo/README.md @@ -0,0 +1,11 @@ +# Quickstart with undocumented image publishing targets + +This quickstart publishes all images to `quay.io/rh-ai-quickstart` on pushes to +`main` and `dev`, and supports manual dispatch for either branch. It uses +`QUAY_USERNAME` and `QUAY_PASSWORD` repository secrets and does not require +Python or `uv` on the hosted runner. + +The Makefile contains three legacy image-release targets, but their roles and +variable contract are not documented here. Ask the quickstart owner to confirm +which target prints the image release version, which builds all images, and +which publishes them before using those targets in CI. diff --git a/test/skills/rh-qs-test-suite/build-and-push-missing-targets/input-repo/helm/Chart.yaml b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/input-repo/helm/Chart.yaml new file mode 100644 index 0000000..5b79c7b --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/input-repo/helm/Chart.yaml @@ -0,0 +1,4 @@ +apiVersion: v2 +name: qs-test-quickstart +version: 0.1.0 +description: Quickstart fixture without image publishing targets diff --git a/test/skills/rh-qs-test-suite/build-and-push-missing-targets/test-config.yaml b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/test-config.yaml new file mode 100644 index 0000000..8105947 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-missing-targets/test-config.yaml @@ -0,0 +1,18 @@ +run-skill-command: Create only the build-and-push images workflow for qs-test-quickstart. +question-answer-context: >- + The Makefile targets are release-tag-source for the version, which prints a + valid release tag; assemble-container-set for building all images; and + replicate-container-set for publishing all images. The build and publish + targets accept VERSION and REGISTRY, and build/publish + quay.io/rh-ai-quickstart/qs-test-quickstart. +validation-options: + mode: full-baseline + additional-instructions: | + Confirm the conversation shows that the skill asks the user to map the + legacy Makefile targets to version/build/publish, receives the mapping, and + creates the workflow using make release-tag-source, make + assemble-container-set, and make replicate-container-set with VERSION and + REGISTRY. Confirm the Makefile lacks build-push-images-needed and the + workflow proceeds with publishing by default when the optional target is + absent. Preserve all input files and create no files other than the + requested workflow. diff --git a/test/skills/rh-qs-test-suite/build-and-push-no-python/expected-output-repo/.github/workflows/build-and-push.yaml b/test/skills/rh-qs-test-suite/build-and-push-no-python/expected-output-repo/.github/workflows/build-and-push.yaml new file mode 100644 index 0000000..feeea74 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-no-python/expected-output-repo/.github/workflows/build-and-push.yaml @@ -0,0 +1,159 @@ +# Canonical local reference for rh-qs-test-suite image publishing. +# Adapt Make target names, variable names, REGISTRY, and toolchain setup to the +# quickstart's Makefile and README. Set IMAGE_REPOSITORIES to every image +# repository built by build-all-images, using fully qualified registry names. +# This workflow is self-contained; it is not a reusable workflow. +name: Build and push images + +on: + push: + branches: + - main + - dev + workflow_dispatch: + inputs: + target_branch: + description: Branch whose images should be built + required: true + default: main + type: choice + options: + - main + - dev + force_build_all: + description: Build and publish every image even when the optional Make target says none are needed + required: true + default: 'FALSE' + type: choice + options: + - 'FALSE' + - 'TRUE' + +concurrency: + group: ${{ github.workflow }}-${{ github.repository }}-${{ inputs.target_branch || github.ref_name }} + cancel-in-progress: false + queue: max + +permissions: + contents: read + +env: + REGISTRY: quay.io/rh-ai-quickstart + FORCE_BUILD_ALL: ${{ inputs.force_build_all || 'FALSE' }} + IMAGE_REPOSITORIES: "quay.io/rh-ai-quickstart/qs-test-quickstart" # Space-separated; include every image built by the Makefile. + +jobs: + build-and-push: + if: github.repository_owner == 'rh-ai-quickstart' && github.event.repository.fork == false + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v5 + with: + ref: ${{ inputs.target_branch || github.sha }} + + - name: Check whether image publishing is needed + id: image-check + run: | + MAKE_STATUS=0 + MAKE_DATABASE="$(make -qp 2>/dev/null)" || MAKE_STATUS=$? + if [ "${MAKE_STATUS}" -gt 1 ]; then + echo "Unable to inspect Makefile targets (make exited ${MAKE_STATUS})." >&2 + exit "${MAKE_STATUS}" + fi + + if grep -Eq '^build-push-images-needed::?([[:space:]]|$)' <<< "${MAKE_DATABASE}"; then + echo "Found build-push-images-needed; checking whether images need publishing." + TARGET_STATUS=0 + if make build-push-images-needed; then + TARGET_STATUS=0 + else + TARGET_STATUS=$? + fi + + if [ "${FORCE_BUILD_ALL}" = "TRUE" ]; then + echo "FORCE_BUILD_ALL=TRUE; building and publishing all images regardless of the optional target result." + echo "needed=true" >> "$GITHUB_OUTPUT" + elif [ "${TARGET_STATUS}" -eq 0 ]; then + echo "Image build and push are needed." + echo "needed=true" >> "$GITHUB_OUTPUT" + else + echo "Image build and push are not needed; skipping them." + echo "needed=false" >> "$GITHUB_OUTPUT" + fi + else + echo "No build-push-images-needed target found; proceeding with image publishing." + echo "needed=true" >> "$GITHUB_OUTPUT" + fi + + - name: Extract image tags + id: version + if: steps.image-check.outputs.needed == 'true' + env: + TARGET_BRANCH: ${{ inputs.target_branch || github.ref_name }} + run: | + BASE_VERSION=$(make version) + if [[ ! "${BASE_VERSION}" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]]; then + echo "make version must return a valid OCI tag (1-128 characters, starting with a letter, number, or underscore)." >&2 + exit 1 + fi + if [ "${TARGET_BRANCH}" = "main" ]; then + TAG_BASE="${BASE_VERSION}" + elif [ "${TARGET_BRANCH}" = "dev" ]; then + TAG_BASE="${BASE_VERSION}-dev" + else + echo "Unsupported target branch: ${TARGET_BRANCH}" >&2 + exit 1 + fi + COMMIT_SHA=$(git rev-parse HEAD) + VERSIONS="${TAG_BASE} ${TAG_BASE}-${COMMIT_SHA}" + if [ "${TARGET_BRANCH}" = "main" ]; then + VERSIONS="${VERSIONS} latest" + else + VERSIONS="${VERSIONS} dev" + fi + for TAG in ${VERSIONS}; do + if [[ ! "${TAG}" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]]; then + echo "Derived image tag is not a valid OCI tag: ${TAG}" >&2 + exit 1 + fi + done + echo "versions=${VERSIONS}" >> "$GITHUB_OUTPUT" + + - name: Set up Docker Buildx + if: steps.image-check.outputs.needed == 'true' + uses: docker/setup-buildx-action@v3 + + - name: Log in to Quay.io + if: steps.image-check.outputs.needed == 'true' + uses: docker/login-action@v3 + with: + registry: quay.io + username: ${{ secrets.QUAY_USERNAME }} + password: ${{ secrets.QUAY_PASSWORD }} + + - name: Build all images + if: steps.image-check.outputs.needed == 'true' + run: | + IFS=' ' read -ra VERSIONS <<< "${{ steps.version.outputs.versions }}" + VERSION="${VERSIONS[0]}" make build-all-images + + - name: Push all images + if: steps.image-check.outputs.needed == 'true' + run: | + IFS=' ' read -ra VERSIONS <<< "${{ steps.version.outputs.versions }}" + VERSION="${VERSIONS[0]}" make push-all-images + + - name: Tag pushed images + if: steps.image-check.outputs.needed == 'true' + env: + IMAGE_TAGS: ${{ steps.version.outputs.versions }} + run: | + IFS=' ' read -ra VERSIONS <<< "${IMAGE_TAGS}" + PRIMARY_TAG="${VERSIONS[0]}" + for IMAGE in ${IMAGE_REPOSITORIES}; do + for TAG in "${VERSIONS[@]:1}"; do + echo "Tagging ${IMAGE}:${PRIMARY_TAG} as ${IMAGE}:${TAG}" + docker buildx imagetools create --tag "${IMAGE}:${TAG}" "${IMAGE}:${PRIMARY_TAG}" + done + done diff --git a/test/skills/rh-qs-test-suite/build-and-push-no-python/expected-output-repo/.github/workflows/pr-checks.yml b/test/skills/rh-qs-test-suite/build-and-push-no-python/expected-output-repo/.github/workflows/pr-checks.yml new file mode 100644 index 0000000..a30e3fd --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-no-python/expected-output-repo/.github/workflows/pr-checks.yml @@ -0,0 +1,11 @@ +name: Existing PR checks + +on: + pull_request: + +jobs: + checks: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - run: make test-all diff --git a/test/skills/rh-qs-test-suite/build-and-push-no-python/expected-output-repo/Makefile b/test/skills/rh-qs-test-suite/build-and-push-no-python/expected-output-repo/Makefile new file mode 100644 index 0000000..88cd416 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-no-python/expected-output-repo/Makefile @@ -0,0 +1,16 @@ +REGISTRY := quay.io/rh-ai-quickstart +IMAGE_NAME := qs-test-quickstart + +.PHONY: version build-all-images push-all-images test-all + +version: + @echo 2.4.0 + +build-all-images: + @echo "Fixture build target; does not invoke a container engine." + +push-all-images: + @echo "Fixture push target; does not access a registry." + +test-all: + @echo "Fixture test target." diff --git a/test/skills/rh-qs-test-suite/build-and-push-no-python/expected-output-repo/README.md b/test/skills/rh-qs-test-suite/build-and-push-no-python/expected-output-repo/README.md new file mode 100644 index 0000000..7d75aec --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-no-python/expected-output-repo/README.md @@ -0,0 +1,16 @@ +# qs-test-quickstart + +## Image publishing + +- Publish images to `quay.io/rh-ai-quickstart`. +- Publish automatically on every push to the `main` and `dev` branches of the + `rh-ai-quickstart` organization, including pushes caused by merged PRs. Do + not skip a push because the release version is unchanged. +- Never publish images from a forked repository. +- Allow a manual workflow dispatch to select either `main` or `dev` and + build and publish the version returned by `make version`. +- `make version` prints the release version used for image tags. +- `make build-all-images` and `make push-all-images` use the hosted runner's + container tooling directly. They do not use Python, `uv`, or another + application runtime. +- GitHub Actions must read `QUAY_USERNAME` and `QUAY_PASSWORD` from repository secrets. diff --git a/test/skills/rh-qs-test-suite/build-and-push-no-python/input-repo/.github/workflows/pr-checks.yml b/test/skills/rh-qs-test-suite/build-and-push-no-python/input-repo/.github/workflows/pr-checks.yml new file mode 100644 index 0000000..a30e3fd --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-no-python/input-repo/.github/workflows/pr-checks.yml @@ -0,0 +1,11 @@ +name: Existing PR checks + +on: + pull_request: + +jobs: + checks: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - run: make test-all diff --git a/test/skills/rh-qs-test-suite/build-and-push-no-python/input-repo/Makefile b/test/skills/rh-qs-test-suite/build-and-push-no-python/input-repo/Makefile new file mode 100644 index 0000000..88cd416 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-no-python/input-repo/Makefile @@ -0,0 +1,16 @@ +REGISTRY := quay.io/rh-ai-quickstart +IMAGE_NAME := qs-test-quickstart + +.PHONY: version build-all-images push-all-images test-all + +version: + @echo 2.4.0 + +build-all-images: + @echo "Fixture build target; does not invoke a container engine." + +push-all-images: + @echo "Fixture push target; does not access a registry." + +test-all: + @echo "Fixture test target." diff --git a/test/skills/rh-qs-test-suite/build-and-push-no-python/input-repo/README.md b/test/skills/rh-qs-test-suite/build-and-push-no-python/input-repo/README.md new file mode 100644 index 0000000..7d75aec --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-no-python/input-repo/README.md @@ -0,0 +1,16 @@ +# qs-test-quickstart + +## Image publishing + +- Publish images to `quay.io/rh-ai-quickstart`. +- Publish automatically on every push to the `main` and `dev` branches of the + `rh-ai-quickstart` organization, including pushes caused by merged PRs. Do + not skip a push because the release version is unchanged. +- Never publish images from a forked repository. +- Allow a manual workflow dispatch to select either `main` or `dev` and + build and publish the version returned by `make version`. +- `make version` prints the release version used for image tags. +- `make build-all-images` and `make push-all-images` use the hosted runner's + container tooling directly. They do not use Python, `uv`, or another + application runtime. +- GitHub Actions must read `QUAY_USERNAME` and `QUAY_PASSWORD` from repository secrets. diff --git a/test/skills/rh-qs-test-suite/build-and-push-no-python/test-config.yaml b/test/skills/rh-qs-test-suite/build-and-push-no-python/test-config.yaml new file mode 100644 index 0000000..45612b6 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-no-python/test-config.yaml @@ -0,0 +1,12 @@ +run-skill-command: Create only the build-and-push images workflow for qs-test-quickstart. +validation-options: + mode: full-baseline + additional-instructions: | + Confirm the expected workflow runs on every main/dev push, supports manual + branch selection, uses make version plus build-all-images/push-all-images, + validates the base and derived OCI tags, and has no uv/Python setup or + UV_VERSION/PYTHON_VERSION values. Confirm it excludes forks, uses the + required Quay secrets, queues same-branch runs without cancellation, and + preserves the existing PR workflow. Confirm the Makefile lacks + build-push-images-needed and the workflow proceeds with publishing by + default when the optional target is absent. diff --git a/test/skills/rh-qs-test-suite/build-and-push-standard/expected-output-repo/.github/workflows/build-and-push.yaml b/test/skills/rh-qs-test-suite/build-and-push-standard/expected-output-repo/.github/workflows/build-and-push.yaml new file mode 100644 index 0000000..996d04d --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-standard/expected-output-repo/.github/workflows/build-and-push.yaml @@ -0,0 +1,170 @@ +# Canonical local reference for rh-qs-test-suite image publishing. +# Adapt Make target names, variable names, REGISTRY, and toolchain setup to the +# quickstart's Makefile and README. Set IMAGE_REPOSITORIES to every image +# repository built by build-all-images, using fully qualified registry names. +# This workflow is self-contained; it is not a reusable workflow. +name: Build and push images + +on: + push: + branches: + - main + - dev + workflow_dispatch: + inputs: + target_branch: + description: Branch whose images should be built + required: true + default: main + type: choice + options: + - main + - dev + force_build_all: + description: Build and publish every image even when the optional Make target says none are needed + required: true + default: 'FALSE' + type: choice + options: + - 'FALSE' + - 'TRUE' + +concurrency: + group: ${{ github.workflow }}-${{ github.repository }}-${{ inputs.target_branch || github.ref_name }} + cancel-in-progress: false + queue: max + +permissions: + contents: read + +env: + REGISTRY: quay.io/rh-ai-quickstart + FORCE_BUILD_ALL: ${{ inputs.force_build_all || 'FALSE' }} + IMAGE_REPOSITORIES: "quay.io/rh-ai-quickstart/qs-test-quickstart" # Space-separated; include every image built by the Makefile. + UV_VERSION: "0.8.9" # Replace with the quickstart's Makefile value when uv is used. + PYTHON_VERSION: "3.12" # Replace with the quickstart's required Python version. + +jobs: + build-and-push: + if: github.repository_owner == 'rh-ai-quickstart' && github.event.repository.fork == false + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v5 + with: + ref: ${{ inputs.target_branch || github.sha }} + + - name: Check whether image publishing is needed + id: image-check + run: | + MAKE_STATUS=0 + MAKE_DATABASE="$(make -qp 2>/dev/null)" || MAKE_STATUS=$? + if [ "${MAKE_STATUS}" -gt 1 ]; then + echo "Unable to inspect Makefile targets (make exited ${MAKE_STATUS})." >&2 + exit "${MAKE_STATUS}" + fi + + if grep -Eq '^build-push-images-needed::?([[:space:]]|$)' <<< "${MAKE_DATABASE}"; then + echo "Found build-push-images-needed; checking whether images need publishing." + TARGET_STATUS=0 + if make build-push-images-needed; then + TARGET_STATUS=0 + else + TARGET_STATUS=$? + fi + + if [ "${FORCE_BUILD_ALL}" = "TRUE" ]; then + echo "FORCE_BUILD_ALL=TRUE; building and publishing all images regardless of the optional target result." + echo "needed=true" >> "$GITHUB_OUTPUT" + elif [ "${TARGET_STATUS}" -eq 0 ]; then + echo "Image build and push are needed." + echo "needed=true" >> "$GITHUB_OUTPUT" + else + echo "Image build and push are not needed; skipping them." + echo "needed=false" >> "$GITHUB_OUTPUT" + fi + else + echo "No build-push-images-needed target found; proceeding with image publishing." + echo "needed=true" >> "$GITHUB_OUTPUT" + fi + + - name: Extract image tags + id: version + if: steps.image-check.outputs.needed == 'true' + env: + TARGET_BRANCH: ${{ inputs.target_branch || github.ref_name }} + run: | + BASE_VERSION=$(make version) + if [[ ! "${BASE_VERSION}" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]]; then + echo "make version must return a valid OCI tag (1-128 characters, starting with a letter, number, or underscore)." >&2 + exit 1 + fi + if [ "${TARGET_BRANCH}" = "main" ]; then + TAG_BASE="${BASE_VERSION}" + elif [ "${TARGET_BRANCH}" = "dev" ]; then + TAG_BASE="${BASE_VERSION}-dev" + else + echo "Unsupported target branch: ${TARGET_BRANCH}" >&2 + exit 1 + fi + COMMIT_SHA=$(git rev-parse HEAD) + VERSIONS="${TAG_BASE} ${TAG_BASE}-${COMMIT_SHA}" + if [ "${TARGET_BRANCH}" = "main" ]; then + VERSIONS="${VERSIONS} latest" + else + VERSIONS="${VERSIONS} dev" + fi + for TAG in ${VERSIONS}; do + if [[ ! "${TAG}" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]]; then + echo "Derived image tag is not a valid OCI tag: ${TAG}" >&2 + exit 1 + fi + done + echo "versions=${VERSIONS}" >> "$GITHUB_OUTPUT" + + - name: Set up Docker Buildx + if: steps.image-check.outputs.needed == 'true' + uses: docker/setup-buildx-action@v3 + + - name: Log in to Quay.io + if: steps.image-check.outputs.needed == 'true' + uses: docker/login-action@v3 + with: + registry: quay.io + username: ${{ secrets.QUAY_USERNAME }} + password: ${{ secrets.QUAY_PASSWORD }} + + # Include this step only if the quickstart uses uv, with versions from + # its Makefile and README. Add other required runtime setup as needed. + - name: Set up uv + if: steps.image-check.outputs.needed == 'true' + uses: astral-sh/setup-uv@v6 + with: + version: ${{ env.UV_VERSION }} + python-version: ${{ env.PYTHON_VERSION }} + + - name: Build all images + if: steps.image-check.outputs.needed == 'true' + run: | + IFS=' ' read -ra VERSIONS <<< "${{ steps.version.outputs.versions }}" + VERSION="${VERSIONS[0]}" make build-all-images + + - name: Push all images + if: steps.image-check.outputs.needed == 'true' + run: | + IFS=' ' read -ra VERSIONS <<< "${{ steps.version.outputs.versions }}" + VERSION="${VERSIONS[0]}" make push-all-images + + - name: Tag pushed images + if: steps.image-check.outputs.needed == 'true' + env: + IMAGE_TAGS: ${{ steps.version.outputs.versions }} + run: | + IFS=' ' read -ra VERSIONS <<< "${IMAGE_TAGS}" + PRIMARY_TAG="${VERSIONS[0]}" + for IMAGE in ${IMAGE_REPOSITORIES}; do + for TAG in "${VERSIONS[@]:1}"; do + echo "Tagging ${IMAGE}:${PRIMARY_TAG} as ${IMAGE}:${TAG}" + docker buildx imagetools create --tag "${IMAGE}:${TAG}" "${IMAGE}:${PRIMARY_TAG}" + done + done diff --git a/test/skills/rh-qs-test-suite/build-and-push-standard/expected-output-repo/Makefile b/test/skills/rh-qs-test-suite/build-and-push-standard/expected-output-repo/Makefile new file mode 100644 index 0000000..a254395 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-standard/expected-output-repo/Makefile @@ -0,0 +1,18 @@ +UV_VERSION := 0.5.24 +PYTHON_VERSION := 3.11 +REGISTRY := quay.io/rh-ai-quickstart +IMAGE_NAME := qs-test-quickstart + +.PHONY: version build-all-images push-all-images build-push-images-needed + +version: + @echo 1.2.3 + +build-all-images: + @echo "Fixture build target; does not invoke a container engine." + +push-all-images: + @echo "Fixture push target; does not access a registry." + +build-push-images-needed: + @test "$${BUILD_PUSH_IMAGES_NEEDED:-0}" = "1" diff --git a/test/skills/rh-qs-test-suite/build-and-push-standard/expected-output-repo/README.md b/test/skills/rh-qs-test-suite/build-and-push-standard/expected-output-repo/README.md new file mode 100644 index 0000000..2977fc7 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-standard/expected-output-repo/README.md @@ -0,0 +1,20 @@ +# qs-test-quickstart + +## Image publishing + +- Publish images to `quay.io/rh-ai-quickstart`. +- Publish automatically on every push to the `main` and `dev` branches of the + `rh-ai-quickstart` organization, including pushes caused by merged PRs. Do + not skip a push because the release version is unchanged. +- Never publish images from a forked repository. +- Allow a manual workflow dispatch to select either `main` or `dev` and + build and publish the version returned by `make version`. +- `make version` prints the release version used for image tags. +- On `main`, publish the release version, a version-plus-commit tag, and + `latest`. On `dev`, publish `-dev`, a version-plus-commit tag, and + `dev`. +- `make build-all-images` builds every quickstart image. +- `make push-all-images` publishes the built images using `REGISTRY` and `VERSION`. +- `make build-push-images-needed` exits 0 when image build/push is needed and nonzero otherwise. +- The image build uses Python 3.11 and `uv` 0.5.24. +- GitHub Actions must read `QUAY_USERNAME` and `QUAY_PASSWORD` from repository secrets. diff --git a/test/skills/rh-qs-test-suite/build-and-push-standard/expected-output-repo/helm/Chart.yaml b/test/skills/rh-qs-test-suite/build-and-push-standard/expected-output-repo/helm/Chart.yaml new file mode 100644 index 0000000..2c797c0 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-standard/expected-output-repo/helm/Chart.yaml @@ -0,0 +1,3 @@ +apiVersion: v2 +name: qs-test-quickstart +version: 0.1.0 diff --git a/test/skills/rh-qs-test-suite/build-and-push-standard/input-repo/Makefile b/test/skills/rh-qs-test-suite/build-and-push-standard/input-repo/Makefile new file mode 100644 index 0000000..a254395 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-standard/input-repo/Makefile @@ -0,0 +1,18 @@ +UV_VERSION := 0.5.24 +PYTHON_VERSION := 3.11 +REGISTRY := quay.io/rh-ai-quickstart +IMAGE_NAME := qs-test-quickstart + +.PHONY: version build-all-images push-all-images build-push-images-needed + +version: + @echo 1.2.3 + +build-all-images: + @echo "Fixture build target; does not invoke a container engine." + +push-all-images: + @echo "Fixture push target; does not access a registry." + +build-push-images-needed: + @test "$${BUILD_PUSH_IMAGES_NEEDED:-0}" = "1" diff --git a/test/skills/rh-qs-test-suite/build-and-push-standard/input-repo/README.md b/test/skills/rh-qs-test-suite/build-and-push-standard/input-repo/README.md new file mode 100644 index 0000000..2977fc7 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-standard/input-repo/README.md @@ -0,0 +1,20 @@ +# qs-test-quickstart + +## Image publishing + +- Publish images to `quay.io/rh-ai-quickstart`. +- Publish automatically on every push to the `main` and `dev` branches of the + `rh-ai-quickstart` organization, including pushes caused by merged PRs. Do + not skip a push because the release version is unchanged. +- Never publish images from a forked repository. +- Allow a manual workflow dispatch to select either `main` or `dev` and + build and publish the version returned by `make version`. +- `make version` prints the release version used for image tags. +- On `main`, publish the release version, a version-plus-commit tag, and + `latest`. On `dev`, publish `-dev`, a version-plus-commit tag, and + `dev`. +- `make build-all-images` builds every quickstart image. +- `make push-all-images` publishes the built images using `REGISTRY` and `VERSION`. +- `make build-push-images-needed` exits 0 when image build/push is needed and nonzero otherwise. +- The image build uses Python 3.11 and `uv` 0.5.24. +- GitHub Actions must read `QUAY_USERNAME` and `QUAY_PASSWORD` from repository secrets. diff --git a/test/skills/rh-qs-test-suite/build-and-push-standard/input-repo/helm/Chart.yaml b/test/skills/rh-qs-test-suite/build-and-push-standard/input-repo/helm/Chart.yaml new file mode 100644 index 0000000..2c797c0 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-standard/input-repo/helm/Chart.yaml @@ -0,0 +1,3 @@ +apiVersion: v2 +name: qs-test-quickstart +version: 0.1.0 diff --git a/test/skills/rh-qs-test-suite/build-and-push-standard/test-config.yaml b/test/skills/rh-qs-test-suite/build-and-push-standard/test-config.yaml new file mode 100644 index 0000000..bf96224 --- /dev/null +++ b/test/skills/rh-qs-test-suite/build-and-push-standard/test-config.yaml @@ -0,0 +1,21 @@ +run-skill-command: Create only the build-and-push images workflow for qs-test-quickstart. +validation-options: + mode: full-baseline + additional-instructions: | + In addition to matching the full expected tree, confirm the build-and-push + workflow runs on every main and dev push with no version-change guard, + supports workflow_dispatch with branch selection and uses make version, + serializes each branch with queue: max and no cancellation, is self-contained + with no workflow_call, limits publishing to + the rh-ai-quickstart owner and excludes + forked repositories, uses the QUAY_USERNAME and QUAY_PASSWORD secrets, + and invokes the fixture's make + version, make build-all-images, and make push-all-images targets. Confirm + the workflow rejects an invalid make version and invalid derived OCI tags + before image build or push steps. Confirm it detects the optional + build-push-images-needed target and gates build/push/retag as follows: + when FORCE_BUILD_ALL is not TRUE, a zero exit runs those steps and a + nonzero exit skips them; when FORCE_BUILD_ALL is TRUE, those steps run + even if the target exits nonzero. Confirm the force_build_all manual input + defaults to FALSE. The fixture target can return either status based on + BUILD_PUSH_IMAGES_NEEDED. diff --git a/test/skills/rh-qs-test-suite/standard-pattern/expected-output-repo/.qs-factory/pipeline/nightly-tests-manifest.yaml b/test/skills/rh-qs-test-suite/standard-pattern/expected-output-repo/.qs-factory/pipeline/nightly-tests-manifest.yaml new file mode 100644 index 0000000..9adfc32 --- /dev/null +++ b/test/skills/rh-qs-test-suite/standard-pattern/expected-output-repo/.qs-factory/pipeline/nightly-tests-manifest.yaml @@ -0,0 +1,52 @@ +slug: qs-test-quickstart +resolved_targets: + deploy: deploy + test: test + undeploy: undeploy +toolchain_setup_steps: +- action: astral-sh/setup-uv@v6 + with: + version: 0.5.24 + source: Makefile UV_VERSION and README prerequisites +branch_model: + main: main + dev: null +namespace: qs-test-quickstart-ci-project-1 +service_account: qs-test-quickstart-ci-1 +cluster_admin_check: + status: clean + detail: No cluster-admin requirement detected; chart contains a namespaced Deployment + and has no subchart dependencies. +required_secrets: +- name: PROD_TOKEN + reason: ServiceAccount token for the nightly CI namespace +- name: PROD_SERVER + reason: OpenShift API server URL for the cluster nightly deploys to +- name: MODEL_API_KEY + reason: Required by check-model-config prerequisite of deploy and test; passed to + Helm model.apiKey by deploy. +required_variables: +- name: MODEL_ID + reason: Required by check-model-config prerequisite of deploy and test; passed to + Helm model.id by deploy. +- name: MODEL_URL + reason: Required by check-model-config prerequisite of deploy and test; passed to + Helm model.url by deploy. +unconfirmed: [] +files: + actions_copied: + - .github/actions/cleanup + - .github/actions/clone-quickstart + - .github/actions/prepare-runner + - .github/actions/run-quickstart-tests + - .github/actions/run-tests + - .github/actions/setup-and-deploy + - .github/actions/setup-kind + - .github/actions/setup-openshift + actions_skipped_identical: [] + actions_updated: [] + workflows_created: + - .github/workflows/test-qs-test-quickstart.yml + - .github/workflows/nightly-main.yml + workflows_skipped_or_updated: [] + provisioning_script: scripts/ci/setup-nightly-namespace.sh diff --git a/test/skills/rh-qs-test-suite/standard-pattern/expected-output-repo/pipeline/nightly-tests-manifest.yaml b/test/skills/rh-qs-test-suite/standard-pattern/expected-output-repo/pipeline/nightly-tests-manifest.yaml deleted file mode 100644 index b7996e8..0000000 --- a/test/skills/rh-qs-test-suite/standard-pattern/expected-output-repo/pipeline/nightly-tests-manifest.yaml +++ /dev/null @@ -1,48 +0,0 @@ -slug: qs-test-quickstart -resolved_targets: - deploy: deploy - test: test - undeploy: undeploy -toolchain_setup_steps: - - action: astral-sh/setup-uv@v6 - with: - version: "0.5.24" - source: README.md and Makefile UV_VERSION -branch_model: - main: main - dev: null -namespace: qs-test-quickstart-ci-project-1 -service_account: qs-test-quickstart-ci-1 -cluster_admin_check: - status: clean - detail: No cluster-admin requirement detected. -required_secrets: - - name: PROD_TOKEN - reason: ServiceAccount token for the nightly CI namespace - - name: PROD_SERVER - reason: OpenShift API server URL for the cluster nightly deploys to - - name: MODEL_API_KEY - reason: API key required by the deploy and test Makefile targets -required_variables: - - name: MODEL_ID - reason: Model deployment ID required by the deploy and test Makefile targets - - name: MODEL_URL - reason: Model API URL required by the deploy and test Makefile targets -unconfirmed: [] -files: - actions_copied: - - cleanup - - clone-quickstart - - prepare-runner - - run-quickstart-tests - - run-tests - - setup-and-deploy - - setup-kind - - setup-openshift - actions_skipped_identical: [] - actions_updated: [] - workflows_created: - - .github/workflows/test-qs-test-quickstart.yml - - .github/workflows/nightly-main.yml - workflows_skipped_or_updated: [] - provisioning_script: scripts/ci/setup-nightly-namespace.sh