Skip to content

ci: replace generated artifact-matrix workflows with one reusable + thin callers#475

Open
igorpecovnik wants to merge 1 commit into
mainfrom
feat/gha-matrix-generator-rewrite
Open

ci: replace generated artifact-matrix workflows with one reusable + thin callers#475
igorpecovnik wants to merge 1 commit into
mainfrom
feat/gha-matrix-generator-rewrite

Conversation

@igorpecovnik

Copy link
Copy Markdown
Member

What

Replaces the six generated complete-artifact-matrix-*.yml workflows (~9,400 lines each) and their generator with one hand-written reusable pipeline plus six thin per-track callers. No more code generation, no committed 9k-line giants, no 6× duplication.

Before After
6 generated workflows (~56k lines) 6 thin callers (~60 lines each)
recreate-matrix.yml (600-line bash/awk generator) — (deleted)
8 Jinja chunk templates + 8 gha_config_*.yaml complete-artifact-matrix.yml + 2 reusable chunk bodies
watchdog.yml (15-min cron, reruns failed jobs) retry-on-failure inlined in the build steps

2,160 lines total, hand-maintained.

How it works

  • complete-artifact-matrix.yml — reusable (workflow_call) machinery: team_check → version_prep → matrix_prep → {artifact chunks} → {image chunks} → publish-debs-to-repo → closing.
  • build-artifacts-chunk.yml / build-images-chunk.yml — the per-chunk build bodies, written once and fanned out over a chunk: [1..17] / [1..16] matrix that indexes matrix_prep's named outputs (outputs[format('artifacts-chunk-json-{0}', matrix.chunk)]).
  • 6 callers (build-nightly, build-standard-support, build-community, build-apps, build-all, build-all-stable) — each holds its own schedule, concurrency, dispatch inputs, and a single uses: with the track's values.

Per-track behaviour (schedules, repos, upload paths, menu defaults, the CHECK_OCI-uses-nightlybuild-default quirk) is preserved. The empty all/all-stable board dropdown (a missing-file bug in the old generator) is gone; use targetsFilterInclude to scope a board.

Targets (targets-release-*.yaml) are already fetched at runtime from github.armbian.com/release-targets/, so board classification / targets generation no longer lives in this repo. targets-release-*.yaml, the templates, and complete-artifact-one-by-one.yml are left untouched.

watchdog.yml is dropped; both build steps now retry the compile.sh invocation BUILD_ATTEMPTS (default 3) times with backoff. watchdog's entry list moved with it.

Validation

  • actionlint clean (custom self-hosted runner labels declared in .github/actionlint.yaml).
  • The matrix + reusable-workflow dynamic-output-indexing mechanic was smoke-tested on real GitHub Actions (temporary push-triggered workflow, since removed): chunk 1 fanned out into its matrix items via the dynamic index, the empty chunk was correctly skipped.
  • Not yet exercised: a real compile.sh gha-matrix run on the super self-hosted runner with NetBox secrets — that needs the workflows on the default branch (workflow_dispatch requirement). The first nightly/dispatch after merge is the true end-to-end check. If the dynamic index ever misbehaves at scale, the fallback is a static matrix.include enumeration (same structure).

…hin callers

Drop the 6 generated ~9400-line complete-artifact-matrix-*.yml files, the
600-line recreate-matrix.yml bash/awk generator, the Jinja chunk templates and
the per-track gha_config_*.yaml. Replace with one hand-written reusable pipeline
(complete-artifact-matrix.yml) plus two reusable chunk bodies
(build-artifacts-chunk.yml, build-images-chunk.yml) fanned out over a chunk
matrix, and six thin per-track callers (nightly, standard-support, community,
apps, all, all-stable).

Targets are already fetched at runtime from github.armbian.com/release-targets/,
so no board classification / targets generation is needed here. Per-track
behaviour (schedules, repos, upload paths, menu defaults) is preserved.

Also drop the external watchdog.yml cron and integrate repeat-on-failure
directly into the build steps (retry the compile.sh invocation BUILD_ATTEMPTS
times). Update nothing else.

Signed-off-by: Igor Pecovnik <igor@armbian.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant