Skip to content

Commit aaaa6dd

Browse files
authored
fix(release): simplify candidate verification (#531)
Remove the duplicate cold candidate build from the blocking release path and scan immutable-image SBOMs without recataloging private images. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
1 parent 8d61be6 commit aaaa6dd

7 files changed

Lines changed: 167 additions & 265 deletions

File tree

‎.github/workflows/release-candidate.yml‎

Lines changed: 20 additions & 169 deletions
Original file line numberDiff line numberDiff line change
@@ -444,121 +444,6 @@ jobs:
444444
if-no-files-found: error
445445
retention-days: 7
446446

447-
build-b:
448-
name: Build B cold canonical candidate
449-
needs: validate
450-
runs-on: ubuntu-24.04
451-
permissions:
452-
contents: read
453-
steps:
454-
- name: Checkout exact candidate source
455-
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4.2.2
456-
with:
457-
ref: ${{ needs.validate.outputs.source_sha }}
458-
fetch-depth: 0
459-
submodules: false
460-
461-
- name: Storage preflight before cold build
462-
shell: bash
463-
run: |
464-
set -euo pipefail
465-
args=(preflight --budget release/storage-budget.json --workspace "${GITHUB_WORKSPACE}" --output dist/storage-preflight-b.json)
466-
if [[ "${{ needs.validate.outputs.measurement_bootstrap }}" == "true" ]]; then args+=(--measurement-run); fi
467-
python3 release/scripts/check-release-storage.py "${args[@]}"
468-
469-
- name: Start peak-storage sampler
470-
run: |
471-
rm -f "${RUNNER_TEMP}/storage-stop-b"
472-
RUNNER_TRACKING_ID="" python3 release/scripts/check-release-storage.py sample \
473-
--workspace "${GITHUB_WORKSPACE}" \
474-
--output "${RUNNER_TEMP}/storage-measurement-b.json" \
475-
--stop-file "${RUNNER_TEMP}/storage-stop-b" \
476-
--interval-seconds 15 \
477-
--label build-b &
478-
echo "$!" > "${RUNNER_TEMP}/storage-sampler-b.pid"
479-
480-
- name: Assert cold compiled-output state
481-
shell: bash
482-
run: |
483-
set -euo pipefail
484-
if [[ -e target || -e .cargo-home ]]; then
485-
echo "cold Build B checkout unexpectedly contains compiled or Cargo cache state" >&2
486-
exit 1
487-
fi
488-
489-
- name: Build canonical Linux payload independently
490-
env:
491-
RELEASE_CARGO_HOME: ${{ runner.temp }}/candidate-b-cargo
492-
RELEASE_TARGET_DIR: ${{ runner.temp }}/candidate-b-target
493-
run: release/scripts/build-release-binaries.sh "${{ needs.validate.outputs.version }}"
494-
495-
- name: Set up Docker Buildx
496-
id: buildx
497-
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v3.11.1
498-
with:
499-
version: ${{ env.RELEASE_BUILDX_VERSION }}
500-
driver-opts: image=${{ env.RELEASE_BUILDKIT_IMAGE }}
501-
502-
- name: Export cold comparison OCI layouts
503-
shell: bash
504-
run: |
505-
set -euo pipefail
506-
mkdir -p dist/layouts dist/images
507-
for name in registry-notary registry-relay; do
508-
RELEASE_IMAGE_NO_CACHE=true \
509-
RELEASE_IMAGE_OCI_LAYOUT="${GITHUB_WORKSPACE}/dist/layouts/${name}" \
510-
RELEASE_BUILDX_BUILDER="${{ steps.buildx.outputs.name }}" \
511-
release/scripts/build-release-image.sh \
512-
"${name}" "${name}:candidate-build-b" \
513-
"https://github.com/${GITHUB_REPOSITORY}" \
514-
"${{ needs.validate.outputs.source_sha }}" \
515-
"${{ needs.validate.outputs.version }}" \
516-
"dist/images/${name}.metadata.json"
517-
done
518-
519-
- name: Stop peak-storage sampler
520-
if: ${{ always() }}
521-
shell: bash
522-
run: |
523-
touch "${RUNNER_TEMP}/storage-stop-b"
524-
if [[ -f "${RUNNER_TEMP}/storage-sampler-b.pid" ]]; then
525-
sampler_pid="$(cat "${RUNNER_TEMP}/storage-sampler-b.pid")"
526-
for _ in {1..30}; do
527-
if ! kill -0 "${sampler_pid}" 2>/dev/null; then break; fi
528-
sleep 1
529-
done
530-
if kill -0 "${sampler_pid}" 2>/dev/null; then
531-
echo "Build B storage sampler did not stop" >&2
532-
exit 1
533-
fi
534-
fi
535-
mkdir -p candidate-build-b/storage
536-
cp "${RUNNER_TEMP}/storage-measurement-b.json" candidate-build-b/storage/measurement.json
537-
cp dist/storage-preflight-b.json candidate-build-b/storage/preflight.json
538-
539-
- name: Stage exact Build B artifact
540-
if: ${{ success() }}
541-
shell: bash
542-
run: |
543-
set -euo pipefail
544-
mkdir -p candidate-build-b/dist
545-
cp -R dist/bin candidate-build-b/dist/bin
546-
cp -R dist/image-bin candidate-build-b/dist/image-bin
547-
cp -R dist/layouts candidate-build-b/dist/layouts
548-
jq -n \
549-
--arg job_id "build-b:${GITHUB_RUN_ID}:${GITHUB_RUN_ATTEMPT}" \
550-
'{job_id: $job_id, cargo_cache: {mode: "cold", primary_key: null, exact_key_hit: false}}' \
551-
> candidate-build-b/build.json
552-
553-
- name: Upload exact Build B artifact
554-
if: ${{ success() }}
555-
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
556-
with:
557-
name: registry-stack-candidate-build-b-${{ github.run_id }}-${{ github.run_attempt }}
558-
path: candidate-build-b
559-
if-no-files-found: error
560-
retention-days: 7
561-
562447
other-platforms:
563448
name: Build candidate registryctl ${{ matrix.asset }}
564449
needs: validate
@@ -660,7 +545,6 @@ jobs:
660545
needs:
661546
- validate
662547
- build-a
663-
- build-b
664548
- other-platforms
665549
runs-on: ubuntu-24.04
666550
permissions:
@@ -709,12 +593,6 @@ jobs:
709593
name: registry-stack-candidate-build-a-${{ github.run_id }}-${{ github.run_attempt }}
710594
path: inputs/build-a
711595

712-
- name: Download Build B
713-
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
714-
with:
715-
name: registry-stack-candidate-build-b-${{ github.run_id }}-${{ github.run_attempt }}
716-
path: inputs/build-b
717-
718596
- name: Download macOS arm64 payload
719597
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
720598
with:
@@ -727,7 +605,7 @@ jobs:
727605
name: registry-stack-candidate-linux-arm64-${{ github.run_id }}-${{ github.run_attempt }}
728606
path: inputs/linux-arm64
729607

730-
- name: Compare canonical binaries
608+
- name: Validate canonical binary inventory
731609
shell: bash
732610
run: |
733611
set -euo pipefail
@@ -742,25 +620,15 @@ jobs:
742620
"registryctl-${{ needs.validate.outputs.tag }}-linux-amd64" \
743621
| sort
744622
)"
745-
for build in a b; do
746-
actual="$(find "inputs/build-${build}/dist/bin" -maxdepth 1 -type f -printf '%f\n' | sort)"
747-
if [[ "${actual}" != "${expected}" ]]; then
748-
echo "Build ${build} binary inventory is not exact" >&2
749-
diff -u <(printf '%s\n' "${expected}") <(printf '%s\n' "${actual}") || true
750-
exit 1
751-
fi
752-
done
753-
while IFS= read -r name; do
754-
cmp "inputs/build-a/dist/bin/${name}" "inputs/build-b/dist/bin/${name}"
755-
done <<<"${expected}"
623+
actual="$(find inputs/build-a/dist/bin -maxdepth 1 -type f -printf '%f\n' | sort)"
624+
if [[ "${actual}" != "${expected}" ]]; then
625+
echo "Build A binary inventory is not exact" >&2
626+
diff -u <(printf '%s\n' "${expected}") <(printf '%s\n' "${actual}") || true
627+
exit 1
628+
fi
756629
expected_image_bin="$(printf '%s\n' RELEASE_BUILDER_IMAGE SHA256SUMS registry-notary registry-notary-cel-worker registry-relay registry-relay-rhai-worker)"
757-
for build in a b; do
758-
actual="$(find "inputs/build-${build}/dist/image-bin" -maxdepth 1 -type f -printf '%f\n' | sort)"
759-
[[ "${actual}" == "${expected_image_bin}" ]]
760-
done
761-
while IFS= read -r name; do
762-
cmp "inputs/build-a/dist/image-bin/${name}" "inputs/build-b/dist/image-bin/${name}"
763-
done <<<"${expected_image_bin}"
630+
actual="$(find inputs/build-a/dist/image-bin -maxdepth 1 -type f -printf '%f\n' | sort)"
631+
[[ "${actual}" == "${expected_image_bin}" ]]
764632
765633
- name: Install pinned candidate inspection tools
766634
env:
@@ -831,16 +699,6 @@ jobs:
831699
--require-provenance > "dist/topology/${name}.json"
832700
done
833701
834-
- name: Compare image config and ordered layers
835-
shell: bash
836-
run: |
837-
set -euo pipefail
838-
for name in registry-notary registry-relay; do
839-
python3 release/scripts/compare-release-image-layouts.py \
840-
"dist/pulled-layouts/${name}" \
841-
"inputs/build-b/dist/layouts/${name}"
842-
done
843-
844702
- name: Scan immutable staging digests
845703
shell: bash
846704
run: |
@@ -887,7 +745,8 @@ jobs:
887745
"dist/candidate/dist/sbom/${name}.spdx.json" \
888746
--image-name "${name}" \
889747
--digest-ref "${digest_ref}"
890-
grype "${digest_ref}" -o json > "dist/candidate/dist/grype/${name}.grype.json"
748+
grype "sbom:dist/candidate/dist/sbom/${name}.syft.json" -o json \
749+
> "dist/candidate/dist/grype/${name}.grype.json"
891750
mkdir -p "dist/candidate/dist/rootfs/${name}"
892751
crane export "${digest_ref}" - \
893752
| tar --extract --file=- --directory="dist/candidate/dist/rootfs/${name}" \
@@ -916,7 +775,7 @@ jobs:
916775
dist/candidate/dist/sbom/postgresql.spdx.json \
917776
--image-name postgresql \
918777
--digest-ref "${postgresql_ref}"
919-
grype "${postgresql_ref}" -o json \
778+
grype "sbom:dist/candidate/dist/sbom/postgresql.syft.json" -o json \
920779
> dist/candidate/dist/grype/postgresql.grype.json
921780
mkdir -p dist/candidate/dist/rootfs/postgresql
922781
crane export "${postgresql_ref}" - \
@@ -1059,7 +918,6 @@ jobs:
1059918
jq -s \
1060919
'{schema_version:"registry-stack.release-storage-measurements.v1", measurements:.}' \
1061920
inputs/build-a/storage/measurement.json \
1062-
inputs/build-b/storage/measurement.json \
1063921
inputs/macos-arm64/storage/measurement.json \
1064922
inputs/linux-arm64/storage/measurement.json \
1065923
dist/candidate/dist/storage/verification-measurement.json \
@@ -1109,21 +967,19 @@ jobs:
1109967
gh api "/repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/artifacts?per_page=100" > dist/receipt/artifact-metadata.json
1110968
names=(
1111969
"registry-stack-candidate-build-a-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
1112-
"registry-stack-candidate-build-b-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
1113970
"registry-stack-candidate-macos-arm64-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
1114971
"registry-stack-candidate-linux-arm64-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
1115972
"registry-stack-release-candidate-payload-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
1116973
)
1117974
roots=(
1118975
inputs/build-a
1119-
inputs/build-b
1120976
inputs/macos-arm64
1121977
inputs/linux-arm64
1122978
dist/candidate
1123979
)
1124-
if [[ "${#names[@]}" -ne 5 || "${#roots[@]}" -ne 5 ||
980+
if [[ "${#names[@]}" -ne 4 || "${#roots[@]}" -ne 4 ||
1125981
"${#names[@]}" -ne "${#roots[@]}" ]]; then
1126-
echo "Candidate receipt requires exact five-name/five-root artifact parity" >&2
982+
echo "Candidate receipt requires exact four-name/four-root artifact parity" >&2
1127983
exit 1
1128984
fi
1129985
records=()
@@ -1235,7 +1091,7 @@ jobs:
12351091
status_sha256:$db_status_sha256
12361092
}
12371093
},
1238-
comparison:{config_equal:true,layers_equal:true}
1094+
comparison:{config_equal:false,layers_equal:false}
12391095
}'
12401096
}
12411097
image_record registry-notary > dist/receipt-parts/registry-notary.json
@@ -1258,7 +1114,6 @@ jobs:
12581114
--arg storage_sha256 "$(sha256sum dist/candidate/dist/storage/storage-measurement.json | awk '{print $1}')" \
12591115
--slurpfile builders dist/receipt/expected-builders.json \
12601116
--slurpfile build_a inputs/build-a/build.json \
1261-
--slurpfile build_b inputs/build-b/build.json \
12621117
--slurpfile other dist/receipt-parts/other-platforms.json \
12631118
--slurpfile artifacts dist/receipt-parts/artifacts.json \
12641119
--slurpfile notary dist/receipt-parts/registry-notary.json \
@@ -1283,10 +1138,10 @@ jobs:
12831138
},
12841139
validity:{created_at:$created_at,expires_at:$expires_at},
12851140
builders:$builders[0],
1286-
builds:{a:$build_a[0],b:$build_b[0],other_platforms:$other[0]},
1141+
builds:{a:$build_a[0],other_platforms:$other[0]},
12871142
artifacts:$artifacts[0],
12881143
images:[$notary[0],$relay[0]],
1289-
comparisons:{binary_bytes:true,image_config_and_layers:true},
1144+
comparisons:{binary_bytes:false,image_config_and_layers:false},
12901145
scans:{policy:"passed",immutable_digests:true},
12911146
storage:{
12921147
budget_status:$storage_budget_status,
@@ -1321,7 +1176,6 @@ jobs:
13211176
mkdir -p dist/candidate-telemetry-evidence
13221177
jq -n \
13231178
--slurpfile build_a inputs/build-a/build.json \
1324-
--slurpfile build_b inputs/build-b/build.json \
13251179
--slurpfile macos inputs/macos-arm64/build.json \
13261180
--slurpfile linux inputs/linux-arm64/build.json \
13271181
--slurpfile storage \
@@ -1330,14 +1184,13 @@ jobs:
13301184
schema_version:"registry-stack.release-candidate-telemetry-evidence.v1",
13311185
builds:{
13321186
a:$build_a[0],
1333-
b:$build_b[0],
13341187
other_platforms:[$macos[0],$linux[0]]
13351188
},
13361189
peak_storage_measurements:$storage[0].measurements
13371190
}' > dist/candidate-telemetry-evidence/evidence.json
13381191
jq -e '
13391192
.builds.a.cargo_cache.mode == "exact-key-restore" and
1340-
(.peak_storage_measurements | length == 5)
1193+
(.peak_storage_measurements | length == 4)
13411194
' dist/candidate-telemetry-evidence/evidence.json >/dev/null
13421195
13431196
- name: Upload compact candidate telemetry evidence
@@ -1569,7 +1422,6 @@ jobs:
15691422
needs:
15701423
- validate
15711424
- build-a
1572-
- build-b
15731425
- other-platforms
15741426
- verify-candidate
15751427
- verify-cli-platforms
@@ -1590,7 +1442,6 @@ jobs:
15901442
- name: Collect candidate wall clock, queue delay, and runner occupancy
15911443
env:
15921444
BUILD_A_RESULT: ${{ needs.build-a.result }}
1593-
BUILD_B_RESULT: ${{ needs.build-b.result }}
15941445
OTHER_PLATFORMS_RESULT: ${{ needs.other-platforms.result }}
15951446
VERIFY_RESULT: ${{ needs.verify-candidate.result }}
15961447
GH_TOKEN: ${{ github.token }}
@@ -1611,10 +1462,10 @@ jobs:
16111462
if [[ -f "${evidence}" ]]; then
16121463
cache_state="$(jq -c '.builds.a.cargo_cache' "${evidence}")"
16131464
storage_evidence="$(jq -c '.peak_storage_measurements' "${evidence}")"
1614-
test "$(jq -r '.peak_storage_measurements | length' "${evidence}")" = 5
1465+
test "$(jq -r '.peak_storage_measurements | length' "${evidence}")" = 4
16151466
else
16161467
cache_state='{"status":"unavailable","build_result":"'"${BUILD_A_RESULT}"'"}'
1617-
storage_evidence='[{"status":"unavailable","build_a_result":"'"${BUILD_A_RESULT}"'","build_b_result":"'"${BUILD_B_RESULT}"'","other_platforms_result":"'"${OTHER_PLATFORMS_RESULT}"'","verify_result":"'"${VERIFY_RESULT}"'"}]'
1468+
storage_evidence='[{"status":"unavailable","build_a_result":"'"${BUILD_A_RESULT}"'","other_platforms_result":"'"${OTHER_PLATFORMS_RESULT}"'","verify_result":"'"${VERIFY_RESULT}"'"}]'
16181469
fi
16191470
collected_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
16201471
jq -n \

‎release/OPERATIONS.md‎

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -23,10 +23,11 @@ The selector requires extended proof when any of these conditions applies:
2323
not an ancestor of the candidate source
2424
- The operator requests `extended`
2525

26-
An actual disagreement between the two builds in the current candidate fails
27-
the candidate. Extended proof is additional evidence, not a bypass for
28-
non-identical candidate bytes or image config and layers. A promotable receipt
29-
is emitted only after the current comparison passes.
26+
A candidate uses one canonical build and fails on incomplete binary inventory,
27+
invalid immutable image topology, an unreviewed advisory, or any source and
28+
artifact binding mismatch. Extended proof is additional evidence produced by
29+
the separate repeatability workflow, not a second build in the blocking
30+
candidate path.
3031

3132
The selector writes `registry-stack.release-proof-selection.v1` JSON into the
3233
candidate evidence.
@@ -35,11 +36,9 @@ base.
3536
The exact previous promoted tag is the fallback and a cross-check when both are
3637
available.
3738

38-
Two candidate builds on separate hosted runners prove build determinism under
39-
the declared build contract.
40-
They do not prove environment independence.
41-
The later clean rebuild in `.github/workflows/release-repeatability.yml` carries
42-
that separate claim.
39+
The candidate path does not claim build determinism from two builds in one
40+
workflow. The clean rebuild in `.github/workflows/release-repeatability.yml`
41+
carries that separate claim without delaying publication.
4342

4443
## Check The Storage Gate
4544

@@ -71,8 +70,8 @@ The budget loader checks both equations and rejects arbitrary thresholds,
7170
including a one-byte placeholder. Absolute filesystem-used bytes are retained
7271
as audit evidence; they are not compared with available bytes.
7372

74-
Splitting build A and build B across separate runners limits the budget to
75-
per-job peak pressure.
73+
Splitting the canonical, platform, and verification work across separate
74+
runners limits the budget to per-job peak pressure.
7675
The gate is preventive; no prior hosted candidate disk exhaustion is recorded.
7776

7877
## Request A Candidate
@@ -155,7 +154,8 @@ release/scripts/registry-release finalize \
155154

156155
The verifier fetches the exact candidate artifact IDs, verifies the closed
157156
receipt, run attempt, workflow identity, GitHub attestations, artifact bytes,
158-
image digests, scans, comparison results, and release identity.
157+
image digests, scans, whether same-run comparisons were performed, and release
158+
identity.
159159
On success, it prints one exact annotated `git tag -a` command whose message
160160
binds the run ID, run attempt, and receipt SHA-256.
161161
Run that printed command, inspect the tag, and push only that exact ref:

0 commit comments

Comments
 (0)