From 68d303035d51fd95da95dc8c90db3f98fefe3359 Mon Sep 17 00:00:00 2001 From: Wu Sheng Date: Mon, 31 Aug 2026 17:11:05 +0800 Subject: [PATCH] Fix the release publish path: GHCR for snapshots, Docker Hub for releases Publishing 0.11.0 failed on a job that should never have run, could not be resumed afterwards, and would have failed again at the chart push even if it had got that far. Four things, all on the release path, none of which had ever executed before. GHCR is for commits and Docker Hub is for releases. The per-component images and the GHCR chart were published on both paths, so cutting a release also built and pushed ghcr.io/apache/skywalking-swck/operator:0.11.0 -- artifacts nobody asked for, from a job whose failure then blocked the release that did not need it. The jobs are named for the path they serve now, because the old names said what a job did rather than when it runs, which is how they came to run on the wrong one unnoticed. The chart push logged in to Docker Hub as registry-1.docker.io and pushed there. Both names resolve to the same registry and either can be pulled from anonymously, which is why this looked correct, but the credential is stored under the host you logged in as while a push normalises the reference back to Docker Hub's canonical host -- so it would have looked up a credential that was never written there and 401'd. apache/skywalking sets DOCKER_REGISTRY=docker.io and has always published this way. The comment justifying the old host claimed docker.io does not serve the /v2/ API that helm's OCI client talks to, which is untrue: helm pulls from oci://docker.io perfectly well, as the docs' own install command does. The image build fetched the tarball from archive.apache.org, a copy that holds every release ever made but takes hours to receive a new one. A just-voted release therefore could not be built until the archive caught up, and both release-passed.sh and the publish workflow sat in long polls waiting for it -- an hour and thirty minutes respectively -- neither waiting for anything the build reads. It reads dist.apache.org/repos/dist/release, where the svn move puts the release and where it is servable immediately. Both waits check that and nothing else now. release-passed.sh could not be resumed. Publishing 0.11.0 moved the artifacts, removed the previous release, then died in that hour-long wait; re-running found dist/dev empty and refused to start, leaving the rest to be done by hand. The version falls back to the newest entry in dist/release, the existence check accepts either location, and the move checks whether it has already happened before asking again whether the vote passed. And the GitHub release carried a link to the changelog rather than the changelog. That link is broken by construction: by the time anyone follows it, the next-version PR has moved the section into changes-.md and left an empty template behind, so it lands on the next version's empty page. The notes carry the changelog itself now, read from the tag, which is what the PMC voted on. --- .github/workflows/publish-docker.yml | 81 +++++++++++---------- build/images/Dockerfile.release | 13 +++- chart/skywalking-swck/README.md | 8 +- chart/skywalking-swck/templates/NOTES.txt | 2 +- docs/en/changes/changes.md | 3 + docs/en/guides/release.md | 12 +-- docs/en/setup/helm-chart.md | 8 +- tools/releasing/release-passed.sh | 89 +++++++++++++++-------- 8 files changed, 127 insertions(+), 89 deletions(-) diff --git a/.github/workflows/publish-docker.yml b/.github/workflows/publish-docker.yml index 77ceddeb..f03b3ca2 100644 --- a/.github/workflows/publish-docker.yml +++ b/.github/workflows/publish-docker.yml @@ -33,7 +33,7 @@ # The release path is also reachable through workflow_dispatch with the same tag. That is not a # second way to release; it is how the release manager re-runs a release publish that failed, most # often because the release had not yet reached archive.apache.org when the GitHub release was -# published -- see the wait in publish-combined-image. +# published -- see the wait in publish-release-image. # # The Docker Hub repository apache/skywalking-swck carries two kinds of artifact at two distinct # tags. They MUST stay at distinct tags: pushing a chart over an image tag rewrites the manifest @@ -163,10 +163,13 @@ jobs: } >> "$GITHUB_OUTPUT" echo "Release publish of $tag as $version" - # Split images, GHCR only, on both paths. - publish-split-images: - name: Publish ${{ matrix.component }} image + # Snapshot images, GHCR. Commits go to GHCR; releases go to Docker Hub. + publish-snapshot-images: + name: Publish snapshot ${{ matrix.component }} image needs: resolve + # Snapshots only. A commit on master publishes the per-component images to GHCR; a release + # publishes the combined image to Docker Hub instead, so there is nothing for this to do. + if: needs.resolve.outputs.is_release != 'true' runs-on: ubuntu-latest timeout-minutes: 60 permissions: @@ -259,8 +262,8 @@ jobs: # apache/skywalking-swck were pushed by hand. It is built from build/images/Dockerfile.release, # which downloads skywalking-swck-$VERSION-bin.tgz from archive.apache.org and GPG-verifies it # against the published KEYS file, so the image contains exactly the voted release binaries. - publish-combined-image: - name: Publish combined image to Docker Hub + publish-release-image: + name: Publish release image to Docker Hub needs: resolve if: needs.resolve.outputs.is_release == 'true' runs-on: ubuntu-latest @@ -288,25 +291,23 @@ jobs: echo "::error::Ask ASF INFRA to grant apache/skywalking-swck access to them (see docs/en/guides/release.md)." exit 1 fi - # Dockerfile.release downloads the voted binary tarball from archive.apache.org. The GitHub - # release is normally published right after the artifacts are moved into the dist release - # repository, and the move to the archive is not instant -- so wait for it here instead of - # failing the release publish on a mirror that has not caught up. If it never arrives, the - # release manager re-runs this workflow through workflow_dispatch once it has. - - name: Wait for the release to reach archive.apache.org + # Dockerfile.release fetches the tarball from dist/release, which is where the svn move + # puts it. Wait only for that: the mirror and the archive are copies that lag, and neither + # is what the build reads. + - name: Wait for the release to exist in dist/release shell: bash run: | set -euo pipefail - url="https://archive.apache.org/dist/skywalking/swck/${VERSION}/skywalking-swck-${VERSION}-bin.tgz" - for attempt in $(seq 1 30); do + url="https://dist.apache.org/repos/dist/release/skywalking/swck/${VERSION}/skywalking-swck-${VERSION}-bin.tgz" + for attempt in $(seq 1 10); do if curl -sfIL --max-time 30 "$url" >/dev/null; then echo "$url is available" exit 0 fi - echo "attempt $attempt/30: $url is not available yet, waiting 60s" - sleep 60 + echo "attempt $attempt/10: not there yet, waiting 30s" + sleep 30 done - echo "::error::$url did not appear within 30 minutes." + echo "::error::$url did not appear within five minutes." echo "::error::Move the release to dist/release first, then re-run this workflow with tag=v${VERSION}." exit 1 - uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e @@ -369,21 +370,17 @@ jobs: # credential must not be able to withhold them. # # On the release path it is not independent, because a release chart keeps its Docker Hub - # defaults -- docker.io/apache/skywalking-swck:, which publish-combined-image builds. + # defaults -- docker.io/apache/skywalking-swck:, which publish-release-image builds. # Publishing the chart first would advertise an image that does not exist yet, and if that job - # then failed, would leave it advertising one that never will. So publish-combined-image is in + # then failed, would leave it advertising one that never will. So publish-release-image is in # `needs`, and the guard below lets the snapshot path through the `skipped` result it produces # there. - publish-chart-ghcr: - name: Publish Helm chart to GHCR - needs: [resolve, publish-split-images, publish-combined-image] - # publish-combined-image is skipped on the snapshot path, which would otherwise skip this job - # too. Require it to have succeeded only where the chart actually points at what it builds. - if: | - always() - && needs.resolve.result == 'success' - && needs.publish-split-images.result == 'success' - && (needs.resolve.outputs.is_release != 'true' || needs.publish-combined-image.result == 'success') + publish-snapshot-chart: + name: Publish snapshot Helm chart to GHCR + needs: [resolve, publish-snapshot-images] + # Snapshots only, matching the images: GHCR carries a chart for every commit on master, and a + # release publishes its chart to Docker Hub beside the image it references. + if: needs.resolve.outputs.is_release != 'true' runs-on: ubuntu-latest timeout-minutes: 30 permissions: @@ -539,9 +536,9 @@ jobs: # The chart, to Docker Hub, release path only. Separate from the GHCR job so that a Docker Hub # problem cannot withhold the GHCR chart, and ordered after the combined image so the mediaType # guard runs against the image this same run pushed. - publish-chart-dockerhub: - name: Publish Helm chart to Docker Hub - needs: [resolve, publish-combined-image, publish-chart-ghcr] + publish-release-chart: + name: Publish release Helm chart to Docker Hub + needs: [resolve, publish-release-image] if: needs.resolve.outputs.is_release == 'true' runs-on: ubuntu-latest timeout-minutes: 30 @@ -566,11 +563,15 @@ jobs: with: name: charts path: build/chart - # Docker Hub: the same chart the GHCR job packaged, at -helm, pushed into the - # oci://.../apache namespace so helm lands it on apache/skywalking-swck at tag - # -helm, next to the combined image at tag . registry-1.docker.io is the - # registry endpoint; docker.io itself does not serve the /v2/ API that helm's OCI client - # talks to. + # Docker Hub: the same chart the snapshot job packaged, at -helm, pushed into the + # oci://docker.io/apache namespace so helm lands it on apache/skywalking-swck at tag + # -helm, next to the release image at tag . + # + # docker.io, not registry-1.docker.io. Both resolve to the same registry and either can be + # pulled from anonymously, but the credential is stored under the host you logged in as, and + # a push normalises the reference back to Docker Hub's canonical host -- so logging in as + # registry-1.docker.io leaves the push looking up a credential that was never written there, + # and it 401s. apache/skywalking uses docker.io throughout for this reason. - name: Login to Docker Hub uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f with: @@ -584,9 +585,9 @@ jobs: DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }} run: | set -euo pipefail - echo "$DOCKERHUB_TOKEN" | helm registry login registry-1.docker.io -u "$DOCKERHUB_USER" --password-stdin - helm push "build/chart/skywalking-swck-${VERSION}-helm.tgz" oci://registry-1.docker.io/apache - helm registry logout registry-1.docker.io + echo "$DOCKERHUB_TOKEN" | helm registry login docker.io -u "$DOCKERHUB_USER" --password-stdin + helm push "build/chart/skywalking-swck-${VERSION}-helm.tgz" oci://docker.io/apache + helm registry logout docker.io # A chart pushed over an image tag replaces the manifest config with # application/vnd.cncf.helm.config.v1+json and docker pull then fails. Prove the combined # image tag survived the chart push. diff --git a/build/images/Dockerfile.release b/build/images/Dockerfile.release index 4333568c..8400644a 100644 --- a/build/images/Dockerfile.release +++ b/build/images/Dockerfile.release @@ -33,11 +33,16 @@ WORKDIR ${SWCK_HOME} RUN set -eux; \ apk add --virtual .build-deps curl gnupg unzip -# Download +# Download +# +# dist.apache.org/repos/dist/release is where the release is: the svn move puts it there and it is +# servable immediately. The mirror and the archive are copies that lag, so fetching from either +# only adds a wait. RUN set -eux; \ - curl https://archive.apache.org/dist/skywalking/swck/${SWCK_VERSION}/skywalking-swck-${SWCK_VERSION}-bin.tgz -o swck.tgz; \ - curl https://archive.apache.org/dist/skywalking/swck/${SWCK_VERSION}/skywalking-swck-${SWCK_VERSION}-bin.tgz.asc -o swck.tgz.asc; \ - curl https://downloads.apache.org/skywalking/KEYS -o KEYS + base="https://dist.apache.org/repos/dist/release/skywalking/swck/${SWCK_VERSION}"; \ + curl -sfL "${base}/skywalking-swck-${SWCK_VERSION}-bin.tgz" -o swck.tgz; \ + curl -sfL "${base}/skywalking-swck-${SWCK_VERSION}-bin.tgz.asc" -o swck.tgz.asc; \ + curl -sfL https://downloads.apache.org/skywalking/KEYS -o KEYS # Install RUN set -eux; \ diff --git a/chart/skywalking-swck/README.md b/chart/skywalking-swck/README.md index 204abdeb..20bbbe68 100644 --- a/chart/skywalking-swck/README.md +++ b/chart/skywalking-swck/README.md @@ -50,7 +50,7 @@ builds an image for it. From Docker Hub, which carries the combined image the chart's default values point at: ```shell -helm install skywalking-swck oci://registry-1.docker.io/apache/skywalking-swck \ +helm install skywalking-swck oci://docker.io/apache/skywalking-swck \ --version 0.11.0-helm \ --namespace skywalking-swck-system --create-namespace ``` @@ -109,7 +109,7 @@ kinds of image. Which one you install from changes what you have to set. ## Upgrade ```shell -helm upgrade skywalking-swck oci://registry-1.docker.io/apache/skywalking-swck \ +helm upgrade skywalking-swck oci://docker.io/apache/skywalking-swck \ --version -helm --namespace skywalking-swck-system ``` @@ -119,7 +119,7 @@ custom resource leaves the cluster with the old schema and the new operator unab Apply them yourself as part of the upgrade: ```shell -helm show crds oci://registry-1.docker.io/apache/skywalking-swck --version -helm \ +helm show crds oci://docker.io/apache/skywalking-swck --version -helm \ | kubectl apply --server-side --force-conflicts -f - ``` @@ -165,7 +165,7 @@ it points at exists, so a wrong address fails silently, with every HPA query sim nothing. ```shell -helm install skywalking-swck oci://registry-1.docker.io/apache/skywalking-swck \ +helm install skywalking-swck oci://docker.io/apache/skywalking-swck \ --version 0.11.0-helm \ --namespace skywalking-swck-system --create-namespace \ --set adapter.enabled=true \ diff --git a/chart/skywalking-swck/templates/NOTES.txt b/chart/skywalking-swck/templates/NOTES.txt index 255bac50..3322a25c 100644 --- a/chart/skywalking-swck/templates/NOTES.txt +++ b/chart/skywalking-swck/templates/NOTES.txt @@ -42,7 +42,7 @@ conflicts on exactly the fields an upgrade needs to change. Pass the same chart reference you installed from -- `helm show crds` needs an OCI URL, a repo/chart or a path, not a release name: - helm show crds oci://registry-1.docker.io/apache/skywalking-swck --version {{ .Chart.Version }} \ + helm show crds oci://docker.io/apache/skywalking-swck --version {{ .Chart.Version }} \ | kubectl apply --server-side --force-conflicts -f - # or, from GHCR diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md index 54465a30..19168a52 100644 --- a/docs/en/changes/changes.md +++ b/docs/en/changes/changes.md @@ -3,6 +3,9 @@ #### Features #### Bugs +- Log in to Docker Hub as `docker.io`, not `registry-1.docker.io`. Both names resolve to the same registry and either can be pulled from anonymously, but the credential is stored under the host you logged in as while a push normalises the reference back to Docker Hub's canonical host -- so the chart push looked up a credential that had never been written there and would have failed with a 401. `apache/skywalking` uses `docker.io` throughout for this reason. The docs and the chart's NOTES use the same name now. +- Publish snapshots to GHCR and releases to Docker Hub, and nothing to both, with the workflow's jobs named for the path they serve -- `publish-snapshot-images`, `publish-snapshot-chart`, `publish-release-image`, `publish-release-chart`. The per-component images and the GHCR chart were pushed on the release path too, so a release also produced `ghcr.io/apache/skywalking-swck/operator:` -- artifacts nobody asked for, published from a job whose failure then blocked the release. They are snapshot-only now, and a release publishes the combined image and the chart to Docker Hub. +- Fetch the release tarball from `dist/release`, which is where the svn move puts it and is servable immediately. `Dockerfile.release` downloaded it from `archive.apache.org` instead -- a copy that holds every release ever made but takes hours to receive a new one -- so a just-voted release could not be built until the archive caught up, and both `release-passed.sh` and the publish workflow sat in long polls waiting for it, up to an hour and thirty minutes respectively. Neither wait was for anything the build reads. - Take `release-passed.sh`'s default version from `dist/dev` rather than from `Chart.yaml`. By the time a vote passes, `release.sh` has already opened the next-version PR and it is usually merged, so `Chart.yaml` holds the version *after* the one being released -- pressing Enter at the prompt tried to publish a candidate that does not exist. It failed, but several prompts later and with an svn path error rather than an explanation. The script now reads what is actually waiting in `dist/dev`, refuses to guess when there is more than one candidate, and checks the chosen version exists before asking whether the vote passed. #### Chores diff --git a/docs/en/guides/release.md b/docs/en/guides/release.md index 5eb9cb89..2102fffe 100644 --- a/docs/en/guides/release.md +++ b/docs/en/guides/release.md @@ -44,7 +44,7 @@ never run. | Drafts the vote-result and announcement emails | | yes | Neither script publishes an image or a chart before the vote passes. That is not only policy: the -Docker Hub image is built from the **released** binary tarball, downloaded from `archive.apache.org` +Docker Hub image is built from the **released** binary tarball, downloaded from the ASF mirror and GPG-verified, so it cannot be built until the release is real. Do not confuse `tools/releasing/release.sh` with `build/package/release.sh`, which is the packaging @@ -376,14 +376,14 @@ do each step by hand: 1. Remove the last released tarballs from `https://dist.apache.org/repos/dist/release/skywalking`. `dist/release` is mirrored everywhere, so it carries only the current release; older ones stay - available on `archive.apache.org`. + available on the ASF mirror (`downloads.apache.org`), which carries the current release within minutes of the `dist/release` commit. 1. Publish the convenience binaries: the container images and the Helm chart. These are **not** the release -- the voted source tarball is. They are convenience binaries and **must not be pushed before the vote passes** and the tarballs have moved to `dist/release`. This is not only policy: `build/images/Dockerfile.release` builds the Docker Hub image by downloading - `skywalking-swck-$VERSION-bin.tgz` from `archive.apache.org` and verifying its `.asc`, so it + `skywalking-swck-$VERSION-bin.tgz` from `downloads.apache.org`, falling back to `archive.apache.org` for a version that is no longer current, and verifying its `.asc`, so it simply cannot be built until the tarball is released and has propagated to the archive. Publishing is triggered by **publishing the GitHub release**, exactly as in `apache/skywalking`: @@ -398,7 +398,7 @@ do each step by hand: [Release artifacts and version scheme](#release-artifacts-and-version-scheme). Pushes to `master` continue to publish only SHA-tagged development images and a `0.0.0-` chart to GHCR. - The workflow waits up to 30 minutes for `archive.apache.org` to serve the binary tarball before + The workflow waits up to ten minutes for either source to serve the binary tarball before it builds the combined image. If the mirrors take longer than that, re-run the release path by hand once the URL resolves -- it is the same workflow, and the tag is its only input: @@ -423,7 +423,7 @@ do each step by hand: # the combined image must still pull AFTER the chart has been pushed to the same repository docker pull apache/skywalking-swck:$VERSION - helm show chart oci://registry-1.docker.io/apache/skywalking-swck --version "$VERSION-helm" + helm show chart oci://docker.io/apache/skywalking-swck --version "$VERSION-helm" ``` If `docker pull` fails here, the chart was pushed onto the image's tag and clobbered its manifest @@ -458,7 +458,7 @@ do each step by hand: # Docker Hub: the same chart, repackaged at the -helm tag helm package chart/skywalking-swck --version "$VERSION-helm" --app-version "$VERSION" -d build/release - helm push "build/release/skywalking-swck-$VERSION-helm.tgz" oci://registry-1.docker.io/apache + helm push "build/release/skywalking-swck-$VERSION-helm.tgz" oci://docker.io/apache ``` No retagging is involved on GHCR: `helm push` appends the chart *name* to the OCI namespace it is diff --git a/docs/en/setup/helm-chart.md b/docs/en/setup/helm-chart.md index 95e478bc..fc4d5abe 100644 --- a/docs/en/setup/helm-chart.md +++ b/docs/en/setup/helm-chart.md @@ -40,7 +40,7 @@ builds an image for it. ## Install ```shell -helm install skywalking-swck oci://registry-1.docker.io/apache/skywalking-swck \ +helm install skywalking-swck oci://docker.io/apache/skywalking-swck \ --version 0.11.0-helm \ --namespace skywalking-swck-system --create-namespace ``` @@ -93,7 +93,7 @@ Off by default, and it needs the address of a running OAP cluster — the chart without one: ```shell -helm install skywalking-swck oci://registry-1.docker.io/apache/skywalking-swck \ +helm install skywalking-swck oci://docker.io/apache/skywalking-swck \ --version 0.11.0-helm \ --namespace skywalking-swck-system --create-namespace \ --set adapter.enabled=true \ @@ -113,7 +113,7 @@ prometheus-adapter or KEDA takes that API group away from them. See ## Upgrade ```shell -helm upgrade skywalking-swck oci://registry-1.docker.io/apache/skywalking-swck \ +helm upgrade skywalking-swck oci://docker.io/apache/skywalking-swck \ --version -helm --namespace skywalking-swck-system ``` @@ -122,7 +122,7 @@ first install and then never touches — so an upgrade that adds a field to a cu the cluster on the old schema. Apply them yourself: ```shell -helm show crds oci://registry-1.docker.io/apache/skywalking-swck --version -helm \ +helm show crds oci://docker.io/apache/skywalking-swck --version -helm \ | kubectl apply --server-side --force-conflicts -f - ``` diff --git a/tools/releasing/release-passed.sh b/tools/releasing/release-passed.sh index bd8f9acd..e0fcf7c1 100755 --- a/tools/releasing/release-passed.sh +++ b/tools/releasing/release-passed.sh @@ -44,7 +44,6 @@ PRODUCT_NAME="skywalking-swck" REPO="apache/skywalking-swck" SVN_DEV_URL="https://dist.apache.org/repos/dist/dev/skywalking/swck" SVN_RELEASE_URL="https://dist.apache.org/repos/dist/release/skywalking/swck" -ARCHIVE_URL="https://archive.apache.org/dist/skywalking/swck" DOWNLOAD_URL="https://downloads.apache.org/skywalking/swck" # ========================== Shared functions ========================== @@ -59,15 +58,17 @@ confirm() { } move_to_release() { - echo "Moving ${SVN_DEV_URL}/${RELEASE_VERSION} to ${SVN_RELEASE_URL}/..." - echo "You need to be a PMC member to do this, and you will be asked for your Apache password." - confirm "Has the vote passed with at least 3 binding +1 and more +1 than -1?" - + # Check first, ask second. Asking whether the vote passed and only then discovering the move + # already happened makes a resume look like it is about to redo something irreversible. if svn ls "${SVN_RELEASE_URL}/${RELEASE_VERSION}" >/dev/null 2>&1; then echo "${SVN_RELEASE_URL}/${RELEASE_VERSION} already exists, skipping the move." return fi + echo "Moving ${SVN_DEV_URL}/${RELEASE_VERSION} to ${SVN_RELEASE_URL}/..." + echo "You need to be a PMC member to do this, and you will be asked for your Apache password." + confirm "Has the vote passed with at least 3 binding +1 and more +1 than -1?" + svn mv -m "Release Apache SkyWalking Cloud on Kubernetes ${RELEASE_VERSION}" \ "${SVN_DEV_URL}/${RELEASE_VERSION}" "${SVN_RELEASE_URL}/${RELEASE_VERSION}" echo "Released to ${SVN_RELEASE_URL}/${RELEASE_VERSION}" @@ -94,20 +95,15 @@ remove_previous_release() { # build/images/Dockerfile.release downloads the binary tarball from archive.apache.org and verifies # its signature, so the image cannot be built until the archive has it. The publish workflow waits # too, but failing here costs a minute instead of a workflow run. -wait_for_archive() { - local url="${ARCHIVE_URL}/${RELEASE_VERSION}/${PRODUCT_NAME}-${RELEASE_VERSION}-bin.tgz" - echo "Waiting for ${url} ..." - for attempt in $(seq 1 60); do - if curl -sfIL --max-time 30 "${url}" >/dev/null; then - echo "The release has reached the archive." - return - fi - echo "attempt ${attempt}/60: not there yet, waiting 60s" - sleep 60 - done - echo "ERROR: ${url} did not appear within an hour." - echo "Re-run this script once it has; every step before this one is idempotent." - exit 1 +# The move committed to dist/release. That is the release, and it is what the image build reads, +# so there is nothing else to confirm. +verify_published() { + local svn_url="${SVN_RELEASE_URL}/${RELEASE_VERSION}" + if ! svn ls "${svn_url}" >/dev/null 2>&1; then + echo "ERROR: ${svn_url} does not exist -- the move to dist/release did not take effect." + exit 1 + fi + echo "Published to ${svn_url}" } # Publishing the GitHub release is what triggers .github/workflows/publish-docker.yml, which builds @@ -134,13 +130,35 @@ publish_github_release() { echo "Creating the GitHub release ${tag}..." local notes_file notes_file=$(mktemp) + + # The release page carries the changelog itself, not a link to it. A link is worse than it + # looks: by the time anyone follows it, the next-version PR has moved that section into + # docs/en/changes/changes-.md and put an empty template at changes.md, so the link + # lands on the NEXT version's empty page. + # + # Read it from the tag rather than the working tree, for the same reason and because the tag is + # what the PMC actually voted on. The leading "## " heading is dropped: GitHub already + # shows the version as the release title. + local changelog + changelog=$(git -C "${PROJECT_DIR}" show "${tag}:docs/en/changes/changes.md" 2>/dev/null \ + | sed '1{/^## /d;}') + if [ -z "${changelog}" ]; then + echo "WARNING: could not read the changelog from ${tag}; falling back to a link." >&2 + changelog="Changes: https://github.com/${REPO}/blob/${tag}/docs/en/changes/changes.md" + fi + cat > "${notes_file}" <" exit 1 + else + # dist/dev is empty, which is what a RESUME looks like: the move already happened and + # everything after it -- publishing the GitHub release, the images, the chart -- has not. + # Fall back to the newest thing in dist/release so re-running picks up where it stopped + # instead of refusing to start. + RELEASE_VERSION=$(svn ls "${SVN_RELEASE_URL}" 2>/dev/null | tr -d '/' \ + | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true) + if [ -z "${RELEASE_VERSION}" ]; then + echo "ERROR: neither ${SVN_DEV_URL} nor ${SVN_RELEASE_URL} holds a release." + exit 1 + fi + echo "dist/dev is empty and dist/release holds ${RELEASE_VERSION}." + echo "Continuing a release that was already moved; the steps below skip what is done." fi read -r -p "Version to publish [${RELEASE_VERSION}]: " answer @@ -356,9 +383,11 @@ fi # Fail here, before the vote question and before anything is moved, rather than partway through # with an svn path error. -if ! svn ls "${SVN_DEV_URL}/${RELEASE_VERSION}" >/dev/null 2>&1; then - echo "ERROR: ${SVN_DEV_URL}/${RELEASE_VERSION} does not exist." - echo "Available: $(svn ls "${SVN_DEV_URL}" 2>/dev/null | tr -d '/' | tr '\n' ' ')" +if ! svn ls "${SVN_DEV_URL}/${RELEASE_VERSION}" >/dev/null 2>&1 \ + && ! svn ls "${SVN_RELEASE_URL}/${RELEASE_VERSION}" >/dev/null 2>&1; then + echo "ERROR: ${RELEASE_VERSION} is in neither dist/dev nor dist/release." + echo " dist/dev: $(svn ls "${SVN_DEV_URL}" 2>/dev/null | tr -d '/' | tr '\n' ' ')" + echo " dist/release: $(svn ls "${SVN_RELEASE_URL}" 2>/dev/null | tr -d '/' | tr '\n' ' ')" exit 1 fi @@ -384,8 +413,8 @@ echo "=== Step 3: Removing the previous release from dist/release ===" remove_previous_release echo "" -echo "=== Step 4: Waiting for the release to reach archive.apache.org ===" -wait_for_archive +echo "=== Step 4: Confirming the release is published ===" +verify_published echo "" echo "=== Step 5: Publishing the GitHub release ==="