Historical Artifact Migration: Script Plan

[SeanTAllen]

Plan for the one-off script that moves existing release-channel artifacts from Cloudsmith to GitHub releases. Scoped to workstream 2 of the migration overview in #405 — just the historical binary move. No ponyup code changes, no CI changes, no cutover work.

Principles at play

• How do you know that you know that? Verify assumptions — API shape, download auth, tag format, pagination — empirically before scripting, and again during the dry-run before the real run.
• It is easier to give than take away. Keep the script minimal: per-tool loop, download, upload, log. No features not needed for this one-off.
• Errors are data. Every skipped or failed item has an explicit reason in the log so the post-run audit is clean.
• Make illegal states unrepresentable. Asset-presence check is per-file; defensive filename-prefix assertion; local SHA-512 re-check after download.

Goal

For each of the four tool repos, walk Cloudsmith release-channel packages that already have a matching GitHub release tag, and upload the archive plus a generated .sha512 sibling as release assets. Skip anything without a matching GitHub release. Additive only — Cloudsmith is untouched.

Per-tool loop

For each of ponylang/ponyc, ponylang/corral, ponylang/ponyup, ponylang/changelog-tool:

1. Fetch the full list of GitHub release tags via gh api /repos/<owner>/<repo>/releases --paginate --jq '.[].tag_name'. Default gh release list --limit 30 would silently miss tags; ponyc has 100+.
2. Page through Cloudsmith packages/ponylang/releases/?query=<tool>%20status:completed — server-side tool-name filter. Loop until page > x-pagination-pagetotal (from the response header) or a page returns []. Page size 100.
3. Defensive check per returned package: assert filename starts with <tool>-. Guards against a substring-match surprise. Skip with log if not.
4. For each Cloudsmith package:
   • Version has no matching GitHub tag → skip, log skipped (no tag).
   • Check existing assets on the release: gh api /repos/<owner>/<repo>/releases/tags/<tag> --jq '.assets[].name'. Per-file check:
     • Archive present AND <archive>.sha512 present → skip, log already present (not verified).
     • Archive present, sibling missing → generate and upload sibling only.
     • Archive missing → download and upload both.
   • Download from cdn_url with Cloudsmith auth.
   • Compute local SHA-512 of the downloaded file; compare to Cloudsmith's checksum_sha512. Mismatch → skip, log failed (checksum mismatch on download). No upload.
   • Write <filename>.sha512 containing Cloudsmith's checksum_sha512 — raw hex, no re-compute.
   • gh release upload <tag> <archive> <archive>.sha512 on the tool's repo.

Credentials

• GitHub upload: classic PAT on ponylang-main, scope public_repo, expiration 1 week. Exported as GH_TOKEN for the invocation only — does not touch the local gh auth login. Assumption: migration completes inside the PAT's validity window; if a later re-run is needed, mint a new one.
• Cloudsmith read: ponylang Cloudsmith API token, exported as CLOUDSMITH_API_KEY. Passed to each download as -H "X-Api-Key: $CLOUDSMITH_API_KEY". Anonymous reads return 402 (bandwidth cap), confirmed empirically, so auth is mandatory.
• Script logs the authenticated GitHub identity once at startup (gh api user --jq .login) so the run-results comment can confirm uploaded as ponylang-main.

Mechanics

• Language: bash. curl + jq + gh.
• Location: posted as a comment on this discussion before the run.
• Staging: STAGING=$(mktemp -d) — respects $TMPDIR. trap 'rm -rf "$STAGING"' EXIT. Delete each archive right after its upload step so disk usage at any moment is bounded.
• Idempotency: per-file asset-presence check; safe to re-run.
• Dry-run: --dry-run prints the full action list (per package: resolved tag, planned action) without downloading or uploading.
• Testing strategy: run --dry-run against the smallest population (changelog-tool) first. Eyeball the output. Then real run on that tool. Verify the uploaded assets on its GitHub release. Then repeat for corral, ponyup, ponyc in that order — increasing size.
• Failure handling: per-package failures log and continue. Script exits non-zero at the end if any package failed. No per-package retry — re-running the script covers it.
• Logging: one line per package — uploaded, uploaded (sibling only), skipped (no tag), skipped (prefix mismatch), already present (not verified), failed (checksum mismatch on download), failed (upload). Counts per tool at the end.

Artifacts produced on this discussion

Three comments, posted in this order:

1. The script — before the run. If revised, post the new version as a new comment rather than editing in place, so the thread carries history.
2. Skipped-items log — Cloudsmith packages with no matching GitHub release.
3. Run results — summary of uploaded / skipped / failed with per-tool counts.

Assumptions to validate at dry-run time

• Every Cloudsmith package in the filtered set includes checksum_sha512.
• Every Cloudsmith version matches a GitHub tagName exactly for the intersection we expect to migrate. All four repos use bare x.y.z today, and Cloudsmith matches.
• X-Api-Key header works on cdn_url downloads (Cloudsmith's documented auth method). Probe one archive before the real run.

[SeanTAllen]

Completed.