action: download latest version of apk (with fallback)#4375
Conversation
| APK_PKG_ID=$(curl -fsSL --retry 5 --retry-all-errors "${APK_TOOLS_API}/packages?package_name=${APK_VERSION}&package_type=generic" | jq '.[] | select(.version == "x86_64") | .id' | sort | tail -1) | ||
| APK_SHA256=$(curl -fsSL --retry 5 --retry-all-errors "${APK_TOOLS_API}/packages/${APK_PKG_ID}/package_files" | jq -r '.[0].file_sha256') | ||
| # If the API fails, fall back to a known-good version | ||
| if [ -z "$APK_VERSION" ] || [ -z "$APK_SHA256" ] |
There was a problem hiding this comment.
This fallback path can never happen. shell snippets use set -e and pipefail by default, so failure to download apk from above would just abort the script right there.
There was a problem hiding this comment.
Hoisted by my own petard. Having SC2155 always in my mind I thought the return value would get clobbered. I wrapped it in set +e/set -e. (I'm actually no fan of set -e, I think it makes error handling in shell in harder than it already is—people don't just panic in Rust everywhere either.)
There was a problem hiding this comment.
Heh, I actually think quite the opposite.. Error handling without set -e is unbearably hard. With the new set +e, if the APK_VERSION curl call fails, you then do a curl call with package_name= (empty), instead of aborting. I think a safer pattern would be
if X=$(curl ...) &&
Y=$(curl ... $X ...) &&
...; then
echo I got it!
else
echo I am so sad nowThere was a problem hiding this comment.
I.e I'm concerned that package_name= might actually be interpreted as "not set"/"no package name filter" instead of "searching for empty package name", and you actually do get something. That's how most web services work?
There was a problem hiding this comment.
Yes, for package_name= you get all the packages and then basically a random APK_PKG_ID and therefore sha256sum. This hinges on APK_VERSION being empty, though, which will trigger the first check in the fallback check.
| APK_TOOLS_API="https://gitlab.alpinelinux.org/api/v4/projects/5" | ||
| APK_VERSION=$(curl -fsSL --retry 5 --retry-all-errors "${APK_TOOLS_API}/repository/tags?per_page=1" | jq -r '.[].name' | sort -V | tail -1) | ||
| APK_PKG_ID=$(curl -fsSL --retry 5 --retry-all-errors "${APK_TOOLS_API}/packages?package_name=${APK_VERSION}&package_type=generic" | jq '.[] | select(.version == "x86_64") | .id' | sort | tail -1) | ||
| APK_SHA256=$(curl -fsSL --retry 5 --retry-all-errors "${APK_TOOLS_API}/packages/${APK_PKG_ID}/package_files" | jq -r '.[0].file_sha256') |
There was a problem hiding this comment.
An absent .file_sha256 gets rendered as string "null", failing the [ -z ] check below. jq has an // empty guard for that.
Also, is it safe to blindly take the first ([0]) file? I suppose good enough if they only ever publish one file.
There was a problem hiding this comment.
I hope that should be safe, since it's only a single package ID.
There was a problem hiding this comment.
I went for adding another check for APK_SHA256 being the actual string null. Unlikely, but then it'll cover the key not being there and the value being empty.
| sudo curl -fsSL -o /usr/bin/apk https://gitlab.alpinelinux.org/api/v4/projects/5/packages/generic/v3.0.6/x86_64/apk.static | ||
| echo 'f1489e05bace7d7dd0a687fcd38d50b585ac660af4231668b123649bef3718c4 /usr/bin/apk' | sha256sum --check | ||
| APK_TOOLS_API="https://gitlab.alpinelinux.org/api/v4/projects/5" | ||
| APK_VERSION=$(curl -fsSL --retry 5 --retry-all-errors "${APK_TOOLS_API}/repository/tags?per_page=1" | jq -r '.[].name' | sort -V | tail -1) |
There was a problem hiding this comment.
This gets the most recent tag, not the highest version. I.e. if "2.0.2" is the current highest version, and someone pushes an "1.0.3" maintenance update, the latter will the be most recent tag.
Also, the tail -1 should be redundant with per_page=1?
There was a problem hiding this comment.
Good catch, the per_page=1 actually still needed to go. Getting the latest version is done by sort -V. I've also added a filter to get out the rc tags.
This uses the gitlab API to fetch the latest release of apk from the upstream apk-tools repo, instead of hardcoding a version. Since gitlab API can be flaky, I put it in a retry loop. set -x in a subshell in the loop because you can never have too much info if/when things break. I also made the whole thing non-fatal if it still fails to get apk after retrying, because not all of them need apk (even if it means other later jobs using apk might fail). I added a warning for that situation so hopefully it makes it a bit more obvious why a later job using apk might fail if that was the case.
4 participants
This is a version of #4364 with my comments worked in.
/cc @craftyguy
Supersedes: #4364