Fix version race in the image build, and add image/service tests - #58
Merged
Conversation
tests/smoke-test.sh exercises a running webservice: the version it reports for itself, the organism and datasource catalogues, and real Derby-backed mappings for human and mouse. tests/verify-image.sh wraps it, first checking what is baked into an image (the version setup.sh pinned, a valid JAR, a present gdb.config) and then starting the container and running the smoke tests against it. Checks assert on response bodies rather than status codes. The webservice answers HTTP 200 for unknown paths with an "Unrecognized query" page, so a status-code assertion passes even against a server that resolves nothing. /swagger.yaml is the only endpoint that reports the running JAR's version, which makes it the one check able to detect an image tagged for a release it does not actually contain.
…latest The docker job read the webservice version from the setup.sh in its own checkout, which actions/checkout resolves at the SHA that triggered the run -- before the update-version job commits the new version. Builds therefore baked the previous release's JAR while tagging the image with the current version, and nothing detected it because both halves were internally consistent. The image published as 3.0.31-2.1.9 contains the 2.1.8 JAR. Versions are now resolved once in a resolve-version job and passed to the others as outputs. The docker job pins setup.sh in its own workspace from that value and fails if the pin does not take, so the build no longer depends on a commit made by a concurrent job. resolve-version also fails fast when the webservice release JAR has not been published yet, rather than part-way through an hour-long build. latest was previously a second, independent build-push invocation, so it was a different image from the version tag -- it had been serving the 2.1.7 image. It is now promoted by re-tagging the tested digest, after test-and-promote verifies the published version tag, so latest is always bit-for-bit an image that passed its tests. verify-published-image.yml runs the same verification weekly, and on demand for any tag, against whatever is currently on Docker Hub.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
The image published as
3.0.31-2.1.9contains the 2.1.8 webservice JAR.The
dockerjob took the webservice version from thesetup.shin its own checkout.actions/checkoutresolves the SHA that triggered the run, which predates the commitupdate-versionmakes — so the image tag came from a freshly downloadedpom.xml(correctly 2.1.9) while the JAR came from the stale checked-outsetup.sh(still 2.1.8). Nothing detected it, because each half was internally consistent.This is structural rather than a one-off: every run bakes the previous release's JAR while labelling it with the current version.
Verified against the published image:
Separately,
:latestwas built by a second, independentbuild-push-actioninvocation, so it was never the same image as the version tag — its digest matched the 2.1.7 image.The fix
resolve-versionjob reads the pom once and passesbdbversion/wsversionto the other jobs as outputs, so they cannot disagree.dockerjob pinssetup.shin its own workspace from that value and fails if the pin does not take. The build no longer depends on a commit made by a concurrent job.resolve-versionfails fast when the webservice release JAR is not yet published, instead of dying part-way through an hour-long build.:latestis no longer built separately.test-and-promotepulls and verifies the published version tag, then re-tags that exact digest as:latest, and asserts both tags share a digest. A mislabelled or broken build can no longer become:latest.update-versionstill commitssetup.shback tomain, but that commit is now bookkeeping rather than something the build reads.The tests
tests/smoke-test.shexercises a running service;tests/verify-image.shwraps it, first checking what is baked into an image (pinned version, valid JAR, presentgdb.config) and then starting the container and running the smoke tests. Both are plain shell and runnable locally:Coverage: the version the service reports for itself, the organism and datasource catalogues, Derby-backed human and mouse mappings, the attributes endpoint, and target-restricted mapping.
verify-published-image.ymlruns the same verification weekly, and on demand for any tag, against whatever is on Docker Hub.Two things worth knowing about these tests
Assertions are on response bodies, not status codes. The webservice returns HTTP 200 for unknown paths with an "Unrecognized query" page rather than a 404, so a status-code assertion passes even against a server that resolves nothing at all.
/swagger.yamlis the only endpoint reporting the running JAR's version, which makes it the single check capable of catching an image tagged for a release it does not contain.Note on merging
Merging does not by itself correct Docker Hub —
3.0.31-2.1.9andlatestboth still carry 2.1.8 code. The build needs re-running after this lands, which will republish the version tag correctly and promotelatestto it once the tests pass.The
pat_external_workflowsecret inbridgedb/BridgeDbWebserviceis still invalid (Bad credentials), so releases will continue to fail to trigger this workflow until it is regenerated. That is outside this repo.