Skip to content

ci: verify main itself, and check the built gem before it ships - #65

Open
tas50 wants to merge 2 commits into
mainfrom
ci-verify-main-and-package
Open

ci: verify main itself, and check the built gem before it ships#65
tas50 wants to merge 2 commits into
mainfrom
ci-verify-main-and-package

Conversation

@tas50

@tas50 tas50 commented Aug 30, 2026

Copy link
Copy Markdown
Member

Two gaps in what CI actually covers. Neither is about testing against a real
CloudStack — see the note at the bottom on why I did not go that way.

main is never verified

.github/workflows/lint.yml only triggers on pull_request. Two pull requests
that are each green against their own merge base can still break main
together, and release-please cuts a release straight off whatever is sitting on
main — so a break there ships rather than being caught. Added
push: branches: [main].

Nothing looks at the built gem before it goes to RubyGems

The release path is fully automated: release-please merges, publish.yml builds
the gem and pushes it to GitHub Packages and RubyGems. No human sees the package
in between.

spec.files is built by grepping git ls-files:

spec.files = `git ls-files`.split($/).grep(/LICENSE|^lib/)

That is fine today — I built the gem and it contains LICENSE and all six
files under lib/. But it is a regex against filenames, so renaming the license
file to LICENSE.txt or LICENSE.md would silently ship every subsequent
release with no license in it, and the first anyone would know is a downstream
packaging complaint.

Added a job that builds the gem on every pull request and asserts LICENSE and
lib/kitchen/driver/cloudstack.rb are in the packaged file list. I checked it
actually catches the thing by pointing the glob at LICENSE.txt:

$ gem spec kitchen-cloudstack-1.1.0.gem files
---
- lib/kitchen/driver/cloudstack.rb
...
FAILS AS EXPECTED

What I looked at and left alone

Unit tests across the supported Rubies: already fine. The shared
test-kitchen/.github lint-unit.yml runs bundle exec rake test on Ruby 3.1,
3.2, 3.3, 3.4 and 4.0, which matches the gemspec's required_ruby_version >= 3.1 exactly. It also runs cookstyle --chefstyle, yamllint and markdownlint.

No orphan tests. rake test uses spec/**/*_spec.rb, so every spec file in
the repo runs, spec/integration/lifecycle_spec.rb included. There are no test
scripts sitting outside that pattern.

Integration against a fake CloudStack: already here, and #62 owns it.
spec/integration/lifecycle_spec.rb already stubs the HTTP boundary and drives
a real Kitchen::Instance through create/status/destroy, so Test Kitchen's
lifecycle, the driver, the client gem and real response parsing all execute
against a faked cloud on every pull request. Broadening it — signature
verification, doctor, a repeat destroy, the associate_public_ip path — would
mean rewriting exactly the file and exactly the client layer that #62 is
replacing. That is worth doing, but on top of #62, not underneath it.

tas50 added 2 commits August 29, 2026 19:12
The workflow only ran on pull_request, so main itself was never verified.
Two pull requests that are each green can still break main together, and
release-please cuts a release straight off whatever is on main, so a
break there goes out rather than being caught.

Signed-off-by: Tim Smith <tim@mondoo.com>
Nothing inspects the package between a merge and RubyGems: release-please
merges, the publish workflow builds and pushes, and that is the whole
path. A gemspec whose spec.files glob stops matching would therefore ship
silently -- the file list here is built by grepping git ls-files, so a
rename of LICENSE is all it would take.

Build the gem on every pull request and assert LICENSE and lib/ are in
it. Verified by pointing the glob at LICENSE.txt, which the check
rejects.

Signed-off-by: Tim Smith <tim@mondoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant