Skip to content

Rewrite the test suite and fix seven driver bugs - #162

Merged
tas50 merged 1 commit into
mainfrom
test-suite-rewrite
Aug 22, 2026
Merged

Rewrite the test suite and fix seven driver bugs#162
tas50 merged 1 commit into
mainfrom
test-suite-rewrite

Conversation

@tas50

@tas50 tas50 commented Aug 22, 2026

Copy link
Copy Markdown
Member

What this does

Replaces the unit test suite from the ground up and fixes the bugs that writing it uncovered.

The previous suite was one Minitest file with a single assertion (api_version == 2), and it could not run at all — the test bundle group pulled in Berkshelf, Vagrant, Dokken and Inspec, so bundle install failed to resolve before a single assertion ran.

  • RSpec: 103 examples, replacing the Minitest/mocha/minitest-stub-const stack
  • Pester 5: 35 tests, ported from Pester 3 syntax and extended
  • 7 bug fixes, each with a regression test
  • YARD on the public API — 100% documented, no warnings
  • Heavy plugins moved to an integration bundle group so unit work installs fast

How the specs are built

The driver's job is turning configuration into PowerShell and running it in the right order. So the specs fake only the outer boundary — the Train connection — and let everything above it run for real. Each command travels the genuine run_pswrap_commandencode_command path, and the fake decodes the -encodedcommand payload back to readable PowerShell:

driver.create(state)

expect(connection.scripts_matching(/New-KitchenVM/).first)
  .to include(%{SwitchName = "Default Switch"})

Stubbing run_ps instead would assert that the driver called a mock, not that it generated the right PowerShell.

Bugs fixed

# Bug Impact
1 powershell.rb used Base64.strict_encode64 without requiring base64 Worked only because Train requires it first. base64 is a bundled gem from Ruby 3.4 on, so this was one dependency change away from breaking every run_ps call. Now required and declared in the gemspec.
2 update_state did @vm["Id"] on nil Get-VmDetail returning nothing surfaced as undefined method '[]' for nil instead of anything actionable.
3 Additional disks ignored remote_vm_path On a remote host they were created under the local kitchen root, where the Hyper-V server cannot see them — unlike the differencing disk.
4 destroy left a stale state[:id] Returning early when the VM was already gone kept the id in the state file permanently.
5 Support script uploaded on the local backend The local backend reads the in-gem copy via base_script_path and never looks at the uploaded one.
6 between? on raw config values A number quoted in kitchen.yml raised comparison of String with 1 failed rather than the intended validation message. Both memory and VLAN bounds now coerce first.
7 local_script_path kept its .. segments; differencing_disk_exists returned nil Cosmetic, but the unnormalized path showed up in every debug log.

The Pester port is semantic, not cosmetic

Pester 5 changed three things that each silently produce tests which pass no matter what the code does. Ported literally, the old tests would have asserted nothing:

  • Discovery is separate from execution. The old tests call New-KitchenVM in the Describe/Context body, which runs during discovery — before any Mock is active.
  • Should -Invoke defaults to -Scope It and counts only invocations from the same It block. Calls made in a BeforeAll are not counted.
  • -Times N means "at least N" since Pester 4, so -Times 0 can never fail. Roughly half the original assertions are "should not have been called" — all now use -Exactly.

spec/powershell/TestHelper.ps1 supplies mockable cmdlet shims: Pester builds CommandMetadata for a mocked command, which forces resolution of every parameter type, and the stub module's signatures name types like [Microsoft.HyperV.PowerShell.VirtualMachine] that only exist on a real Hyper-V host.

Verification

Both suites were run locally, and mutation-tested to confirm they can actually fail:

Mutation Result
Drop the $VlanId -ne $null guard 3 Pester tests fail
Drop the iso_path guard 1 RSpec example fails
Revert the stale-state[:id] fix 1 RSpec example fails
rspec   103 examples, 0 failures
pester  35 passed, 0 failed        (pwsh 7.5.4 / Pester 6.1.0)
yard    100.00% documented, no warnings
rubocop 12 files inspected, no offenses
rake    exit 0

Both suites run on any platform — neither needs Hyper-V, Windows, or an elevated shell.

Rake tasks

rake yard, rake yard:stats, rake yard:server, and rake pester are added. None are prerequisites of rake default, so documentation coverage and PowerShell availability cannot fail a build. rake test and rake unit remain as aliases for rake spec, so the shared CI workflow is unaffected.

Notes for review

  • The Pester files moved from spec/support/ to spec/powershell/. spec/support/ is now RSpec's auto-loaded Ruby helper directory, so the two could not share it.
  • .rubocop.yml no longer excludes spec/**/* — the specs lint clean.
  • CONTRIBUTING.md had a stale claim that bundle exec rake changelog exists. It never did, and github_changelog_generator is not in the Gemfile; release-please handles the changelog.
  • CI workflows are untouched. support/ci/windows_ci.bat did need updating — it ran invoke-pester ./spec directly and installed Pester unversioned.
  • Labelled feat: so release-please cuts a minor bump — the seven fixes change runtime behavior and should not ship silently.

🤖 Generated with Claude Code

Replaces the Minitest suite (one assertion, and unrunnable because the
`test` bundle group pulled in Berkshelf, Vagrant and Dokken) with an
RSpec suite of 103 examples, and ports the PowerShell tests to Pester 5.

The specs fake only the Train connection, so every command travels the
real run_ps -> wrap_command -> encode_command path and the fake decodes
the -encodedcommand payload back to readable PowerShell. Assertions are
on the script that would have reached the host, not on a mock.

Bugs found while writing the tests, each with a regression test:

- powershell.rb called Base64.strict_encode64 without requiring base64;
  it worked only because Train happens to require it first. base64 is a
  bundled gem from Ruby 3.4 on, so it is now required and declared.
- update_state raised `undefined method '[]' for nil` when Get-VmDetail
  returned nothing. It now reports what actually went wrong.
- Additional disks always used the local kitchen root, so against a
  remote host they were created where the Hyper-V server cannot see
  them. They now sit under remote_vm_path beside the differencing disk.
- destroy returned early when the VM was already gone without clearing
  state[:id], leaving a stale id in the state file permanently.
- The support script was uploaded even on the local backend, which
  reads the in-gem copy and never looks at the uploaded one.
- Memory and VLAN bounds were checked with between? on the raw config,
  so a value quoted in kitchen.yml raised "comparison of String with 1
  failed" instead of the intended message. Both are coerced first.
- local_script_path kept its ".." segments, and
  differencing_disk_exists returned nil rather than false.

The Pester port is a semantic change, not a syntax refresh. Pester 5
splits discovery from execution, so calls made in a Describe body run
before any Mock is active; `Should -Invoke` defaults to `-Scope It` and
counts only invocations from the same It block; and `-Times N` means
"at least N", so `-Times 0` can never fail without `-Exactly`. Ported
literally, the old tests would have asserted nothing. TestHelper.ps1
supplies mockable shims because Pester builds CommandMetadata for a
mocked command, which cannot resolve the Hyper-V parameter types the
stub module declares.

Also: YARD documentation on the public API (100% documented, no
warnings), `rake yard`, `rake yard:stats`, `rake yard:server` and
`rake pester` tasks, none of them prerequisites of `rake default` so
docs and PowerShell cannot fail a build; the heavy plugins moved to an
`integration` bundle group; and specs are no longer excluded from
Cookstyle.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@tas50
tas50 merged commit 6616094 into main Aug 22, 2026
8 checks passed
@tas50
tas50 deleted the test-suite-rewrite branch August 22, 2026 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant