diff --git a/packages/test-helpers/CONTRIBUTING.md b/packages/test-helpers/CONTRIBUTING.md index 382bd1f2402..0161d013d46 100644 --- a/packages/test-helpers/CONTRIBUTING.md +++ b/packages/test-helpers/CONTRIBUTING.md @@ -103,6 +103,28 @@ yarn workspace @elastic/eui-test-helpers show-report > The local workflow above is unchanged: Playwright's `webServer` is configured with `reuseExistingServer` (off in CI), so when you already have the dev server on `:6006` it is reused. In CI — where no dev server runs — `webServer` serves EUI's prebuilt static Storybook from `packages/eui/storybook-static`. That directory is a gitignored build artifact, so it must be produced first with `yarn workspace @elastic/eui build-storybook` (the `test-helpers` CI task does this automatically). +## Validating against Kibana before publishing + +The Storybook validation tests prove a helper works in isolation. They do **not** prove it works against a real consumer, where production DOM, parent-controlled state, and stricter test assertions surface issues Storybook can't. Validate against Kibana (the primary Scout consumer) *before* publishing — don't publish and fix the fallout later. + +**1. Prototype the Component Object in Kibana first.** Build it in Kibana's `kbn-scout` package, iterating against real specs, before porting it here: + +- Add the object under `src/platform/packages/shared/kbn-scout/src/playwright/eui_components/`, then register it on the `page.components` fixture so specs call `page.components.(testSubj)` (the combo box is the reference: `page.components.comboBox('myComboBox')`). +- Write/convert the Scout specs that use it and run them locally. + +Iterating in Kibana means you exercise the helper against the DOM it must actually support, not a curated story. + +**2. Open a draft Kibana PR and run CI.** Local runs cover a slice; CI runs every stateful/serverless lane. Trigger it and confirm the specs that use the helper pass across lanes. + +**3. Port to EUI and publish via a snapshot.** Once proven, move the object into this package (with `selectors.ts`, validation specs, and a README per [Adding a new Component Object](#adding-a-new-component-object)) and open the EUI PR. To run the *published* helper through Kibana CI you need a **snapshot** — Kibana CI installs from the npm registry, not a git ref, and this package is one workspace in the EUI monorepo, so you can't point Kibana at an EUI branch or commit. + +- **Nightly:** EUI auto-publishes a snapshot on weekdays under the `snapshot` dist-tag (see [`.github/workflows/update_kibana_dependencies.yml`](../../.github/workflows/update_kibana_dependencies.yml) for the schedule and dist-tag). Pin Kibana's `@elastic/eui-test-helpers` to that exact version. +- **On demand:** label the EUI PR `ci:regression-integration-test-kibana` to build a snapshot from the PR head and kick off the Kibana integration chain (requires write/label access on `elastic/eui`). + +Repin Kibana to the official release before merging its PR — snapshots are moving prereleases and get pruned. + +**4. Read first-attempt failures, not just the final status.** Scout retries a failed spec once, so a lane can be green overall while its first attempt failed. A first-attempt failure in a spec that uses your helper often points to helper flakiness (a race or timing bug the retry hides) — inspect those and fix before publishing. + ## CI integration These tests run in EUI's Buildkite CI on every PR, with flake detection when a component changes. See [Testing → EUI test helpers](../../wiki/contributing-to-eui/testing/eui-test-helpers.md) in the wiki. diff --git a/packages/test-helpers/README.md b/packages/test-helpers/README.md index d78cf01fbbc..9c6dd6be271 100644 --- a/packages/test-helpers/README.md +++ b/packages/test-helpers/README.md @@ -1,8 +1,9 @@ # EUI Test Helpers -`@elastic/eui-test-helpers` is a library of test helpers to use when testing EUI components -with [Cypress], [React Testing Library], or [Scout]. It provides assertion, find, and query helpers to make interacting -with EUI components easy and reliable. Say goodbye to finding CSS selectors to query that _seem_ right. +`@elastic/eui-test-helpers` provides helpers for interacting with EUI components reliably in +consumer tests — no more guessing at CSS selectors that only _seem_ right. Today it ships +**Playwright Component Objects** for [Scout]/[Playwright] consumers; [Cypress] and +[React Testing Library] helpers may follow. > [!NOTE] > This library is in early stages of development and is missing many useful utilities. Please contribute or @@ -51,6 +52,7 @@ their own version at runtime. See [CONTRIBUTING.md](CONTRIBUTING.md) for design principles, directory structure, and how to add new Component Objects. +[Playwright]: https://playwright.dev [Cypress]: https://github.com/cypress-io/cypress [React Testing Library]: https://github.com/testing-library/react-testing-library [Scout]: https://github.com/elastic/kibana/tree/main/src/platform/packages/shared/kbn-scout