test(serenity): API-level e2e against the Semrush vendor mocks

[rainer-friederich]

1. Abstract

Adds API-level end-to-end tests for the /serenity/* routes that drive the real controller through the public Semrush vendor mocks (Project Engine + User Manager) in the postgres integration-test harness, replacing the prior 400/401-only contract suite.

2. Reasoning

The /serenity/* surface sits behind IMS-only authentication and a live Semrush gateway, so the existing integration tests could only assert the route gate (400) and the IMS-only contract (401) — never the actual handler behaviour. Now that the Semrush Project Engine and User Manager APIs are published as stateful Counterfact mock Docker images, the routes can be exercised end to end, both locally and in CI, without a real IMS account or a real vendor.

3. High-level overview of the changes

• The integration-test stack now boots the two Semrush mocks alongside Postgres/PostgREST and MinIO, and the serenity suite drives the controller through them over HTTPS.
• Auth (test-only): requireImsBearer honours a SERENITY_ALLOW_NON_IMS_AUTH flag that skips the IMS-type gate so the harness's locally-signed JWT can reach the handlers. This is sound only against the mocks, which do not validate the forwarded bearer (the token value never matters). The flag is never written to Vault / any deployed environment — it mirrors the existing SERENITY_ALLOW_WORKSPACE_DELETE opt-in.
• Base-URL split (issue 2656): the User Manager gateway origin is resolved from SEMRUSH_USERS_BASE_URL, falling back to SEMRUSH_PROJECTS_BASE_URL when unset. This lets the two mocks live on separate hosts with no path-routing reverse proxy. Production/stage/dev (a single shared host today) are unchanged via the fallback.
• Version-alignment convention: the mock image tag is derived from the installed client dependency version — the harness reads @adobe/spacecat-shared-project-engine-client / -user-manager-client from node_modules and exports SERENITY_PE_MOCK_TAG / SERENITY_UM_MOCK_TAG, which the compose interpolates. The mock is published from the same package as the typed client, so this guarantees the integration test exercises the exact contract version we ship; it can never silently drift. If a client is bumped to a version whose mock image is not yet published, the image pull fails loudly — by design, forcing the mock to be released in lockstep.
• Dependency bump: the two Semrush clients are moved to 1.3.0, the version for which the mock images are published (and the version whose typed surface the transport already targets).
• First test increment: route-gate validation, the brand-independent model/language catalog read live through the Project Engine mock, and brand resolution after the relaxed auth. The mutating sub-workspace flows (activate/deactivate, market create/delete) are the next increment; resetSemrushMocks is already wired for them.

4. Required information

• Other:
  • Base-URL split (implemented here): refactor(serenity): split Semrush base URLs (Project Engine vs User Manager) #2656
  • Cross-repo e2e harness architecture spec (related context): https://github.com/adobe/mysticat-architecture/pull/169
  • Semrush vendor mock images: feat(project-engine-client): publish stateful mock as an HTTPS Docker image (LLMO-5460) spacecat-shared#1732 and feat(user-manager-client): publish stateful mock as an HTTPS Docker image spacecat-shared#1735

5. Affected / used mysticat-workspace projects

• spacecat-shared — consumed. Provides both the typed Semrush clients and the mock Docker images they are published alongside. This PR bumps the two clients to 1.3.0 and pins the mock images to the same version. No change to spacecat-shared is required (the 1.3.0 clients and mocks are already published).

6. Additional information outside the code

• The serenity integration suite was run end to end through the harness against the real mock containers (pulled anonymously from GHCR): the route gate, the org-level model and language catalogs returning live data through the Project Engine mock, and brand resolution proceeding past the relaxed auth. It was also run alongside a sibling integration suite to confirm the always-on mocks and the global integration-test env do not regress other integration tests.
• The Project Engine mock's catalog endpoints were smoke-tested directly over HTTPS to confirm the served shapes match what the controller reads.

7. Test plan

• Local: bring up the integration stack and run the serenity suite — npx mocha --require test/it/postgres/harness.js --timeout 60000 test/it/postgres/serenity.test.js — which boots both mocks (tag derived from the installed client version), points the transport at them, and drives the routes through to the live mock responses. Verified green this session.
• CI: runs automatically through the mysticat-ci reusable workflow's existing it-postgres target — it brings up the same compose (the mock images are public, so no registry credentials are needed) with no workflow change.
• dev / stage / prod: nothing to verify — this is test-only infrastructure. The only production-path code is the base-URL split, which is a no-op while SEMRUSH_USERS_BASE_URL is unset (fallback to the existing host); the auth flag and the split target are never set in any deployed environment.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 96.96970% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/controllers/serenity.js	90.32%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[github-actions]

This PR will trigger a patch release when merged.

[MysticatBot]

Hey @rainer-friederich,

Verdict: Approve - well-structured test infrastructure with sound production changes; only non-blocking notes below.
Complexity: HIGH - large diff; dependency bump + API surface change.
Changes: Adds API-level e2e tests for /serenity/* routes against Semrush vendor mocks, splits the User Manager base URL, and bumps Semrush clients to 1.3.0 (12 files).
Note: CI checks are currently failing - resolve before merge.

Non-blocking (5): minor issues and suggestions

• nit: resetSemrushMocks() swallows all errors with .catch(() => {}) - when the mutating-lifecycle tests land (next increment), a silently failed reset will produce flaky order-dependent tests. Consider checking response.ok and throwing on non-2xx. - test/it/postgres/setup.js:141
• suggestion: The SERENITY_ALLOW_NON_IMS_AUTH env flag bypasses the IMS-type gate at runtime with no deploy-time structural guard. The defense-in-depth model (Semrush validates upstream) makes this acceptable today, but a debug-level log when the flag is active would make accidental enablement visible in production logs without adding a hard gate. - src/controllers/serenity.js:199
• nit: The old IT suite tested 401 on each endpoint individually (markets, prompts, tags, models, activate, deactivate, DELETE). The new suite proves auth bypass via a single 404 assertion. Unit tests still cover per-endpoint enforcement, so this is a noted tradeoff rather than a gap - consider adding one more brand-level endpoint (e.g. GET prompts) to the e2e suite for breadth. - test/it/shared/tests/serenity.js
• nit: GET /serenity/languages asserts only expect(res.body).to.be.an('object') - passes for any JSON object including error responses. Adding expect(res.body).to.have.property('items') would catch schema drift. - test/it/shared/tests/serenity.js:72
• nit: usersBaseUrl silently falls back to SEMRUSH_PROJECTS_BASE_URL when SEMRUSH_USERS_BASE_URL is unset. A debug-level log on fallback would help distinguish "intentionally shared host" from "forgot to set the new var" during troubleshooting. - src/support/serenity/rest-transport.js:209

---

_{Skill: pr-review | Model: us.anthropic.claude-opus-4-6-v1[1m] | Duration: 1m 59s | Cost: $9.16 | Commit: a467c9c37c5378c132b6fc998089e47f30bd2674
If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[rainer-friederich]

Thanks @MysticatBot — addressed the non-blocking notes in 40366566 (all 7 serenity IT cases green against the mocks). Per-item:

• resetSemrushMocks swallowed errors — fixed. It now throws on a non-2xx reset instead of .catch(() => {}), so a failed reset surfaces immediately rather than producing flaky order-dependent tests when the mutating-lifecycle increment lands.
• SERENITY_ALLOW_NON_IMS_AUTH visibility — fixed. Logs once per process when the flag is active. Used warn rather than debug on purpose: the whole point is to surface an accidental enablement in a deployed env, where the default log level would drop debug. Gated to one line per process so it isn't per-request noise.
• More brand-level e2e breadth — fixed. Added a second brand-level case (GET /serenity/prompts with query params) alongside markets, both proving the relaxed-auth path reaches the handler (404 on an unknown brand, not 401/500).
• /serenity/languages weak assertion — fixed. Now asserts the { items: [...] } envelope (non-empty), matching the models check, so an error body slipping through as 200 would fail.
• usersBaseUrl silent fallback log — declined. The fallback to SEMRUSH_PROJECTS_BASE_URL is the intended default and the production path today (single shared host), so it fires on every request; logging it even at debug is per-request noise for the normal case. usersBaseUrl is also a pure typed helper with no logger injected (kept clean per this dir's type-check convention), and the fallback is already documented in its JSDoc. A "forgot to set it" mistake is harmless here — it resolves to the same host that works today.

[MysticatBot]

Hey @rainer-friederich,

Verdict: Approve - all prior non-blocking findings addressed; no new issues found on re-review.
Complexity: HIGH - large diff; dependency bump + API surface + infrastructure signals.
Changes: Addresses prior review nits - adds warn log for auth flag, strengthens test assertions, adds GET prompts breadth test, and fixes resetSemrushMocks error propagation (12 files).
Note: CI checks are currently pending - resolve before merge.

Previously flagged, now resolved

• resetSemrushMocks now throws on non-2xx instead of swallowing errors
• SERENITY_ALLOW_NON_IMS_AUTH logs once at warn level when active (surfaces accidental enablement)
• Added GET /serenity/prompts brand-level e2e test for breadth
• /serenity/languages assertion strengthened to check { items: [...] } envelope
• usersBaseUrl fallback log declined (sound reasoning: normal production path, typed helper with no logger, documented in JSDoc)

Non-blocking (3): minor issues and suggestions

• nit: CLAUDE.md documents --timeout 30000 for the full IT glob, but the serenity suite was run with --timeout 60000 in the test plan. The harness hook timeouts are independently managed, so the documented command likely works, but consider adding a note for the serenity-specific timeout or updating the value. - CLAUDE.md
• nit: NODE_TLS_REJECT_UNAUTHORIZED save/restore in waitForSemrushMocks() and resetSemrushMocks() is redundant given buildEnv() already sets it to '0' for the entire IT process. The save/restore implies scoped mutation that is not actually needed. - test/it/postgres/setup.js:100
• suggestion: The "relaxed auth" 404 tests only assert status code. Asserting the body shape (e.g., res.body.error === 'notFound') would distinguish "handler ran and returned a clean 404" from a generic middleware 404 for an unmatched route. - test/it/shared/tests/serenity.js:85

---

_{Skill: pr-review | Model: us.anthropic.claude-opus-4-6-v1[1m] | Duration: 1m 41s | Cost: $7.90 | Commit: 40366566cd629331394dd9d0591914237d158317
If this code review was useful, please react with 👍. Otherwise, react with 👎.}