Skip to content

Commit 2fb981b

Browse files
authored
feat(everything-dev): regression runs on wiped file-backed pglite dirs — the seed phase reaches CI (#869)
.env.test's in-memory pglite marker now substitutes per-workspace pglite dirs under .bos/regression/db (absolute — services resolve relative pglite paths against their own workspace cwd, which is how the dev layout scatters), and start-stack wipes the root per run. An in-memory database cannot be seeded: nothing exists before its service process creates it, so the pre-spawn seed phase (#866) had nothing to address. A wiped dir keeps in-memory's isolation while opening the phase's window — apps that declare a seed block get it on every regression boot, which is the migration path off direct-DB seed helpers and the postgres service containers they force. Explicit database URLs pass through untouched; dev-root URLs stay refused.
1 parent 189080a commit 2fb981b

7 files changed

Lines changed: 133 additions & 22 deletions

File tree

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
"everything-dev": minor
3+
---
4+
5+
Regression harness runs on wiped file-backed pglite dirs — seed-phase integration
6+
7+
`.env.test`'s in-memory pglite marker now substitutes per-workspace pglite dirs
8+
under `.bos/regression/db` (absolute paths; start-stack wipes the root per run),
9+
because an in-memory database cannot be seeded — nothing exists before its
10+
service process creates it, so the pre-spawn seed phase (#866) had nothing to
11+
address. A wiped dir keeps in-memory's isolation while opening the pre-spawn
12+
window: **apps that declare a `seed` block get it applied on every regression
13+
boot automatically**, which is also the migration path off direct-DB seed
14+
helpers and the postgres service containers they force. Explicit database URLs
15+
(postgres or custom pglite dirs) still pass through untouched, and dev-root
16+
URLs stay refused.

‎AGENTS.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -266,7 +266,10 @@ zero-config, but drizzle-kit commands (`db:studio`/`db:migrate`/`db:push`) requi
266266
divergence sets `TEST_DATABASE=postgres` + the external URL — no committed compose exists.
267267

268268
**Dev/test isolation:** a committed `.env.test` (generated by `bos dev`) maps every
269-
`*_DATABASE_URL` to in-memory pglite and `BETTER_AUTH_SECRET` to the test secret. Test
269+
`*_DATABASE_URL` to in-memory pglite and `BETTER_AUTH_SECRET` to the test secret; the
270+
regression harness substitutes in-memory markers for wiped per-workspace pglite dirs
271+
under `.bos/regression/db` (start-stack wipes them per run) so a declared `seed` block
272+
runs pre-spawn on every regression boot. Test
270273
suites load `.env.test` instead of `.env`: the regression stack
271274
(`tests/regression/lib/start-stack.mjs`) injects it into spawned stacks,
272275
`tests/regression/lib/regression-env.mjs` resolves it with fail-fast guards that refuse

‎docs/agents/testing.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -106,8 +106,10 @@ boundary above; until it lands, this document is the contract.
106106
cached session; mint a new state only when the fixtures don't cover it, and
107107
route the seeding through the shared HTTP helpers
108108
(`tests/regression/lib/http-seed.mjs` and the bootstrap-admin API key is
109-
the supported path — the stack database is in-process pglite, so direct DB
110-
writes are impossible).
109+
the supported path — the stack database is a pglite data dir held open by
110+
its service (ADR 0044's file lock), so direct DB writes are impossible
111+
post-spawn. App fixtures belong in declared `seed` steps instead, which
112+
the phase runs pre-spawn — see `docs/seed-phase.md).
111113
- **Serial by design**: `workers: 1`, `fullyParallel: false`, `retries: 1` —
112114
specs share ports and the seeded cookie cache. Don't write a spec that
113115
assumes a clean global state another spec might have breathed on; seed

‎docs/seed-phase.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,21 @@ admin user instead of minting a second user. Multi-segment accounts (whose
104104
sign-in email is unpredictable) are linked by the `near_account` row, not the
105105
email.
106106

107+
## Regression: the phase runs on every boot
108+
109+
The regression harness substitutes `.env.test`'s in-memory pglite markers for
110+
wiped per-workspace pglite dirs (`.bos/regression/db/<slug>`, absolute paths —
111+
services resolve relative pglite paths against their own workspace cwd), and
112+
start-stack wipes the root before the stack spawns. A fresh dir holds no
113+
marker, so **a declared `seed` block runs in full on every regression boot** —
114+
the phase's pre-spawn window is exactly what makes this possible.
115+
116+
That is also the migration path for consumer apps whose test helpers open
117+
`pg` connections directly (e.g. seeding fixtures for browser specs): a direct
118+
connection requires a shared, externally-reachable database, which is what
119+
keeps postgres service containers in CI. Port those helpers to declared
120+
`seed` steps and the suite runs on pglite with no services at all.
121+
107122
## Production: supported, never implicit
108123

109124
Production seeding never happens as a side effect of boot or deploy. Run it

‎tests/regression/lib/regression-env.mjs‎

Lines changed: 52 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -83,23 +83,26 @@ export function readDotEnv(repoRoot, fileName = ".env") {
8383
return parse(fs.readFileSync(envPath));
8484
}
8585

86-
function databaseSecrets(config) {
86+
function databaseSecretEntries(config) {
8787
const sections = [
88-
config.app?.api?.secrets,
89-
config.app?.auth?.secrets,
90-
...Object.values(config.plugins ?? {}).map((plugin) => plugin?.secrets),
88+
{ key: "api", secrets: config.app?.api?.secrets },
89+
{ key: "auth", secrets: config.app?.auth?.secrets },
90+
...Object.entries(config.plugins ?? {}).map(([key, plugin]) => ({
91+
key,
92+
secrets: plugin?.secrets,
93+
})),
9194
];
9295
const seen = new Set();
93-
const secrets = [];
94-
for (const list of sections) {
95-
for (const secret of list ?? []) {
96+
const entries = [];
97+
for (const { key, secrets } of sections) {
98+
for (const secret of secrets ?? []) {
9699
if (typeof secret === "string" && secret.endsWith("_DATABASE_URL") && !seen.has(secret)) {
97100
seen.add(secret);
98-
secrets.push(secret);
101+
entries.push({ secret, key });
99102
}
100103
}
101104
}
102-
return secrets;
105+
return entries;
103106
}
104107
/**
105108
* The compose-era local convention (`postgres://everythingdev:…@localhost:{5432,5433}/…`
@@ -125,26 +128,61 @@ function isDevDatabaseUrl(url) {
125128
);
126129
}
127130

131+
/**
132+
* Mirror of the framework's normalizeSlug (db/core) — the regression dirs
133+
* must name themselves exactly as the dev layout (ADR 0044) does, so the
134+
* pre-spawn seed phase and the services address the same files.
135+
*/
136+
function normalizeSlug(name) {
137+
const basename = name.split("/").pop() ?? name;
138+
return basename
139+
.replace(/^@/, "")
140+
.replace(/[^a-zA-Z0-9_-]/g, "_")
141+
.replace(/-plugin$/i, "")
142+
.replace(/-/g, "_")
143+
.toLowerCase();
144+
}
145+
146+
/**
147+
* The regression database root: scratch, wiped per run (start-stack), one
148+
* subdirectory per workspace. `.env.test`'s in-memory pglite marker means
149+
* "harness-picked pglite" and is substituted for these dirs, because an
150+
* in-memory database cannot be seeded — nothing exists before its service
151+
* process creates it, so a pre-spawn seed phase (#866) has nothing to
152+
* address. A wiped dir keeps in-memory's isolation (nothing survives a
153+
* run) while opening the pre-spawn window the seed phase needs.
154+
*
155+
* The substituted URLs are ABSOLUTE: each service resolves a relative pglite
156+
* path against its own workspace cwd (that is how the dev layout scatters
157+
* `api/.bos/db/api` vs `.bos/db/auth`), and the wipe must own one root.
158+
*/
159+
const REGRESSION_DB_SUBDIR = path.join(".bos", "regression", "db");
160+
const IN_MEMORY_PGLITE_URLS = new Set(["pglite::memory:", "pglite:memory:"]);
161+
128162
export function computeRegressionEnv({ repoRoot, env = process.env } = {}) {
129163
const root = repoRoot ?? findRepoRoot();
130164
if (!root) throw new Error("No authored config (bos.app.ts) found in any parent directory");
131165
const config = loadRegressionConfig(root);
132166
const fileEnv = readDotEnv(root);
133167
const testEnv = readDotEnv(root, ".env.test");
134168
const allowDevDb = env.REGRESSION_ALLOW_DEV_DB === "1";
169+
const regressionDbRoot = path.join(root, REGRESSION_DB_SUBDIR);
135170

136171
const dbUrls = {};
137-
for (const secret of databaseSecrets(config)) {
172+
for (const { secret, key } of databaseSecretEntries(config)) {
138173
const resolved = testEnv[secret] ?? env[secret] ?? fileEnv[secret] ?? TEST_DATABASE_DEFAULT;
139-
if (!allowDevDb && isDevDatabaseUrl(resolved)) {
174+
const url = IN_MEMORY_PGLITE_URLS.has(resolved)
175+
? `pglite:${regressionDbRoot}/${normalizeSlug(key)}`
176+
: resolved;
177+
if (!allowDevDb && isDevDatabaseUrl(url)) {
140178
throw new Error(
141-
`refusing to run: ${secret} resolves to the dev database (${resolved}). ` +
179+
`refusing to run: ${secret} resolves to the dev database (${url}). ` +
142180
"Regression tests must stay isolated from dev data. " +
143181
"Restore the generated .env.test (in-memory pglite — run `pnpm run bos dev` to regenerate), " +
144182
"or set REGRESSION_ALLOW_DEV_DB=1 to override deliberately.",
145183
);
146184
}
147-
dbUrls[secret] = resolved;
185+
dbUrls[secret] = url;
148186
}
149187

150188
const baseUrl = env.REGRESSION_BASE_URL ?? fileEnv.REGRESSION_BASE_URL ?? DEFAULT_BASE_URL;
@@ -180,6 +218,7 @@ export function computeRegressionEnv({ repoRoot, env = process.env } = {}) {
180218
baseUrl,
181219
basePort,
182220
dbUrls,
221+
dbRoot: regressionDbRoot,
183222
authSecret,
184223
bootstrapAdminKey,
185224
stalePorts,

‎tests/regression/lib/regression-env.test.mjs‎

Lines changed: 34 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -113,15 +113,46 @@ void test("per-identity pglite dirs under the dev root are dev data too", () =>
113113
);
114114
});
115115

116-
void test("in-memory pglite and external postgres URLs are accepted test targets", () => {
116+
void test("the in-memory test marker substitutes wiped per-workspace pglite dirs", () => {
117117
const repoRoot = fixture();
118118
writeFileSync(
119119
join(repoRoot, ".env.test"),
120120
"API_DATABASE_URL=pglite::memory:\nBETTER_AUTH_SECRET=test-secret\n",
121121
);
122122
const config = computeRegressionEnv({ repoRoot, env: {} });
123-
assert.equal(config.dbUrls.API_DATABASE_URL, "pglite::memory:");
124-
assert.equal(config.dbUrls.AUTH_DATABASE_URL, "pglite::memory:");
123+
const dbRoot = join(repoRoot, ".bos", "regression", "db");
124+
assert.equal(config.dbUrls.API_DATABASE_URL, `pglite:${dbRoot}/api`);
125+
assert.equal(config.dbUrls.AUTH_DATABASE_URL, `pglite:${dbRoot}/auth`);
126+
assert.equal(config.dbRoot, dbRoot);
127+
});
128+
129+
void test("explicit database URLs pass through untouched (sslmode param preserved)", () => {
130+
const repoRoot = fixture();
131+
const url = "postgres://test:test@postgres-api-test:5432/api_test_db?sslmode=disable";
132+
const config = computeRegressionEnv({
133+
repoRoot,
134+
env: { API_DATABASE_URL: url, BETTER_AUTH_SECRET: "test-secret" },
135+
});
136+
assert.equal(config.dbUrls.API_DATABASE_URL, url);
137+
});
138+
139+
void test("plugin keys slug per-workspace regression dirs like the dev layout", () => {
140+
const repoRoot = fixture();
141+
writeFileSync(
142+
join(repoRoot, "bos.config.json"),
143+
JSON.stringify({
144+
app: { api: { secrets: ["API_DATABASE_URL"] } },
145+
plugins: {
146+
"@acme/widget-plugin": { secrets: ["WIDGET_DATABASE_URL"] },
147+
},
148+
}),
149+
);
150+
writeFileSync(join(repoRoot, ".env.test"), "BETTER_AUTH_SECRET=test-secret\n");
151+
const config = computeRegressionEnv({ repoRoot, env: {} });
152+
assert.equal(
153+
config.dbUrls.WIDGET_DATABASE_URL,
154+
`pglite:${join(repoRoot, ".bos", "regression", "db")}/widget`,
155+
);
125156
});
126157

127158
void test("invalid modes fail before starting a stack", () => {

‎tests/regression/lib/start-stack.mjs‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
#!/usr/bin/env node
22
// Boots the regression stack (dev/prod/backcompat) with the test environment
3-
// from .env.test (in-memory pglite) so regression runs never touch dev data
4-
// or dev ports.
3+
// from .env.test (wiped per-workspace pglite dirs) so regression runs never
4+
// touch dev data or dev ports.
55
import { spawn } from "node:child_process";
6-
import { mkdirSync, openSync } from "node:fs";
6+
import { mkdirSync, openSync, rmSync } from "node:fs";
77
import net from "node:net";
88
import { join } from "node:path";
99
import { setTimeout as sleep } from "node:timers/promises";
@@ -72,6 +72,11 @@ if (!root) {
7272
}
7373

7474
const regressionEnv = computeRegressionEnv({ repoRoot: root });
75+
// The regression DB root is scratch: wiped per run so every boot starts from
76+
// empty — the seed phase's marker lives in the api database, so a fresh dir
77+
// means the phase runs again. Wiped before the stack spawns; nothing holds
78+
// these dirs open yet.
79+
rmSync(regressionEnv.dbRoot, { recursive: true, force: true });
7580
const spec = regressionStackOptions(regressionEnv, mode);
7681
await waitForDatabases(regressionEnv.dbUrls);
7782
killStalePorts(regressionEnv.stalePorts);

0 commit comments

Comments
 (0)