Skip to content

Feat: negotiate the sessions database schema version and add an in-place upgrade step - #836

Open
AmaadMartin wants to merge 3 commits into
mainfrom
feat/session-db-schema-version-negotiation
Open

Feat: negotiate the sessions database schema version and add an in-place upgrade step#836
AmaadMartin wants to merge 3 commits into
mainfrom
feat/session-db-schema-version-negotiation

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):

N/A

  1. Or, if no issue exists, describe the change:

Problem: DatabaseSessionService accepts exactly one stored schema version, '1'. Bumping that constant would make init() throw for every existing deployment, and nothing can re-stamp the row. The failure comes from init(), so an operator cannot even build a working service to repair the database from.

Solution: core/src/sessions/db/schema_version.ts now owns a set of accepted versions, the version this build stamps, and upgradeSessionDatabaseSchema(), which applies the additive DDL and re-stamps a database in place. The upgrade step opens its own connection, so it does not depend on init() succeeding. Behaviour is unchanged for every database today, because the set has one member and it is the latest.

Collision check: gh pr list over all 300 open fork PRs found no PR implementing schema-version negotiation. #740 (events cascade foreign key) and #797 (MikroORM v7) also edit operations.ts and schema.ts, but only ensureDatabaseCreated, StorageSession and StorageEvent. Both target main, so this branches from main too.

Design notes:

  • One assertCompatibleVersion() membership check guards both entry points. An earlier revision modelled four states in an enum with a parameterized classifier; the complexity review showed the supported-but-older state cannot occur while the accepted set has one member, so that state machine was removed. The next bump adds the constant, adds it to the set, repoints LATEST_SCHEMA_VERSION, and adds one logger.warn line, which is step 6 of the README runbook.
  • No second entity set. adk-python selects between schemas/v0.py and schemas/v1.py at runtime; MikroORM freezes entity metadata at init(), so the equivalent needs a pre-init probe connection and an indirection object over every query. core/src/sessions/db/README.md records the decision, its cost, and the additive-bump alternative that replaces it.
  • The upgrade step mutates in place and takes one URI. adk-python's upgrade() refuses in-place migration because it rewrites pickled rows into JSON; this package has no pickle legacy.
  • The error prefix ADK Database schema version <v> is not compatible. is preserved verbatim, so the two pre-existing tests that assert it pass unedited.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.
npx vitest run --project unit:core core/test/sessions/db/schema_version_test.ts \
  core/test/sessions/db/operations_test.ts \
  core/test/sessions/database_session_service_test.ts     # 54 passed
npm run ts:check && npm run lint && npm run format:check && npm run docs:check && npm run build

core/test/sessions/db/schema_version_test.ts reaches 100% statement, branch, function and line coverage of schema_version.ts (v8), with the finally kept. The three validateDatabaseSchemaVersion cases moved verbatim from operations_test.ts; no assertion changed.

Proof the tests can fail. Each mutation was applied to schema_version.ts, the tests were run, then the mutation was reverted.

Mutation Test that failed Message
assertCompatibleVersion never throws 5 tests, including the pre-existing, unedited DatabaseSessionService > should fail with incompatible schema version promise resolved "undefined" instead of rejecting
validateDatabaseSchemaVersion stamps a present row instead of a missing one should initialize schema version if missing, keeps a single version row when called twice expected +0 to be 1
upgradeSessionDatabaseSchema never reaches the stamp 3 upgradeSessionDatabaseSchema cases expected [] to have a length of 1 but got +0
Always em.create in stampSchemaVersion replaces the stored value instead of adding a row expected '1' to be '2'
Drop finally { await orm.close(); } closes the connection when the stored version is rejected expected "close" to be called 1 times, but got 0 times
Reword the is not compatible prefix 4 tests, including the same pre-existing one expected [Function] to throw error including 'ADK Database schema version 999 is no…'

Every remaining test pins a reachable behaviour: each of the six mutations kills at least one.

Manual End-to-End (E2E) Tests:

Run against the built package (npm run build) with a real SQLite file, reading the metadata row with the sqlite3 client rather than the ORM:

import {
  DatabaseSessionService,
  upgradeSessionDatabaseSchema,
} from '@google/adk';

const uri = `sqlite://${dbPath}`;
const service = new DatabaseSessionService(uri);
await service.createSession({appName: 'app', userId: 'u'}); // stamps '1'
await upgradeSessionDatabaseSchema(uri); // no-op
// then: UPDATE adk_internal_metadata SET value = '999', and retry both

Transcript:

1. stamped: [ { key: 'schema_version', value: '1' } ]
2. after upgrade: [ { key: 'schema_version', value: '1' } ]
3. sessions readable: 1
4. init rejected: ADK Database schema version 999 is not compatible. This build of ADK
   supports schema version(s) 1. Upgrade the @google/adk package if the database was
   written by a newer release, or call upgradeSessionDatabaseSchema() to bring an older
   database up to version 1.
5. upgrade rejected: <same message>
5. row untouched: [ { key: 'schema_version', value: '999' } ]
6. restamped: [ { key: 'schema_version', value: '1' } ]

Step 5 is the important one: the upgrade step refuses an unknown version instead of stamping over it.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.

CI note: green on 3d291af3run-tests on Linux, macOS and Windows, plus cross-language and the license check. The branch is rebased onto current main, which now carries #602; operations.ts keeps its redactUriPassword call and this PR only trims the imports the moved function used.

Two unrelated flakes needed a re-run on an earlier head: an app_loader integration timeout on macOS and a prebuild-install network timeout for sqlite3. Neither touches this diff.

@AmaadMartin AmaadMartin closed this Aug 9, 2026
@AmaadMartin AmaadMartin reopened this Aug 9, 2026
@AmaadMartin AmaadMartin closed this Aug 9, 2026
@AmaadMartin AmaadMartin reopened this Aug 9, 2026
Amaad Martin added 3 commits August 8, 2026 20:21
DatabaseSessionService pinned the stored version to exactly '1', so a future
bump would throw at init() for every existing deployment with no way to
recover. schema_version.ts now owns an accepted-version set, the version this
build stamps, and upgradeSessionDatabaseSchema() to move a database up in
place.
Records the steps for the next version bump, the backward-compatibility
window carried over from adk-python, and why this package keeps one MikroORM
entity set instead of one per version.
…uard

SUPPORTED_SCHEMA_VERSIONS has one member and it is LATEST_SCHEMA_VERSION, so
the SUPPORTED state could never occur and both callers passed the same two
constants. A membership check replaces the enum and the classifier. Behaviour
is unchanged on every reachable input.
@AmaadMartin
AmaadMartin force-pushed the feat/session-db-schema-version-negotiation branch from e280399 to 3d291af Compare August 9, 2026 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant