Skip to content

Add scheduled and targeted refresh for BReg external projections - #2650

Merged
jeremi merged 4 commits into
mainfrom
feat/breg-external-refresh
Nov 11, 2026
Merged

jeremi merged 4 commits into
mainfrom
feat/breg-external-refresh

Conversation

@jeremi

@jeremi jeremi commented Nov 11, 2026

Copy link
Copy Markdown
Member

Summary

Add deployment-owned externalRefresh plans for indexed BReg links. Operators can reuse a scoped service worker, schedule reconciliation with bregctl external-projection refresh, and admit signed source-change hints through a separate notifier. Local Record changes enqueue work in the same transaction. PostgreSQL owns the lease, checkpoint, retries, and replay state.

Targeted work resolves current authorised links before contacting the source. The existing projection grant, per-key publication fence, and expiry rules still govern the retained results. Generated runtime schema, OpenAPI and CLI reference describe the configuration and driver.

Evidence

  • cargo test --locked -p registry-breg --features postgres-test,tooling,schema --lib external_: 94 passed on the integrated branch against real PostgreSQL, including scoped acquisition, retry and expiry, queue fencing, label membership, and schedule edits.
  • External-view and runtime-config integration tests: 9 and 104 passed. CLI refresh and CLI catalog snapshot tests: 2 and 3 passed. The exact integrated binaries passed the TLS managed-refresh smoke journey.
  • The signed webhook receiver suite passed all 12 tests. The Record, Action and Proposal PostgreSQL regression suites passed all 102 tests, and Clippy passed for both BReg crates.
  • run_candidate.py --scale --managed-refresh: the paired scale candidate covered 1,000,000 eligible and 100,000 excluded roots. Cold indexed readiness took 491 seconds, followed by repeats at 401 and 406 seconds. A source hint fetched one selector and updated all 20 linked roots. The final branch separately verified the subsequent schedule registration changes and latest-main integration.
  • Actual pg_dump / pg_restore preserved ordinary Records and restored all seven excluded cache and queue tables empty. The 100,000-receipt SQL probe refused new admission while allowing exact replay and settlement.

Hosted CI will run on this PR.

Notes

Existing indexed deployments need a successor activation to install the queue and managed lookup indexes. After the queue is provisioned, removing the final indexed projection declaration requires a separate retirement migration. Removing the source binding and plans stops acquisition without discarding queue state during activation.

DCO

  • Every commit includes a Signed-off-by trailer.
  • I reviewed the submitted changes and am responsible for the contribution.

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-11T18:08:04.478339Z bbbf0db New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 36d2301e19

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
@jeremi
jeremi added this pull request to the merge queue Nov 11, 2026
Merged via the queue into main with commit 9cc762b Nov 11, 2026
63 checks passed
@jeremi
jeremi deleted the feat/breg-external-refresh branch November 11, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants