Skip to content

feat: send read-freshness signal on the lance-namespace path#3551

Merged
brendanclement merged 1 commit into
mainfrom
feat/namespace-read-freshness
Jun 17, 2026
Merged

feat: send read-freshness signal on the lance-namespace path#3551
brendanclement merged 1 commit into
mainfrom
feat/namespace-read-freshness

Conversation

@brendanclement

Copy link
Copy Markdown
Contributor

Description

db://-style connections that use the lance-namespace path (LanceNamespaceDatabaseNativeTable + the lance-namespace REST client) never sent a read-freshness signal. Against a server configured to serve cached table metadata up to some staleness window, this allows stale-read-after-write across handles and processes. The remote table path already solved this (#3439). This brings the namespace path to parity.

The namespace REST client doesn't let callers attach headers directly, but it forwards a DynamicContextProvider's headers.* context entries as HTTP headers per request. So:

  • A shared per-table baseline map is created before the namespace client. I built and installed on the ConnectBuilder via a context provider.
  • On read operations the provider emits ·x-lancedb-min-timestamp = max(baseline, now − read_consistency_interval)(RFC3339), keyed by the operation'sobject_id`.
  • Each table handle bumps its baseline (monotonically) on checkout_latest(), restore(), and every data/schema write. checkout_latest() is the primary hook: consumers refresh a handle there after writing elsewhere, then poll.

Read operations that carry the floor: describe_table, list_table_versions, query_table, list_tables. list_table_versions is what resolves "latest" for managed-versioning tables (get_latest_version), so it's the op that makes checkout_latest() actually observe a prior write. describe_table_version is excluded (pinned to an immutable version). This mirrors #3439 (timestamp baseline, max(baseline, now − interval), monotonic); no min_version and no body channel, since the namespace path has no version-returning write responses.

Testing

  • Unit tests for compute_min_timestamp / next_freshness_baseline and the provider (header at/after a bumped baseline; nothing for an empty baseline + no interval; interval floor applies; non-read ops emit nothing; list_tables uses only the interval floor).
  • Verified end-to-end against a local server that honors the header: reads carry x-lancedb-min-timestamp, writes don't, and read-your-write holds.

@brendanclement brendanclement marked this pull request as ready for review June 17, 2026 16:54

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@github-actions github-actions Bot added enhancement New feature or request Rust Rust related issues labels Jun 17, 2026
@brendanclement brendanclement merged commit 0abf641 into main Jun 17, 2026
29 checks passed
@brendanclement brendanclement deleted the feat/namespace-read-freshness branch June 17, 2026 17:30
@mintlify

mintlify Bot commented Jun 17, 2026

Copy link
Copy Markdown

Docs PR opened: lancedb/docs#276

Clarified that read_consistency_interval applies to all db:// remote tables and updated the read-your-write description to match current client behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Rust Rust related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants