Bound and exhaust Content document discovery

[3mdistal]

Purpose

Content is shared by two kinds of callers: people navigating the workspace and agents trying to find, reason about, or safely create documents. Both need the same basic guarantee: after discovery finishes, “I found nothing” must mean that no authorized match exists—not merely that the caller received a clipped first page.

That guarantee matters most for create-if-absent workflows. An agent commonly searches for a document or database before deciding whether to create it. If an incomplete search looks complete, the agent can falsely conclude that the object does not exist, create a duplicate, omit existing material from an analysis, or report that a projection is complete when records were silently missed.

Problem

Before this change, Content had two opposite discovery failures:

• The workspace document list read every authorized document in one unbounded query. It could be complete, but its cost grew with the workspace.
• Document search stopped at its requested limit without saying whether more matches existed. A successful response containing zero or fifty results looked final even when it was only a prefix.

Callers therefore had no common way to ask for a finite amount of work and then prove that they had reached the end. Worse, the responses looked ordinary: no timeout, error, or “partial” marker warned an agent that an absence claim was unsafe. One path in the UI could also turn an unexpected response shape into an empty array, making a contract failure resemble a genuinely empty workspace.

Before and after

Before

• Listing attempted to return the whole visible corpus in one call.
• Search returned at most its limit, with no total or continuation.
• “No result” and “not all results were examined” were indistinguishable.
• The UI tolerated malformed or legacy response shapes by producing a plausible empty list.

After

• Listing and search each return one bounded, deterministic, access-scoped page.
• Every page reports offset, limit, totalItems, returnedItems, hasMore, and nextOffset.
• Callers follow nextOffset until hasMore is false; only then is absence or completeness established.
• The Content UI exhausts all pages and rejects missing pagination, total drift, duplicate IDs, wrong boundaries, early exhaustion, and non-advancing continuations instead of treating them as success.
• Exact lookup can be scoped by title, parent, Content space, and document type, so create-if-absent callers can distinguish zero, one, and multiple authorized matches.

How the contract works

Access and Content visibility filters are applied before both the count and the page are constructed. This is important for privacy: an unauthorized caller receives neither private rows nor a total that reveals their existence.

Pages have a maximum size of 200 and stable ordering with an ID tie-breaker. The workspace UI requests pages of 200, accumulates them, and verifies that the declared total stays fixed for the read. If documents change during traversal, it fails and asks for a retry rather than presenting a mixed snapshot as complete.

The React Query cache remains object-shaped—{ documents, pagination }—because optimistic create/update code writes that contract. UI consumers receive only the selected, fully exhausted Document[]. This prevents an optimistic create from briefly handing an object to components that iterate an array.

Exact 124-Capability proof

The database proof deliberately uses a complete, non-trivial corpus rather than a three-row pagination toy:

1. It loads the 124 unique Capability IDs from the merged Content product graph.
2. It performs an exact, private, root-level lookup and proves the projection is initially absent: zero rows, total zero, and no continuation.
3. It creates one private Content database and upserts all 124 Capability records by stable Capability ID. Every first write reports created, and the test records each item/document identity.
4. It changes one Capability and restores it. Both writes report updated while preserving the original identities.
5. It replays all 124 original records. Every write reports unchanged with the same identities, proving stable-key idempotency rather than duplicate creation.
6. It reads the database back in pages of 37 at offsets 0, 37, 74, and 111. Every page reports a total of 124; no ID repeats; the exhausted ID set exactly equals all 124 source IDs; and every read-back identity matches the first upsert receipt.
7. It proves the owner’s exact discovery returns one match, while an unrelated user sees a total of zero and cannot read the private database.

This is an isolated local test projection. It does not write Alice’s Content workspace or vault.

What changed

• Added a shared discovery boundary for access-first filters, deterministic paging, totals, and continuation metadata.
• Bounded list-documents and excluded trashed or soft-deleted database artifacts before counting and paging.
• Added paginated and exact scoped lookup to search-documents, including all Content organizations the caller belongs to.
• Made the workspace UI exhaust the bounded listing contract and fail loudly on ambiguous completeness.
• Preserved the object-shaped optimistic cache contract while selecting the final array for consumers.
• Added private-corpus, access-boundary, stable-key replay, complete-readback, and optimistic cache-shape regressions.

Compatibility, safety, and limitations

• This changes the direct list-documents Action contract: omitting pagination now returns the first 50 records, not the whole corpus. The shipped Content UI follows every page. Other direct Action callers must do the same.
• Search remains bounded SQL title/content matching. This PR does not claim indexed search, ranking, freshness semantics, or completion of the broader Search roadmap capability.
• There are no schema changes, migrations, credentials, production writes, backfills, deletion jobs, background workers, or feature flags.
• Rollback restores the previous unbounded-list and silently limited-search behavior. It cannot strand new stored state.
• This PR does not implement an importer or live private-roadmap synchronization. It has no dependency on PR Add atomic Content database row migrations #2573 and does not include Slack service-principal capture.
• Documentation, agent guidance, and the user-facing changelog remain intentionally separated into a later PR.
• Deployment-backed private real-interface acceptance has not run and remains required at the later terminal lifecycle gate. This draft does not claim that acceptance or authorize merge/deployment.

Product roadmap impact

content_product_impact:
  lane: contract_fulfillment
  features:
    - content.feature.durable-foundations
  capabilities:
    - content.knowledge.search
    - content.agent.action-parity
  record_change: none
  proof:
    - 109 focused Content discovery, database, lifecycle, and action-registration tests
    - Content typecheck and all 42 repository guards
    - private 124-Capability stable-key projection with paginated readback
  rationale: Hardens the shared UI and agent discovery Action contract without claiming complete indexed Search or cross-surface parity verification.

No roadmap record changes state in this PR.

Verification

Evidence is bound to head f712a7b6c3b68d6e34a3e46523b0f7232c49f442:

• 109 focused tests pass across bounded discovery, private access, stable-key replay, database lifecycle behavior, Action registration, and the optimistic cache-shape regression.
• The 124-Capability projection proves complete readback and stable identities as described above.
• Content typecheck passes.
• Content product-impact tests pass 30/30.
• All 42 repository guards pass.
• Oxfmt and git diff --check pass for the implementation diff.
• GitHub’s required matrix is green, including Content DB tests, Content parity, privacy leakage evaluations, build, typecheck, lint/format, security guards, static template checks, and the trusted acceptance substrate.
• Deployment-backed private real-interface acceptance was not run; it is a stated remaining gate, not inferred from green CI.

Review focus

• Does the access-first count/page construction preserve the intended personal and multi-organization privacy boundary without leaking private totals?
• Are the ordering and continuation invariants strong enough to make “fully exhausted” trustworthy, especially when data changes during traversal?
• Is the default first page of 50 an acceptable compatibility change for direct Action callers, given that the shipped UI now exhausts pages?
• Does the object-shaped cache plus array selection preserve every optimistic writer and prevent transient object-to-array consumer failures?
• Are the limitations stated narrowly enough that this PR cannot be mistaken for complete indexed Search or deployed private-roadmap synchronization?

[github-actions]

Visual recap — generation failed

The visual recap could not be generated for this pull request. This is informational only and does not block the PR.

Diagnostic:

No plan URL: Repair changed too much of targeted file plan.mdx; expected a localized parser fix.

Agent output: Repaired recap-source.json: structural \n escapes now form real MDX paragraph breaks; code payload escapes remain intact. --- ⠀ 🟢 Corrected recap source is ready for CI to republish. stderr: Reading additional input from stdin...

[builder-io-integration]

Builder reviewed your changes — looks good ✅

Review Details

Code Review Summary

PR #2659 bounds Content document discovery with deterministic, access-scoped pagination, explicit totals/continuations, exact-title filters, and a shared predicate used consistently by count and page queries. The UI now exhausts list pages before exposing the document tree and rejects missing, inconsistent, duplicated, or non-advancing pagination metadata rather than treating a clipped response as complete. The optimistic cache contract remains object-shaped internally while consumers continue to receive a document array.

Risk: Standard — this changes shared discovery/business logic and privacy-sensitive filtering, though it does not alter authentication or public API security boundaries.

No confirmed high- or medium-severity issues were found. The implementation has strong coverage for multi-page list/search results, exact-match cardinality, outsider isolation, soft-deleted database filtering, stable-key projection replay, and optimistic cache compatibility. Code-review agents also verified the relevant Content tests pass.

🧪 Browser testing: Skipped — dev server was available, but browser automation tooling was unavailable to all executors; backend endpoints were independently verified, while UI flows could not be exercised.