Fix single-source chat missing full document content by mike-foucault · Pull Request #1209 · lfnovo/open-notebook

mike-foucault · 2026-07-23T09:05:33Z

Problem

Chatting/asking questions about an individual source (not a notebook) silently failed to use the document's actual content when no transformation/insight had been generated for it — the LLM only ever saw the source's title, with no error or warning. Users would get generic "I have no document to analyze" style answers without understanding why.

Root cause

build_source_context() (in open_notebook/utils/context_builder.py, used by the source-chat graph) requested context_size="short" from Source.get_context(). That mode never includes full_text — only id, title, insights. Without insights (i.e. no transformation run on the source), the context was reduced to just the title.

Switching to context_size="long" alone was not enough: it exposed a second, previously latent bug. The function's token-budget truncation loop dropped the entire source whenever its full_text alone exceeded max_tokens (very common for real documents, since the budget is 50k tokens) — leaving an empty context for exactly the documents most likely to have real content.

Fix

Use context_size="long" in build_source_context so full_text is always included, regardless of whether the source has insights.
Fix the truncation loop to only drop insights (last-fetched first) for budget — the source itself is never dropped. The prompt formatter (_format_source_context in open_notebook/graphs/source_chat.py) already caps full_text at a fixed size before it reaches the LLM, so this doesn't risk oversized prompts.

Tests

Added/updated tests in tests/test_utils.py:

test_source_and_insights_shape now asserts context_size="long" is requested.
New test_keeps_source_when_it_alone_exceeds_budget: a source whose full_text alone exceeds max_tokens is still returned instead of being dropped.
Existing test_truncates_insights_to_token_budget and test_missing_source_yields_empty_context still pass unchanged in behavior.

tests/test_utils.py::TestBuildSourceContext::test_source_and_insights_shape PASSED
tests/test_utils.py::TestBuildSourceContext::test_truncates_insights_to_token_budget PASSED
tests/test_utils.py::TestBuildSourceContext::test_keeps_source_when_it_alone_exceeds_budget PASSED
tests/test_utils.py::TestBuildSourceContext::test_missing_source_yields_empty_context PASSED
4 passed

Also manually verified end-to-end locally: uploaded a document with no transformation, opened its chat, and confirmed the LLM's response now reflects the document's actual content instead of the "no document" fallback.

This is a small, self-contained bug fix (per CONTRIBUTING.md — "small, obvious bug fixes" are exempt from requiring a prior Discussion/Issue).

build_source_context() (used by per-source chat/ask) requested 'short' context from Source.get_context(), which never includes full_text. A source without a transformation/insight left the LLM with only the title, making single-document chat silently useless. - Use context_size="long" so full_text is always included. - Fix the token-budget truncation loop to never drop the source itself when it alone exceeds max_tokens (previously it could, leaving an empty context for large documents). Only insights are trimmed; the prompt formatter already caps full_text length.

Copilot

Pull request overview

Fixes source-level chat context so the LLM receives the actual document content even when no transformations/insights exist, and prevents the source from being dropped when token truncation is applied.

Changes:

Switch build_source_context() to request context_size="long" so full_text is included.
Adjust truncation logic to drop insights to meet a token budget while keeping the source.
Update/add tests to validate long-context usage and the “oversized source” behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
`open_notebook/utils/context_builder.py`	Uses long source context and changes token-budget truncation behavior for source-chat context building.
`tests/test_utils.py`	Updates characterization tests and adds coverage for oversized-source behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

        total_tokens = sum(item_tokens)
        if max_tokens:
-            while total_tokens > max_tokens and item_tokens:
+            while total_tokens > max_tokens and insights:
                total_tokens -= item_tokens.pop()
-                if insights:
-                    insights.pop()
-                else:
-                    sources.pop()
+                insights.pop()


+    @pytest.mark.asyncio
+    async def test_keeps_source_when_it_alone_exceeds_budget(self):
+        """A source whose full_text alone exceeds max_tokens is still
+        returned (not dropped) — an empty context is worse than one over
+        budget, and the prompt formatter caps full_text length downstream."""
+        source = SimpleNamespace(id="source:123")
+        big_text = "word " * 20000  # far bigger than the token budget below
+        source.get_context = AsyncMock(
+            return_value={"id": "source:123", "title": "T", "full_text": big_text}
+        )
+        source.get_insights = AsyncMock(return_value=[])
+
+        with patch(
+            "open_notebook.utils.context_builder.Source.get",
+            new=AsyncMock(return_value=source),
+        ):
+            result = await build_source_context("source:123", max_tokens=600)
+
+        assert len(result["sources"]) == 1
+        assert result["sources"][0]["full_text"] == big_text
+        assert result["total_tokens"] > 600
+


cubic-dev-ai

No issues found across 2 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You’re at about 99% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.}

_{Re-trigger cubic}

Addresses PR lfnovo#1209 review comment: dropping insights when the source alone already exceeds max_tokens can't bring the total under budget, so it was discarding context for no benefit. Also switch the max_tokens check to 'is not None' so max_tokens=0 isn't silently treated the same as no limit.

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

        if source:
-            source_context = await source.get_context(context_size="short")
+            source_context = await source.get_context(context_size="long")
            sources.append(source_context)
            item_tokens.append(token_count(str(source_context)))


+        # Long context is used so full_text is always included — a source
+        # with no insights would otherwise leave the LLM with only the title.
+        source.get_context.assert_awaited_once_with(context_size="long")


Source.get_context() fetches and embeds insights itself when not given a list, so build_source_context() (which fetches and represents insights separately) was hitting the DB twice and double-counting insight tokens toward max_tokens. Pass insights=[] so get_context() leaves insight representation to the caller.

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

+            source_context = await source.get_context(
+                context_size="long", insights=[]
+            )
            sources.append(source_context)
            item_tokens.append(token_count(str(source_context)))


+    Used by the source-chat graph. Uses the "long" context so the source's
+    full text is always included — a source with no insights/transformations
+    would otherwise leave the LLM with only the title. If `max_tokens` is
+    given, insights are dropped (last-fetched first) until the total fits.
+    The source itself is never dropped — a large full_text can alone exceed
+    the budget, and dropping it would leave an empty context; the caller
+    (source-chat's prompt formatter) already caps full_text at a fixed size
+    before it reaches the LLM.


Copilot AI review requested due to automatic review settings July 23, 2026 09:05

Copilot started reviewing on behalf of mike-foucault July 23, 2026 09:06 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

cubic-dev-ai Bot reviewed Jul 23, 2026

View reviewed changes

mike-foucault mentioned this pull request Jul 23, 2026

[Bug]: Document Search Only Accesses Document Title Without Default Transformation #1207

Closed

1 task

mike-foucault requested a review from Copilot July 23, 2026 09:28

Copilot started reviewing on behalf of mike-foucault July 23, 2026 09:29 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

mike-foucault requested a review from Copilot July 23, 2026 09:57

Copilot started reviewing on behalf of mike-foucault July 23, 2026 09:58 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Navigation Menu

Fix single-source chat missing full document content - #1209

Fix single-source chat missing full document content#1209
mike-foucault wants to merge 3 commits into
lfnovo:mainfrom
mike-foucault:fix/source-chat-missing-full-text

mike-foucault commented Jul 23, 2026 •

edited by cubic-dev-ai Bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mike-foucault commented Jul 23, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Root cause

Fix

Tests

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mike-foucault commented Jul 23, 2026 •

edited by cubic-dev-ai Bot

Loading