Repository navigation
fix: bound excerpt recovery on long paragraphs - #422
Merged
Merged
Conversation
what changed Prose candidates retain their physical paragraph start during the existing scan. Excerpt recovery runs forward from that start and stops once the existing 420-character cap is exceeded, eliminating repeated unshift calls and whole-block excerpt construction. Finding selection, blank/heading boundaries, Markdown filtering, table excerpts and truncation spelling remain the same. why Fixes #334. Repeated front insertion made a vendor-controlled unbroken paragraph quadratic. This is an internal performance defect with one direct remedy that preserves the existing output contract; no new command, flag, format, provider or check is introduced. what it touched research-kit/lib/finding.mjs; two finding regressions covering an 8 MB paragraph and exact boundary/cap output; the finding architecture row; CHANGELOG.md; the registered offline test count in research-kit/README.md (1847). what you verified verified: the unchanged-code baseline remained inside the 8 MB regression beyond its 8-second performance bound and was terminated after 303.7s. No final suite count was produced for that deliberately bounded baseline run. verified: node research-kit/bin/selftest.mjs finding passed all 32 tests, 0 failed, in 2.5s after the fix, including the same 8 MB input and pinned finding/excerpt. verified: root handoff OK (33 ledger entries, 27 evidence rows), preflight PASS (28 passing, 1 existing unstamped-brief warning), and git diff --check. Cwd: C:\Users\PC\Desktop\gittoken\Research-Kit-issue-334. verified: the shared machine was observed as doctor READY from the issue-326 checkout before implementation, collector role and firecrawl-cli transport on Node 24.20.0. expected: the normal pre-commit gate will run the full staged-tree offline suite and enforce the architecture map and README count before this commit can be recorded; its exit and CI results will accompany the PR. untested: no live collection or external pages were fetched; all extractor inputs are synthetic strings. what you got wrong and fixed nothing to report
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Finding extraction now remembers the physical paragraph start and builds the excerpt forward only until the existing cap is exceeded. Long unbroken pages avoid quadratic front insertion while preserving finding selection and excerpt output.
Fixes #334.
what changed: Prose candidates retain their paragraph boundary during the existing scan. Excerpt recovery runs forward and stops once truncation is established; table excerpts and physical blank/heading boundaries retain their behavior.
why: Repeated array
unshiftmade the recovery cost quadratic on accepted captures.what it touched:
finding.mjs, two regressions, its architecture row, changelog and registered offline test count (1847).what you verified:
what you got wrong and fixed: nothing to report.