Repository navigation
fix: distinguish unreadable ledgers from missing evidence - #421
Merged
Merged
Conversation
what changed readLedger tolerates only an editor's leading byte-order mark outside hashed entries and never rewrites the file. Handoff distinguishes unreadable ledger entries from proven missing files, names their physical lines, and chooses the existing tail-repair or restore-from-git remedy. Secondary chain and unledgered-capture findings cannot establish travel failure or entry loss while parsing is unresolved; independent missing captures still receive the travel remedy. Tail repair recognizes an intact one-entry BOM ledger and preserves its original prefix when dropping a later torn append. why Fixes #296, including the torn-tail case in its issue comment. The bytes had arrived but could not be parsed, so collector-push advice sent the same unreadable file again. Entry hashes cover canonical entries, not an editor's leading marker. The existing reader and sanctioned tail repair must agree about that marker to avoid dropping valid evidence. These are corrections to existing reading and remedy behavior; no new check, flag or file format is introduced. what it touched research-kit/lib/corpus.mjs, handoff.mjs and provenance.mjs; five regressions across corpus, handoff and provenance tests; those three architecture rows; CHANGELOG.md; the registered offline test count in research-kit/README.md (1850). what you verified verified: the initial focused run reproduced all four new reader/remedy regressions (91 passed, 4 failed, 1 unsupported). verified: after correcting the fixture's verification layer, handoff/provenance/corpus passed 95 tests, 0 failed, with 1 unsupported symlink case (61.2s, LOCAL RESEARCH_KIT_ALLOW_UNSUP=1). verified: before the tail fix, a disposable one-entry BOM ledger was wrongly dropped from 546 bytes to zero. After the fix, the same probe returns tail-parses and preserves the file byte-for-byte; a later torn append is dropped while preserving the prefix and a valid chain. verified: final handoff/provenance/corpus group including the tail-repair regression passed 96 tests, 0 failed, with 1 unsupported symlink case in 45.7s (LOCAL RESEARCH_KIT_ALLOW_UNSUP=1). verified: root handoff OK (33 ledger entries, 27 evidence rows), preflight PASS (28 passing, 1 existing unstamped-brief warning), and git diff --check. Cwd: C:\Users\PC\Desktop\gittoken\Research-Kit-issue-296. verified: the shared machine was observed as doctor READY from the issue-326 checkout before implementation, collector role and firecrawl-cli transport on Node 24.20.0. expected: the normal pre-commit gate will run the full staged-tree offline suite and enforce the architecture map and README count before this commit can be recorded; its exit and CI results will accompany the PR. untested: local file symlink creation is unavailable under this Windows account; CI must verify that capability. No live collection was performed. what you got wrong and fixed mistake: the first BOM acceptance test invoked the commit gate against an unstaged disposable fixture. where: research-kit/test/handoff.test.mjs, new BOM acceptance assertion. impact: one false regression failure after the reader fix; no delivered code or real corpus was affected. cause: selected the commit verification layer instead of the fixture's provenance preflight. fix: assert runPreflight(dir).pass for the same fixture. verified: the corrected focused run passed 95 tests, 0 failed, with one unsupported symlink case. mistake: the first implementation did not align the independent tail parser with the BOM-tolerant reader. where: research-kit/lib/provenance.mjs, repairLedgerTail. impact: the existing tail-repair path removed a valid first entry from a disposable 546-byte fixture; found before publication, with no real corpus changes. cause: the separate bare JSON.parse(tail) check had not been included in the initial reader review. fix: tolerate the marker only when the tail is the file's first line, leave an intact ledger untouched, and preserve its prefix when dropping a later torn append. verified: the direct final probe preserves the intact file, removes only the torn append, and verifies the chain; the added regression pins both cases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A ledger with a leading editor byte-order mark now reads and verifies without rewriting the file. Handoff gives unreadable ledger lines the appropriate local tail-repair or restore-from-git remedy, while separately missing captures still receive the travel remedy.
Fixes #296, including the torn-tail example in its issue comment.
what changed: The reader tolerates only the leading marker outside hashed entries. Handoff retains parse failures and secondary chain findings as failures, but does not label unresolved parsing as proven entry loss or failed travel. Tail repair leaves an intact one-entry BOM ledger alone and preserves the marker when dropping a later torn append.
why: Pushing the same unreadable bytes from the collector cannot repair an editor artifact or interrupted append.
what it touched:
corpus.mjs,handoff.mjs,provenance.mjs, five regressions, their architecture rows, changelog and registered count.what you verified:
what you got wrong and fixed:
Mistake: the first BOM acceptance assertion used the commit gate on an unstaged disposable fixture. Where: the new handoff test. Impact: one false test failure, no published code or real corpus affected. Cause: wrong verification layer. Fix: assert the fixture's provenance preflight. Verified: the corrected focused run passed.
Mistake: the initial reader fix did not align the separate tail parser. Where:
repairLedgerTail. Impact: the existing path dropped a valid 546-byte disposable BOM ledger; found before publication, no real evidence affected. Cause: its bare tail JSON parser was missed in the first review. Fix: recognize a leading marker only on the first physical line and preserve it during later tail repair. Verified: the final probe and focused regression preserve the valid entry and chain.