Skip to content

test: exclude word sets a parse meta cannot represent - #145

Merged
thedavidmeister merged 1 commit into
mainfrom
2026-10-02-parse-meta-fingerprint-fuzz-bound
Oct 2, 2026
Merged

thedavidmeister merged 1 commit into
mainfrom
2026-10-02-parse-meta-fingerprint-fuzz-bound

Conversation

@thedavidmeister

@thedavidmeister thedavidmeister commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

main's test job fails on some fuzz seeds with DuplicateFingerprint() in testRoundMetaExpanderDeeper, which blocks autopublish.

It is a test bound, not a defect, and that was proven rather than assumed. The CI counterexample (run 36937526403) is 114 words. A brute-force scan of all 256 seeds against all 6441 pairs found exactly one colliding pair — indices 30 and 66, agreeing on expansion bit 199 and fingerprint 0x9540FF under seed 59, and on nothing under seed 0. The collision is legitimate: wordBitmapped takes the bitmap from the high byte of keccak(word‖seed) and the fingerprint from the low three, so 32 independent bits must agree, which FINGERPRINT_MASK's own comment rates at solidity selector resistance. Two such words cannot both live in a 4 byte meta item, and DuplicateFingerprint exists to say so at codegen time. A real word set that hits it is fixed by renaming a word.

LibParseMetaFingerprint.findsDupes scans every seed in [0, 255] and chains words on the low fingerprint byte, comparing the full fingerprint and the full bitmap, so a false result guarantees buildParseMetaV2 cannot revert with DuplicateFingerprint. Scanning every seed is the cheap direction: predicting the seeds findBestExpander picks costs a full expander chain. The six fuzz tests that build a parse meta from a fuzzed word set now assume on it, after the existing bloom.

Cost, measured rather than estimated: the two heavy parse-meta suites take 7m38 with the guard. An assembly version of the guard was drafted and is not shipped — it has never compiled, and trusting it would need differential fuzz tests holding its inlined hashing to wordBitmapped. The gas is Solidity per-iteration overhead, not hashing: the guard already hashes each word once per seed (29k keccaks at n=114), not once per pair.

Two findings filed separately rather than folded in: #143 (buildParseMetaV2 can write one item past the items region) and #144 (checkNoOOBPointers fuzz tests get TruncatedSource on some seeds — also red on main, unrelated to this suite).

QA

  • Discriminating tests: 15 new, all passing. testCiCounterexampleBuildReverts pins that the 114-word set does revert; testCiCounterexampleIsGuarded that the guard flags it; testCiCounterexampleBuildsWithoutCollidingWord that dropping index 66 leaves 113 words that pass the guard, build and round-trip — the discriminator separating "guard works" from "guard rejects everything". testFindsDupesCollisionOnlyAtNonZeroSeed pins that the pair is clean under seed 0, so a seed-0-only scan is not enough.
  • Mutations applied: 9 authored, 6 applied (3 skipped for non-unique patterns, reported not killed). Killed: scan seed 0 only (3 tests), ignore fingerprint (2), no bucket clear (3), and in production revert DuplicateFingerprint() → assignment (testCiCounterexampleBuildReverts). Survived: ignore the bitmap compare — equivalent, it only over-rejects and the guard's contract is one-directional. Compare the chain head only initially survived, which was a real gap: testFindsDupesCollisionBuriedInSet buries the pair behind sequential fillers, so whether any filler shares the colliding bucket is a property of their hashes rather than of the test. Added testFindsDupesCollisionBehindAChainHead, which constructs a filler that does share the bucket; it kills that mutant.
  • Oracle: the counterexample calldata from the CI run, decoded independently of the fix, and wordBitmapped's own documented split of the hash into bitmap and fingerprint bytes. The guard is exact, not bloomed, so its false is a guarantee rather than a probability.
  • Category check: asks whether the collision is a legitimate input the test should exclude or a hashing defect; answered with the brute-force scan and the bit accounting — legitimate, excluded. Swept all 7 fuzz tests that build a word set: 6 bounded, findExpander.t.sol left alone because findBestExpander has no DuplicateFingerprint path. No assertion weakened, no production code changed, no try/catch around the build.

forge fmt --check, forge lint -D warnings, slither (61 contracts, 0 results) and reuse lint all clean. 141 of 143 tests pass; the 2 failures are #144 on main.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes

    • Parse metadata generation now detects distinct words that would be indistinguishable due to fingerprint collisions and rejects those sets, preventing ambiguous metadata.
  • Tests

    • Added coverage for collision detection across varied word sets, including collisions that appear only with particular seeds and collisions within larger sets.
    • Added checks that valid sets still build and round-trip correctly.

`buildParseMetaV2` reverts `DuplicateFingerprint` when two words share an
expansion bit and a 3 byte fingerprint under a seed it picks, because the
4 byte meta item cannot tell them apart. That is the documented outcome of
a 1 byte bitmap plus 3 byte fingerprint, which the mask's own comment
rates at solidity selector resistance; a real word set that hits it is
fixed by renaming a word.

A fuzzer drawing uniform `bytes32` words hits the ~2^-32 per pair
coincidence often enough to flake CI. The counterexample from run
36937526403 is 114 words with exactly one colliding pair, at indices 30
and 66, which agree on expansion bit 199 and fingerprint 0x9540FF under
seed 59 and on nothing under seed 0.

`LibParseMetaFingerprint.findsDupes` scans every seed in [0, 255] and
chains words on the low fingerprint byte, so a false result guarantees
`buildParseMetaV2` cannot revert for that set. Scanning every seed is the
cheap direction: predicting the seeds `findBestExpander` picks costs a
full expander chain. The six fuzz tests that build a parse meta from a
fuzzed word set assume on it, after the existing bloom.

No production code changes and no assertion is weakened.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

🧰 Additional context used
📚 Code guidelines (1)
CLAUDE.md — auto-discovered

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: rainlanguage/rainlang.interface/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 7604bee4-368b-4af6-aac1-53620bf715d0

📥 Commits

Reviewing files that changed from the base of the PR and between 43cc249 and d82968d.

📒 Files selected for processing (5)
  • test/lib/meta/LibParseMetaFingerprint.sol
  • test/lib/meta/LibParseMetaFingerprint.t.sol
  • test/src/lib/codegen/LibGenParseMeta.buildMeta.t.sol
  • test/src/lib/codegen/LibGenParseMeta.duplicateFingerprint.t.sol
  • test/src/lib/parse/LibParseMeta.lookupWord.t.sol

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The changes add a helper that detects word pairs with matching masked fingerprints and expansion bitmaps across 256 seeds. Tests cover direct detection, a pinned metadata-generation counterexample, and fuzz-test exclusions.

Changes

Parse-meta fingerprint duplicate handling

Layer / File(s) Summary
Fingerprint guard and direct tests
test/lib/meta/LibParseMetaFingerprint.sol, test/lib/meta/LibParseMetaFingerprint.t.sol
Adds findsDupes and tests empty, single-word, repeated-word, clean-set, and collision cases.
Pinned generation counterexample
test/src/lib/codegen/LibGenParseMeta.duplicateFingerprint.t.sol
Adds a 114-word fixture with a colliding pair. Tests the guard, expects metadata generation to revert with DuplicateFingerprint, and verifies building and lookup after removing one colliding word.
Parse-meta fuzz-test exclusions
test/src/lib/codegen/LibGenParseMeta.buildMeta.t.sol, test/src/lib/parse/LibParseMeta.lookupWord.t.sol
Adds findsDupes exclusions to parse-meta build, round-trip, structural-validation, and lookup fuzz tests.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to d8296

No actionable merge-blocking issue is identified; the change is ready to merge after normal checks.

Architecture Summary

Architecture risk: 🔵 Low · up to d8296

The change affects 1 system.

Changed systems: test

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — test (service) was modified; 5 changed files map to changed impact.

Before / after behavior

  • observed — Modified behavior in test/lib/meta/LibParseMetaFingerprint.sol: Added LibParseMetaFingerprint and its internal findsDupes(bytes32[] memory words) guard. It checks all 256 seeds for pairs with identical masked fingerprints and expansion bitmaps, returning true on a match; inputs shorter than two words and sets with no matching pair return false.
  • observed — Modified behavior in test/lib/meta/LibParseMetaFingerprint.t.sol: Adds the Solidity test file, imports the metadata and fingerprint libraries, and declares the test contract.
  • observed — Modified behavior in test/lib/meta/LibParseMetaFingerprint.t.sol: Defines the two collision-test words and seed 59 as internal constants.
  • observed — Modified behavior in test/lib/meta/LibParseMetaFingerprint.t.sol: Adds tests asserting that empty and single-word inputs and clean sets of up to 128 sequential words are not flagged, while repeated words—including a non-adjacent repeat—are flagged. The repeated-word fuzz test bounds its count to 2–8.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely describes the main change: excluding word sets that parse metadata cannot represent.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@thedavidmeister
thedavidmeister merged commit d15e659 into main Oct 2, 2026
4 of 6 checks passed
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown

@coderabbitai assess this PR size classification for the totality of the PR with the following criterias and report it in your comment:

S/M/L PR Classification Guidelines:

This guide helps classify merged pull requests by effort and complexity rather than just line count. The goal is to assess the difficulty and scope of changes after they have been completed.

Small (S)

Characteristics:

  • Simple bug fixes, typos, or minor refactoring
  • Single-purpose changes affecting 1-2 files
  • Documentation updates
  • Configuration tweaks
  • Changes that require minimal context to review

Review Effort: Would have taken 5-10 minutes

Examples:

  • Fix typo in variable name
  • Update README with new instructions
  • Adjust configuration values
  • Simple one-line bug fixes
  • Import statement cleanup

Medium (M)

Characteristics:

  • Feature additions or enhancements
  • Refactoring that touches multiple files but maintains existing behavior
  • Breaking changes with backward compatibility
  • Changes requiring some domain knowledge to review

Review Effort: Would have taken 15-30 minutes

Examples:

  • Add new feature or component
  • Refactor common utility functions
  • Update dependencies with minor breaking changes
  • Add new component with tests
  • Performance optimizations
  • More complex bug fixes

Large (L)

Characteristics:

  • Major feature implementations
  • Breaking changes or API redesigns
  • Complex refactoring across multiple modules
  • New architectural patterns or significant design changes
  • Changes requiring deep context and multiple review rounds

Review Effort: Would have taken 45+ minutes

Examples:

  • Complete new feature with frontend/backend changes
  • Protocol upgrades or breaking changes
  • Major architectural refactoring
  • Framework or technology upgrades

Additional Factors to Consider

When deciding between sizes, also consider:

  • Test coverage impact: More comprehensive test changes lean toward larger classification
  • Risk level: Changes to critical systems bump up a size category
  • Team familiarity: Novel patterns or technologies increase complexity

Notes:

  • the assessment must be for the totality of the PR, that means comparing the base branch to the last commit of the PR
  • the assessment output must be exactly one of: S, M or L (single-line comment) in format of: SIZE={S/M/L}
  • do not include any additional text, only the size classification
  • your assessment comment must not include tips or additional sections
  • do NOT tag me or anyone else on your comment

@linear

linear Bot commented Oct 2, 2026

Copy link
Copy Markdown

RAI-2848

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant