[nonirosenfeldredis]

Describe the changes in the pull request
Remove id reuse from the tiered HNSW index.

Today an overwrite reserves the replaced id, and the insert that follows clears the slot under an exclusive main index guard. That acquisition waits out every graph scan, and while it is pending it blocks the shared acquisitions of the repair jobs and of deleteLabelFromHNSW, which serialises the workers under an update load. Measured locally (100k documents, 120,000 random-key updates, 16 connections): about 1.1k ops/s with the update-path reuse against about 4.4k with it off, and the main thread spent about 70% of its samples blocked on the flat or main index guard.

After this PR the tiered index never reuses a deleted element's id: an overwrite deletes the old element and the insert appends a fresh id, and the GC compacts the deleted elements as it did before id reuse existed.

• Removed: reserved swap jobs, the deferred insert submission, the exclusive barrier in the insert job, the id reuse in addVector, updateVectors and write-in-place, and setReuseOnUpdate. updateVectors is a delete followed by adds.
• Kept: the HNSW-level slot helpers (clearDeletedSlot, storeNewElementInClearedSlot), which the stacked follow-up uses.
• Follow-up: the stacked PR adds reuse back on the insert path, from a cache of slots the GC clears, off by default.

Supersedes #1084.

Which issues this PR fixes

1. MOD-19327

Main objects this PR modified

1. TieredHNSWIndex: the update-path reuse is removed (reuseIdOnUpdate, reserved swap jobs, insertVectorToHNSWReusingId, updateVectorInPlace, updateMultiValueInPlace, updateMultiValueAsync).
2. Tests: the reuse-on-update tests are removed.

Mark if applicable

• This PR introduces API changes
• This PR introduces serialization changes

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes core tiered HNSW update/delete/insert job ordering and compaction behavior under async load; correctness risk is mitigated by large test deletions/additions but behavior no longer matches the previous reuse semantics.

Overview
Removes the tiered HNSW “reuse deleted id on overwrite/update” path so updates no longer reserve swap jobs, defer insert jobs until repairs finish, or fill cleared slots via insertVectorToHNSWReusingId (which took an exclusive main index guard and contended with repair/delete work).

Overwrites and updateVectors now follow delete + fresh append: deleteLabelFromHNSW no longer pairs deleted ids with pending insert jobs, swap jobs always become ready for GC when repairs complete, and insert jobs always call insertVectorToHNSW. Write-in-place addVectorInPlace deletes the label first (including when it exists in both flat and HNSW) then addVectors; multi-value updateVectors is always deleteVector plus per-vector addVector. setReuseOnUpdate / reuseIdOnUpdate and related helpers (updateVectorInPlace, updateMultiValueInPlace/Async, deferred reuse submission) are dropped; HNSW slot helpers stay for a planned follow-up.

Tests covering id reuse and reserved swap jobs are removed; addVectorInPlaceOverwritesLabelInBothTiers asserts overwrite return semantics when a label spans both tiers.

^{Reviewed by Cursor Bugbot for commit a19f216. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Stale Bugbot comment from a previous run.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.37%. Comparing base (12e3c03) to head (a19f216).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1085      +/-   ##
==========================================
- Coverage   97.55%   97.37%   -0.18%     
==========================================
  Files         143      143              
  Lines        9715     9549     -166     
==========================================
- Hits         9477     9298     -179     
- Misses        238      251      +13     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[nonirosenfeldredis]

Local measurements of this experiment (VecSim 281a86cb, RediSearch with OPTIMIZE_PARTIAL_UPDATE on, 100k documents, one run each, Mac M4 Max). Reuse on is the build with the reuseIdOnInsert default flipped to true.

Vector updates only (120,000 updates to random keys, 16 connections)

	Reuse off	Reuse on
ops/s	5,448	5,391
CPU (of one core)	981%	996%
Inserts that took a cleared slot	0	59,997
Defrag runs / time	118 / 352 ms	118 / 269 ms
Backend size at the end (100,000 labels)	100,096	100,515
After a forced GC: backend size / marked deleted	100,000 / 0	100,258 / 258

Mixed: KNN queries plus rate-limited vector updates (memtier with the arguments of search-vector-field-update-knn-mixed, 180 s)

	Reuse off	Reuse on
KNN ops/s (avg latency)	2,263 (28.3 ms)	2,314 (27.6 ms)
Background updates	299/s	300/s
Inserts that took a cleared slot	0	49,664
Defrag runs / time	97 / 272 ms	97 / 200 ms
After a forced GC: backend size / marked deleted	100,000 / 0	100,274 / 274

Reading:

• Throughput and CPU match reuse off. For comparison, the update-path reuse on main gave about 1.1k ops/s against about 4.4k without reuse in the same updates-only setup ([MOD-19327] Clear the reused slot without the flat guard, and disable id reuse by default #1084).
• About half of the inserts take a cleared slot, as expected from compacting half of the cache each GC.
• Defrag time is about 25% lower with reuse on in both runs.
• The cache keeps cleared slots: a forced GC leaves about 260 of them, because each GC compacts half. An idle index never shrinks to zero by itself.

Single local runs, so differences of a few percent are noise. Not run on CI or with large documents yet.

🤖 Generated with Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 070fb42. Configure here.}

[dor-forer]

Looks good