Skip to content

Add TurboQuant KV cache compression on KVCacheSimple#287

Closed
joelnishanth wants to merge 2 commits into
ml-explore:mainfrom
offlyn-ai:feature/turboquant
Closed

Add TurboQuant KV cache compression on KVCacheSimple#287
joelnishanth wants to merge 2 commits into
ml-explore:mainfrom
offlyn-ai:feature/turboquant

Add configurable turboMinActivationTokens and turboHotWindowSize on C…

1a7cf0d
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs