Skip to content

feat(llama-cpp-localai-paged): paged KV cache llama.cpp backend + cross-request prefix sharing + GB10 decode optimization [WIP]#10462

Open
localai-bot wants to merge 321 commits into
masterfrom
worktree-feat+paged-attention
Open

feat(llama-cpp-localai-paged): paged KV cache llama.cpp backend + cross-request prefix sharing + GB10 decode optimization [WIP]#10462
localai-bot wants to merge 321 commits into
masterfrom
worktree-feat+paged-attention

docs(paged): record P6 fp8-KV measured NO-GO - throughput dead end, c…

3159ed0
Select commit
Loading
Failed to load commit list.
GitHub Advanced Security / gosec failed Jul 2, 2026 in 2s

1 new alert including 1 error

New alerts in code changed by this pull request

  • 1 error

Alerts not introduced by this pull request might have been detected because the code changes were too large.

See annotations below for details.

View all branch alerts.

Annotations

Check failure on line 185 in core/backend/options.go

See this annotation in the file changed.

Code scanning / gosec

Use of weak random number generator (math/rand or math/rand/v2 instead of crypto/rand) Error

Use of weak random number generator (math/rand or math/rand/v2 instead of crypto/rand)