Skip to content

docs: name the first upstream llama.cpp release for Q2_0 (b11556) - #285

Merged
bri-prism merged 1 commit into
mainfrom
docs/upstream-q2_0-b11556
Nov 5, 2026
Merged

bri-prism merged 1 commit into
mainfrom
docs/upstream-q2_0-b11556

Conversation

@bri-prism

Copy link
Copy Markdown
Collaborator

What

Docs only. Updates the upstream llama.cpp support claims for Bonsai 2.

  • The community benchmark README and the llama.cpp template still said the Q2_0 file "currently requires our fork" and linked the old dev repo copy. Both now link Ternary-Bonsai-2-27B-Q2_0.gguf in the main repo and say it runs on upstream llama.cpp b11556 or newer.
  • README, AGENTS.md, BACKEND-SUPPORT.md and MODEL-FORMATS.md said "current official llama.cpp". They now name release b11556.
  • The upstream support section now lists all four sizes. It also says the MTP and DFlash2 drafters on the model cards need our fork.

Why

People checking whether Bonsai 2 runs on stock llama.cpp were reading the stale "requires our fork" text. A release number gives them a concrete check.

How it was checked

Upstream master and release b11556 were tested against the prism fork on the 2B, 4B, 9B and 27B Q2_0 files:

  • Mean KLD is about 0 and the top token matches 99.9% of the time, on Metal and CPU.
  • Greedy text and image output are identical.
  • The commit before b11556 gives garbage (perplexity above 1 million).
  • Upstream rejects PTQ1_0 and PQ2_0 as unknown types.
  • Upstream loads the DFlash2 drafters but accepts none of their drafts.

The community benchmark README and template still said the Q2_0 file requires our fork and linked the old dev repo. Q2_0 runs on upstream llama.cpp from b11556 onward, so point them at the main repo file and say so. Replace "current official llama.cpp" with the release number elsewhere, and name the MTP and DFlash2 drafters as fork-only.

@khosravipasha khosravipasha left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks LGTM

@bri-prism
bri-prism merged commit de058d0 into main Nov 5, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants