glm 5.3 flash metal not optimized?

[badelon]

i am testing glm 5.3 flash on m1 ultra
i get 100 tok/sec prefil which is kinda slow
on deepseek i got 170-200 tok/sec prefil
i guess for glm we need to wait optimizing update?

[trueimage]

I'm done a bunch of work to optimize prefill and generation on pre-M5 metal, I'm using an M3 Ultra.

I created this PR #964 and am waiting for a review. if you could test it that would be helpful. I mainly tested using the Q4_K gguf but the Q2 data would be ok too. here's how:

Please run a short benchmark on main, then on PR 964, and paste both raw results. Use the same GLM 5.3 Flash Q4_K model for both runs. On an M1 Ultra, this model needs
--ssd-streaming.

MODEL="/absolute/path/to/GLM-5.3-Flash-Q4_K.gguf"

git switch main
git pull --ff-only
make clean && make -j "$(sysctl -n hw.logicalcpu)" ds4-bench

./ds4-bench -m "$MODEL" \
  --prompt-file speed-bench/promessi_sposi.txt \
  --ctx-start 512 --ctx-max 4096 --step-mul 2 \
  --gen-tokens 128 --ssd-streaming \
  2>&1 | tee ~/ds4-main-bench.txt

Then fetch and build PR 964:

git switch --detach FETCH_HEAD
make clean && make -j "$(sysctl -n hw.logicalcpu)" ds4-bench

./ds4-bench -m "$MODEL" \
  --prompt-file speed-bench/promessi_sposi.txt \
  --ctx-start 512 --ctx-max 4096 --step-mul 2 \
  --gen-tokens 128 --ssd-streaming \
  2>&1 | tee ~/ds4-pr964-bench.txt

Please paste both saved outputs, including the startup memory/cache information, and tell me your RAM capacity, GPU core count, and macOS version. Keep other GPU-heavy
applications closed during both runs.

[badelon]

#964

very nice do you have telegram? i also have m3 ultra 512 and m1 128gb
we can work and test together

also i have q2 version which is 96gb so i guess i dont need ssd streaming

[badelon]

your pr is optimized for m3 right? still i got nice gains in prefil