feat(qwen): local Qwen3-ASR provider (POC, CPU-only) - #28
Draft
sumerc wants to merge 3 commits into
Draft
Conversation
A local provider read its model ID from config.json and hard-errored when that ID belonged to a different engine, which is the normal state right after any provider switch: the persisted ID is whatever the PREVIOUS provider had selected, and it reaches the new provider via SetModel before the tray ever offers this engine's model list. The result was that every recording failed with `unknown <engine> model "<other-engine-id>"` until the user happened to open the model menu and pick something. Fall back to the provider's own default instead. Falling back is safe because the default is by construction the right engine's file. What must never happen is loading another engine's weights, and that is still rejected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Vendors antirez/qwen-asr (b00b789) into third_party/qwen-asr as plain C
sources and wraps it in internal/qwen, following the layout the other two
local engines already use. Dependency-free C, so the build is cc + ar — no
cmake, no submodule, and none of the ggml-sharing hazard whisper-lib exists
to prevent: it links its own kernels plus Accelerate and shares nothing
with parakeet/whisper.
Works end to end, 30 languages including Turkish, and is wired into the
tray like any other local provider. Draft rather than merge-ready — see
docs/design-notes.md for the measurements. In short:
- CPU only. No GPU backend exists upstream; matmuls go through
cblas_sgemm. Parakeet and Whisper both run on Metal here.
- Weights are bf16 and unquantized: 1.8 GB on disk, ~2.8 GB resident,
against 574 MB for whisper-turbo-q5.
- 23 s clip: 1944 ms vs Whisper's ~1100 ms wall.
- Accuracy on Turkish dictation with English technical vocabulary is
worse than Whisper's: it resolves loanwords through Turkish phonology
("sample" -> "sempol", "check edebilirsin" -> "çekebilirsin"). That is
intrinsic, not configuration — forced language, auto-detect and hints
all produced the same or worse output.
The model is a DIRECTORY (safetensors + tokenizer JSON), which the
single-file + one-sha256 contract behind install.sh and localmodel.Download
cannot express. localmodel.Model.IsDir marks it so it is skipped by the
manifest and refuses to download; `make download-qwen` fetches it by hand.
Includes the local-provider fallback fix (cherry-picked) so the branch is
testable on checkout; it is also proposed separately against main.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The measurement entry was all latency; the reason for parking is accuracy. Records the loanword failure on real dictation, that it is intrinsic rather than configuration (forced language, auto-detect and hints all produce the same or worse output), and the mechanism — an LLM decoder's monolingual prior, which is the same thing that makes it strong on monolingual speech. Marks the earlier "quality was a wash" line superseded rather than deleting it: that claim came from a Turkish TTS clip, and a clean synthetic signal in one language turned out to say nothing about code-switching. Also scopes what was actually measured. Everything here is antirez/qwen-asr — CPU, bf16, the slowest available implementation. qwen3-asr.cpp (ggml + Metal + Q8_0), MLX 4-bit and ONNX were not tested, with the one published qwen3-asr.cpp number and why none of them changes the conclusion. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft — parked deliberately, not awaiting review. Working end-to-end POC of
Qwen3-ASR as a local provider. Opened so the work is recoverable if we revisit;
the accuracy result below is why it is not proposed for merge.
What is here
third_party/qwen-asr/— antirez/qwen-asr @b00b789, vendored as plain C sourcesinternal/qwen/— cgo wrapper + non-darwin stub, matchinginternal/whispertranscriber/qwen.go— provider seam, matchingtranscriber/whisper.golocalmodel.Model.IsDir— directory-model support (see below)make qwen-lib/make download-qwenDependency-free C, so the build is
cc+ar— no cmake, no submodule, andnone of the ggml-sharing hazard
whisper-libexists to prevent: it links its ownkernels plus Accelerate and shares nothing with parakeet/whisper.
Why it is parked
Accuracy on the actual use case. On Turkish dictation containing English
technical vocabulary it resolves loanwords through Turkish phonology:
çekebilirsinis a real Turkish word, so the error is silent. This is intrinsic,not configuration — forced language, auto-detect and vocabulary hints all
produced identical or worse output. Published benchmarks put Qwen3-ASR above
Whisper-large-v3, but those are monolingual read speech and do not measure
code-switching.
Cost. CPU only (no GPU backend exists upstream), bf16 and unquantized:
Faster paths exist (qwen3-asr.cpp on ggml+Metal, MLX 4-bit), but they buy speed, not accuracy —
and quantization moves WER the wrong way. Not worth the work for a model that is
already the weaker one at full precision here.
Notes for a future revisit
install.shandlocalmodel.Downloadcannotexpress.
IsDirskips it in the manifest and refuses download.cblas_sgemm$NEWLAPACK(macOS 13.3+) while zee targets11.0 — fine locally, would fail to load on 11–13.2.
n_threads=1. Missingqwen_set_threads()cost ~2x before it was caught.ZEE_QWEN_THREADS/ZEE_QWEN_VERBOSE=2for experiments.Full measurements in
docs/design-notes.md.Contains the local-provider fallback fix (cherry-picked) so the branch is
testable on checkout; that fix is proposed separately in the companion PR
against
main.🤖 Generated with Claude Code