[Alok-Ranjan23]

Whisper's tools accepted only numeric GPU ordinals, and an explicit device that failed initialization could silently return CPU inference or a CPU fit verdict. Default Hexagon host buffers also left quantized projection weights on CPU. This adds explicit device selection and chooses the backend's existing repack buffers for supported quantized matrix weights.

The runtime remains based on the shared Speech GGML producer; this PR changes no kernels or numerical grading.

Current head: 9dfcfcd5cd88a934e125cac3b2d061af648a06e2. Paired shared producer: GGML PR #128, d1b3e16c0c3b2da178c9c0d04685240bf69d7df5, base speech. The new native CLI/fit default prefers available HTP0 when neither a flag nor WHISPER_ARG_DEVICE selects another target. Local host validation passes 24/24 tests with 60 selector assertions; Android builds pass and independent review found no blocking selector issue. Current-head paired native CI passes 24/24 CPU fixtures and verifies 60 selector assertions. All current-head checks are complete: 32 successful, one intentional skip, zero failures or pending checks. The optional short device default-selection smoke was not rerun because wireless debugging is unreachable; the user was informed no further phone connection is needed now. The previously completed device corpus preserves every original transcript. Ready for human review: on 2026-10-22, Alok accepted the known tiny multilingual Q8 French difference as a task-specific exception because HTP matches OpenCL and the intended meaning is essentially unchanged. Original scores and the numerical gate result are retained below.

Task: https://app.asana.com/1/45238840754660/project/1214153063536860/task/1219734880954142

Changes

• Add --list-devices and exact-name selection such as --device HTP0 to both native CLI/fit tools. Automatic selection now prefers registered HTP0; explicit flags, environment targets and CPU mode override it. Numeric ordinals and the public C API selection contract are preserved.
• Reject explicit device initialization fallback. Add whisper_get_primary_backend_name without changing public parameter structs; primary identity is initialization metadata, while unsupported graph nodes may still use CPU.
• For quantized MUL_MAT weights, probe the selected GPU's extra buffers and use supported repack storage. Preserve F16/F32, tied embeddings, VAD, canonical CPU fallback and OpenCL placement.
• Add an installed-library measurement consumer and release diagnostics, matching host/DSP deployment instructions, regression coverage and native Android Whisper CI.
• Use generic unified-memory capacity with explicit unknown-capacity errors, and add only prospective HTP metadata growth to fit. The paired shared producer prices exact physical allocation chunks and preserves shared-buffer lifetimes on growth.

Validation

One device tested: OnePlus CPH2723 / SM8750, Android 16, Hexagon v79 / Adreno 830. Android arm64 Release/API28 uses NDK r28b, Hexagon SDK6.6.0.0 and Tools19.0.07; DSP skeletons v73/v75/v79/v81 build successfully. No QVAC/SDK or consumer package validation is claimed.

Evidence is attributed separately:

• Original inference/performance: Speech 594dbe6c + shared a6122941.
• Full repeated corpus/operators: Speech 209525b4 + shared d1b3e16c in an isolated deployment.
• Fit-error fix before the selection-only follow-up: Speech 798d15e1 + shared d1b3e16c; 12 fit/regression commands and CPU/HTP/OpenCL real-model smoke tests. The last commit changes fit measurement failure propagation, with no inference math or placement change.

Check	Result
Current host fixtures	24/24 passed locally and in current-head paired native CI, including 60 selector assertions locally
Native model calls	228/228 completed, 76 each CPU/OpenCL/HTP; all 228 transcripts byte-identical to the original same-backend/model/clip results
Models and corpus	tiny.en/base.en/tiny/base, each F16 and Q8_0; 11 fixed English/French, long and silence clips; native Whisper .bin models
Frozen quality gate	11/12 partitions pass numerically; one French partition accepted by Alok as an explicit exception; 24/24 JFK expected-phrase checks pass; no long/silence review flags
Real HTP execution	Original tiny multilingual Q8/JFK returned quantized DSP records increase 0→680; CPU split records 731→78; primary backend identity alone is not execution proof
Explicit REPACK matmul	Per environment: 26 support assertions +5 actual HTP0 Q8 math cases versus CPU; queue/compute stage bits verified; worst NMSE 1.45089067e−5, unchanged limit 5e−4
Memory fit	7 supported cases ×5 components = 35/35 exact projected/measured byte pairs; pressure control correctly returns does-not-fit
Failed graph measurement	Four unsupported full-Whisper HOSTBUF=0 graphs now return Error / measurement-failed instead of a false successful estimate; separate CPU graph rejection/recovery regression passes
Real-model smoke on fit-error head 798d15e1	CPU, HTP and OpenCL each return native0 and the expected JFK phrase; new automatic-selection smoke checks were not rerun because the wireless endpoint was unreachable

The follow-up focused operator selections use the same 73 instances /70 unique cases and unchanged grading:

Environment	Before passed / failed / unsupported	After passed	After failed	After unsupported
Default	58 /7 /8	58	0	15
GGML_HEXAGON_NDEV=1 GGML_HEXAGON_HOSTBUF=0	65 /0 /8	65	0	8

The seven default canonical-Q8 cases are now honestly rejected as unsupported; they were not converted to numerical passes. The added explicit-REPACK test supplies five real DSP math comparisons in each environment. Four F16 GELU and four non-contiguous NORM instances remain unsupported. Buffer and runtime-info contracts also pass in both environments; all 24 follow-up native test commands exit0. No full backend suite was run.

Fit now includes physical allocation guards, reports unified host RAM when HTP capacity is unknown and charges only future runtime metadata growth. Independent runtime snapshot verifies 10,833,920 bytes of prospective growth without double-counting retained allocations. Weights/KV/compute/VAD/host-overflow projections match real loads in every supported case (VAD is zero; no VAD inference claim). HOSTBUF=0 full Whisper still has an unsupported preallocated KV-copy graph; its operator passes do not establish whole-model support. Unknown capacity and failed graph measurement return explicit errors. This is an estimate with a safety margin, not a guarantee of future driver allocation.

Accepted French accuracy exception

Tiny multilingual Q8_0 / French: CPU 26/63 word errors (41.26984% WER), HTP/OpenCL 27/63 (42.85714%; +1.58730 absolute points), above the unchanged +1.0-point gate. One différents→différentes substitution reproduces in fresh runs and remains identical after the shared fixes.

Acceptance recorded 2026-10-22: Alok explicitly accepted this known difference for this task because OpenCL has the same output and the intended meaning is essentially the same. Both adjective forms mean “different”; différentes has incorrect grammatical gender agreement with concurrents. This acceptance covers the documented tiny-Q8 French difference only. The raw partition still exceeds the original numerical gate; neither the score nor the scorer, corpus, model or tolerance has changed.

Read-only traces show identical prefixes before callback 15; the signed différentes−différents logit margin is ARM CPU Q8 −0.038498, HTP +0.055366 and OpenCL +0.059588. Isolated final-dot capture preserves all original output rows; independent FP64/dequantized and Q8-activation arithmetic validates the final projection. Its hidden inputs already reverse the ranking, so no final-dot defect is demonstrated.

As a diagnostic only, every original Q8 weight was exactly dequantized into an F32 model; all 37,760,640 values were independently verified bit-exact. On the same phone CPU this model also selects différentes (margin +0.035530), matching all 63 HTP/OpenCL token prefixes and the final transcript. This supports precision-sensitive accumulated ranking, not a demonstrated HTP-specific faulty operation. It does not identify the first divergent upstream operation or prove every upstream kernel correct. The diagnostic model is not substituted into the frozen quality score. No tolerance, model, CPU baseline or production precision policy has changed.

Six original zero-input API cases pass across CPU/OpenCL/HTP. The helper's subsequent generated-silence call emits you on all three backends; that checks API/state safety, not silence recognition quality.

Performance

Measured on the device above using the installed native library, fixed 11-second JFK audio, four CPU threads and identical deterministic decoding. Times below are native whisper_full calls; loading is separate. No forced suspend. Both benchmark model variants passed their own frozen English accuracy gate.

Warm results are exploratory: one excluded warmup and three timed calls, in fixed CPU/OpenCL/HTP order, without a matched cooldown before every row.

Model	Backend	Call median [min–max], s	RTF	Load, s	Process peak RSS, MiB
tiny.en F16	CPU	1.7966 [1.4762–2.0255]	0.1633	0.1681	171.12
tiny.en F16	OpenCL	1.7410 [1.5935–1.8019]	0.1583	0.2165	175.11
tiny.en F16	HTP	0.1494 [0.1332–0.2705]	0.0136	0.1187	48.29
base.en Q8_0	CPU	1.6863 [1.4341–2.2694]	0.1533	0.1340	197.00
base.en Q8_0	OpenCL	4.4179 [3.3506–4.7203]	0.4016	0.4600	191.62
base.en Q8_0	HTP	0.3986 [0.3963–0.4306]	0.0362	0.1172	48.59

RTF is call time divided by 11 seconds of audio; lower is better. Process RSS incompletely counts GPU/FastRPC/shared allocations and does not establish total-memory savings.

The sustained protocol is complete: one accepted session per backend, each at least 300 native seconds after cooldown and matched sampled starting battery/skin temperatures. Original frozen source is Speech 594dbe6c + shared a6122941; these are not performance measurements of the later fit/support-only follow-up.

Backend	Sustained calls	Native phase, s	Mean call, s	Audio throughput, × realtime	First→last 30s call medians, s	Observed time change
CPU	126	301.611	2.3937	4.60	2.6168→2.3558	-9.97%
OpenCL	92	301.210	3.2739	3.36	3.2041→3.2584	+1.69%
HTP	1,188	300.012	0.2525	43.57	0.1757→0.2170	+23.54%

Mean call time is summed native inference divided by completed sustained calls. Audio throughput is total processed audio divided by summed native inference; 1× means real time. Loading, warmup and three initial measurement calls are excluded. The first/last windows include only calls fully inside those 30 native seconds.

Backend	Encode / decode total medians, s	Load, s	Process peak RSS, MiB
CPU	2.1268 / 0.1252	0.1264	202.64
OpenCL	0.7799 / 2.3812	0.4792	202.45
HTP	0.0560 / 0.0810	0.1463	53.93

Stage columns use bracketed stderr totals per full call, not per-decode API averages; separate medians need not sum to the median full call. Process RSS incompletely counts GPU/FastRPC/shared memory and does not establish total-memory savings.

Backend	Battery service °C, start→end	Skin °C, start→end (sampled max)	Android thermal status, start→end
CPU	36.1→37.3	37.984→41.825 (42.334)	0→0
OpenCL	35.7→36.8	37.345→41.986 (43.682)	0→0
HTP	36.9→39.0	38.607→43.655 (46.668)	0→1

Observed conclusion: HTP delivers 9.48× CPU and 12.97× OpenCL throughput on this repeated base.en Q8/JFK workload. HTP first→last call median rises 23.54%, OpenCL rises 1.69%, and CPU falls 9.97%. These are observed call-time changes; no frequency trace establishes thermal throttling. HTP still has the lowest mean call time.

All 130 CPU, 96 OpenCL and 1,192 HTP calls including initial phases return the expected phrase and one identical transcript per backend. All accepted sessions pass sampled screen/unplugged/battery guards, with no recorded telemetry gaps. Six rejected archive directories remain preserved and excluded. The latest OpenCL screen-interrupted attempt was guard-terminated with native 143; it was not an inference crash. The wake cause is unproved.

Interruption conditions differ: warm/CPU used DND OFF, HTP priority DND, and accepted OpenCL total silence (explicitly approved). These once-per-change verified settings may affect background work; no correction is quantified. DND is now restored to original OFF with a successful command and zen 0 readback. Equal-duration sessions process different amounts of audio, so temperature/battery changes do not establish equal-work thermal or energy benefit. No power, frequency-residency or suspend-residency trace was collected.

Final offline audit reports 9/9 completed performance rows, no validation errors, matching before/after frozen library/skel/driver hashes and verified DND restoration. Bracketing hashes are not per-run hashes; warm rows predate the sustained-specific brackets. Full native output, exit receipts, stage timings, telemetry and excluded-attempt records remain preserved. The French partition exceeds the original numerical gate and is separately accepted by Alok as the explicit exception above.

Native CI

Current selection-only head 9dfcfcd5 has passing local host/Android builds; all automatic Speech checks pass: 32 successful, one intentional skip, zero failures or pending checks. Current paired shared/Speech native workflow passes 24/24 CPU fixtures against this exact Speech SHA; six shared host lanes pass with HTP host arms explicitly skipped. The following green results belong to the prior fit-error head and are preserved separately.

Prior reviewed Speech head 798d15e1: 32 successful checks, one intentional opt-in GPU skip, zero failures or pending checks. Shared head d1b3e16c: 10 successful checks and one intentional paired-job skip on the PR event. No CI execution approval is pending.

• Whisper CI
• Cross-platform CI
• Divergence guard
• Exact-SHA shared + Speech native CI: 24/24 CPU fixtures; six shared host lanes (Ubuntu/macOS/Windows ×shared/static), each 3 host tests passed and 2 HTP arms explicitly skipped. These host skips are separate from actual device execution above.

Review status and scope

Ready for human review following Alok's explicit acceptance of the documented French exception. Independent source/workflow reviews found no remaining blocking implementation issue. Shared operator/fit fixes, fit-error behavior and all three original sustained sessions are validated. The new default preference has passed host contracts and Android compilation; its optional short real-device selector check was not rerun because wireless debugging is unreachable. The original French numerical result remains recorded above. Task completion remains pending review and merge.

Fabric-only. Shared PR #128 is a producer dependency for the paired behavior validated here. QVAC/SDK integration, consumer packages/prebuilds, registry rollout and releases remain deferred until legal review and separate authorization. Full command output, native exit receipts, source/artifact/corpus/model hashes, diagnostic limits and CI JUnit artifacts are retained in the task evidence directory. The acceptance exception is limited to the documented French result.

[github-actions]

Review Status

Current Status: ✅ APPROVED
Approvals so far: Team Lead: 1, Member: 1