Qwen3VL: default per-image resolution to a 2,560 vision-token budget#398
Open
spokvulcan wants to merge 2 commits into
Open
Qwen3VL: default per-image resolution to a 2,560 vision-token budget#398spokvulcan wants to merge 2 commits into
spokvulcan wants to merge 2 commits into
Conversation
4 tasks
Contributor
Author
|
Lint fixed. |
…udget The qwen3_5 ViT runs global O(patches^2) attention with no window structure, so an uncapped full-resolution image (PARO ships a ~16 MP longest_edge that never clamps) can allocate a tens-of-GB attention score matrix. Default each image -- and each video frame -- to a 2,560 vision-token budget (min(config.maxPixels, 2560 * factor^2)), mirroring the Qwen2.5-VL clamp, and honor an explicit processing.maxPixels override. Downscales only what the tower sees; the stored bytes and image digest are untouched. The default budget multiply (2560 * factor^2) is guarded with multipliedReportingOverflow so a pathological patch/merge size degrades to the config ceiling instead of trapping, and both the image and video preprocess paths clamp through config.maxPixels / config.minPixels so the two clamp sites cannot drift. See tesseract ADR-0014. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Rebased onto current main. The |
3ad5fbe to
c424edd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #396.
The qwen3_5 ViT runs global O(patches²) attention with no window structure, so an uncapped full-resolution image (several published configs ship a ~16 MP longest_edge that never clamps) can allocate a tens-of-GB attention score matrix. Measured on a 27B Qwen3.6 checkpoint: 47 GB peak / 121 s prefill for one full-screen screenshot; ~21 GB with this cap.
min(config.maxPixels, 2560 * factor²), mirroring the Qwen2.5-VL clamp.processing.maxPixelsoverride, so full resolution stays available as an opt-in.multipliedReportingOverflowso a pathological patch/merge size degrades to the config ceiling instead of trapping, and the image and video preprocess paths clamp through the sameconfig.maxPixels/config.minPixelsaccessors so the two clamp sites cannot drift.On the default value: if you'd rather keep upstream's current behavior as the default, I'm happy to reshape this as a pure opt-in knob — the part that matters for on-device use is that a per-image budget exists.