fix: use image path placeholders for text-only models - #1750
Open
wangwingzero wants to merge 1 commit into
Open
Conversation
Replace stripped image blocks with local file path placeholders so text-only models (e.g. DeepSeek V4 Flash) can invoke an image-reading MCP tool instead of receiving an unsupported image_url block. - Prefer Codex clipboard paths under %TEMP% when present - Decode data URLs and download http(s) images into %TEMP%\codex-plus-vision-images - Fall back to path placeholders when VLM config is incomplete or VLM analysis fails - Add integration tests for strip mode and incomplete VLM config
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
纯文本模型(如 DeepSeek V4 Flash)不支持图片输入。旧逻辑在以下场景存在两个问题:
strip images只把图片替换成[图片已省略],模型完全看不到图片,也无法调用读图 MCP 读取本地文件。image_url会被原样发给纯文本模型,导致 OpenAI-compatible 上游报错:unknown variant 'image_url', expected 'text'改动
参考 cursor-byok 的图片路径占位实现,把文本模型的图片块替换为本地路径占位:
replace_images_with_path_placeholders,图片块替换为:[图片文件: 绝对路径] 用户刚刚发送了这张图片。请使用你可用的读图工具读取该文件路径来查看图片内容...analyze_image等),不再写死工具名。%TEMP%\codex-clipboard-*.png路径。%TEMP%\codex-plus-vision-images\;http(s) 图片下载后落盘;本地路径 canonicalize 为绝对路径。strip模式、VLM 配置不完整、VLM 当前轮失败均改为路径占位,不再向上游发送image_url。测试
cargo test -p codex-plus-core --lib:206 passedcargo test -p codex-plus-core --test protocol_proxy:51 passedcargo fmt --all -- --check/git diff --check:通过npm run check/npm run vite:build:通过cargo build --release:通过涉及文件
crates/codex-plus-core/src/vision.rscrates/codex-plus-core/src/protocol_proxy.rscrates/codex-plus-core/tests/protocol_proxy.rs