Skip to content

fix(grok): preserve Responses vision tool output images - #6325

Open
251392463 wants to merge 1 commit into
Wei-Shaw:mainfrom
251392463:fix/grok-responses-vision-tool-output
Open

fix(grok): preserve Responses vision tool output images#6325
251392463 wants to merge 1 commit into
Wei-Shaw:mainfrom
251392463:fix/grok-responses-vision-tool-output

Conversation

@251392463

@251392463 251392463 commented Aug 28, 2026

Copy link
Copy Markdown

Summary

  • preserve images sent by Grok Build in function_call_output.images
  • extract input_image parts from structured function_call_output.output arrays
  • keep function_call_output.output as text while lifting media into a following user message in call order
  • cover parallel tool outputs and invalid/empty image payloads with regression tests

Root cause

Grok Build 1.0.5 can return local read_file results as either top-level images[] or structured output[] parts. The Grok Responses sanitizer previously reduced every tool result to a string and silently discarded those images (or serialized image data as ordinary text). The next continuation therefore reached xAI without the vision context that the client had supplied, which can lead to an unrelated continuation as described in #6185.

This change keeps the xAI-compatible textual tool output and sends the extracted images as standard Responses input_image content immediately after the consecutive tool outputs, preserving per-call attribution and order.

Validation

  • go test ./internal/service -run 'TestSanitizeGrokResponsesModelInput|TestPatchGrokResponsesBody.*Images' -count=1
  • go test -tags=unit ./internal/service ./internal/pkg/apicompat -count=1
  • go vet -tags=unit ./internal/service

Fixes #6185

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA. ✅
Posted by the CLA Assistant Lite bot.

@251392463

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@251392463
251392463 force-pushed the fix/grok-responses-vision-tool-output branch from 0050563 to b4b5371 Compare August 28, 2026 07:38
github-actions Bot added a commit that referenced this pull request Aug 28, 2026
@251392463

Copy link
Copy Markdown
Author

公司内部环境验证,问题已经解决了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Grok][Responses] 并行 vision tool_result 后续写会串到无关会话(0.1.181 & 0.1.182 复现)

1 participant