Skip to content

Fix: treat an absent File.contentEncoding as base64 so sandbox output files are written as bytes - #678

Open
AmaadMartin wants to merge 1 commit into
mainfrom
fix/file-content-encoding-base64-default
Open

Fix: treat an absent File.contentEncoding as base64 so sandbox output files are written as bytes#678
AmaadMartin wants to merge 1 commit into
mainfrom
fix/file-content-encoding-base64-default

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):
    N/A
  2. Or, if no issue exists, describe the change:
    Problem: File.contentEncoding is optional, and the repo disagreed on what an absent value meant. materializeFiles passed it straight to Buffer.from, which falls back to Node's utf8 default, while AgentEngineSandboxCodeExecutor pushed base64 output files without the field. Every binary file the sandbox produced was written to disk as its base64 text, so a PNG from a skill script was ASCII, not an image. The sandbox input path had the mirror bug: it forwarded raw text as data where the API wants base64.

Solution: An absent contentEncoding now means FileContentEncoding.BASE64. base64 is the lossless reading, it is what the File.content docstring and two of the three consumers already assumed, and arbitrary bytes cannot survive a utf-8 round-trip. The rule lives in exactly one expression, decodeFileContent() in core/src/utils/file_utils.ts. Both in-repo producers that omitted the field now state it, so the default only ever governs File objects built outside this package.

Behavior change: a caller that builds CodeExecutionInput.inputFiles with plain text, omits contentEncoding, and runs UnsafeLocalCodeExecutor now has that text decoded as base64. Such a caller must add contentEncoding: FileContentEncoding.UTF8. This is the deliberate cost of having one rule. Input files that declare UTF8 are now base64-encoded before being sent to the sandbox; files that were already base64 go through a byte-identical round-trip, so no working caller breaks. The public API surface is unchanged: no signature, export, or enum value moves, and decodeFileContent stays internal.

Collision check: I listed all 577 open PRs on this fork and diffed every adjacent one. None lands this change. #677 fixes the separate artifact-save consumer (postProcessCodeExecutionResult), which this change deliberately does not touch. #409, #523 and #371 rework collision and path containment in materializeFiles and all keep the buggy Buffer.from(file.content, file.contentEncoding) decode. #409 hoists that call above the collision loop, so whichever of the two lands second rebases. I based this on main.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.

Six new tests, no existing test edited or deleted:

core/test/utils/file_utils_test.ts

  • writes a file with no contentEncoding as decoded base64 bytes
  • writes a file that declares base64 as decoded bytes
  • writes a file that declares utf-8 verbatim

core/test/code_executors/agent_engine_sandbox_code_executor_test.ts

  • marks output files as base64
  • base64-encodes an input file that declares utf-8
  • passes a base64 input file through unchanged
npx vitest run --project unit:core \
  core/test/utils/file_utils_test.ts \
  core/test/code_executors/agent_engine_sandbox_code_executor_test.ts \
  core/test/code_executors/unsafe_local_code_executor_test.ts \
  core/test/agents/processors/code_execution_request_processor_test.ts \
  core/test/code_executors/code_execution_utils_test.ts
# Test Files 5 passed (5), Tests 95 passed (95)

The first two file tests compare Buffers, not strings, because a string comparison passes on mojibake. The fixture is the PNG signature, which is not valid utf-8.

Mutation proof. I ran every new test against a mutated source line and recorded the failure.

Mutation Test that fails Failure
decodeFileContent reverted to Buffer.from(file.content, file.contentEncoding) no contentEncoding as decoded base64 bytes received Buffer[105, 86, 66, 79, 82, 119, ...], the ASCII of iVBORw, expected Buffer[137, 80, 78, 71, 13, 10, ...]
decode forced to FileContentEncoding.UTF8 declares base64 as decoded bytes; passes a base64 input file through unchanged 3 failed
decode forced to FileContentEncoding.BASE64, ignoring the declared value declares utf-8 verbatim; base64-encodes an input file that declares utf-8 5 failed, including 3 pre-existing utf-8 tests
input chunk reverted to data: file.content base64-encodes an input file that declares utf-8 chunk data was "# hi", expected "IyBoaQ=="
contentEncoding dropped from the output push marks output files as base64 expected undefined to be 'base64'

Coverage. decodeFileContent is at 100%: tests 1 and 2/3 cover both branches of the ??. file_utils.ts measures 95.18% lines overall; the shortfall is the pre-existing second path-traversal guard at lines 92-95, which this change does not touch.

Manual End-to-End (E2E) Tests:
I did not run the end-to-end test. It needs Google Cloud credentials and a live reasoning-engine sandbox, which I do not have. To run it: configure an LlmAgent with AgentEngineSandboxCodeExecutor, invoke a skill script that calls matplotlib savefig('plot.png'), then run file plot.png. It must report PNG image data and the image must open. Before this change it reported ASCII text.

I validated locally instead, on the exact commit pushed:

  • npm run build — passed
  • npm run lint — passed
  • npm run format:check — passed
  • npm run docs:check — passed
  • npx tsc --noEmit — 0 errors
  • the targeted suites above — 95 passed

Checklist

[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.

CI note: the Windows job failed twice before passing, each time on a different unrelated integration test that timed out waiting on a spawned process (build_setup_test.ts at 20s, then app_loader_test.ts at 40s). Neither test imports any module this change touches, and both live in the integration project while the new tests are unit:core. The third run passed with no code change.

materializeFiles passed file.contentEncoding straight to Buffer.from, so
an absent value fell back to Node's utf8 default. Every binary file the
Agent Engine sandbox produced was written to disk as its base64 text.

Put the rule in one place: decodeFileContent() in file_utils. Both
in-repo producers that omitted the field now state it, so the default
only ever governs File objects built outside this package. The sandbox
input chunk now honors a declared encoding instead of assuming base64.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant