Skip to content

Add redirect_stdout_to_stderr() utility for native libraries - #1635

Open
fly1d wants to merge 1 commit into
simonw:mainfrom
fly1d:codex/redirect-native-stdout
Open

Add redirect_stdout_to_stderr() utility for native libraries#1635
fly1d wants to merge 1 commit into
simonw:mainfrom
fly1d:codex/redirect-native-stdout

Conversation

@fly1d

@fly1d fly1d commented Aug 23, 2026

Copy link
Copy Markdown

Summary

Fixes #196.

Some native model backends write log messages straight to file descriptor 1, bypassing both sys.stdout and contextlib.redirect_stdout. This is visible with the llm-llama-cpp and llm-gpt4all plugins, where llama.cpp/gpt4all emit lines such as llama_new_context_with_model: ... directly to stdout.

This adds a small reusable redirect_stdout_to_stderr() context manager to llm.utils that redirects OS-level stdout (fd 1) to stderr for the duration of the block. Plugins can wrap their native init and generate calls with it, so that noise stays on stderr and stdout keeps only the model output.

Verification

  • pytest tests/test_utils.py -q: 86 passed
  • ruff check llm/utils.py tests/test_utils.py: passed
  • black --check llm/utils.py tests/test_utils.py: passed

The new test uses os.pipe() to observe fd-level writes directly, so it verifies the redirect works for writes that bypass Python's sys.stdout.

Prepared with OpenAI Codex (GPT-5) assistance.

Some native model backends (llama.cpp, gpt4all) write log lines directly to file descriptor 1, bypassing sys.stdout and contextlib.redirect_stdout. Expose a reusable context manager that redirects fd 1 to stderr so plugins can keep that noise off stdout.

Assisted-by: OpenAI Codex (GPT-5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

some log messages are getting output to stdout

1 participant