Skip to content

Resolve glibc libc_nonshared.a symbols for the in-process JIT - #1077

Open
conrade-ctc wants to merge 2 commits into
compiler-research:mainfrom
conrade-ctc:glibc-nonshared-symbols
Open

Resolve glibc libc_nonshared.a symbols for the in-process JIT#1077
conrade-ctc wants to merge 2 commits into
compiler-research:mainfrom
conrade-ctc:glibc-nonshared-symbols

Conversation

@conrade-ctc

@conrade-ctc conrade-ctc commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

The in-process JIT cannot resolve at_quick_exit. glibc keeps it (with atexit, pthread_atfork, and __stack_chk_fail_local) in libc_nonshared.a, so every executable and DSO links its own private copy. dlsym never sees these, and the ORC process-symbol generator fails the lookup:

JIT session error: Symbols not found: [ at_quick_exit ]

Repro: Cpp::Process("extern \"C\" int at_quick_exit(void (*)(void)); int r = at_quick_exit([]{});") — any header that calls std::at_quick_exit inline hits this.

The fix (in-process, glibc-only, installed in Interpreter::create):

  • __stack_chk_fail_local is plain code, so a definition generator resolves it to this library's own copy.
  • at_quick_exit and pthread_atfork register callbacks, and a JIT'd callback must never end up inside host glibc: glibc would call it after the JIT is gone. A small IR module in the main JITDylib defines both (the same pattern LLJIT uses for atexit). The shims store the callbacks in a host-side registry, tagged by owning interpreter.
  • Quick-exit callbacks run exactly once: on a real quick_exit, or at interpreter teardown, whichever comes first.
  • Atfork callbacks fire through fixed host-side hooks while their interpreter is alive and are dropped at teardown.
  • atexit needs no entry: LLJIT's platform support already defines a JIT-aware atexit and runs its handlers at deinitialize().

Tests cover symbol resolution, the run-exactly-once rule, live fork dispatch, and two death tests for quick_exit/fork after teardown (previously SIGSEGV in freed JIT memory).

🤖 Done with the help of Claude Code (Fable 5, human in the loop)

@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.21488% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.80%. Comparing base (1795887) to head (e07d11f).

Files with missing lines Patch % Lines
lib/CppInterOp/CompatibilityGLIBC.h 93.33% 6 Missing ⚠️
lib/CppInterOp/CppInterOpInterpreter.h 96.77% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1077      +/-   ##
==========================================
+ Coverage   87.68%   87.80%   +0.11%     
==========================================
  Files          23       24       +1     
  Lines        6382     6502     +120     
==========================================
+ Hits         5596     5709     +113     
- Misses        786      793       +7     
Files with missing lines Coverage Δ
lib/CppInterOp/CppInterOpInterpreter.h 86.56% <96.77%> (+1.27%) ⬆️
lib/CppInterOp/CompatibilityGLIBC.h 93.33% <93.33%> (ø)
Files with missing lines Coverage Δ
lib/CppInterOp/CppInterOpInterpreter.h 86.56% <96.77%> (+1.27%) ⬆️
lib/CppInterOp/CompatibilityGLIBC.h 93.33% <93.33%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread lib/CppInterOp/CppInterOpInterpreter.h
Comment thread lib/CppInterOp/CppInterOpInterpreter.h
Comment thread lib/CppInterOp/CppInterOpInterpreter.h
Comment thread lib/CppInterOp/CppInterOpInterpreter.h
Comment thread lib/CppInterOp/CppInterOpInterpreter.h
Comment thread lib/CppInterOp/CppInterOpInterpreter.h
Comment thread lib/CppInterOp/CppInterOpInterpreter.h
Comment thread lib/CppInterOp/CppInterOpInterpreter.h
Comment thread lib/CppInterOp/CppInterOpInterpreter.h
Comment thread lib/CppInterOp/CppInterOpInterpreter.h
@lhames

lhames commented Aug 20, 2026

Copy link
Copy Markdown

If I'm reading it right this is unsafe: It's pointing JIT'd code to register with the host atexit, but by the time that callback happens the JIT'd code may already have been torn down -- leading to a use-after-free.

You can already inject C code: I'd say that you should just build a small glibc_nonshared implementation that you load into your JIT as a first step. As long as that code doesn't depend on atexit, etc. itself, you can then arrange to run the JIT-registered atexit callbacks before you tear the JIT down.

@conrade-ctc

conrade-ctc commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

You're right — reproduced: JIT'd code registers at_quick_exit(&jit_fn), the interpreter is destroyed, then quick_exit(0) faults with $pc = jit_fn in an unmapped page. pthread_atfork has the same hazard (fork after teardown faults in the freed prepare handler). atexit was already safe: the generic platform's per-JITDylib atexit shadows the fallback and runs its handlers at deinitialize().

I am iterating on a fix along the lines you suggest and will update the PR.

Great catch, thanks!

@conrade-ctc
conrade-ctc force-pushed the glibc-nonshared-symbols branch from 2c8a6fb to 5a9b042 Compare August 21, 2026 19:51
@conrade-ctc

conrade-ctc commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

Updated per the review, @lhames for viz/re-review

The absolute-symbol table now only carries __stack_chk_fail_local, which is plain code and safe to share. at_quick_exit and pthread_atfork no longer hand JIT'd function pointers to glibc. A small IR module in the main JITDylib defines both, the same way the LLJIT platform defines atexit. The shims store the callbacks in a host-side registry, tagged with the interpreter that owns them.

Quick-exit callbacks run exactly once: on a real quick_exit if one happens, otherwise when the interpreter is destroyed — while the JIT can still run them. Atfork callbacks fire through fixed host-side hooks while their interpreter is alive, and are dropped when it goes away. atexit needs nothing: the per-JITDylib platform version already handles it.

Tests: two death tests cover the post-teardown paths (quick_exit or fork after DeleteInterpreter used to fault in freed JIT memory; both exit cleanly now), plus tests that callbacks run exactly once before teardown and that live fork dispatch works.

@github-actions

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@conrade-ctc
conrade-ctc force-pushed the glibc-nonshared-symbols branch from 5a9b042 to 0b4796b Compare August 21, 2026 20:29
@github-actions

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

Comment thread lib/CppInterOp/Compatibility.h Outdated
#include <pthread.h>
#endif

#ifdef __GLIBC__

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can move that into a separate header CompatibilityGLIBC.h?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved: the glibc block now lives in lib/CppInterOp/CompatibilityGLIBC.h, so Compatibility.h is untouched by this PR. CppInterOpInterpreter.h and the test include the new header directly.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread lib/CppInterOp/CompatibilityGLIBC.h
Emery Conrad added 2 commits August 27, 2026 09:27
glibc keeps at_quick_exit, atexit, pthread_atfork and
__stack_chk_fail_local in libc_nonshared.a. Every ELF module links a
private copy that dlsym cannot see, so jitted references fail with
"Symbols not found: [ at_quick_exit ]".

__stack_chk_fail_local is plain code; a definition generator resolves
it to this library's own copy. at_quick_exit and pthread_atfork
register callbacks, and a jitted callback must never sit inside host
glibc: glibc would call it after the JIT is destroyed. A small IR
module in the main JITDylib defines both, the same way LLJIT's
platform defines atexit. The shims store callbacks in a host-side
registry keyed by the owning interpreter. Quick-exit callbacks run
exactly once, on a real quick_exit or at interpreter teardown,
whichever comes first. Atfork callbacks fire through fixed host-side
hooks while the interpreter lives and are dropped at teardown. atexit
needs no entry: LLJIT's platform already defines a JIT-aware atexit.

Co-developed-with-the-help-of: Claude Code (Fable 5, human in the loop)
Requested in review. Compatibility.h returns to the upstream text.
The new header includes <cstdlib> before the __GLIBC__ test; the macro
is undefined until a libc header is seen, and the body silently
compiles away without it.

Co-developed-with-the-help-of: Claude Code (Fable 5, human in the loop)
@conrade-ctc
conrade-ctc force-pushed the glibc-nonshared-symbols branch from 8735da7 to e07d11f Compare August 27, 2026 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants