Resolve glibc libc_nonshared.a symbols for the in-process JIT - #1077
Resolve glibc libc_nonshared.a symbols for the in-process JIT#1077conrade-ctc wants to merge 2 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1077 +/- ##
==========================================
+ Coverage 87.68% 87.80% +0.11%
==========================================
Files 23 24 +1
Lines 6382 6502 +120
==========================================
+ Hits 5596 5709 +113
- Misses 786 793 +7
🚀 New features to boost your workflow:
|
|
If I'm reading it right this is unsafe: It's pointing JIT'd code to register with the host You can already inject C code: I'd say that you should just build a small glibc_nonshared implementation that you load into your JIT as a first step. As long as that code doesn't depend on |
|
You're right — reproduced: JIT'd code registers I am iterating on a fix along the lines you suggest and will update the PR. Great catch, thanks! |
2c8a6fb to
5a9b042
Compare
|
Updated per the review, @lhames for viz/re-review The absolute-symbol table now only carries Quick-exit callbacks run exactly once: on a real Tests: two death tests cover the post-teardown paths ( |
|
clang-tidy review says "All clean, LGTM! 👍" |
5a9b042 to
0b4796b
Compare
|
clang-tidy review says "All clean, LGTM! 👍" |
| #include <pthread.h> | ||
| #endif | ||
|
|
||
| #ifdef __GLIBC__ |
There was a problem hiding this comment.
Maybe we can move that into a separate header CompatibilityGLIBC.h?
There was a problem hiding this comment.
Moved: the glibc block now lives in lib/CppInterOp/CompatibilityGLIBC.h, so Compatibility.h is untouched by this PR. CppInterOpInterpreter.h and the test include the new header directly.
glibc keeps at_quick_exit, atexit, pthread_atfork and __stack_chk_fail_local in libc_nonshared.a. Every ELF module links a private copy that dlsym cannot see, so jitted references fail with "Symbols not found: [ at_quick_exit ]". __stack_chk_fail_local is plain code; a definition generator resolves it to this library's own copy. at_quick_exit and pthread_atfork register callbacks, and a jitted callback must never sit inside host glibc: glibc would call it after the JIT is destroyed. A small IR module in the main JITDylib defines both, the same way LLJIT's platform defines atexit. The shims store callbacks in a host-side registry keyed by the owning interpreter. Quick-exit callbacks run exactly once, on a real quick_exit or at interpreter teardown, whichever comes first. Atfork callbacks fire through fixed host-side hooks while the interpreter lives and are dropped at teardown. atexit needs no entry: LLJIT's platform already defines a JIT-aware atexit. Co-developed-with-the-help-of: Claude Code (Fable 5, human in the loop)
Requested in review. Compatibility.h returns to the upstream text. The new header includes <cstdlib> before the __GLIBC__ test; the macro is undefined until a libc header is seen, and the body silently compiles away without it. Co-developed-with-the-help-of: Claude Code (Fable 5, human in the loop)
8735da7 to
e07d11f
Compare
The in-process JIT cannot resolve
at_quick_exit. glibc keeps it (withatexit,pthread_atfork, and__stack_chk_fail_local) inlibc_nonshared.a, so every executable and DSO links its own private copy. dlsym never sees these, and the ORC process-symbol generator fails the lookup:Repro:
Cpp::Process("extern \"C\" int at_quick_exit(void (*)(void)); int r = at_quick_exit([]{});")— any header that callsstd::at_quick_exitinline hits this.The fix (in-process, glibc-only, installed in
Interpreter::create):__stack_chk_fail_localis plain code, so a definition generator resolves it to this library's own copy.at_quick_exitandpthread_atforkregister callbacks, and a JIT'd callback must never end up inside host glibc: glibc would call it after the JIT is gone. A small IR module in the main JITDylib defines both (the same pattern LLJIT uses foratexit). The shims store the callbacks in a host-side registry, tagged by owning interpreter.quick_exit, or at interpreter teardown, whichever comes first.atexitneeds no entry: LLJIT's platform support already defines a JIT-awareatexitand runs its handlers atdeinitialize().Tests cover symbol resolution, the run-exactly-once rule, live fork dispatch, and two death tests for
quick_exit/forkafter teardown (previously SIGSEGV in freed JIT memory).🤖 Done with the help of Claude Code (Fable 5, human in the loop)