Support aarch64 (AWS Graviton, Apple Silicon) - #96
Open
lorenzo wants to merge 1 commit into
Open
Conversation
The C SIMD kernels use a small subset of x86 intrinsics (AVX2 / SSSE3 / BMI2). This adds bit-exact NEON / scalar equivalents so the library builds and runs on aarch64: - cbits/neon_shim.h: NEON implementations of the vector intrinsics and scalar implementations of the BMI2 bit-gather ops (_pext / _pdep) and _lzcnt. - cbits/intrinsics.h: include the NEON shim on aarch64; keep immintrin on x86. - cbits/simd.h: drop the intrinsics.h include (its prototypes use no vector types) so c2hs, which parses this header, does not pull in arm_neon.h. - cbits/simd-spliced.c: take the vector branch of hw_json_simd_summarise on the NEON path. - cbits/simd.c: report the indexing kernels as available on aarch64. - hw-json-simd.cabal: emit the x86 -m flags only for x86 targets; NEON is baseline on ARMv8-A. Remove the base < 0 constraint that blocked ARM. - CI: add aarch64 Linux and macOS runners; key the cabal cache by runner.arch. Verified on aarch64: every kernel's output is byte-for-byte identical to the x86 AVX2/BMI2 build (compared via an x86_64 build run under Rosetta), and the executable produces valid indexes for both the simple and standard methods.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds aarch64 support (AWS Graviton on Linux, Apple Silicon on macOS). The C SIMD kernels use a small subset of x86 intrinsics (AVX2 / SSSE3 / BMI2); this PR provides bit-exact NEON / scalar equivalents so the library builds and runs on aarch64. NEON (Advanced SIMD) is baseline on every ARMv8-A core, so no runtime feature detection is required. The x86 build path is unchanged.
Changes
cbits/neon_shim.h(new): NEON implementations of the vector intrinsics used (_mm256_cmpeq_epi8,_mm256_movemask_epi8,_mm256_slli/srli_epi64,_mm_shuffle_epi8,_mm_set*,_mm_extract_epi32) and portable scalar implementations of the BMI2 bit-gather ops (_pext_u64/u32,_pdep_u64) and_lzcnt_u64, which have no NEON counterpart.cbits/intrinsics.h: include the NEON shim on aarch64; keepimmintrin.h/mmintrin.hon x86.cbits/simd.h: drop theintrinsics.hinclude — its prototypes use no vector types, and dropping it keepsc2hs(which parses this header viaForeign.chs) from pulling inarm_neon.h, which its C parser cannot handle.cbits/simd-spliced.c: take the vector branch ofhw_json_simd_summariseon the NEON path.cbits/simd.c: report the indexing kernels as available on aarch64 (they run on the NEON path).hw-json-simd.cabal: emit the x86-mavx2/-mbmi2/-msse4.2/-mssse3/-mlzcntflags only for x86 targets; remove thebase < 0constraint that forced a build failure on ARM.ubuntu-24.04-arm,macos-latest) and key the cabal cache byrunner.arch.0.1.2.0+ ChangeLog entry.Verification
On aarch64 (Apple Silicon):
hw_json_simd_summarise, thesm_process_chunk/sm_make_ib_op_cl_chunksstate machine,sm_write_bp_chunk, and the splicedprocess_chunkwith cross-chunk carries) with identical deterministic inputs, compiled both as arm64 (NEON shim) and x86_64 (real AVX2/BMI2/SSSE3 intrinsics, run under Rosetta). Output is byte-for-byte identical.hw-json-simdexecutable produces valid indexes with both thesimpleandstandardmethods, including multi-chunk documents; the balanced-parens bit count is consistent with the interesting-bits count, and running balance never underflows.cabal build allandcabal test allpass.The new aarch64 CI jobs exercise this on Linux (Graviton class) and macOS.