Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 20 additions & 9 deletions .github/workflows/e2e-dapp-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,16 @@ name: E2E dApp Integration
# every supported (link, transport) combination, asserts both exit cleanly,
# and publishes a per-combination summary for the consolidated CI report.
#
# POSIX/SCTP is excluded because ubuntu-latest runners lack the sctp kernel
# module. The code path is still built; only the CI matrix excludes it.
# The POSIX and the io_uring link run on IPC, TCP and SCTP (the hosted runner
# opens SCTP sockets; only the headers are installed here). ZeroMQ has no SCTP.

on:
workflow_call:
workflow_dispatch:

jobs:
asn1c-cache:
# Builds/warms the asn1c cache once, ahead of the 4 matrix legs below, so
# Builds/warms the asn1c cache once, ahead of the 8 matrix legs below, so
# they don't race each other rebuilding it from source on a cold cache.
name: Warm asn1c cache
runs-on: ubuntu-latest
Expand Down Expand Up @@ -52,11 +52,11 @@ jobs:
strategy:
fail-fast: false
matrix:
# POSIX/SCTP is intentionally excluded — ubuntu-latest runners lack
# the sctp kernel module. The code path is built; we just don't
# exercise it in CI.
link: [zmq, posix]
transport: [ipc, tcp]
link: [zmq, posix, uring]
transport: [ipc, tcp, sctp]
exclude:
- link: zmq
transport: sctp

steps:
- name: Checkout repository
Expand All @@ -77,11 +77,22 @@ jobs:
run: |
chmod +x ./build_libe3
./build_libe3 -I
sudo apt-get install -y libsctp-dev

- name: Build Release with examples
run: |
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DLIBE3_ENABLE_SCTP=ON
cmake --build build -j $(nproc)
grep -q '^LIBE3_WITH_URING:INTERNAL=ON' build/CMakeCache.txt || {
echo "the io_uring link was not built (liburing missing?)"; exit 1; }

# What a failing uring or sctp cell needs to be read: the kernel, and whether it lets this job use them.
- name: Kernel, io_uring and SCTP on this runner
run: |
uname -r
echo "kernel.io_uring_disabled=$(cat /proc/sys/kernel/io_uring_disabled 2>/dev/null || echo absent)"
python3 -c "import socket; socket.socket(socket.AF_INET, socket.SOCK_STREAM, 132).close(); print('SCTP sockets: yes')" \
|| echo "SCTP sockets: no"

- name: Prepare IPC directory
run: |
Expand Down
20 changes: 18 additions & 2 deletions .github/workflows/full-loop-bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@ name: Full-loop Latency Benchmark
# Runs the full-loop control-loop latency benchmark (tests/integration/
# bench_full_loop_latency.cpp) on every PR and publishes the per-phase
# markdown table for the consolidated CI report, mirroring the MPMC queue
# benchmark workflow.
# benchmark workflow. It also runs the benchmark over every link (ZeroMQ,
# POSIX, io_uring and two io_uring variants) and transport on this one runner,
# interleaved (tools/bench_matrix.py), and reports which link is fastest.

on:
workflow_call:
Expand Down Expand Up @@ -39,13 +41,15 @@ jobs:
run: |
chmod +x ./build_libe3
./build_libe3 -I
sudo apt-get install -y libsctp-dev

- name: Build Release with integration tests
run: |
cmake -S . -B build \
-DCMAKE_BUILD_TYPE=Release \
-DLIBE3_BUILD_INTEGRATION_TESTS=ON \
-DLIBE3_ENABLE_LATREC=ON
-DLIBE3_ENABLE_LATREC=ON \
-DLIBE3_ENABLE_SCTP=ON
cmake --build build --target test_bench_full_loop_latency -j $(nproc)

- name: Run benchmark
Expand All @@ -55,6 +59,14 @@ jobs:
| grep -v "^\[20" > bench_results.md
cat bench_results.md

- name: Compare the links
run: |
uname -r
echo "kernel.io_uring_disabled=$(cat /proc/sys/kernel/io_uring_disabled 2>/dev/null || echo absent)"
python3 tools/bench_matrix.py --bench ./build/test_bench_full_loop_latency \
--links zmq,posix,uring,uring-busy,uring-sqpoll-busy --transports ipc,tcp,sctp \
--rounds 3 --iterations 1000 --out-md bench_matrix.md --out-csv bench_matrix.csv

- name: Assemble report fragment
if: ${{ always() }}
run: |
Expand All @@ -66,6 +78,10 @@ jobs:
echo ""
echo "_ubuntu-latest, Release build, ZMQ + IPC, ASN.1 APER._"
echo ""
if [ -f bench_matrix.md ]; then
sed 's/^## /#### /; s/^### /#### /' bench_matrix.md
echo ""
fi
echo "_These numbers are measured inside a GitHub Actions container and should be treated as an upper bound on E3AP's and the library's own latency, not a representative deployment measurement._"
} > ci-report-full-loop.md

Expand Down
24 changes: 24 additions & 0 deletions .github/workflows/pr-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ jobs:
needs: asn1c-cache
runs-on: ${{ matrix.os }}
timeout-minutes: 20
# The io_uring tests skip (exit 77) where the kernel refuses io_uring; this runner is meant
# to run them, so a skip fails here instead of passing unseen.
env:
LIBE3_REQUIRE_URING: "1"
strategy:
matrix:
os: [ubuntu-latest]
Expand All @@ -64,6 +68,11 @@ jobs:
chmod +x ./build_libe3
./build_libe3 -I

- name: Kernel and io_uring on this runner
run: |
uname -r
echo "kernel.io_uring_disabled=$(cat /proc/sys/kernel/io_uring_disabled 2>/dev/null || echo absent)"

- name: Build and test with build_libe3 (${{ matrix.build_type }})
shell: bash
run: |
Expand All @@ -80,6 +89,8 @@ jobs:
runs-on: ubuntu-latest
needs: test
timeout-minutes: 20
env:
LIBE3_REQUIRE_URING: "1"
steps:
- name: Checkout repository
uses: actions/checkout@v5
Expand Down Expand Up @@ -170,6 +181,19 @@ jobs:
cmake --build build-nozmq -j $(nproc)
cd build-nozmq && ctest --output-on-failure

# The io_uring link is optional too: without it the library must build, `--link uring`
# must fail cleanly (test_link_availability), and no test may need liburing.
- name: Configure, build and test with the io_uring link disabled
run: |
cmake -S . -B build-nouring \
-DCMAKE_BUILD_TYPE=Release \
-DLIBE3_ENABLE_URING=OFF
cmake --build build-nouring -j $(nproc)
if grep -q '^LIBE3_WITH_URING:INTERNAL=ON' build-nouring/CMakeCache.txt; then
echo "LIBE3_ENABLE_URING=OFF still built the io_uring link"; exit 1
fi
cd build-nouring && ctest --output-on-failure

swig:
name: SWIG Python Bindings
runs-on: ubuntu-latest
Expand Down
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

## What libe3 is

libe3 is a vendor-neutral C++17 library that implements the E3AP (E3 Application Protocol) on both sides of the O-RAN E3 interface: the RAN function (server) and the dApp (client). A single entry-point class backs both roles, switched by one config field. Transport (ZeroMQ or POSIX TCP/SCTP/Unix-domain IPC) and encoding (ASN.1 APER, JSON, and Protocol Buffers) are pluggable and selected at runtime. The data path is built for sub-millisecond, inference-driven control loops (lock-free queues, dedicated I/O threads, optional CPU pinning). It ships a C API and Python (SWIG) bindings.
libe3 is a vendor-neutral C++17 library that implements the E3AP (E3 Application Protocol) on both sides of the O-RAN E3 interface: the RAN function (server) and the dApp (client). A single entry-point class backs both roles, switched by one config field. Transport (ZeroMQ, or POSIX TCP/SCTP/Unix-domain IPC driven by plain system calls or by io_uring) and encoding (ASN.1 APER, JSON, and Protocol Buffers) are pluggable and selected at runtime. The data path is built for sub-millisecond, inference-driven control loops (lock-free queues, dedicated I/O threads, optional CPU pinning). It ships a C API and Python (SWIG) bindings.

## Build & test commands

Expand Down Expand Up @@ -49,7 +49,7 @@ ctest -LE integration # only unit tests
PYTHONPATH=build/swig python3 tests/test_swig_smoke.py # SWIG bindings smoke test
```

**Build options** live in `cmake/libe3Options.cmake`: `LIBE3_BUILD_TESTS` (ON), `LIBE3_BUILD_EXAMPLES` (ON), `LIBE3_BUILD_INTEGRATION_TESTS` (OFF), `LIBE3_ENABLE_SWIG` (OFF), `LIBE3_ENABLE_ZMQ` (ON), `LIBE3_ENABLE_ASN1` (ON), `LIBE3_ENABLE_E2SM_DAPP` (ON, needs ASN1; builds the E2SM-DAPP codec libraries, see `docs/e2sm-dapp/`), `LIBE3_ENABLE_JSON` (OFF), `LIBE3_ENABLE_PROTOBUF` (OFF), `LIBE3_ENABLE_ASAN`/`LIBE3_ENABLE_TSAN` (OFF), `LIBE3_BUILD_DOCS` (OFF). **At least one of `LIBE3_ENABLE_ASN1` / `LIBE3_ENABLE_JSON` / `LIBE3_ENABLE_PROTOBUF` must be ON**; any combination can be built together and the encoder is chosen at runtime. `build_libe3` has a dedicated `--enable-protobuf` flag; JSON and SWIG use `--cmake-opt`.
**Build options** live in `cmake/libe3Options.cmake`: `LIBE3_BUILD_TESTS` (ON), `LIBE3_BUILD_EXAMPLES` (ON), `LIBE3_BUILD_INTEGRATION_TESTS` (OFF), `LIBE3_ENABLE_SWIG` (OFF), `LIBE3_ENABLE_ZMQ` (ON), `LIBE3_ENABLE_URING` (ON when Linux and liburing allow it), `LIBE3_ENABLE_ASN1` (ON), `LIBE3_ENABLE_E2SM_DAPP` (ON, needs ASN1; builds the E2SM-DAPP codec libraries, see `docs/e2sm-dapp/`), `LIBE3_ENABLE_JSON` (OFF), `LIBE3_ENABLE_PROTOBUF` (OFF), `LIBE3_ENABLE_ASAN`/`LIBE3_ENABLE_TSAN` (OFF), `LIBE3_BUILD_DOCS` (OFF). **At least one of `LIBE3_ENABLE_ASN1` / `LIBE3_ENABLE_JSON` / `LIBE3_ENABLE_PROTOBUF` must be ON**; any combination can be built together and the encoder is chosen at runtime. `build_libe3` has a dedicated `--enable-protobuf` flag; JSON and SWIG use `--cmake-opt`.

## Architecture (big picture)

Expand All @@ -58,7 +58,7 @@ The design has one façade over three pluggable seams, plus a real-time data pat
**Dual-role façade.** `E3Agent` (`include/libe3/e3_agent.hpp`) is the single public class for **both** the RAN and dApp roles; the role is chosen by `E3Config.role` (RAN vs DAPP), not by a separate class. It is a Pimpl over the internal `E3Interface` (`src/core/e3_interface.cpp`), which coordinates the full protocol lifecycle (Setup → Subscription → Control/Indication → Release) and owns the worker threads and queues. `E3Config` (in `types.hpp`) is the one struct that selects role, transport, encoding, endpoints, and real-time tuning.

**Three pluggable seams**, each behind an interface with a factory:
- **Transport** — `E3Connector` (`include/libe3/e3_connector.hpp`), implemented by the ZeroMQ and POSIX (TCP/SCTP/IPC) connectors in `src/connector/`, built by `connector_factory.cpp`.
- **Transport** — `E3Connector` (`include/libe3/e3_connector.hpp`), implemented by the ZeroMQ, POSIX and io_uring (TCP/SCTP/IPC) connectors in `src/connector/`, built by `connector_factory.cpp`. The POSIX and io_uring connectors share `socket_link.{hpp,cpp}` (endpoint, socket options, framing) so they stay the same wire; `uring_ring.cpp` is the only file that includes `<liburing.h>`. See `docs/connectors/uring.md`.
- **Encoding** — `E3Encoder` (`include/libe3/e3_encoder.hpp`), implemented by the encoders in `src/encoder/` (ASN.1 APER, JSON, and Protocol Buffers), built by `encoder_factory.cpp`.
- **Service Model** — `ServiceModel` (`include/libe3/sm_interface.hpp`) is the RAN-side extension point for opaque, application-defined payload semantics. Its lifecycle is subscription-driven (`start()` on first subscriber to a RAN function, `stop()` on the last), managed via `SmRegistry` (`src/core/sm_registry.cpp`).

Expand Down
39 changes: 35 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@
- **Vendor-Neutral API**: Clean E3Agent facade that hides implementation details
- **Both Roles**: Single `E3Agent` class acts as either **RAN** (server, binds sockets) or **dApp** (client, connects to a remote RAN), switched by `E3Config.role`
- **Multi-Peer dApps**: One dApp process can hold N `E3Agent` instances connecting to N different RAN agents (e.g. a DU-HIGH and a DU-LOW)
- **Multiple Transports**: Support for ZeroMQ and POSIX sockets (TCP, SCTP, Unix Domain)
- **Multiple Transports**: Support for ZeroMQ and POSIX sockets (TCP, SCTP, Unix Domain), the latter
also driven through io_uring on Linux (`E3LinkLayer::URING`, the same wire; see
[docs/connectors/uring.md](docs/connectors/uring.md))
- **Multiple Encodings**: ASN.1 APER (primary), JSON, and Protocol Buffers encoders
- **Service Model Extensions**: Easy-to-implement SM interface for custom functionality
- **Python Bindings**: Optional SWIG bindings (`LIBE3_ENABLE_SWIG=ON`) so the same C++ library can back Python dApp clients
Expand All @@ -34,7 +36,7 @@
▼ ▼
┌───────────────────┐ ◄─── E3AP PDUs ───► ┌──────────────────────┐
│ E3Agent │◄──────────────────────────►│ E3Agent │
│ │ ZMQ / POSIX │ │
│ │ ZMQ / POSIX / io_uring │ │
│ register_sm(...) │ IPC · TCP · SCTP │ set_indication_ │
│ send_indication │ ASN.1 · JSON · Protobuf │ handler(fn) │
│ send_xapp_ctrl │ │ subscribe(rfid, …) │
Expand Down Expand Up @@ -63,7 +65,8 @@
├────────────────────┤ ├──────────────────────┤ │ extension │
│ ZmqConnector │ │ Asn1Encoder │ │ point) │
│ PosixConnector │ │ JsonEncoder │ ├──────────────────┤
└────────────────────┘ └──────────────────────┘ │ SimpleSM │
│ UringConnector │ │ ProtobufEncoder │ │ SimpleSM │
└────────────────────┘ └──────────────────────┘ │ │
│ custom SMs … │
└──────────────────┘
```
Expand All @@ -80,6 +83,7 @@
- `protobuf-compiler` + `libprotobuf-dev` — `protoc` and the Protocol Buffers runtime (only when `LIBE3_ENABLE_PROTOBUF=ON`)
- `libzmq3-dev` for ZMQ transport
- `libsctp-dev` — SCTP headers for the POSIX/SCTP transport (only when `LIBE3_ENABLE_SCTP=ON`; **not installed by `./build_libe3 -I`**)
- `liburing-dev` (2.0 or later, Linux) for the io_uring link; without it the build goes on without the link

### Install Dependencies

Expand Down Expand Up @@ -170,6 +174,7 @@ pkg-config --modversion libe3
| `LIBE3_ENABLE_TSAN` | OFF | Enable ThreadSanitizer |
| `LIBE3_ENABLE_SWIG` | OFF | Build the SWIG-generated Python bindings (`_libe3py.so` + `libe3py.py`) |
| `LIBE3_ENABLE_SCTP` | OFF | Enable the SCTP POSIX transport (needs `libsctp-dev`) |
| `LIBE3_ENABLE_URING` | ON | Build the io_uring link when it can be: on Linux with liburing found, otherwise the configure says so and goes on without it. `LIBE3_HAS_URING` (0 or 1) tells code and consumers which happened |

> **Note on SCTP:** the transport is **off by default**. It is the only dependency that needs a
> kernel module plus a distro `-dev` package for a single system header, and the deployed E3 links
Expand Down Expand Up @@ -346,6 +351,32 @@ private:
agent.register_sm(std::make_unique<MyKpmServiceModel>());
```

### The io_uring link

Set `link_layer = E3LinkLayer::URING` (`--link uring` in the examples, `2` in the C API). It speaks the POSIX
wire, so it works against a RAN or a dApp on `posix`. It needs a libe3 built with liburing (`liburing-dev`),
Linux 5.11 or later, and a kernel that lets the process create a ring. Where it does not, `create_connector`
returns null and the log says which of these is missing; nothing falls back to another link.

Red Hat Enterprise Linux 9 and OpenShift's nodes ship with io_uring switched off for everyone
(`kernel.io_uring_disabled=2`). The least that has to be opened is one group, for the length of the run, as root
on the host or in a privileged pod:

```bash
GID=61234 # a group nothing else on the node uses
sysctl -w kernel.io_uring_group=$GID # set it first
sysctl -w kernel.io_uring_disabled=1 # 0 = everyone, 1 = root and that group, 2 = nobody

setpriv --reuid=$UID --regid=$GID --groups=$GID --inh-caps=-all --bounding-set=-all --no-new-privs \
./example_simple_agent --link uring --transport tcp --encoding asn1 # a member of the group needs no capability

sysctl -w kernel.io_uring_disabled=2 kernel.io_uring_group=-1 # close it again: the values you read before
```

The setting is the host kernel's, not the container's, and it applies to every container on the node. Ring
options, the checks that it took, a `trap` that closes it when a run fails, pods, and persisting it are in
[docs/connectors/uring.md](docs/connectors/uring.md#enabling-io_uring-on-a-host-that-disables-it).

### Notes on Simulation / Local Testing

The library exposes IPC endpoints by default (see `E3Config` defaults) so
Expand Down Expand Up @@ -410,7 +441,7 @@ for local development (IPC) or production deployments.
| `dapp_version` | string | dApp version advertised in `SetupRequest` |
| `vendor` | string | Vendor name for `SetupRequest` |
| `e3ap_version` | string | E3AP protocol version string |
| `link_layer` | E3LinkLayer | ZMQ or POSIX |
| `link_layer` | E3LinkLayer | ZMQ, POSIX or URING. URING is the POSIX link's wire driven through io_uring: every field below that names the POSIX link applies to it too, and its ring options come from `LIBE3_URING_*` environment variables ([docs/connectors/uring.md](docs/connectors/uring.md)) |
| `transport_layer` | E3TransportLayer | SCTP, TCP, IPC |
| `setup_endpoint` | string | Setup connection endpoint (default: ipc:///tmp/dapps/setup). The POSIX link serves everything on this one endpoint, with the IPC path /tmp/dapps/e3 while the field keeps its default; for TCP and SCTP `tcp://host:port` or `sctp://host:port` names the host and port |
| `subscriber_endpoint` | string | Subscriber endpoint, ZeroMQ only (default: ipc:///tmp/dapps/dapp_socket) |
Expand Down
Loading
Loading