Skip to content

development.md

Latest commit

 

History

History
276 lines (205 loc) · 7.25 KB

development.md

File metadata and controls

276 lines (205 loc) · 7.25 KB
Raw

Development

Setup

brew install meson qemu
python3 -m venv .venv
source .venv/bin/activate
pip install pytest scapy pyyaml black matplotlib

Building

./build.sh build

This creates a universal binary (arm64 + x86_64) at build/vmnet-helper.tar.gz.

Installing

To install from a locally built tarball:

./install.sh build/vmnet-helper.tar.gz

Tests

To run the tests, activate the virtual environment first:

source .venv/bin/activate
pytest -v

Formatting

./fmt.sh

Benchmarking

Activate the virtual environment first:

source .venv/bin/activate

Create VMs for benchmarking (see examples for details on the run tool):

./bench create

To run all benchmarks with all drivers and all operation modes and store iperf3 results in json format use:

./bench run performance/benchmarks/full.yaml

The benchmark results are stored under out/bench/vmnet-helper.

See the benchmarks directory for additional configurations.

When done you can delete the vms using:

./bench delete

Creating plots

To create plots from benchmark results run:

./bench plot -o out performance/plots/drivers.yaml

The plots use the results stored under out/bench and created under out/plot.

See the plots directory for additional configurations.

socket_vmnet

Running socket_vmnet as launchd service, creating virtual machines with lima 1.0.6.

Tests run using socket_vmnet test/perf.sh script:

test/perf.sh create
test/perf.sh run

To include socket_vmnet results in the plots copy the test results to the output directory:

cp ~/src/socket_vmnet/test/perf.out/socket_vmnet out/bench/

run: start a virtual machine for testing

The run script starts vmnet-helper and a Linux virtual machine. It allows for quick integration testing with various distributions and helper options. All helper options are supported. By default, the script starts a Ubuntu 26.04 VM on a shared network. To see all options, use ./run -h.

% ./run test
[   0.035] INFO Starting vmnet-helper for 'test' with interface id '83fa1a6e-13ec-408f-ae44-2c26bc31
7160'
[   0.115] INFO Creating image '/Users/user/.vmnet-helper/vms/test/disk.img'
[   0.121] INFO Creating cloud-init iso '/Users/user/.vmnet-helper/vms/test/cidata.iso'
[   0.128] INFO Starting 'vfkit' virtual machine 'test' with mac address '1a:ad:75:f7:ca:a2'
[   0.128] INFO Creating ssh config '/Users/user/.vmnet-helper/vms/test/ssh.config'
[  17.123] INFO VM is ready at test-vmnet-helper.local

Virtual machine resources can be customized. The following example sets 4 vcpus and 4 GiB of memory:

% ./run test --cpus 4 --memory 4096

By default, run uses vfkit, connected to the helper using a file descriptor. The following example uses the qemu driver, and connects using vmnet-run:

% ./run test --driver qemu --connection runner
[   0.031] INFO Starting 'qemu' virtual machine 'test' with mac address '9a:a9:fe:3c:db:46'
[  16.630] INFO VM is ready at test-vmnet-helper.local

VMs use DHCP by default. To assign a static IP address, restrict the DHCP range, then select an address outside of that range:

% ./run test \
    --start-address 192.168.200.1 \
    --end-address 192.168.200.127 \
    --subnet-mask 255.255.255.0 \
    --ip-address 192.168.200.128

Note

Setting --ip-address to a value inside the DHCP range may work, but may cause conflicts.

Note

--ip-address must be difrerent from --start-address, but in the same subnet.

When changing a VM's IP address or switching to DHCP, the instance ID and host key will be reset. Remove the old host key before you ssh again:

% ssh-keygen -R test-vmnet-helper.local
# Host test-vmnet-helper.local found: line 161
/Users/user/.ssh/known_hosts updated.
Original contents retained as /Users/user/.ssh/known_hosts.old

Performance tuning

By default, VMs use interrupt-driven packet processing. During high throughput TX benchmarks (host sending to the VM), ksoftirqd saturates a single CPU core in the guest, limiting throughput to ~30 Gbps. RX throughput (VM sending to the host) is not affected since the host does the heavy packet processing.

The --busy-poll option enables busy polling in the VM, shifting packet processing from ksoftirqd to the application threads:

% ./run test --busy-poll

With busy polling, TX throughput improves to ~37 Gbps. This works by configuring net.core.busy_poll and net.core.busy_read via cloud-init. When the application calls recv() or poll() and no data is ready, the kernel polls the virtio RX ring for up to 50 microseconds before falling back to interrupt-driven processing.

Busy polling increases CPU usage when the VM is idle, so the default (without --busy-poll) represents the expected performance for general purpose workloads. Use --busy-poll for benchmarking to measure vmnet-helper TX throughput without guest-side bottlenecks.

Note

Requires CONFIG_NET_RX_BUSY_POLL=y in the guest kernel. This is enabled by default in Ubuntu, Fedora, Debian, and Alpine.

Storage and debugging

The script downloads cloud images to ~/.vmnet-helper/cache/images, converts them to RAW, and uses APFS reflinks to provision individual VM images. Each VM has a directory under ~/.vmnet-helper/vms for storage, configuration, and logs.

For a given VM $VM and driver $DRIVER, the following logs are available for debugging relative to the VM storage directory ~/.vmnet-helper/$VM:

  • serial.log: the output of the VM's serial console
  • $DRIVER.command: the driver command used to start the VM
  • $DRIVER.log: any logs from the hypervisor driver
  • vmnet-helper.log: logs from vmnet-helper itself
% tree ~/.vmnet-helper/vms/test
/Users/tofugarden/.vmnet-helper/vms/test
├── cidata.iso
├── disk.img
├── efi-variable-store
├── meta-data
├── network-config
├── serial.log
├── ssh.config
├── user-data
├── vfkit.command
├── vfkit.log
└── vmnet-helper.log

stats: analyze vmnet-helper stats

The stats script parses vmnet-helper logs and computes per-interval deltas for network counters. It requires the helper to be started with --stats-interval SECONDS.

Start a VM with stats enabled:

% ./run test --stats-interval 1

Show human-readable stats (transfer, bitrate, packets, drops):

% ./stats ~/.vmnet-helper/vms/test/vmnet-helper.log
- time: 2026-07-25T01:27:25.664+03:00
  host:
    transfer: 26.25 GBytes
    bitrate: 21.00 Gbits/sec
    packets: 69383 pkt/sec
    drops: 0
  vm:
    transfer: 20.81 GBytes
    bitrate: 16.65 Gbits/sec
    packets: 52335 pkt/sec
    drops: 0

Show raw JSON deltas for programmatic processing:

% ./stats -o json ~/.vmnet-helper/vms/test/vmnet-helper.log | jq .

Limitations

The qemu driver is not compatible with --connection set to socket.

Multicast DNS setup requires network access. To use mDNS with --operation-mode set to host, create the VM in shared mode, then restart it in host mode.