Skip to content

fix(network/shape): bandwidth-shaper unit runs before netplan-created egress devices exist and never retries #980

Description

@brunodam

Problem

solo-provisioner-bandwidth-shaper.service is ordered After=network-pre.target Before=network.target. Physical NICs exist by then (udev settles during sysinit.target), but devices created by netplan / systemd-networkd — a bond, bridge, or VLAN — do not: systemd-networkd.service is itself ordered After=network-pre.target, so it has not yet created them when the shaper unit runs.

On such a host the boot replay fails:

Cannot find device "bond0"

solo-provisioner-bandwidth-shaper.sh runs under set -e, and while the teardown line is guarded (tc qdisc del dev "$NIC" root 2>/dev/null || true) the build lines are not:

tc qdisc add dev "$NIC" root handle 1: htb default {{.Device.DefaultMinor}}
tc class  add dev "$NIC" parent 1:   classid 1:1  htb rate "{{.Device.Rate}}" ceil "{{.Device.Rate}}"

The unit is Type=oneshot with no Restart=, so nothing retries. The egress NIC stays completely unshaped for the life of the boot, and because Before= is ordering-only (not a requirement) solo-provisioner-daemon.service starts anyway and reports healthy — the daemon only reconciles the per-pod $VETH hierarchy and nft set membership, never $EGRESS. The failure is visible only in systemctl status solo-provisioner-bandwidth-shaper.service.

Related, smaller: the unit sets DefaultDependencies=no without Wants=network-pre.target, so the After=network-pre.target is inert unless something else pulls that target into the boot transaction.

Proposed fix

Render the unit with a dependency on the configured egress device rather than on a network target — After=sys-subsystem-net-devices-<nic>.device plus Wants=/BindsTo= as appropriate — or move it to After=network-online.target and accept the later start. Either way add a bounded retry (Restart=on-failure with RestartSec=, or a wait-for-device loop in the script) so a transient miss self-heals, and keep Before=solo-provisioner-daemon.service.

The device name is already known at render time (--egress-interface), so the unit template can name it directly.

Acceptance

  • On a host whose egress interface is a netplan-created bond, bridge, or VLAN, the $EGRESS HTB hierarchy is present after a reboot (tc class show dev <nic> shows the trunk and leaf classes).
  • A transient device-not-ready failure is retried rather than leaving the NIC unshaped for the whole boot.
  • The rendered unit pulls its ordering target into the transaction (no inert After=).
  • Unit-render test covering a virtual device name.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions