Skip to content

feat(daemon): re-assert the weaver nft tables and $EGRESS HTB when a third party removes them #981

Description

@brunodam

Problem

Both nft templates use the scoped add table / delete table / add table idiom rather than flush ruleset, so weaver never destroys another subsystem's rules. But nothing recovers weaver's own state when a third party wipes it after boot, and every such loss is silent.

nftables side:

  • Stock Debian/Ubuntu /etc/nftables.conf begins with flush ruleset. If nftables.service is enabled or restarted at any point, both inet weaver-host-firewall and inet weaver-workload-policy are destroyed.
  • The same applies to a firewalld reload, or any operator running nft -f with a flush at the top.

tc side:

  • netplan apply can recreate the egress device; a driver reload, an ip link replace, or a stray tc qdisc del dev <nic> root removes the HTB hierarchy.

Neither is repaired. The daemon's hourly force-resync goes through policy.Manager.ApplySets, which reconciles set membership and assumes the table exists; the TrafficShaperMonitor covers only the per-pod $VETH hierarchy. There is no code path that re-asserts table presence or the $EGRESS root qdisc.

The impact is invisible from the outside:

  • Host firewall gone → the node's INPUT filter disappears and it is wide open.
  • Workload policy gone → no meta priority stamping, so every block-node flow falls into the HTB default class. Traffic keeps flowing at wire speed; nothing errors, no counter moves, and the shaping guarantee is quietly void.
  • $EGRESS HTB gone → no egress shaping at all, same silence.

Proposed fix

Add a cheap presence check to the daemon on the existing force-resync cadence:

  • nft list table inet weaver-host-firewall / inet weaver-workload-policy — on absence, restart solo-provisioner-network-nft.service (which re-applies the persisted .nft files) and then let the normal statusz reconcile rehydrate set membership.
  • The $EGRESS root qdisc handle — on absence, restart solo-provisioner-bandwidth-shaper.service.

Every re-assertion should log at WARN and be reflected in the daemon's GET /status so the event is attributable rather than inferred.

Acceptance

  • nft flush ruleset on a provisioned node: both weaver tables are restored within one resync interval, with set membership rehydrated.
  • tc qdisc del dev $EGRESS root on a provisioned node: the HTB hierarchy is restored within one resync interval.
  • Each re-assertion is logged at WARN and surfaced in GET /status over daemon.sock.
  • The presence check is cheap enough to run on the force-resync cadence without a measurable cost when nothing is wrong.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions