Report bundle deployment state as Kubernetes events

[0xavi0]

Fleet exposes deployment failures only in bundle status, which is easy to miss. This adds events on the bundle when its deployments fail, and when they all become ready again, so kubectl describe bundle and any event- based tooling can see them.

Events are created directly rather than through client-go's recorder: the recorder keeps the note of the first event in a series, which would drop notes describing different failures of the same bundle. Deduplication happens on a fingerprint of the failure causes and how many deployments are affected instead, so a burst collapses into one accurate event, and comparing against the last persisted status keeps that true across controller restarts.

Configurable via the deploymentEvents block in the fleet chart: debounce, minimum interval per object, recovery reporting, how many causes an event names, and optional per-bundle-deployment events (off by default). Reporting is on by default and can be turned off entirely.

Refers to: #4455

Additional Information

Checklist

• I have updated the documentation via a pull request in the fleet-product-docs repository.

[Copilot]

Pull request overview

Adds Kubernetes Events reporting for bundle deployment failures (and optional recovery/per-deployment reporting) so failures are visible via kubectl describe and event tooling, with burst debouncing and deduplication to avoid event floods.

Changes:

• Introduces bundleevents emitter to generate aggregated Bundle events (and optional per-BundleDeployment events) with debounce/min-interval behavior.
• Wires bundle/bundledeployment reconcilers and controller operator to observe persisted status transitions and emit/forget events appropriately.
• Adds deploymentEvents chart/config support and improves duration sanitization to handle nested duration fields.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
pkg/durations/durations.go	Adds default debounce/min-interval durations for bundle deployment events.
internal/config/config.go	Adds deploymentEvents config block and sanitizes nested duration keys.
internal/config/config_test.go	Tests nested duration sanitization behavior for deploymentEvents.
internal/cmd/controller/reconciler/bundledeployment_controller.go	Notifies event emitter after persisted BundleDeployment status; forgets on deletion.
internal/cmd/controller/reconciler/bundle_controller.go	Notifies event emitter after persisted Bundle status summary; forgets on deletion.
internal/cmd/controller/reconciler/bundle_controller_test.go	Adds notifier stub to ensure persisted summaries are observed on early-return paths.
internal/cmd/controller/operator.go	Instantiates and registers the bundleevents runnable; injects into reconcilers.
internal/cmd/controller/bundleevents/options.go	Defines emitter options and config mapping defaults.
internal/cmd/controller/bundleevents/options_test.go	Validates options parsing from rendered config JSON and defaults/disable behavior.
internal/cmd/controller/bundleevents/notifier.go	Implements the event emitter (queueing, deduping, retries, eviction/TTL).
internal/cmd/controller/bundleevents/notifier_test.go	Tests burst aggregation, dedup, recovery reporting, rate limiting, retries, eviction, etc.
internal/cmd/controller/bundleevents/note.go	Builds bounded event notes and fingerprints from bundle/deployment status.
internal/cmd/controller/bundleevents/note_test.go	Tests note formatting, limits, cause bucketing, and UTF-8 truncation.
charts/fleet/values.yaml	Documents and exposes deploymentEvents configuration values.
charts/fleet/values.schema.json	Adds schema for deploymentEvents and allows empty-string durations as “unset”.
charts/fleet/templates/rbac.yaml	Grants permission to create events.k8s.io Events.
charts/fleet/templates/configmap.yaml	Renders deploymentEvents into the controller configmap.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[p-se]

I've tested it and I'm a bit confused about the events it reports:

  Summary:
    Desired Ready:  1
    Ready:          1
  Unavailable:      0
Events:
  Type     Reason                  Age   From               Message
  ----     ------                  ----  ----               -------
  Warning  BundleNotReady          85s   fleet-bundle-ctrl  1/1 bundle deployments failing. fleet-default/bds-events-multi-ds1: NotReady: ;
  Normal   BundleDeploymentsReady  24s   fleet-bundle-ctrl  0/1 bundle deployments ready

Reporting ready when it is not

Normal BundleDeploymentsReady 38m fleet-bundle-ctrl 0/1 bundle deployments ready

I'm confused that it reports ready while it isn't (0/1). Could we resolve this by adding a generic "InProgress" state? Or is that an issue with our states?

No Reason

It looks like there should be a reason for why it isn't ready, but it's missing in all cases where BundleNotReady is emitted.

How I reproduced it

Two clusters, no special timing, the agent stays running and the whole thing takes about a minute.

A bundle that fails, because the image does not exist:

apiVersion: fleet.cattle.io/v1alpha1
kind: Bundle
metadata:
  name: repro-5515
  namespace: fleet-default
spec:
  resources:
    - name: deploy.yaml
      content: |
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: repro-5515
          namespace: default
        spec:
          replicas: 1
          selector:
            matchLabels:
              app: repro-5515
          template:
            metadata:
              labels:
                app: repro-5515
            spec:
              containers:
                - name: app
                  image: registry.invalid/nope:v1
  targets:
    - clusterSelector: {}

A fix that is ready the moment it is applied:

apiVersion: fleet.cattle.io/v1alpha1
kind: Bundle
metadata:
  name: repro-5515
  namespace: fleet-default
spec:
  resources:
    - name: cm.yaml
      content: |
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: repro-5515
          namespace: default
        data:
          fixed: "true"
  targets:
    - clusterSelector: {}

kubectl apply -f broken.yaml   # goes NotReady
sleep 15                       # let the failure event be written
kubectl apply -f fixed.yaml    # ready ~2s later
sleep 60
kubectl -n fleet-default describe bundle repro-5515

[0xavi0]

Thanks for catching this, @p-se! It is indeed a bug when aggregating the BDs.
I'm working on fixing this and also noticed the event reason could be better.

[0xavi0]

I've added changes to report correctly the number of BDs that failed as well as changes for reporting the events reasons.

I can see the events as below now:

  Type     Reason          Age   From               Message
  ----     ------          ----  ----               -------
  Warning  BundleNotReady  69s   fleet-bundle-ctrl  2/2 bundle deployments failing. fleet-default/dev: NotReady: ; fleet-default/prod: NotReady: ;
  Normal   BundleReady     9s    fleet-bundle-ctrl  2/2 bundle deployments ready