Skip to content

Regression in 2.2.8: operator restart may no longer trigger recovery for clusters already OFFLINE at startup #52

Description

@G-NamanGupta

Summary

After upgrading from 2.2.6 to 2.2.8, an existing InnoDBCluster that is already OFFLINE when the operator restarts may remain OFFLINE indefinitely if no further pod status change occurs after operator startup.

Versions

  • works as expected: 2.2.6 (9.5.0-2.2.6)
  • problematic: 2.2.8 (9.7.0-2.2.8)

Test setup

The reproduction I used was with an InnoDBCluster configured with 1 instance.

Steps to reproduce

  1. Deploy MySQL Operator and create an InnoDBCluster with 1 instance
  2. Put the cluster into a state where it is already OFFLINE when the operator comes back up
    • for example, after a node reboot or a simultaneous operator/pod restart scenario
  3. Let the operator restart
  4. Ensure no additional pod status change occurs after startup
  5. Observe the cluster status

Expected behavior

After operator restart, an existing cluster that is already OFFLINE should still be re-evaluated for recovery.

Actual behavior

In 2.2.8, the cluster may remain OFFLINE indefinitely unless a later pod status change happens.

Additional note

I compared the relevant source between 2.2.6 and 2.2.8, and this looks related to a change in the pod status watch / restart detection behavior after operator restart. I can provide the exact code comparison if that would be helpful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions