[ENH] Estimator checks let a constant-score or random anomaly detector pass

[mateenali66]

The series anomaly detector checks in _yield_series_anomaly_detection_checks.py verify output shape and dtype but never compare scores to the labels that UNIVARIATE_SERIES_ANOMALY already provides. A detector that returns the same float at every timepoint passes every check today, and so does one that returns random noise. The metric layer already treats constant scores as a failure (roc_auc_score returns 0.0 with a warning), so the estimator checks are weaker than the benchmark metrics that sit beside them. I would like to add one check that fits on the labeled fixture and asserts that scores are not constant, that ranked scores place the labeled anomalies above the prevalence baseline, and for binary detectors that the positive rate is neither zero nor one. Detectors that cannot rank the synthetic fixture would opt out through an EXCLUDED_TESTS entry in aeon/testing/testing_config.py. This is the degenerate-detector guard from the FDES benchmark specification I maintain, which came out of a shipped detector that returned identical scores for 220 days until a user report, not a test, caught it (open-telemetry/opentelemetry-collector-contrib#46988, reported by pathcl). This is separate from #2801, which asks for a _fit_predict consistency check in the same yield function and has been waiting since #2326 merged. If this is welcome I will fold that check into the same PR so both land together. Happy to adjust the fixture or the baseline form to whatever the anomaly module maintainers prefer.

[TonyBagnall]

thanks for this, is a constant anomaly score always wrong? Its not my area, but seems to me if you pass a constant time series, you would expect a constant anomaly score?

[mateenali66]

You are right for that input. A constant series should give a constant score, and the check would not fail it.

The check only runs on the labeled anomaly fixture the test suite already uses, and that series does contain an anomaly. If a detector gives the same score to every point of that series, it cannot rank the anomaly above anything else. The metric layer already treats that as a failure (roc_auc_score returns 0.0 with a warning when all scores are equal). So the rule is not "scores can never be constant". It is "on a series with a known anomaly, the scores have to separate something".

A detector with a good reason to fail this could opt out through EXCLUDED_TESTS in aeon/testing/testing_config.py, like the existing per-estimator exclusions.

I also fixed two mistakes in the issue text above, edited in place. The opt-out is EXCLUDED_TESTS, not _unit_test_skips, which does not exist in aeon. And the shipped detector went unnoticed for 220 days, not 142. The 142 days was the time from report to fix.