Repository navigation
Commit 2187401
Fix SR-CNN missing anomalies when Period > 0 (#7610)
## Root cause
Two domain-mismatch bugs in the `Period > 0` code path of
`SrCnnEntireModeler`:
**1. `Train()` — z-score statistics computed in the wrong domain.**
`_mean` / `_std` were computed from the raw input series *before*
deseasonality ran. The z-score "false anomaly" filter inside
`SpectralResidual` then compared `_seriesToDetect[i]` (the
deseasonalized residual) against those raw-domain stats, producing
meaningless z-scores that suppressed real anomalies.
**2. `GetMarginPeriod()` — `CalculateAnomalyScore` called with `exp` in
the wrong domain.**
`_ifftRe[i]` was passed as the expected-value argument.
`CalculateAnomalyScore` computes `|exp - value| / unit`, which is only
meaningful when both arguments share a domain.
In the no-period sibling `GetMargin()`, this call is correct:
`CalculateExpectedValueByFft(_deAnomalyData)` is invoked immediately
beforehand and overwrites `_ifftRe` with the raw-domain expected value
of the de-anomalized series — so `_ifftRe[i]` and `values[i]` share a
domain there.
`GetMarginPeriod()` was added later (#5202) and reused the same call
shape without that prior overwrite. In the period > 0 path `_ifftRe`
holds the SR saliency map's real component computed on the
deseasonalized residual series, not a raw-domain expected value. The
resulting score collapses to roughly `|raw_value| / unit` — proportional
to raw magnitude rather than deviation from expectation — so real
anomalies often score near zero. This looks like a copy-paste of the
no-period call site without re-deriving what `_ifftRe` actually holds in
the new path.
## Fix
1. Move the `sum` / `_mean` / `_std` computation in `Train()` to *after*
the deseasonality call and compute the statistics against
`_seriesToDetect` (the same series SR will actually score).
2. In `GetMarginPeriod()` pass `results[i][3]` (the raw-domain expected
value just produced by `GetExpectedValuePeriod`) instead of
`_ifftRe[i]`.
The no-period path (`Train` with `_period == 0` and `GetMargin`) is
correct as written and is unchanged.
## Test
Adds `TestSrCnnAnomalyDetectorPhoneCalls`, ported from the
[dotnet/samples PhoneCallsAnomalyDetection
tutorial](https://learn.microsoft.com/dotnet/machine-learning/tutorials/phone-calls-anomaly-detection).
Data file is taken verbatim from
[dotnet/samples](https://github.com/dotnet/samples/tree/main/machine-learning/tutorials/PhoneCallsAnomalyDetection/Data).
The test asserts `DetectSeasonality` returns 7 and that
`DetectEntireAnomalyBySrCnn` flags exactly indices `{28, 44, 56, 70}`.
Prior to this fix the same call detects zero anomalies on the same
input.
`Sensitivity = 87.0` is used to obtain the boundary width the tutorial
was originally calibrated for under the v1.5.2 `_factors` table; the
table was rewritten in #5436, so the tutorial's literal `Sensitivity =
64.0` now produces a much wider boundary that masks all but the
strongest spike at index 28.
---------
Co-authored-by: Matous Kozak <55735845+matouskozak@users.noreply.github.com>
Co-authored-by: Matous Kozak <matouskozak@seznam.cz>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>1 parent 6b55808 commit 2187401
3 files changed
Lines changed: 163 additions & 9 deletions
File tree
- src/Microsoft.ML.TimeSeries
- test
- Microsoft.ML.TimeSeries.Tests
- data/Timeseries
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
496 | 496 | | |
497 | 497 | | |
498 | 498 | | |
499 | | - | |
500 | | - | |
501 | | - | |
502 | 499 | | |
503 | 500 | | |
504 | 501 | | |
505 | 502 | | |
506 | 503 | | |
507 | 504 | | |
508 | 505 | | |
509 | | - | |
510 | | - | |
511 | 506 | | |
512 | 507 | | |
513 | | - | |
514 | | - | |
515 | | - | |
516 | 508 | | |
517 | 509 | | |
518 | 510 | | |
519 | 511 | | |
520 | 512 | | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
521 | 525 | | |
522 | 526 | | |
523 | 527 | | |
| |||
812 | 816 | | |
813 | 817 | | |
814 | 818 | | |
815 | | - | |
| 819 | + | |
816 | 820 | | |
817 | 821 | | |
818 | 822 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
772 | 772 | | |
773 | 773 | | |
774 | 774 | | |
| 775 | + | |
| 776 | + | |
| 777 | + | |
| 778 | + | |
| 779 | + | |
| 780 | + | |
| 781 | + | |
| 782 | + | |
| 783 | + | |
| 784 | + | |
| 785 | + | |
| 786 | + | |
| 787 | + | |
| 788 | + | |
| 789 | + | |
| 790 | + | |
| 791 | + | |
| 792 | + | |
| 793 | + | |
| 794 | + | |
| 795 | + | |
| 796 | + | |
| 797 | + | |
| 798 | + | |
| 799 | + | |
| 800 | + | |
| 801 | + | |
| 802 | + | |
| 803 | + | |
| 804 | + | |
| 805 | + | |
| 806 | + | |
| 807 | + | |
| 808 | + | |
| 809 | + | |
| 810 | + | |
| 811 | + | |
| 812 | + | |
| 813 | + | |
| 814 | + | |
| 815 | + | |
| 816 | + | |
| 817 | + | |
| 818 | + | |
| 819 | + | |
| 820 | + | |
| 821 | + | |
| 822 | + | |
| 823 | + | |
| 824 | + | |
| 825 | + | |
| 826 | + | |
| 827 | + | |
| 828 | + | |
| 829 | + | |
| 830 | + | |
| 831 | + | |
| 832 | + | |
| 833 | + | |
| 834 | + | |
| 835 | + | |
| 836 | + | |
| 837 | + | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
| 843 | + | |
| 844 | + | |
| 845 | + | |
775 | 846 | | |
776 | 847 | | |
777 | 848 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
0 commit comments