[ENH] Add the Torsk anomaly detector - #3758
Conversation
aeon-actions-bot
Bot
added
anomaly detection
Thank you for contributing to
|
There was a problem hiding this comment.
Pull request overview
Adds the Torsk echo-state-network (ESN) anomaly detector to aeon.anomaly_detection.series under the new deep_learning subpackage, aligning the implementation and defaults with the TimeEval configuration and exposing it in the anomaly detection API docs.
Changes:
- Introduces
Torskdetector implementation (_torsk.py) with reservoir construction, sliding readout fitting, and normality-based scoring. - Adds a new
deep_learninganomaly detection subpackage with tests (output shape/localisation, multivariate support, determinism, and validation). - Updates the anomaly detection API reference docs to include a new Deep-learning section.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
aeon/anomaly_detection/series/deep_learning/_torsk.py |
Implements the Torsk detector and its scoring logic. |
aeon/anomaly_detection/series/deep_learning/__init__.py |
Exposes Torsk at the deep-learning subpackage level. |
aeon/anomaly_detection/series/deep_learning/tests/test_torsk.py |
Adds unit tests for output, multivariate behavior, determinism, and validation. |
aeon/anomaly_detection/series/deep_learning/tests/__init__.py |
Initializes the deep learning test package. |
docs/api_reference/anomaly_detection.rst |
Documents the new Deep-learning section and Torsk API entry. |
This pull request includes code written with the assistance of AI.
The code has not yet been reviewed by a human.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| if self.spectral_radius <= 0.0: | ||
| raise ValueError("spectral_radius must be positive") | ||
| if self.transient_window_size >= self.train_window_size: | ||
| raise ValueError( | ||
| "transient_window_size must be smaller than train_window_size" | ||
| ) | ||
| if self.normality_small_window < 1 or self.normality_large_window < 1: | ||
| raise ValueError("normality window sizes must be at least 1") | ||
| if not 0.0 <= self.rcond < 1.0: | ||
| raise ValueError("rcond must be in [0, 1)") | ||
|
|
| normality_small_window: int = 10, | ||
| normality_large_window: int = 100, | ||
| rcond: float = 1e-4, | ||
| random_state: int | None = None, | ||
| ): |
Responds to the two automated review comments on aeon-toolkit#3758. prediction_window_size=0 previously reached a division by zero and transient_window_size=train_window_size-1 an IndexError inside the SVD; both now fail fast with a ValueError, as does a negative transient_window_size, which was silently accepted. random_state is annotated int | np.random.RandomState | None to match the docstring and the other detectors.
|
Both points were valid, addressed in a9bece9.
|
Implements Torsk (Heim & Avery 2019, arXiv:1909.01709) as a series anomaly detector following the TimeEval configuration: a fixed random sparse ESN reservoir, a ridge readout refitted at every sliding position, autoregressive prediction, and a Gaussian tail normality score over prediction errors. Deviations from the paper and both reference implementations are documented in the class docstring Notes section. Pure numpy and scipy, no new dependencies. File is unregistered pending the placement decision in aeon-toolkit#1636.
…t#1636) Places the detector under deep_learning per the VLDB TSAD taxonomy discussed in the issue, adds the subpackage, registers Torsk in the API reference, and sets the module maintainer.
Replaces the ridge normal equations solver with a truncated SVD of the design matrix, matching the reference implementation's default pinv_svd path. Forming the Gram matrix squares the condition number of an already underdetermined system, which measurably degrades accuracy and runtime once window_size * n_channels grows large: on Exathlon series with 165 and 239 channels the SVD solver improves ROC AUC from 0.55/0.52 to 0.63/0.64 (reference parity) and is 7 to 11 times faster. Renames tikhonov_beta to rcond, the relative singular value cutoff.
Responds to the two automated review comments on aeon-toolkit#3758. prediction_window_size=0 previously reached a division by zero and transient_window_size=train_window_size-1 an IndexError inside the SVD; both now fail fast with a ValueError, as does a negative transient_window_size, which was silently accepted. random_state is annotated int | np.random.RandomState | None to match the docstring and the other detectors.
a9bece9 to
8f41f80
Compare
Closes #1636
Why
Torsk (Heim and Avery 2019) is an echo state network detector that TimeEval includes in its benchmark set, and #1636 asks for it in aeon. The placement and fidelity questions were settled in the issue thread: it lives under
series/deep_learningfollowing the VLDB TSAD taxonomy, and it follows the TimeEval configuration rather than the full spatial input map stack of the paper.Design and deviations from the paper
The detector keeps a fixed random sparse reservoir and a random input map drawn once in
_fit._predictslides over the series, refits a linear readout at every position, runs it freely forprediction_window_sizeframes, and turns the prediction errors into a Gaussian tail normality score that is mapped back onto the time points each prediction covers.Every deviation from the paper is listed in the class docstring Notes section, in the same style as
DWT_MLEAD. The ones worth a reviewer's attention:pinv_svdpath. The difference is not cosmetic: on the two Exathlon series with 165 and 239 channels, ridge normal equations scored 0.55 and 0.52 ROC AUC and took 62 and 130 seconds; truncated SVD scores 0.63 and 0.64, which is reference parity, in 8 and 12 seconds.input_scaledepends on that range, so it is kept, and the leak is stated in Notes rather than hidden.The method is strongly seed sensitive. The docstring says so and recommends setting
random_state.What
aeon/anomaly_detection/series/deep_learning/_torsk.py, theTorskclassaeon/anomaly_detection/series/deep_learning/__init__.pyandtests/__init__.py, new subpackageaeon/anomaly_detection/series/deep_learning/tests/test_torsk.py, output, multivariate, determinism, input validation and channel mismatch testsdocs/api_reference/anomaly_detection.rst, new Deep learning sectionNo new dependencies. The core is numpy plus
scipy.sparse,scipy.sparse.linalg.eigsandscipy.special.erf, all already core.check_estimator(Torsk)passes all 22 checks, the doctest runs, and the fullanomaly_detectionsuite is green.One determinism detail worth knowing:
scipy.sparse.linalg.eigsdraws its ARPACK start vector from the global RNG unlessv0is passed, which silently breaksrandom_statereproducibility. The reservoir construction passes a seededv0, andtest_torsk_determinismguards it.Evidence
As asked in the thread, the implementation was evaluated against the TimeEval reference on real benchmark datasets rather than only on synthetic series. Both implementations ran on identical data, 27 datasets from 19 TimeEval collections loaded through
aeon.datasets.load_anomaly_detectionand the curated loaders, 7 seeds each, single threaded.Summary: all 27 datasets agree within the pooled seed standard deviation. A Wilcoxon signed rank test on the per dataset differences gives p = 0.229, so no significant difference between the two implementations. Mean absolute difference in ROC AUC is 0.041. Mean runtime is 1.65 s for this implementation against 4.72 s for the reference, 2.9 times faster, mostly from the solver.
The reference was run from the vendored TimeEval code with marshmallow pinned below 3.13, since that code predates the
defaultfield rename. The evaluation script is self contained and I am happy to attach it or turn it into a benchmarking example if that is useful.Per dataset ROC AUC, mean ± sd over 7 seeds
Open questions
check_estimatorbut by being degenerate rather than by working. Would you rather it return zeros there or raise for series below some minimum length?spectral_radiusdefault. The paper and every nmheim experiment use 1.5; nmheim'sdefault_params.jsonand the TimeEval manifest use 2.0. Benchmarking could not separate them. This ships 2.0 for consistency with the TimeEval configuration, and switching is one line.AI disclosure
AI assistance was used for navigating the codebase, drafting the docstring and test scaffolding, and running the benchmark comparison. The algorithm reading, parameter choices, the deviations list, the solver investigation and every line of the final code were reviewed and run by me. Numbers in the Evidence section are from local runs on this branch.