Repository navigation
Model examples with Early Stopping breaking one-class-classification assumption? #3317
Replies: 2 comments 3 replies
|
@samet-akcay Could you please help assign this discussion to the team? |
|
@DeclanMcIntosh unfortunately that is still an open problem in this field. It affects the threshold selection as well. We use max F1 score to select the pixel-level and image-level thresholds. F1 score needs "anomaly" images from the validation set. The single-label method to go for now would be to use something like a proxy signal for anomalous labels. Anomalib supports synthetic anomaly generation. So you can set the val split to synthetic (https://anomalib.readthedocs.io/en/latest/markdown/guides/reference/data/utils/image_video.html#anomalib.data.utils.split.ValSplitMode.SYNTHETIC). Other option would be to use just the loss for early stopping. The threshold would then be the maximum score on the normal images |
Uh oh!
There was an error while loading. Please reload this page.
The YAML default configs for the models often use early stopping as below, based on image_AUROC or pixel_AUROC. Is this not using the labelled data and breaking the OCC assumption of the original method? Would this not make these methods rely on labelled test or validation data of both nominal and anomalous structures to replicate the results?
Example from examples/configs/model/reverse_distillation.yaml
All reactions