Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Threshold values are ignored when using non adaptive threshold #139

Closed
lorenzomammana opened this issue Mar 9, 2022 · 2 comments · Fixed by #152
Closed

Threshold values are ignored when using non adaptive threshold #139

lorenzomammana opened this issue Mar 9, 2022 · 2 comments · Fixed by #152
Assignees
Labels
Bug Something isn't working

Comments

@lorenzomammana
Copy link

Describe the bug
If in the config file I disable the adaptive threshold to use my custom threshold that gives a better tradeoff compared to the adaptive one, the values that I specify are ignored and the default threshold of 0.5 if used (even if I'm not using minmax or cdf).

Looking at the code the values of my custom threshold are used only to initialize the AdaptiveThreshold object

self.image_threshold = AdaptiveThreshold(self.hparams.model.threshold.image_default).cpu()
self.pixel_threshold = AdaptiveThreshold(self.hparams.model.threshold.pixel_default).cpu()

But then only the values from

self.image_metrics.F1.threshold
self.pixel_metrics.F1.threshold

Are used, which are only updated when the adaptive_threshold flag is set to True, otherwise are just set as 0.5 as by F1 default.

To Reproduce
Perform a training with no anomaly score normalization, disabling adaptive threshold and setting values of your own, add debug logs to check which value of the threshold are being used, it should use 0.5 while ignoring own values.

Expected behavior
If no adaptive threshold is set and model.threshold.image_default or model.threshold.pixel_default, these values should be the one used to compute metrics and to reconstruct images.

Additional context
I'm using the updated development branch

@samet-akcay
Copy link
Contributor

Hi @lorenzomammana, thanks for reporting the issue. We'll have a look at and address the issue at our earliest convenience.

@samet-akcay
Copy link
Contributor

@djdameln, will you be able to look at this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants