-
Notifications
You must be signed in to change notification settings - Fork 639
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add metric visualizations #429
Add metric visualizations #429
Commits on Jul 11, 2022
-
Refactor
VisualizerCallback
, fix Wandb bug* `VisualizerCallback` is refactored into a base `VisualizerCallbackBase` class which is used to construct `VisualizerCallbackImage` via inheritance. * Fix bug in `VisualizerCallback.on_test_end`, where `pl_module.logger` was accessed instead of iterating over `trainer.loggers` (wandb errort silently before iirc) * Skeleton for `VisualizerCallbackMetric` is added
Configuration menu - View commit details
-
Copy full SHA for bc793aa - Browse repository at this point
Copy the full SHA bc793aaView commit details -
Configuration menu - View commit details
-
Copy full SHA for cfd92b1 - Browse repository at this point
Copy the full SHA cfd92b1View commit details
Commits on Jul 12, 2022
-
Log Figure with built-in SummaryWriter funtcion
PyTorch provides a function for logging Figure objects, so lets not reinvent the wheel. https://pytorch.org/docs/stable/tensorboard.html?highlight=add_figure#torch.utils.tensorboard.writer.SummaryWriter.add_figure
Configuration menu - View commit details
-
Copy full SHA for 302a728 - Browse repository at this point
Copy the full SHA 302a728View commit details -
Configuration menu - View commit details
-
Copy full SHA for fe19315 - Browse repository at this point
Copy the full SHA fe19315View commit details -
Explicitly close figure in
ImageGrid.generate
Otherwise, matplotlib starts to complain and objects aren't gced properly
Configuration menu - View commit details
-
Copy full SHA for b4bbde7 - Browse repository at this point
Copy the full SHA b4bbde7View commit details -
Shift
Visualizer
toVisualizerCallbackBase
`Visualizer` is needed by all Callbacks for writing images to disk
Configuration menu - View commit details
-
Copy full SHA for c0d7f59 - Browse repository at this point
Copy the full SHA c0d7f59View commit details -
Add first metric visualization
Every metric that should be visualized needs to implement its own `generate_figure` function, and the resulting plot is then saved by `VisualizerCallbackMetric`
Configuration menu - View commit details
-
Copy full SHA for 6fe5999 - Browse repository at this point
Copy the full SHA 6fe5999View commit details -
Merge branch 'feature/metrics_visualizer' of github.com:ORippler/anom…
…alib-1 into feature/metrics_visualizer
Configuration menu - View commit details
-
Copy full SHA for cee564f - Browse repository at this point
Copy the full SHA cee564fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7b20171 - Browse repository at this point
Copy the full SHA 7b20171View commit details -
Configuration menu - View commit details
-
Copy full SHA for 01286f4 - Browse repository at this point
Copy the full SHA 01286f4View commit details -
Configuration menu - View commit details
-
Copy full SHA for f9522a0 - Browse repository at this point
Copy the full SHA f9522a0View commit details -
Add AUPRO metric and its vizualiation
* Needed to change signature of `AnomalyModule._collect_outputs` for pixel-wise metrics as we need spatial dimensions for connected-componenta-analysis in AUPRO * Add AUPRO, which uses kornia and the fact that per-region overlap == per-region tpr for fast AUPRO computation * Updated docstrings for AUPR/AUROC
Configuration menu - View commit details
-
Copy full SHA for 629056b - Browse repository at this point
Copy the full SHA 629056bView commit details -
Adjust tests, CLI and notebooks
Due to bugs in CLI, feature could not be tested well
Configuration menu - View commit details
-
Copy full SHA for bbbb188 - Browse repository at this point
Copy the full SHA bbbb188View commit details
Commits on Jul 13, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 2b6a391 - Browse repository at this point
Copy the full SHA 2b6a391View commit details -
Bugfix of AUPRO implementation
Since unique fpr/tpr curves are generated for each label, we cannot use fpr_index across calls ro `roc`. Instead, we now bilinearly resample generated pro curves at `fpr <= self.fpr_limit` to fixed sampling points, and then aggregate over the resampled curve.
Configuration menu - View commit details
-
Copy full SHA for f69b719 - Browse repository at this point
Copy the full SHA f69b719View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0da986f - Browse repository at this point
Copy the full SHA 0da986fView commit details -
Improve variable naming/wording in AUPRO
We follow a scheme similar to the ROC curve, where the PRO curve is composed of per-region TPR plotted against global FPR.
Configuration menu - View commit details
-
Copy full SHA for 30b1c17 - Browse repository at this point
Copy the full SHA 30b1c17View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4c5a795 - Browse repository at this point
Copy the full SHA 4c5a795View commit details -
Configuration menu - View commit details
-
Copy full SHA for 87b3a2a - Browse repository at this point
Copy the full SHA 87b3a2aView commit details -
We now prepend the last part, i.e. `VisualizerCallbackBase` -> `BaseVisualizerCallback`
Configuration menu - View commit details
-
Copy full SHA for 469f4ef - Browse repository at this point
Copy the full SHA 469f4efView commit details -
Configuration menu - View commit details
-
Copy full SHA for e2cf6c2 - Browse repository at this point
Copy the full SHA e2cf6c2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1dacd8a - Browse repository at this point
Copy the full SHA 1dacd8aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 43d8e4a - Browse repository at this point
Copy the full SHA 43d8e4aView commit details