Remove task parameter

[djdameln]

Description

• This PR removes the config.dataset.task parameter, which was redundant.

• The data modules now check if ground truth mask annotations are available to determine the task type.

• The anomaly module now checks if the pixel metrics have been updated before attempting to compute the metrics.

• Fixes Get rid of task_type parameter #34

Changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist

• My code follows the pre-commit style and check guidelines of this project.
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing tests pass locally with my changes

[alexriedel1]

I think this will have a negative impact on the workflow when using anomalib.
When doing experiments on say MVTec and want to quickly change between classification and segmentation with the same model, i will have to remove the ground truth mask directory right?

[LukasBommes]

What about resolving this problem with an optional 'ignore_masks' flag in the config?

[djdameln]

@alexriedel1 @LukasBommes Thanks for your feedback.

Our motivation for removing this parameter was that the task type does not actually affect the training process, but just the post-processing steps. Most models in anomalib are essentially segmentation models that predict a pixel-level anomaly map. The image-level classification predictions are obtained by aggregating in some way these pixel-level anomaly scores (usually this is done by taking the maximum pixel value). Since we have the anomaly maps available anyway, it just takes a few postprocessing steps (apply threshold, compute performance) to also obtain the segmentation predictions and the corresponding performance.

Of course there are also some models in Anomalib that do not generate pixel-level predictions (DFKDE, DFM). Those will however not be affected by this PR, because the segmentation predictions and performance is only computed when anomaly maps are present in the models output dictionary.

So our reasoning was that, since the pixel-level anomaly scores are computed anyway, we might as well use them to show the segmentation results. The image-level results will not be affected by the computation of the pixel-level results, so users who are only interested in the classification performance could safely ignore the pixel-level predictions and performance.

If you see any problems with this approach, could you please provide some more details about your usecase and why it would be problematic for you if the pixel-level predictions and performance are generated based on availability rather than an explicit parameter?

[samet-akcay]

@djdameln, it might be an idea if we take into account this issue #183 as well. There could be a usecase, where the users want to be able to choose any metric they want. In this case, we might need to reconsider hardcoded metrics here. Any thoughts?

[alexriedel1]

If you see any problems with this approach, could you please provide some more details about your usecase and why it would be problematic for you if the pixel-level predictions and performance are generated based on availability rather than an explicit parameter?

Yes, actually you're right I guess. I'm just a fan of explicity (opposed to implicitly define the task if no mask dir is set..)

[djdameln]

Closing this PR after feedback from the community pointed out that this change was not desirable. We will look for another way to make the task_type parameter less confusing.