Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panoptic Metrics Bugfix: correctly ignoring void segments #12

Merged
merged 2 commits into from
Jun 27, 2022

Conversation

VSainteuf
Copy link
Owner

Fix for issue #11

Problem

Void instances where not correctly matched to predicted instances. Predicted instances matched with void instances were counted as False Positives instead of being ignored. This results in an artificial decrease of the Recognition Quality.

Fix

Iterate over the correct tensor in l.55 of src/panoptic/metrics.py.

Impact

  • The metrics reported in the original paper are impacted by this error. I re-runed the evaluation of the different methods of the paper. Across methods, the bug fix entails a ~2 point increase of Panoptic Quality. The results will be updated in a new Arxiv version.
  • The panoptic metrics are not involved in the training loss so this bug does not affect training procedure. Models that were trained with the previous implementation just need to be re-evaluated with the corrected metrics.

@VSainteuf VSainteuf merged commit 4628267 into main Jun 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant