Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ap calculation bug #8464 #8484

Merged
merged 3 commits into from
Jul 7, 2022
Merged

Conversation

UnglvKitDe
Copy link
Contributor

@UnglvKitDe UnglvKitDe commented Jul 5, 2022

πŸ› οΈ PR Summary

Made with ❀️ by Ultralytics Actions

🌟 Summary

Refinement of validation stats recording in YOLOv5's evaluation script.

πŸ“Š Key Changes

  • Altered the structure of the stats list to include label class indices.

🎯 Purpose & Impact

  • 🎩 This change enables more detailed analysis of performance by allowing evaluators to see not only if a prediction was correct, but which class it was correct for.
  • πŸ” It can improve debugging and enhance the granularity of model evaluation, potentially impacting users by providing deeper insights into model performance per class.
  • πŸ‘ Users making use of the validation outputs for detailed analysis will benefit from more informative statistics.

@UnglvKitDe
Copy link
Contributor Author

Before:
image

After:
image

@glenn-jocher
Copy link
Member

@UnglvKitDe got it, thanks for the PR! So with the default settings it appears results are unchanged correct?

Seems good to merge.

@UnglvKitDe
Copy link
Contributor Author

@UnglvKitDe got it, thanks for the PR! So with the default settings it appears results are unchanged correct?

Seems good to merge.

@glenn-jocher In this case, it does not look like that. In the general case, it leads to a "reduction" of the AP because you have more total positives, but so the AP calculation would be wrong. But it is generally very rare that no prediction is made for an image with a label.

@glenn-jocher glenn-jocher linked an issue Jul 7, 2022 that may be closed by this pull request
2 tasks
@glenn-jocher glenn-jocher merged commit 39d7a93 into ultralytics:master Jul 7, 2022
@glenn-jocher
Copy link
Member

@UnglvKitDe yes I understand. PR is merged. Thank you for your contributions to YOLOv5 πŸš€ and Vision AI ⭐

@glenn-jocher glenn-jocher removed the TODO label Jul 7, 2022
Shivvrat pushed a commit to Shivvrat/epic-yolov5 that referenced this pull request Jul 12, 2022
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
ctjanuhowski pushed a commit to ctjanuhowski/yolov5 that referenced this pull request Sep 8, 2022
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Val.py: No prediction on image follows in wrong AP calculation
2 participants