Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zero-mAP fix remove torch.empty() forward pass in .train() mode #9068

Merged
merged 2 commits into from
Aug 21, 2022

Conversation

0zppd
Copy link
Contributor

@0zppd 0zppd commented Aug 21, 2022

Signed-off-by: 0zppd 111682241+0zppd@users.noreply.github.com

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Upgraded logging to use zero-initialized tensor for improved stability during TensorBoard graph generation.

📊 Key Changes

  • Changed input image initialization from torch.empty() to torch.zeros().

🎯 Purpose & Impact

  • Ensures more consistent initialization by using zero-filled tensors, which can help avoid any potential issues caused by uninitialized memory.
  • The change could lead to more reliable model graph visualization in TensorBoard, benefiting users who analyze model architectures and debug them.

Signed-off-by: 0zppd <111682241+0zppd@users.noreply.github.com>
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👋 Hello @0zppd, thank you for submitting a YOLOv5 🚀 PR! To allow your work to be integrated as seamlessly as possible, we advise you to:

  • ✅ Verify your PR is up-to-date with upstream/master. If your PR is behind upstream/master an automatic GitHub Actions merge may be attempted by writing /rebase in a new comment, or by running the following code, replacing 'feature' with the name of your local branch:
git remote add upstream https://github.com/ultralytics/yolov5.git
git fetch upstream
# git checkout feature  # <--- replace 'feature' with local branch name
git merge upstream/master
git push -u origin -f
  • ✅ Verify all Continuous Integration (CI) checks are passing.
  • ✅ Reduce changes to the absolute minimum required for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." -Bruce Lee

@AyushExel
Copy link
Contributor

I think #9067 includes these changes

@glenn-jocher
Copy link
Member

@0zppd thanks for the PR! Are you sure this is the only place that torch.empty() is causing problems? See #9067

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
@glenn-jocher
Copy link
Member

glenn-jocher commented Aug 21, 2022

@AyushExel if this is the only source of error then this PR is preferred as #9067 introduces some slight speed improvements, torch.empty() profiles faster than torch.zeros(), which was the original purpose behind #9067, but it seems the danger is when the model BN layers are in train mode and they are updating based on the inputs.

EDIT: testing this now x10 in Colab

@glenn-jocher glenn-jocher changed the title Fix Zero Map Issue zero-mAP fix remove torch.empty() forward pass in .train() mode Aug 21, 2022
@0zppd
Copy link
Contributor Author

0zppd commented Aug 21, 2022

I have extensively tested this, and making only this change consistently removes the issue (and reverting the change brings it back ) in the same Colab session.

@glenn-jocher
Copy link
Member

Confirmed on 10x colab training all good Screenshot 2022-08-21 at 15 40 00

@glenn-jocher glenn-jocher merged commit 87e8dea into ultralytics:master Aug 21, 2022
@glenn-jocher glenn-jocher linked an issue Aug 21, 2022 that may be closed by this pull request
1 task
@glenn-jocher
Copy link
Member

@0zppd PR is merged. Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐

This was referenced Aug 21, 2022
@robotaiguy
Copy link

Great work! You guys handled the business!

@pourmand1376
Copy link
Contributor

I tested this PR on colab 30 times and I found no problem. I think this is finally resolved.

ctjanuhowski pushed a commit to ctjanuhowski/yolov5 that referenced this pull request Sep 8, 2022
…ltralytics#9068)

* Fix Zero Map Issue

Signed-off-by: 0zppd <111682241+0zppd@users.noreply.github.com>

* Update __init__.py

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

Signed-off-by: 0zppd <111682241+0zppd@users.noreply.github.com>
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
@glenn-jocher
Copy link
Member

@pourmand1376 that's fantastic news! I'm glad to hear that the changes have been thoroughly tested and are proving effective. Your input and testing are invaluable to the YOLOv5 community. Thank you for your dedication to improving the project!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants