Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TensorRT PyTorch Hub inference fix #7560

Merged
merged 1 commit into from
Apr 24, 2022
Merged

TensorRT PyTorch Hub inference fix #7560

merged 1 commit into from
Apr 24, 2022

Conversation

glenn-jocher
Copy link
Member

@glenn-jocher glenn-jocher commented Apr 24, 2022

Solution proposed in #7128 to TRT PyTorch Hub CUDA illegal memory errors.

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Improvement in device compatibility for AMP inferences within YOLOv5.

📊 Key Changes

  • Ensured that the dummy tensor p is on the same device as self.model when self.pt is False.

🎯 Purpose & Impact

  • Purpose: To fix a potential issue where the dummy tensor for device and type determination was not being consistently placed on the correct device (e.g., GPU or CPU).
  • Impact: Enhanced compatibility and stability when running Automatic Mixed Precision (AMP) inferences, preventing errors related to device mismatches. This change is particularly relevant when self.pt is False, ensuring that users who are performing inferences without a .pt model (PyTorch format) do not face device-related issues.

Solution proposed in #7128 to TRT PyTorch Hub CUDA illegal memory errors.
@glenn-jocher glenn-jocher self-assigned this Apr 24, 2022
@glenn-jocher
Copy link
Member Author

Fix works!

Screen Shot 2022-04-24 at 12 44 49 PM

@glenn-jocher glenn-jocher merged commit 950a85d into master Apr 24, 2022
@glenn-jocher glenn-jocher deleted the update/trt_hub branch April 24, 2022 19:45
@glenn-jocher glenn-jocher linked an issue Apr 24, 2022 that may be closed by this pull request
1 task
@glenn-jocher glenn-jocher linked an issue Apr 24, 2022 that may be closed by this pull request
1 task
@glenn-jocher glenn-jocher linked an issue Apr 24, 2022 that may be closed by this pull request
1 task
BjarneKuehl pushed a commit to fhkiel-mlaip/yolov5 that referenced this pull request Aug 26, 2022
Solution proposed in ultralytics#7128 to TRT PyTorch Hub CUDA illegal memory errors.
ctjanuhowski pushed a commit to ctjanuhowski/yolov5 that referenced this pull request Sep 8, 2022
Solution proposed in ultralytics#7128 to TRT PyTorch Hub CUDA illegal memory errors.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant