fix tensor mismatch error in multithread inference

[CanKorkut]

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Refactored grid and anchor grid initialization during YOLO model inference.

📊 Key Changes

• Removed conditional checks for dynamic grid resizing.
• Replaced class attributes self.grid and self.anchor_grid with local variables grid and anchor_grid.
• Ensured that grid and anchor_grid are recalculated each time during inference, avoiding the use of potentially outdated attributes.

🎯 Purpose & Impact

• Reduces Memory Footprint: By not storing the grid and anchor grid in the model instance, memory usage is potentially reduced.
• Enhances Clarity: Localizes variables to where they are needed, improving code readability and maintainability.
• Potential Performance Optimization: Recalculating grids only when necessary could optimize the speed and resource consumption of the model during inference.

[github-actions]

👋 Hello @CanKorkut, thank you for submitting a YOLOv5 🚀 PR! To allow your work to be integrated as seamlessly as possible, we advise you to:

• ✅ Verify your PR is up-to-date with ultralytics/yolov5 master branch. If your PR is behind you can update your code by clicking the 'Update branch' button or by running git pull and git merge master locally.
• ✅ Verify all YOLOv5 Continuous Integration (CI) checks are passing.
• ✅ Reduce changes to the absolute minimum required for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee

[glenn-jocher]

@CanKorkut thanks for the PR! Grid construction is a resource intensive task that should only occur if required, i.e. if an image with a new size is passed for inference. Your code would reconstruct a new grid for every image, which is not desire-able from a FLOPs/speed/export perspective.

[glenn-jocher]

@CanKorkut are there solutions that only reconstruct the grid when needed, i.e. when it's the wrong shape?

[CanKorkut]

@glenn-jocher I'm working on that. What do you think about using parameter like "multithread_safe"? Reconstruct grid only when its true.

[glenn-jocher]

@CanKorkut well we already have an attribute that forces grid reconstruction called self.dynamic:

yolov5/models/yolo.py

Lines 64 to 66 in f038ad7

	if self.dynamic or self.grid[i].shape[2:4] != x[i].shape[2:4]:
	self.grid[i], self.anchor_grid[i] = self._make_grid(nx, ny, i)

[glenn-jocher]

@CanKorkut is it the list type that contains the grids that's causing the problem? Should they be stored as a torch object instead, like a ModuleList or something else?

yolov5/models/yolo.py

Lines 50 to 51 in f038ad7

	self.grid = [torch.empty(1)] * self.nl # init grid
	self.anchor_grid = [torch.empty(1)] * self.nl # init anchor grid

EDIT: or maybe its the way they are initialized? Maybe we should initialize them with a for loop instead of a multiplication?

[glenn-jocher]

@CanKorkut maybe we should use nn.ParameterList:

https://pytorch.org/docs/1.10.0/generated/torch.nn.ParameterList.html#torch.nn.ParameterList

EDIT: Opened PR #9492 with proposed solution. Can you try it out and see if it resolves your issue?

[CanKorkut]

@glenn-jocher Yeah, I think nn.ParameterList looks like good idea. I will try it for the issue.

[glenn-jocher]

@CanKorkut opened up a second PR #9494 which might have a simpler solution. ParameterList turns out to be complicated, initializes with requires_grad=True by default and modifying the parameter list is unsupported in torch 1.7

Can you test #9494?

[glenn-jocher]

@CanKorkut any updates on this?

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions YOLOv5 🚀 and Vision AI ⭐.

[github-actions]

👋 Hello there! We wanted to let you know that we've decided to close this pull request due to inactivity. We appreciate the effort you put into contributing to our project, but unfortunately, not all contributions are suitable or aligned with our product roadmap.

We hope you understand our decision, and please don't let it discourage you from contributing to open source projects in the future. We value all of our community members and their contributions, and we encourage you to keep exploring new projects and ways to get involved.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐