Add autoshape parameter to Pytorch hub models

[NanoCode012]

Following #1677 , I quickly added the autoshape code.

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

PR introduces more flexibility with 'autoshape' during YOLOv5 model creation.

📊 Key Changes

• Added autoshape parameter to the create function and respective model functions (yolov5s, yolov5m, yolov5l, yolov5x, custom).
• Changed README.md to remove the autoshape call from the model instantiation.
• Enhanced autoShape class to fetch images from file paths or URIs for inference.
• Allowed the forward method within autoShape to accept a list of images or a single image from various sources like filenames, URIs, or NumPy arrays.
• Added a check to prevent autoshape from being applied if it's already enabled on a model.

🎯 Purpose & Impact

• Flexibility in Autoshaping: Users can now choose whether to apply autoshape when loading a model, rather than it being applied by default.
• Easier Image Inputs for Inference: The update simplifies using images from different sources (file paths, URLs, OpenCV, etc.), making the process more intuitive.
• Seamless Inference Integration: Users can perform inference with varying input methods with less upfront manual image processing.
• Potential Reduction in Memory Usage: By removing the default autoshape call, users who don't need it can avoid the additional overhead, potentially reducing memory usage.
• Overall Impact: Improves the YOLOv5 library's usability and flexibility, enabling users to utilize the models with greater control over input processing. 🚀

[glenn-jocher]

@NanoCode012 awesome, thanks for getting this started! We need to coordinate this change in a additional places too:

• with a second PR to https://github.com/pytorch/hub/blob/master/ultralytics_yolov5.md that will update the https://pytorch.org/hub/ultralytics_yolov5/ webpage
• With an update to the README example (can be added to this PR)
• In the tutorial at Load YOLOv5 from PyTorch Hub ⭐ #36

[NanoCode012]

Sure! I’ll check it out in the morning!

[NanoCode012]

I have one suggestion here that may (hopefully) lighten your load.

I notice that each time you update your requirements (eg. torch), you would need to manually do it for every guide which is quite redundant and time consuming. Perhaps you can create just one Issue/Wiki/on ReadMe and reference it instead of pasting it manually? Same for Environments?

---

I saw that the requirements are different between the hub/yolov5 page and the pytorch hub tutorial.

# hub/yolov5
$ pip install -U PyYAML  # install dependencies

# pytorch hub tutorial
$ pip install -U opencv-python pillow pyyaml tqdm  # install dependencies

---

Unfortunately, I can't edit the pytorchhub tutorial directly, so I'll add the parts here, which you can copy it out.

Under Simple Example and Detailed Example,

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True).autoshape()  # for PIL/cv2/np inputs and NMS

to

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True) # for PIL/cv2/np inputs and NMS

Under Custom Models,

model = torch.hub.load('ultralytics/yolov5', 'custom', path_or_model='yolov5s_voc_best.pt')
model = model.autoshape()  # for PIL/cv2/np inputs and NMS

to

model = torch.hub.load('ultralytics/yolov5', 'custom', path_or_model='yolov5s_voc_best.pt')

Add a new Section,

Autoshape

To prevent autoshape from being added to the YOLOv5 model from PyTorch Hub, pass in autoshape=False as a parameter,

model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True, autoshape=False)

(Is the use case for people who are making their own training code?)

---

Lastly, for the other hub repo, I think it would be better for the owner to make that PR :)

[glenn-jocher]

@NanoCode012 thanks! Yes agree on all points. Will create the torch hub PR and try to get this merged soon.

[glenn-jocher]

/rebase

TODO: had an idea to add direct inference from filepaths and urls in a separate PR
https://stackoverflow.com/questions/7391945/how-do-i-read-image-data-from-a-url-in-python

from PIL import Image
import requests

url = 'https://github.com/ultralytics/yolov5/releases/download/v1.0/bus.jpg'
img1 = Image.open(requests.get(url, stream=True).raw)  # from url

img2 = Image.open('path/to/img.jpg')  # from file

[glenn-jocher]

/rebase

[glenn-jocher]

@NanoCode012 I just realized this will break existing workflows, as if a user tries to .autoshape() an existing autoshape module they will get an error. So I think I need to add a passthrough method to the autoshape class called autoshape, that simply reports to screen that model is already in autoshape mode. Test is basically:

model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True).autoshape() # to be integrated togethor in this PR
model = model.autoshape()  # what existing users will try to do in addition to previous line

[glenn-jocher]

I think this will work:

class autoShape(nn.Module):
    # input-robust model wrapper for passing cv2/np/PIL/torch inputs. Includes preprocessing, inference and NMS
    img_size = 640  # inference size (pixels)
    conf = 0.25  # NMS confidence threshold
    iou = 0.45  # NMS IoU threshold
    classes = None  # (optional list) filter by class

    def __init__(self, model):
        super(autoShape, self).__init__()
        self.model = model.eval()

    def autoshape(self):
        print('autoShape already enabled, skipping... ')  # model already converted to model.autoshape()
        return self

[glenn-jocher]

Ok, I've added the autoshape check passthrough (if a user tries to autoshape an autoshape model, i.e. existing users), and also file/URI input capability:

This should work after the PR is merged:

import torch
from PIL import Image

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True, force_reload=True)  # for file/URI/PIL/cv2/np inputs and NMS

# Images
img1 = 'data/images/zidane.jpg'  # filename
img2 = Image.open('data/images/bus.jpg')  # PIL
img3 = 'https://github.com/ultralytics/yolov5/releases/download/v1.0/zidane.jpg'  # URI
imgs = [img1, img2, img3]  # batched list of images

# Inference
result = model(imgs, size=640)  # includes NMS

results = model(imgs)
results.show()
results.print()

[NanoCode012]

That's great! Do we need to do any separate unit test? I assume it should be fine though.

[glenn-jocher]

That's great! Do we need to do any unit test?

That's the funny thing about PyTorch Hub work, I think it's not possible to easily debug things without merging with master first. The above tests work well in just running hubconf.py though :)

[glenn-jocher]

I'll just merge and then test it out with force_reload true in a few minutes, and if there's any bugs we can just do another PR to patch them up.

[glenn-jocher]

Yay it works!!

@NanoCode012 thank you for your contributions :)