How can I test/evaluate my custom model and custom dataset when my model is loaded via torch.hub?

[Rich2020]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

I have trained a custom model and now I want to evaluate/test it on a different dataset (the classes/labels are the same in this dataset as what the model was trained on).

I load my model via torch.hub and I am trying to evaluate:

results = model.val(data='data/config.yml', conf=0.25, iou=0.45, save_json=True, save_conf=True)

But this just errors:

    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'AutoShape' object has no attribute 'val'. Did you mean: 'eval'?

I tried to find the correct documentation online, but none of the examples using torch say a word about evaluation or testing.

Additional

No response

[glenn-jocher]

@Rich2020 hello,

Thank you for reaching out and for providing detailed information about your issue. It looks like you're trying to evaluate your custom model using the torch.hub interface, but encountering an error due to the absence of the val method in the AutoShape class.

To evaluate your custom model, you should load it without the AutoShape wrapper, which is primarily used for inference. Here's how you can do it:

1. Load the model for evaluation:

   import torch
   
   # Load your custom model without AutoShape
   model = torch.hub.load('ultralytics/yolov5', 'custom', path='path/to/best.pt', autoshape=False)

2. Evaluate the model:
   You can then use the model.val method to evaluate your model on your custom dataset. Ensure that your dataset configuration file (data/config.yml) is correctly set up.

   results = model.val(data='data/config.yml', conf=0.25, iou=0.45, save_json=True, save_conf=True)

If you encounter any issues, please ensure you are using the latest versions of torch and the YOLOv5 repository. You can update them using:

pip install --upgrade torch
pip install --upgrade git+https://github.com/ultralytics/yolov5.git

If the problem persists, please provide a minimum reproducible code example so we can investigate further. You can find more details on creating a minimum reproducible example here.

Thank you for your cooperation, and feel free to reach out if you have any further questions! 😊

[Rich2020]

@glenn-jocher Thank you so much. You always respond so quickly and with such helpful answers. I really appreciate all that you do for the world of CV. autoshape=False was indeed the issue I was facing.

[glenn-jocher]

Hello @Rich2020,

Thank you for your kind words! The credit truly goes to the amazing YOLO community and the dedicated Ultralytics team. I'm glad to hear that setting autoshape=False resolved your issue.

If you have any more questions or run into any other issues, feel free to reach out. We're here to help! 😊

Best of luck with your project, and happy coding!

Warm

[fsafsfda]

ABGJE-F32IJ-KL892O-P870T-UVW9Y

[glenn-jocher]

Hello @fsafsfda-KL892O-P870T-UVW9Y,

Thank you for reaching out and for your detailed report. To assist you effectively, we need a minimum reproducible code example. This helps us understand the context and reproduce the issue on our end. You can find guidelines on how to create one here.

Additionally, please ensure you are using the latest versions of torch and the YOLOv5 repository. You can update them with the following commands:

pip install --upgrade torch
pip install --upgrade git+https://github.com/ultralytics/yolov5.git

Once you've done that, if the issue persists, please share the reproducible code example so we can investigate further.

Thank you for your cooperation, and we look forward to helping you resolve this issue! 😊