Difference on Inferencing the custom models

[Sanath1998]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

Hi @glenn-jocher,

Actually I wanted to know the difference between detect.py while doing the inference on the images from the trained custom model to just directly inferencing from one line (results = model(imgs)).
Is there any specific reason behind doing one line inference like results = model(imgs) by just passing images to model() api.
Can u please get me a glimpse of it.
Reference link for one line inference "#2703 (comment)"

Additional

No response

[glenn-jocher]

@Sanath1998 👋 Hello! Thanks for asking about handling inference results. YOLOv5 🚀 PyTorch Hub models allow for simple model loading and inference in a pure python environment without using detect.py.

Simple Inference Example

This example loads a pretrained YOLOv5s model from PyTorch Hub as model and passes an image for inference. 'yolov5s' is the YOLOv5 'small' model. For details on all available models please see the README. Custom models can also be loaded, including custom trained PyTorch models and their exported variants, i.e. ONNX, TensorRT, TensorFlow, OpenVINO YOLOv5 models.

import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')  # yolov5n - yolov5x6 official model
#                                            'custom', 'path/to/best.pt')  # custom model

# Images
im = 'https://ultralytics.com/images/zidane.jpg'  # or file, Path, URL, PIL, OpenCV, numpy, list

# Inference
results = model(im)

# Results
results.print()  # or .show(), .save(), .crop(), .pandas(), etc.
results.xyxy[0]  # im predictions (tensor)

results.pandas().xyxy[0]  # im predictions (pandas)
#      xmin    ymin    xmax   ymax  confidence  class    name
# 0  749.50   43.50  1148.0  704.5    0.874023      0  person
# 2  114.75  195.75  1095.0  708.0    0.624512      0  person
# 3  986.00  304.00  1028.0  420.0    0.286865     27     tie

results.pandas().xyxy[0].value_counts('name')  # class counts (pandas)
# person    2
# tie       1

See YOLOv5 PyTorch Hub Tutorial for details.

Good luck 🍀 and let us know if you have any other questions!

[Sanath1998]

As u had stated above that model = torch.hub.load('ultralytics/yolov5', 'yolov5s') uses pretrained model to load.
But how to load the custom object detector model which is trained on our dataset. As far as I know torch.hub.load () api loades the pretrained model from internet source. So, I'am using torch.load() api to load the weights file from the local machinebut it is throwing an error.
Can you please sort it out @glenn-jocher ?

[glenn-jocher]

@Sanath1998 I've already explained this in my previous message.

[Sanath1998]

@Sanath1998 I've already explained this in my previous message.

Thanks @glenn-jocher :)

[Sanath1998]

@Sanath1998 I've already explained this in my previous message.

Thanks @glenn-jocher :)

HI @glenn-jocher,
Even I'am seeing slight difference in the values of xmin, ymin, xmax, ymax for bounding box co-ordinates with python hub inference and detect.py inference.
Im not sure the reason behind it. Can u please explain the same so that we can use for our generic use-cases.

[glenn-jocher]

👋 Hello, thanks for asking about the differences between train.py, detect.py and val.py in YOLOv5 🚀.

These 3 files are designed for different purposes and utilize different dataloaders with different settings. train.py dataloaders are designed for a speed-accuracy compromise, val.py is designed to obtain the best mAP on a validation dataset, and detect.py is designed for best real-world inference results. A few important aspects of each:

train.py

• trainloader LoadImagesAndLabels(): designed to load train dataset images and labels. Augmentation capable and enabled.

  yolov5/train.py

  Lines 210 to 213 in 7ee5aed

  	train_loader, dataset = create_dataloader(train_path, imgsz, batch_size // WORLD_SIZE, gs, single_cls,
  	hyp=hyp, augment=True, cache=opt.cache, rect=opt.rect, rank=LOCAL_RANK,
  	workers=workers, image_weights=opt.image_weights, quad=opt.quad,
  	prefix=colorstr('train: '))

  val_loader LoadImagesAndLabels(): designed to load val dataset images and labels. Augmentation capable but disabled.

  yolov5/train.py

  Lines 220 to 223 in 7ee5aed

  	val_loader = create_dataloader(val_path, imgsz, batch_size // WORLD_SIZE * 2, gs, single_cls,
  	hyp=hyp, cache=None if noval else opt.cache, rect=True, rank=-1,
  	workers=workers, pad=0.5,
  	prefix=colorstr('val: '))[0]

• image size: 640
• rectangular inference: False
• confidence threshold: 0.001
• iou threshold: 0.6
• multi-label: True
• padding: None

val.py

• dataloader LoadImagesAndLabels(): designed to load train, val, test dataset images and labels. Augmentation capable but disabled.

• yolov5/val.py

  Lines 152 to 153 in 7ee5aed

  	dataloader = create_dataloader(data[task], imgsz, batch_size, gs, single_cls, pad=pad, rect=True,
  	prefix=colorstr(f'{task}: '))[0]

• image size: 640
• rectangular inference: True
• confidence threshold: 0.001
• iou threshold: 0.6
• multi-label: True
• padding: 0.5 * maximum stride

detect.py

• dataloaders (multiple): designed for loading multiple types of media (images, videos, globs, directories, streams).

  yolov5/detect.py

  Lines 120 to 128 in 7ee5aed

  	# Dataloader
  	if webcam:
  	view_img = check_imshow()
  	cudnn.benchmark = True # set True to speed up constant image size inference
  	dataset = LoadStreams(source, img_size=imgsz, stride=stride, auto=pt)
  	bs = len(dataset) # batch_size
  	else:
  	dataset = LoadImages(source, img_size=imgsz, stride=stride, auto=pt)
  	bs = 1 # batch_size

• image size: 640
• rectangular inference: True
• confidence threshold: 0.25
• iou threshold: 0.45
• multi-label: False
• padding: None

YOLOv5 PyTorch Hub Inference

YOLOv5 PyTorch Hub models areAutoShape() instances used for image loading, preprocessing, inference and NMS. For more info see YOLOv5 PyTorch Hub Tutorial

yolov5/models/common.py

Lines 276 to 282 in 7ee5aed

	class AutoShape(nn.Module):
	# YOLOv5 input-robust model wrapper for passing cv2/np/PIL/torch inputs. Includes preprocessing, inference and NMS
	conf = 0.25 # NMS confidence threshold
	iou = 0.45 # NMS IoU threshold
	classes = None # (optional list) filter by class
	multi_label = False # NMS multiple labels per box
	max_det = 1000 # maximum number of detections per image

• image size: 640
• rectangular inference: True
• confidence threshold: 0.25
• iou threshold: 0.45
• multi-label: False
• padding: None

Good luck 🍀 and let us know if you have any other questions!

[Sanath1998]

Hi @glenn-jocher ,
I wanted to know why there is a huge time difference while inferencing the detector model using detect.py and with single line inference using torch.hub (results = model(imgs).
Note: The inference for the both is being done on same image.

Time Taken to do inference:
detect.py = 11-21 ms
hub.py (single line inference (results = model(imgs)) = 35-45 ms

[glenn-jocher]

@Sanath1998 👋 hi, thanks for letting us know about this possible problem with YOLOv5 🚀. We've created a few short guidelines below to help users provide what we need in order to start investigating a possible problem.

How to create a Minimal, Reproducible Example

When asking a question, people will be better able to provide help if you provide code that they can easily understand and use to reproduce the problem. This is referred to by community members as creating a minimum reproducible example. Your code that reproduces the problem should be:

• ✅ Minimal – Use as little code as possible to produce the problem
• ✅ Complete – Provide all parts someone else needs to reproduce the problem
• ✅ Reproducible – Test the code you're about to provide to make sure it reproduces the problem

For Ultralytics to provide assistance your code should also be:

• ✅ Current – Verify that your code is up-to-date with GitHub master, and if necessary git pull or git clone a new copy to ensure your problem has not already been solved in master.
• ✅ Unmodified – Your problem must be reproducible using official YOLOv5 code without changes. Ultralytics does not provide support for custom code ⚠️.

If you believe your problem meets all the above criteria, please close this issue and raise a new one using the 🐛 Bug Report template with a minimum reproducible example to help us better understand and diagnose your problem.

Thank you! 😃