Difference between 'conf-thresh' and 'iou-thresh'

[aonsafdar]

Hello,
I am training yolov5s with a custom dataset. I have trouble understanding the difference between 'conf-thres' and 'iou_thres' in val.py and detect.py. My question(s):-

1. What is the difference between the two?
2. Why the default value is different in val.py and detect.py?
3. What are the general effects of altering these defaults?
   Thanks.

In val.py
parser.add_argument('--conf-thres', type=float, default=0.25, help='confidence threshold')
parser.add_argument('--iou-thres', type=float, default=0.45, help='NMS IoU threshold')

In detect.py
conf_thres=0.001, # confidence threshold
iou_thres=0.6, # NMS IoU threshold

[glenn-jocher]

@aonsafdar 👋 Hello, thanks for asking about the differences between train.py, detect.py and val.py in YOLOv5 🚀.

These 3 files are designed for different purposes and utilize different dataloaders with different settings. train.py dataloaders are designed for a speed-accuracy compromise, val.py is designed to obtain the best mAP on a validation dataset, and detect.py is designed for best real-world inference results. A few important aspects of each:

train.py

• trainloader LoadImagesAndLabels(): designed to load train dataset images and labels. Augmentation capable and enabled.

  yolov5/train.py

  Lines 210 to 213 in 7ee5aed

  	train_loader, dataset = create_dataloader(train_path, imgsz, batch_size // WORLD_SIZE, gs, single_cls,
  	hyp=hyp, augment=True, cache=opt.cache, rect=opt.rect, rank=LOCAL_RANK,
  	workers=workers, image_weights=opt.image_weights, quad=opt.quad,
  	prefix=colorstr('train: '))

  val_loader LoadImagesAndLabels(): designed to load val dataset images and labels. Augmentation capable but disabled.

  yolov5/train.py

  Lines 220 to 223 in 7ee5aed

  	val_loader = create_dataloader(val_path, imgsz, batch_size // WORLD_SIZE * 2, gs, single_cls,
  	hyp=hyp, cache=None if noval else opt.cache, rect=True, rank=-1,
  	workers=workers, pad=0.5,
  	prefix=colorstr('val: '))[0]

• image size: 640
• rectangular inference: False
• confidence threshold: 0.001
• iou threshold: 0.6
• multi-label: True
• padding: None

val.py

• dataloader LoadImagesAndLabels(): designed to load train, val, test dataset images and labels. Augmentation capable but disabled.

• yolov5/val.py

  Lines 152 to 153 in 7ee5aed

  	dataloader = create_dataloader(data[task], imgsz, batch_size, gs, single_cls, pad=pad, rect=True,
  	prefix=colorstr(f'{task}: '))[0]

• image size: 640
• rectangular inference: True
• confidence threshold: 0.001
• iou threshold: 0.6
• multi-label: True
• padding: 0.5 * maximum stride

detect.py

• dataloaders (multiple): designed for loading multiple types of media (images, videos, globs, directories, streams).

  yolov5/detect.py

  Lines 120 to 128 in 7ee5aed

  	# Dataloader
  	if webcam:
  	view_img = check_imshow()
  	cudnn.benchmark = True # set True to speed up constant image size inference
  	dataset = LoadStreams(source, img_size=imgsz, stride=stride, auto=pt)
  	bs = len(dataset) # batch_size
  	else:
  	dataset = LoadImages(source, img_size=imgsz, stride=stride, auto=pt)
  	bs = 1 # batch_size

• image size: 640
• rectangular inference: True
• confidence threshold: 0.25
• iou threshold: 0.45
• multi-label: False
• padding: None

YOLOv5 PyTorch Hub Inference

YOLOv5 PyTorch Hub models areAutoShape() instances used for image loading, preprocessing, inference and NMS. For more info see YOLOv5 PyTorch Hub Tutorial

yolov5/models/common.py

Lines 276 to 282 in 7ee5aed

	class AutoShape(nn.Module):
	# YOLOv5 input-robust model wrapper for passing cv2/np/PIL/torch inputs. Includes preprocessing, inference and NMS
	conf = 0.25 # NMS confidence threshold
	iou = 0.45 # NMS IoU threshold
	classes = None # (optional list) filter by class
	multi_label = False # NMS multiple labels per box
	max_det = 1000 # maximum number of detections per image

• image size: 640
• rectangular inference: True
• confidence threshold: 0.25
• iou threshold: 0.45
• multi-label: False
• padding: None

Good luck 🍀 and let us know if you have any other questions!

[Alberto1404]

Regarding this... How should be define conf-thres and iou-thres for inference (when running detect.py) ? I mean, I suppose that based on a custom training, the graphs would tell us which value optimize it, but, i don't know how to understand them. Here I show my training results for person detection (only one class).
Thank you in advance. Any help is appreciated

[glenn-jocher]

@Alberto1404 you can use max F1 curve to determine your best confidence threshold. For IoU you can use the default or try different settings in val.py to see it's effect on mAP.

[yesid-acm]

@aonsafdar @glenn-jocher why does a warning to use confi >0.001 appear in val.py?

to better use yolov5 you should know a little more in the tutorial, add the step by step. For example: train with XX thresholds and use those in the val.py which are the best, etc... don't you think glenn?

[lucasvw]

you can use max F1 curve to determine your best confidence threshold.

@glenn-jocher what's your intuition behind this? I can confirm on my custom trained yolo model that this indeed gives best results on the test set, but I'm failing to see the intuition behind this. I have tried searching for this online, but can't seem to find a good explanation. Perhaps you can share your ideas or perhaps you can point me to a resource?

[faisalhazry]

@lucasvw
Precision: Knowns as ability of a model to correctly identify positive predictions out of instances predicted as positive Precision = TP / (TP + FP)

Recall: Known as sensitivity of true positive rate by total total relevant results instances correctly classified Recall = TP / (TP + FN)

In conclusion we cannot possible to maximize both metric recall and precision at the same time as one comes at the cost of another. To analysis precision and recall we need to balancing two separate scores

F-1 scores which is harmonic precision and recall camas help to give a good simplicity matrix for evaluation model.

F-1 scores: Increasing the F1 score indicates an improvement in the overall performance of a model

Confidence -> is IoU Overlapping result. This one not sure

Reference https://medium.com/@shrutisaxena0617/precision-vs-recall-386cf9f89488

@glenn-jocher plz correct me if i am wrong