Metrics and FPS

[Raknash3]

How can I add more metrics apart from the ones already initialized and is it possible to display the FPS while running detect.py on video files?

[glenn-jocher]

@Raknash3 you can compute your FPS by dividing your framecount by your total time displayed at the end of detect.py.

What additional metrics are you interested in?

[Raknash3]

Thanks for replying.

I wish to add recall and precision at different thresholds ( 30%, 60% and 90%) while testing, does setting the '--conf' argument to the required values take care of this . And I have few more questions.

1. Can I use the argument '--verbose' and '--augment' in train.py ?
2. I ran train.py on a custom dataset and in the results.png , I saw a graph with the title 'Box', is it same as GIoU?
3. While training on custom dataset can I use '--evolve' or only after I get the weights after training should I retrain with '--evolve' for hyperparameter tuning?

[glenn-jocher]

@Raknash3 test.py actually computes precision and recall at 1000 different thresholds and then presents the P and R values at the highest F1 value. This is all done here:

yolov5/utils/metrics.py

Lines 40 to 79 in cbd55da

	# Create Precision-Recall curve and compute AP for each class
	px, py = np.linspace(0, 1, 1000), [] # for plotting
	ap, p, r = np.zeros((nc, tp.shape[1])), np.zeros((nc, 1000)), np.zeros((nc, 1000))
	for ci, c in enumerate(unique_classes):
	i = pred_cls == c
	n_l = (target_cls == c).sum() # number of labels
	n_p = i.sum() # number of predictions
	if n_p == 0 or n_l == 0:
	continue
	else:
	# Accumulate FPs and TPs
	fpc = (1 - tp[i]).cumsum(0)
	tpc = tp[i].cumsum(0)
	# Recall
	recall = tpc / (n_l + 1e-16) # recall curve
	r[ci] = np.interp(-px, -conf[i], recall[:, 0], left=0) # negative x, xp because xp decreases
	# Precision
	precision = tpc / (tpc + fpc) # precision curve
	p[ci] = np.interp(-px, -conf[i], precision[:, 0], left=1) # p at pr_score
	# AP from recall-precision curve
	for j in range(tp.shape[1]):
	ap[ci, j], mpre, mrec = compute_ap(recall[:, j], precision[:, j])
	if plot and j == 0:
	py.append(np.interp(px, mrec, mpre)) # precision at mAP@0.5
	# Compute F1 (harmonic mean of precision and recall)
	f1 = 2 * p * r / (p + r + 1e-16)
	if plot:
	plot_pr_curve(px, py, ap, Path(save_dir) / 'PR_curve.png', names)
	plot_mc_curve(px, f1, Path(save_dir) / 'F1_curve.png', names, ylabel='F1')
	plot_mc_curve(px, p, Path(save_dir) / 'P_curve.png', names, ylabel='Precision')
	plot_mc_curve(px, r, Path(save_dir) / 'R_curve.png', names, ylabel='Recall')
	i = f1.mean(0).argmax() # max F1 index
	return p[:, i], r[:, i], ap, f1[:, i], unique_classes.astype('int32')

GIoU loss was renamed to Box loss several months ago.

For evolution see the hyperparameter evolution tutorial:

Tutorials

• Hyperparameter Evolution

[glenn-jocher]

Also, by default P and R are plotted for all thresholds after training completes.

[Raknash3]

Thanks a lot, this really helped me and what about my first question, can I use arguments like '--augment' and '--verbose' on train files. Because, I followed the TTA tutorial and you have used '--augment' only on test dataset.

[glenn-jocher]

@Raknash3 available arguments are shown in each file, i.e. for detect.py arguments:

yolov5/detect.py

Lines 147 to 166 in cbd55da

	if __name__ == '__main__':
	parser = argparse.ArgumentParser()
	parser.add_argument('--weights', nargs='+', type=str, default='yolov5s.pt', help='model.pt path(s)')
	parser.add_argument('--source', type=str, default='data/images', help='source') # file/folder, 0 for webcam
	parser.add_argument('--img-size', type=int, default=640, help='inference size (pixels)')
	parser.add_argument('--conf-thres', type=float, default=0.25, help='object confidence threshold')
	parser.add_argument('--iou-thres', type=float, default=0.45, help='IOU threshold for NMS')
	parser.add_argument('--device', default='', help='cuda device, i.e. 0 or 0,1,2,3 or cpu')
	parser.add_argument('--view-img', action='store_true', help='display results')
	parser.add_argument('--save-txt', action='store_true', help='save results to *.txt')
	parser.add_argument('--save-conf', action='store_true', help='save confidences in --save-txt labels')
	parser.add_argument('--classes', nargs='+', type=int, help='filter by class: --class 0, or --class 0 2 3')
	parser.add_argument('--agnostic-nms', action='store_true', help='class-agnostic NMS')
	parser.add_argument('--augment', action='store_true', help='augmented inference')
	parser.add_argument('--update', action='store_true', help='update all models')
	parser.add_argument('--project', default='runs/detect', help='save results to project/name')
	parser.add_argument('--name', default='exp', help='save results to project/name')
	parser.add_argument('--exist-ok', action='store_true', help='existing project/name ok, do not increment')
	opt = parser.parse_args()
	print(opt)

[Raknash3]

Thank you.