How to cancel the data augmentation of letterbox during training?

[jayer95]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

I know that yolov5 introduced "letterbox" data augmentation and many hyperparameters to increase the strength of the model, but I need to do an experiment.

As title, how can I cancel the data augmentation of "letterbox" during training?

The images size of my training dataset is all 1920x1080. I want to do an experiment to cancel the data augmentation "letterbox" made by yolov5 during training, and resize all 1920x1080 images to 640x640, even if the images are distorted due to resizing.

I checked the argument options and hyperparameters before training, but it did not have the switch of "letterbox", or I missed an important code, please guide me, thank you.

I know that one argument is "--rect", and I don't mean rectangular training.

Additional

No response

[glenn-jocher]

@jayer95 letterboxing is a method for fitting an image within a constrained space dictated by the model requirements. You can modify the arguments into the function here:

yolov5/utils/augmentations.py

Lines 91 to 122 in db6ec66

	def letterbox(im, new_shape=(640, 640), color=(114, 114, 114), auto=True, scaleFill=False, scaleup=True, stride=32):
	# Resize and pad image while meeting stride-multiple constraints
	shape = im.shape[:2] # current shape [height, width]
	if isinstance(new_shape, int):
	new_shape = (new_shape, new_shape)
	# Scale ratio (new / old)
	r = min(new_shape[0] / shape[0], new_shape[1] / shape[1])
	if not scaleup: # only scale down, do not scale up (for better val mAP)
	r = min(r, 1.0)
	# Compute padding
	ratio = r, r # width, height ratios
	new_unpad = int(round(shape[1] * r)), int(round(shape[0] * r))
	dw, dh = new_shape[1] - new_unpad[0], new_shape[0] - new_unpad[1] # wh padding
	if auto: # minimum rectangle
	dw, dh = np.mod(dw, stride), np.mod(dh, stride) # wh padding
	elif scaleFill: # stretch
	dw, dh = 0.0, 0.0
	new_unpad = (new_shape[1], new_shape[0])
	ratio = new_shape[1] / shape[1], new_shape[0] / shape[0] # width, height ratios
	dw /= 2 # divide padding into 2 sides
	dh /= 2
	if shape[::-1] != new_unpad: # resize
	im = cv2.resize(im, new_unpad, interpolation=cv2.INTER_LINEAR)
	top, bottom = int(round(dh - 0.1)), int(round(dh + 0.1))
	left, right = int(round(dw - 0.1)), int(round(dw + 0.1))
	im = cv2.copyMakeBorder(im, top, bottom, left, right, cv2.BORDER_CONSTANT, value=color) # add border
	return im, ratio, (dw, dh)

[jayer95]

Hi @glenn-jocher ,
I'm the author of this article,
#5244

Do you mean to make this function an "argument" and add "--letterbox false" to the instruction during training?

Can I just delete this "letterbox" function (yolov5/utils/augmentations.py)?

I'm currently training with "--rect" and I don't need "letterbox".

[glenn-jocher]

@jayer95 letterbox function adapts input images to meet minimum stride length constraints. It is required.

[jayer95]

@glenn-jocher In order to make up the remainder of the width and height for multiple of 32.

[jayer95]

@glenn-jocher
I understand.

[jayer95]

Hi @glenn-jocher ,
If I use rectangular training to force resize to 384x640,
Or I don't use rectangular training, I use like YOLOv4 (darknet) to force resize into a square like 416x416, how can I cancel the "letterbox"?

[glenn-jocher]

@jayer95 you can customize the letterbox arguments or code if you'd like.

[jayer95]

@glenn-jocher

Hi author, after our in-depth research, we understand the mosaic parameters and letterbox practices you use during training.

I would like to ask you about letterbox when inference (detect.py),
When inferring,

python detect.py --source demo_h264.mp4 --weights runs/train/exp7/weights/last.pt --imgsz 320

demo_h264.mp4 ---> 1920x1080, landscape
will see him do letterbox,

video 1/1 (136/140) /home/gvsai/yolov5/demo_h264.mp4: 192x320 1 license plate, Done. (0.008s)
video 1/1 (137/140) /home/gvsai/yolov5/demo_h264.mp4: 192x320 1 license plate, Done. (0.009s)
video 1/1 (138/140) /home/gvsai/yolov5/demo_h264.mp4: 192x320 1 license plate, Done. (0.008s)

I looked at your code, you are filling the gray border of (114,114,114) up and down to make up the multiple of 32, as shown below,

May I ask why you don't fill all of them on the top, or all of them on the bottom, as shown in the image below?

May I ask whether you have proved through experiments that fill the top and bottom are the best because of the mosaic parameters during training?

[glenn-jocher]

@jayer95 the current letterbox implementation provides the best results compared to the alternatives, i.e. the decision making process was informed via empirical results.

[jayer95]

@glenn-jocher Thanks for your quick reply.

[github-actions]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com/hub
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!