Add random interpolation method augmentation

[developer0hye]

I couldn't test this augmentation on COCO dataset because I don't have computing power to do this test.

I referenced one of the timm's augmentation methods and ross wightman's tweet.

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Enhanced image resizing options during dataset loading in YOLOv5.

📊 Key Changes

• Added a list of random image interpolation methods for resizing images during the dataset loading phase.
• Modified the image resizing function to utilize either a linear interpolation method when augmenting data or a randomly selected method from the new list, rather than always using linear interpolation or area interpolation depending on the augmentation flag and resize ratio.

🎯 Purpose & Impact

• Purpose: To provide a wider variety of interpolation methods for resizing images, which could potentially improve model accuracy by introducing more diversity during training.
• Impact: Users may experience slight variations in training results due to the new randomization of interpolation methods, which could lead to more robust models when dealing with different image resolutions.

[glenn-jocher]

@developer0hye thanks for the idea! The implementation is not quite right though, as the random interp method selection should only apply during training and not during detect/val.

The training distinction can be made here in load_images() L643:

yolov5/utils/datasets.py

Lines 627 to 647 in 8a66eba

	def load_image(self, i):
	# loads 1 image from dataset index 'i', returns (im, original hw, resized hw)
	im = self.imgs[i]
	if im is None: # not cached in RAM
	npy = self.img_npy[i]
	if npy and npy.exists(): # load npy
	im = np.load(npy)
	else: # read image
	f = self.img_files[i]
	im = cv2.imread(f) # BGR
	assert im is not None, f'Image Not Found {f}'
	h0, w0 = im.shape[:2] # orig hw
	r = self.img_size / max(h0, w0) # ratio
	if r != 1: # if sizes are not equal
	im = cv2.resize(im,
	(int(w0 * r), int(h0 * r)),
	interpolation=cv2.INTER_LINEAR if (self.augment or r > 1) else cv2.INTER_AREA)
	return im, (h0, w0), im.shape[:2] # im, hw_original, hw_resized
	else:
	return self.imgs[i], self.img_hw0[i], self.img_hw[i] # im, hw_original, hw_resized

[developer0hye]

@glenn-jocher
Thanks for your review!

Should I edit load_image function to apply this augmentation?

[glenn-jocher]

@developer0hye I'll try to update this accordingly

[glenn-jocher]

@developer0hye ok I've applied some updates. I think this will slow down training somewhat as INTER_CUBIC and INTER_LANCZOS4 are much heavier ops than the current INTER_LINEAR. I'll train a COCO model and we can see the results in a few days.

[glenn-jocher]

@developer0hye interpolation profiling results:

Colab++ V100 High-mem

import cv2
import numpy as np

from utils.general import Profile

im = cv2.imread('data/images/bus.jpg')
method = (cv2.INTER_NEAREST, cv2.INTER_LINEAR, cv2.INTER_CUBIC, cv2.INTER_AREA, cv2.INTER_LANCZOS4)

print('Resize 640 to 320')
for m in method:
    with Profile():
        for _ in range(100):
            cv2.resize(im, (320, 320), interpolation=m)

print('Resize 640 to 1280')
for m in method:
    with Profile():
        for _ in range(100):
            cv2.resize(im, (1280, 1280), interpolation=m)

[developer0hye]

@glenn-jocher Thanks for sharing your results!

[glenn-jocher]

@developer0hye I wasn't able to generate any mAP improvement using this method unfortunately. I trained COCO from scratch before and after here:

train.py --batch 256 --weights '' --cfg yolov5s6.yaml --data coco.yaml --epochs 300 --cache disk --img 640 --project test_conv_reduce --device 7 --name yolov5s6-coco-randinterp

[glenn-jocher]

@developer0hye it did slighly redue obj loss though. Perhaps this simply reduced mAP because the model had not overfitted sufficiently, and it's possible the method would benefit larger models more.

[developer0hye]

@glenn-jocher

Is there any progress on this item?

[glenn-jocher]

@developer0hye based on the above test results (PR produces worse mAP on COCO) we don't want to implement this as a default choice. Perhaps this could be left as an option that is disabled by default.

[developer0hye]

@glenn-jocher

Yes, I agree with you.

Have you ever tested with a larger model?

[glenn-jocher]

@developer0hye yes it's possible this belongs with the other aggressive augmentation techniques that only benefit larger models, like higher 0.9 scale jitter.

It's also possible that the location information gets lost when downsampling on some of these techniques, which might cause it to be more suitable to classification augmentation instead of detection augmentation.

But I'll try to run with a larger model when I get some free GPUs.

[glenn-jocher]

@developer0hye training now with YOLOv5x COCO 640 at https://wandb.ai/glenn-jocher/v7_retrain

[developer0hye]

@glenn-jocher

Thanks a lot! What is yolov5-7bx?

[glenn-jocher]

@developer0hye yolov5x-7bx is just a v7.0 release candidate that we are testing. It's not important, as there are many experiments that we run like this one. The important thing is to compare yolov5x-7bx to yolov5x-7bx-randinterp for an apples to apples comparison of the effect of this PR.

[developer0hye]

@glenn-jocher
I hope yolov5x-7bx-randinterp become a monster beats yolov5-7bx at last epoch!!

[developer0hye]

@glenn-jocher

What do you think of the results?

I think that we should measure mAPs with various resize setting to compare the performance of yolov5x-7bx-randinterp and yolov5x-7bx.

[glenn-jocher]

@developer0hye PR is merged. I retained the current interpolation behavior as was not able to produce mAP gains using the proposed method (may be more suitable for classification than detection), but left the random interp idea commented for possible future use.

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐

[developer0hye]

@glenn-jocher
Thanks!

[glenn-jocher]

@developer0hye you're welcome! If you have any more questions or suggestions, feel free to reach out. Your input is greatly appreciated.