How does HSV augmentation during training work?

[haimat]

I have seen the three hsv_* options in the hyperparameters file and looked at the code - but I don't fully understand it. Could you please be so kind an explain to me how HSV augmentation and these three settings work during training?

[glenn-jocher]

@haimat Hue Saturation Value colorspace augmentation:

yolov5/utils/augmentations.py

Lines 47 to 61 in 4c40933

	def augment_hsv(im, hgain=0.5, sgain=0.5, vgain=0.5):
	# HSV color-space augmentation
	if hgain or sgain or vgain:
	r = np.random.uniform(-1, 1, 3) * [hgain, sgain, vgain] + 1 # random gains
	hue, sat, val = cv2.split(cv2.cvtColor(im, cv2.COLOR_BGR2HSV))
	dtype = im.dtype # uint8
	x = np.arange(0, 256, dtype=r.dtype)
	lut_hue = ((x * r[0]) % 180).astype(dtype)
	lut_sat = np.clip(x * r[1], 0, 255).astype(dtype)
	lut_val = np.clip(x * r[2], 0, 255).astype(dtype)
	im_hsv = cv2.merge((cv2.LUT(hue, lut_hue), cv2.LUT(sat, lut_sat), cv2.LUT(val, lut_val)))
	cv2.cvtColor(im_hsv, cv2.COLOR_HSV2BGR, dst=im) # no return needed

[haimat]

@glenn-jocher So do I understand correctly that with YOLOv5 there is no way to work with images that are saved in HSV format at the moment?

[glenn-jocher]

@haimat supported formats are indicated here:

yolov5/utils/datasets.py

Lines 34 to 38 in 4c40933

	# Parameters
	HELP_URL = 'https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data'
	IMG_FORMATS = ['bmp', 'dng', 'jpeg', 'jpg', 'mpo', 'png', 'tif', 'tiff', 'webp'] # include image suffixes
	VID_FORMATS = ['asf', 'avi', 'gif', 'm4v', 'mkv', 'mov', 'mp4', 'mpeg', 'mpg', 'wmv'] # include video suffixes

[haimat]

@glenn-jocher Thanks for your quick reply, Glenn. So I have a dataset with images saved in HSV format. They have been generated via cv2.cvtColor(im, cv2.COLOR_BGR2HSV) and then saved as usual via cv2.imwrite(). What would you recommend in this case - can YOLOv5 work with them as usual, or is there a "best practice" approach when working with HSV images in YOLOv5?

[glenn-jocher]

@haimat I have no idea, just train normally and find out. To train correctly your data must be in YOLOv5 format. Please see our Train Custom Data tutorial for full documentation on dataset setup and all steps required to start training your first model. A few excerpts from the tutorial:

1.1 Create dataset.yaml

COCO128 is an example small tutorial dataset composed of the first 128 images in COCO train2017. These same 128 images are used for both training and validation to verify our training pipeline is capable of overfitting. data/coco128.yaml, shown below, is the dataset config file that defines 1) the dataset root directory path and relative paths to train / val / test image directories (or *.txt files with image paths), 2) the number of classes nc and 3) a list of class names:

# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: ../datasets/coco128  # dataset root dir
train: images/train2017  # train images (relative to 'path') 128 images
val: images/train2017  # val images (relative to 'path') 128 images
test:  # test images (optional)

# Classes
nc: 80  # number of classes
names: [ 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light',
         'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow',
         'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',
         'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard',
         'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',
         'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',
         'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone',
         'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear',
         'hair drier', 'toothbrush' ]  # class names

1.2 Create Labels

After using a tool like Roboflow Annotate to label your images, export your labels to YOLO format, with one *.txt file per image (if no objects in image, no *.txt file is required). The *.txt file specifications are:

• One row per object
• Each row is class x_center y_center width height format.
• Box coordinates must be in normalized xywh format (from 0 - 1). If your boxes are in pixels, divide x_center and width by image width, and y_center and height by image height.
• Class numbers are zero-indexed (start from 0).

The label file corresponding to the above image contains 2 persons (class 0) and a tie (class 27):

1.3 Organize Directories

Organize your train and val images and labels according to the example below. YOLOv5 assumes /coco128 is inside a /datasets directory next to the /yolov5 directory. YOLOv5 locates labels automatically for each image by replacing the last instance of /images/ in each image path with /labels/. For example:

../datasets/coco128/images/im0.jpg  # image
../datasets/coco128/labels/im0.txt  # label

Good luck 🍀 and let us know if you have any other questions!

[joihn]

@haimat either you preprocess your dataset to transform it to RBG (I would recommend this ) or try to train directly on the HSV (uncommon pipeline, I would not recommend)

[haimat]

Ok, just for the records - it turned out HVS or RGB does not make any difference (at least for our custom dataset). The original RGB images achieved a overall model fitness of 0.81, then training with the same dataset converted to HXV, using the same training settings and hyperparams, we again achieved 0.81 (both tried multiple times). Funny, I was expecting different results ...

[glenn-jocher]

Colorspaces are irrelevant as the YOLOv5 dataloader loads all images as RGB

[haimat]

@glenn-jocher Thanks for the clarification.