Albumentations-Pipeline is applied to BGR not to RGB

[UnglvKitDe]

Search before asking

• I have searched the YOLOv5 issues and found no similar bug report.

YOLOv5 Component

No response

Bug

As written here in step 3, Albumentations internally uses the RGB format and not the BGR format of opencv. However, the data is currently passed internally as BGR:

yolov5/utils/dataloaders.py

Lines 626 to 628 in 92e47b8

	# Albumentations
	img, labels = self.albumentations(img, labels)
	nl = len(labels) # update after albumentations

yolov5/utils/dataloaders.py

Line 654 in 92e47b8

img = img.transpose((2, 0, 1))[::-1] # HWC to CHW, BGR to RGB

Or am I missing something?

Environment

YOLOv5 torch 1.11 (cuda 11.3) and 1.12 (cuda 11.6)

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[seonho]

YOLOv5 uses the following augmentation transforms

yolov5/utils/augmentations.py

Line 28 in 92e47b8

A.CLAHE(p=0.01),

AFAIK, all other transforms except CLAHE are channel order independent.
But, CLAHE considers channel order if image has 3 dimensions.
https://github.com/albumentations-team/albumentations/blob/a8dc46ee29f9573c8569213f0d243254980b02f1/albumentations/augmentations/functional.py#L673

[UnglvKitDe]

@seonho According to the current state YOLOv5 also uses Gray. There it depends on the implementation of cv2.cvtCOLOR() how the RGB image is transformed to a Gray image (I couldn't find the OpenCV implementation). But either way, it is confusing if you want to adjust e.g. the default pipline for your dataset.

[seonho]

@UnglvKitDe Yes, I missed ToGray. According to OpenCV's Color conversion document, RGB2GRAY also affected by channel order.

So, at least two (ToGray, CLAHE) transforms are channel order dependent.

[glenn-jocher]

@UnglvKitDe yes you are correct, there is an inconsistency there is Albumentations is used. The best solution I can think of is to immediately convert to RGB after cv2 imread (and update where required like augment_hsv()). Then we can safely assume RGB-order images throughout all dataloaders.

I think this might be slightly slower than the current implementation which reverses BGR-RGB in the first dimension (CHW) rather than the last (HWC), resulting in a faster BGR-RGB conversion. I'll add a TODO here so I don't forget.

TODO: Fix bug that Albumentations inputs BGR images but expects RGB images.

[UnglvKitDe]

@glenn-jocher I have pushed a first idea.

[glenn-jocher]

@UnglvKitDe thanks! I will take a look at the PR. For reference some BGR to RGB speeds:

import numpy as np
import cv2
import time

im = np.random.rand(640,640,3).astype(np.uint8)

t = time.time()
for _ in range(10000):
  x = im[...,::-1]
print(time.time() - t)

t = time.time()
for _ in range(10000):
  np.flip(im,2)
print(time.time() - t)

t = time.time()
for _ in range(10000):
  cv2.cvtColor(im, cv2.COLOR_BGR2RGB)
print(time.time() - t)

[UnglvKitDe]

@glenn-jocher Oh thats a good point. I thought opencv do something like that. But it looks very slow. Thx!

[glenn-jocher]

@UnglvKitDe you're welcome! Thank you for your understanding. Let me know if you have any questions or need further assistance with the PR.