YOLO format without normalization and denormalization

[Dipet]

Since yolo and albumentations are normalized formats, we don't need to normalize and denormalize the values in the conversion step.
The previous approach gave round-off errors.

These changes should fix the following issues:

• ValueError: Expected x_max for bbox to be in the range [0.0, 1.0], got 1.0025. #922
• Random Crop yields incorrect value for bounding box #903
• Yolo Bboxes will generate tags larger than 1. #862
• Question about YOLO format in convert_bbox_to_albumentations #883
• Yolo boxes format overflowing image and thowing an exception #848
• ValueError: Expected x_min for bbox (-0.02666666666666667, 0.3022222222222222, 0.10666666666666667, 0.48444444444444446, tensor(1)) to be in the range [0.0, 1.0], got -0.02666666666666667. #679

[glenn-jocher]

@Dipet @ternaus what's the current status of the YOLO normalization bug with Albumentations?

I have a YOLOv5 Albumentations PR in progress in ultralytics/yolov5#3882, but scanning the YOLO-related issues here in the albumentations repo raises some concerns currently.

Can you explain the problem and the solution roadmap if this has not been resolved already? Thanks!

[glenn-jocher]

@Dipet @ternaus is this PR present in v1.0.2? I'm getting this error today using v1.0.2 when training with the new YOLOv5 integration, and I'm certain that none of the YOLOv5 labels are out of bounds, as we strictly check for this on dataloader init.

ValueError: Expected y_max for bbox (0.6265270002186298, 0.9481956716626883, 0.700299333781004, 1.000000024214387, 0.0) to be in the range [0.0, 1.0], got 1.000000024214387.

EDIT: Another anomaly here is that I have no spatial transforms, so I don't see why the bounding boxes would be examined or checked/modified by Albumentations. The transforms I use are here:

            self.transform = A.Compose([
                A.CLAHE(p=0.1),
                A.RandomBrightnessContrast(p=0.0),
                A.RandomGamma(p=0.0),
                A.Blur(p=0.0),
                A.MedianBlur(p=0.0),
                A.ToGray(p=0.0),
                A.ImageCompression(quality_lower=75, p=0.0),
            ],

[glenn-jocher]

@Dipet @ternaus I modified all probabilities to 0.0, but somehow I am still seeing the same error from albumentations. Now I'm doubly confused about why the boxes are being examined when 1) all probabilities are 0.0, and 2 ) there are no spatial transforms present that would affect the bboxes.

            self.transform = A.Compose([
                A.CLAHE(p=0.0),
                A.RandomBrightnessContrast(p=0.0),
                A.RandomGamma(p=0.0),
                A.Blur(p=0.0),
                A.MedianBlur(p=0.0),
                A.ToGray(p=0.0),
                A.ImageCompression(quality_lower=75, p=0.0),
            ],

[glenn-jocher]

@Dipet @ternaus to double check that YOLO is not creating out of bounds labels somehow I inserted this code right before the albumentations call:

        if self.augment:
            if np.any(labels[:, 1:5] > 1.0):  # bboxes > 1.0
                print(labels[:, 1:5])
            
            # Albumentations
            img, labels = self.albumentations(img, labels)

And during my training nothing is printed before the albumentations error. So I suspect that albumentations is performing some operations on the labels that are causing them to lose numerical precision, even though no augmentations are asked for (probabilities are all zeros), and even though no spatial transforms are called for (labels should never change).

Since I see this with v1.0.2, which appears to include this PR, I would say that this PR has not resolved the issue.

[Dipet]

This looks like float numbers rounding error, because np.isclose(1, 1.000000024214387) returns True.

This check is incorrect because it is doesn't check real bbox coordinates. Albumentations uses internal bbox format - some kind of normalized pascal_voc.
To correctly check bbox you need do this steps:

x, y, h, w = bbox
x1 = x - w / 2
x2 = x1 + w
y1 = y - h / 2
y2  = y1 + h

if x1 < 0 or x2 > 1 or y1 < 0 or y2 > 1:
    print(f"Bbbox greater than image: {[x1, y1, x2, y2]}")

[Dipet]

I modified all probabilities to 0.0, but somehow I am still seeing the same error from albumentations. Now I'm doubly confused about why the boxes are being examined when 1) all probabilities are 0.0, and 2 ) there are no spatial transforms present that would affect the boxes.

We have simplified our code and so bboxes and keypoints preprocessors will now be called regardless of whether dual transforms exist in the pipeline or not.

[glenn-jocher]

@Dipet it seems like float32 loses precision on xyxy to xywh and back conversions resulting in the error.

I didn't find a good solution so I simply implemented a clip with epsilon in 'Numerical stability fix for Albumentations' PR ultralytics/yolov5#3958. This adds about 50 us of overhead per image unfortunately in the dataloader.

An alternative might be to implement an epsilon value in your Albumentation checks, i.e. for normalized data this might be something like:

eps = 1E-6
assert -eps < x < (1 + eps)

rather than the current hard bounds:

assert 0.0 < x < 1.0

[creafz]

Hey @glenn-jocher

We have released the 1.0.3 version that should deal with the numerical stability issue on our side.

[glenn-jocher]

@creafz thanks! I've updated our requirements to 1.0.3 in ultralytics/yolov5#4068.

[zhangzhihaozi]

Hey @glenn-jocher

We have released the 1.0.3 version that should deal with the numerical stability issue on our side.
version1.0.3 still raise the same error: (-__-) Expected x_max for bbox (0.7486702127659575, 0.18541666666666667, 1.0053191489361701, 0.65, 0.0) to be in the range [0.0, 1.0], got 1.0053191489361701

[aengusng8]

Someone and me still faced this problem #955 #947 when using .BboxParams(format="coco" with RandomCrop, CenterCrop.

[mikel-brostrom]

My issue was that I:

img = img.transpose((2,0,1)) # H X W C -> C X H X W

before the transformations. Albumentations expect the image in H X W X C which messed up the bbox transformation for me

[hiskuDN]

Check if your bbox is inside the image. Here's a short piece of code I wrote to check that.

def check_bbox_in_image(fileName, bbox_tl, bbox_br):
    # open the image
    image = Image.open(os.path.join('./images', fileName))

    # bounding box to be checked
    bbox = box(bbox_tl[0], bbox_tl[1], bbox_br[0],bbox_br[1])

    # image dimensions
    width, height = image.size

    # check if the bounding box is inside the image
    if bbox[0] >= 0 and bbox[1] >= 0 and bbox[2] <= width and bbox[3] <= height:
        # the bounding box is inside the image
        print("Bounding box is inside the image.")
    else:
        # the bounding box is outside the image
        print("Bounding box is outside the image.")

[sainisatish]

I am facing below error ->
Version - 1.3.1
Bounding Box = [[0.56195, 0.738133, 0.125, 0.181567],
[0.046181, 0.64675, 0.092363, 0.131267],
[0.271012, 0.6556, 0.375, 0.457233]]
Format - Yolo
Error - Expected x_min for bbox (-5.000000000005e-07, 0.5811165, 0.0923625, 0.7123835000000001, 5) to be in the range [0.0, 1.0], got -5.000000000005e-07.

[PawelKinczyk]

@sainisatish me too....

# Define transform pipeline
transform = A.Compose([
    A.RandomResizedCrop(width=1280, height=1280, scale=(0.2, 2)),
    A.HorizontalFlip(p=0.5)
], bbox_params=A.BboxParams(format='yolo', min_visibility=0.8))

[sainisatish]

@sainisatish me too....

# Define transform pipeline
transform = A.Compose([
    A.RandomResizedCrop(width=1280, height=1280, scale=(0.2, 2)),
    A.HorizontalFlip(p=0.5)
], bbox_params=A.BboxParams(format='yolo', min_visibility=0.8))

@PawelKinczyk As of now I have dropped the images that display the mention error.

[PawelKinczyk]

@sainisatish yeah I did the same but in 500 data I dropped 60 images...

[martinurbieta]

This comment was helpful for me.
I had to modified my bbox_params from format='coco' to format='pascal_voc'

        bbox_params=dict(
            type='BboxParams',
            format='pascal_voc',
            label_fields=['gt_bboxes_labels', 'gt_ignore_flags'],
            min_visibility=0.0,
            filter_lost_elements=True),