How to get cropped images from --save-crop in sequential order similar to original image?

[ashmalvayani]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

SOLVED, see my last comment.

Question

I've run the inference of detecting multiple digits from an image, and the model is detecting all the numbers correctly; however, when I am cropping these digits and saving them using --save-crop, the outputs that I am getting are not in the order which is necessary for me to further link them to my next step.
the output should be 0 0 1 2 7, whereas the output is 1 7 0 2 0.
Is there any way to preserve the order? or anything through which I can rename the saved cropped images myself?

Additional

[github-actions]

👋 Hello @ashmalvayani, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email support@ultralytics.com.

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@ashmalvayani 👋 Hello! Thanks for asking about cropping results with YOLOv5 🚀. Cropping bounding box detections can be useful for training classification models on box contents for example. This feature was added in PR #2827. You can crop detections using either detect.py or YOLOv5 PyTorch Hub. Crops are generally saved by order of confidence.

detect.py

Crops will be saved under runs/detect/exp/crops, with a directory for each class detected.

python detect.py --save-crop

YOLOv5 PyTorch Hub

Crops will be saved under runs/detect/exp/crops if save=True, and also returned as a dictionary with crops as numpy arrays.

import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')  # or yolov5m, yolov5l, yolov5x, custom

# Images
img = 'https://ultralytics.com/images/zidane.jpg'  # or file, Path, PIL, OpenCV, numpy, list

# Inference
results = model(img)

# Results
crops = results.crop(save=True)  # or .show(), .save(), .print(), .pandas(), etc.

Good luck 🍀 and let us know if you have any other questions!

[Guttappa1238]

Hi @ashmalvayani If you are using torch hub, look at #36. For sorted results i.e. to sort license plate digit detection left-to-right (x-axis):

results = model(im)  # inference
results.pandas().xyxy[0].sort_values('xmin')  # sorted left-right

Hope it helps!

[Andy31798]

I have the same issue as @ashmalvayani. The saved cropped sequence is random. Any ways to let the save crop save from the most left x-axis to the most right x-axis? (Will edit some parts of the code in detect.py help?)

[glenn-jocher]

@Andy31798 not random. Sorted by confidence.

[Andy31798]

@glenn-jocher is there any parts in detect.py that I can modify so that the saved sequence will follow the order from the most left x-axis to the most right x-axis?

[glenn-jocher]

@Andy31798 the for loop that processes the results is here:

yolov5/detect.py

Lines 158 to 172 in c4cb7c6

	# Write results
	for *xyxy, conf, cls in reversed(det):
	if save_txt: # Write to file
	xywh = (xyxy2xywh(torch.tensor(xyxy).view(1, 4)) / gn).view(-1).tolist() # normalized xywh
	line = (cls, *xywh, conf) if save_conf else (cls, *xywh) # label format
	with open(txt_path + '.txt', 'a') as f:
	f.write(('%g ' * len(line)).rstrip() % line + '\n')
	if save_img or save_crop or view_img: # Add bbox to image
	c = int(cls) # integer class
	label = None if hide_labels else (names[c] if hide_conf else f'{names[c]} {conf:.2f}')
	annotator.box_label(xyxy, label, color=colors(c, True))
	if save_crop:
	save_one_box(xyxy, imc, file=save_dir / 'crops' / names[c] / f'{p.stem}.jpg', BGR=True)

[ashmalvayani]

@glenn-jocher can you recommend any changes in lines 158-172 to sort the results. I also found the same problem in this issue, can you tell the logic to sort via x-axis?
#5573

[ashmalvayani]

### Solved

In the yolov5 directory, you'll have a detect.py file, there's a code line for NMS (Non-Max Suppression) in line 126 (depending upon the time you're cloning the GitHub repository it may vary and add the following code.

   # NMS
    pred = non_max_suppression(pred, conf_thres, iou_thres, classes, agnostic_nms, max_det=max_det)

    ## changing code here
    t = pred[0]
    pred = [torch.stack(sorted(t, key=lambda t: t[0]))]
    ## ending changed code here

    dt[2] += time_sync() - t3

This piece of code between ## solved my problem, however, it's giving me reversed sorted output so it's advised to use accordingly to your problem by removing sorted() for eg and many more.

[glenn-jocher]

Hello @Ammar-Azman!

Thanks for sharing the solution that worked best for you. I'm sure it will be helpful to others who are having a similar issue. It's great to see that you were able to make that small modification to detect.py, which is the part of the code that processes detection results. You simply added a line of code that sorts the detections from left to right based on their x-axis coordinates.

It's important to know that sorting the results in a particular way depends on your specific use-case. In the solution you provided, you sorted the detections from left to right based on x-axis, which helped solve your issue. However, be aware that this solution will not work in all cases, as the properties of your input image and what you want to extract from it can vary.

Thanks for reaching out and please let us know if you have any other questions or issues!