Training with images of different sizes?

[ambondarev]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

My use case for yolov5 (6.2) is using IP camera feeds to improve detection for people & vehicles.

1. With IP cameras, they come with a wide arrange of resolutions, from 480p, 720p, 1080p, 4k, etc. I was reading that all the trained images should be the same size for performance reasons, however I am unsure the best approach given the camera footage that is annotated and used for training can range from 480p to 4k and have different aspect ratios. What is the recommended approach here? Would I need to break up the images by size then train them at their respective sizes?
2. When discussing performance are we referring to do the detection performance or just the performance when it comes to training the model? I didn't see much of a performance difference between an image size of 2048 and 640 when it came to detections.
3. Separately, my first model was trained on a 2048 x 1536 (padded) camera footage. I annotated where the people are, but it makes me wonder if a better approach would be to crop the annotations with something like 20px padding and use that instead? The only issue I see here is the model doesn't see the full area and it doesn't do anything for "false positive" reinforcement.

I appreciate any insight here.

Additional

No response

[github-actions]

👋 Hello @ambondarev, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

[github-actions]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

[ambondarev]

If anyone stumbles upon this thread. I asked the following questions to Ultralytics a while back, and I am providing a copy of the responses in the event it helps someone.

Q: My main question is in regards to image size and how it pertains to training and what it means for detection performance. I am building a model using yolov5 to be used with IP cameras for detection, I am using the footage from our warehouses and online camera streams. I have an almost endless supply of data to use for this as a result. The problem is I have footage ranging from 480p, 720p, 1080p, all the way to 3840x2160 (4K). Let's assume the initial data set for my base is images that are 2048x1536 (1500 of them) and maybe some sprinkling of 640x480. After that model is made, I collect more data to improve the model. Suppose the next batch of images are half 4k and half 1080p. Does the existing model require I continue using 2048 for image size? or can I use 4k for my image size on the next dataset for the model?

A: The input size to the YOLO model is typically square and can be adjusted based on the specifics of your dataset and application. Changing the input size can influence both the training speed and the detection performance. You do not necessarily have to continue using the same image size for future data, but a model trained on lower resolution images may not perform as well when applied to higher resolution images.

Q: This leads me to my next question, if the sizes dont matter, then would it be better if I organize my dataset training by image size? For instance, train the model on all 2048 images, then 3840, then 640, etc? Would this create any issues?

A: Organizing your training by image size might lead to some improvements, especially if the objects of interest appear at similar scales in each image size group. However, it may not be necessary and could complicate the training process.

Q: Lastly, let's say hypothetically you have a dataset of 100k images with people at 4k resolution, if I use an image size of 640, I am aware that the images will be resized and training will be faster than if I trained with 100k images at 4k, however they will be less accurate. Since I intend to use these for IP Cameras where a lot of frames will be passed through for detection, will the time to generate a detection result be slower if I trained my model with 4K images? In my testing with 1200 images I didn't notice anything that stood out to me, but 100k is a lot more than 1200k, so I wanted to make sure I didn't waste a ton of time. The 4k resolution also generated a much better result whereas the 640 thought a vacuum was a person.

A: The detection time of the trained model will not be affected by the resolution of the images it was trained on, but rather by the resolution of the images it is being asked to make predictions on. So if you train a model on 4K images but apply it to 640x480 images, it will not take longer to make predictions.

[glenn-jocher]

@ambondarev changing the input size can have an impact on both training speed and detection performance. While you don't necessarily have to continue using the same image size for future data, it's important to note that a model trained on lower resolution images may not perform as well when applied to higher resolution images.

Organizing your training by image size could potentially lead to improvements, especially if the objects of interest appear at similar scales in each image size group. However, it's not necessary and may complicate the training process.

Regarding the time required to generate a detection result, it is not affected by the resolution of the images the model was trained on, but rather by the resolution of the images it is making predictions on. So if you train a model on 4K images but apply it to 640x480 images, it will not take longer to make predictions.

I hope this clarifies your questions. If you have any further inquiries, feel free to ask.

[ambondarev]

@glenn-jocher hahah, seems like we wrote the same thing. I provided your answers from email above. I did shoot you a message though just now, so I will copy the questions here:

Suppose there was a base model already created that had the following classes, a person, a vehicle. I don't have the training media for the model.

1. Would the only way to remove a "vehicle" class itself from an existing model be to make a new model with new classes?

2a. If I wanted to break up the vehicle class into things like buses, cars, trailers, bikes, while still counting them as vehicles, Would I just annotate 2 boxes one as a bus and one as a vehicle on new images?

2b. Second part of that, suppose a model was trained with pictures of cars as vehicles and down the road, I wanted bicycles to be counted as vehicles. If the previous media had pictures of bicycles next to cars that were not annotated would that cause problems if I suddenly wanted to count bicycles as vehicles or would I have to go back and annotate all the bicycles in the previous media?

[glenn-jocher]

Hey @ambondarev! 😄 Indeed, it seems we were on the same page. Regarding your questions:

1. Removing a "vehicle" class from an existing model would require creating a new model with the desired classes.

2a. You can break up the "vehicle" class into subcategories by annotating separate bounding boxes for each subcategory (e.g., buses, cars, trailers, bikes) while still counting them as vehicles.

2b. If you want bicycles to be counted as vehicles in previously unannotated media, it's advisable to go back and annotate the bicycles in the previous media. The model's performance may be impacted if it encounters scenarios it was not trained on.

Feel free to ask anything else. I'm here to help!