Poor FPS Using yolov5n Custom Model?

[MagicalPotato0001]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

PC Specs:
CPU: Ryzen 5 3600 6 Core
GPU1: RTX 2070 8GB
GPU2: RTX 3060Ti 8GB
RAM: 32GB DDR4
CUDA: 11.7
CUDNN: 8.8.1.3
PYTORCH: 2.0

I have trained a custom model using yolov5n, I was hoping to get around at least 30 fps but I am getting closer to 10, so I'm sure its probably user error on my part. All my script should do is take a screenshot of my primary monitor, pass it through the model, and then display it using opencv. Here is my code and any help would be greatly appreciated :)

import pyautogui as pyg
import torch
import numpy as np
import cv2

print('Import Complete')

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(device)
model = torch.hub.load('.', 'custom','best.pt', source='local', force_reload=True) #Imports model
model = model.autoshape()
model.conf = 0.30 #Confidence setting

#Create Window
cv2.namedWindow("Live", cv2.WINDOW_NORMAL) #Creates window
 
#Resize this window
cv2.resizeWindow("Live", 480, 270) #Resizes window

while True:
    img = np.array(pyg.screenshot()) #Takes a screenshot of primary monitor.
    results = model(img) #Passes image through AI.
    r_img = results.render() #Returns a list with the images as np.array.
    img_with_boxes = r_img[0] # image with boxes as np.array
    cv2.imshow('Live', img_with_boxes) #Updates window and displays image.
    if cv2.waitKey(1) == ord('q'):
        break

Additional

No response

[github-actions]

👋 Hello @MagicalPotato0001, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

[glenn-jocher]

@MagicalPotato0001 hi there! Your system specifications seem pretty good and they should provide you with good performance with YOLOv5.

Since you are getting lower performance than expected, there might be a few things you could try. Firstly, I recommend you use a smaller model such as yolov5s, or with fewer anchor boxes, such as yolov5n. These models are faster and may give you the 30 fps you are looking for, especially if your primary use case is object detection.

Another thing you could try is to resize the input images before feeding them into the model, as larger input images require more computations and thus decrease the FPS.

Lastly, you should check that your GPU is being utilized during inference and that there's no other process that might be competing for its resources. You can use your system's task manager to check for this.

I hope this helps! Let me know if you have any other questions.

[MagicalPotato0001]

@glenn-jocher Hi, I am using yolov5n because I tried yolov7 first, I have tried reshaping the images before going into the model but it provided no noticeable differences. As for my GPU utilization while running the script it only uses 3% of my GPU, it seemingly uses my GPU memory but not its processing power.


And on Cuda:

[glenn-jocher]

@MagicalPotato0001 hello there!

It looks like your GPU is not being utilized properly during inference, which might be causing the lower than expected fps. I would suggest ensuring that PyTorch is properly installed with GPU support by running torch.cuda.is_available(). If it returns True, then PyTorch is properly installed with GPU support.

If that's not the case, please try reinstalling PyTorch following the instructions on the official PyTorch website.

If PyTorch is installed properly, then you might want to update your NVIDIA drivers and ensure that CUDA and cuDNN versions match the requirements specified in the YOLOv5 repo requirements.txt file.

Let me know if this helps or if you have any further questions!

[MagicalPotato0001]

@glenn-jocher torch.cuda.is_available() returns true. All drivers are up to date, and i am currently using CUDA v11.7 and CUDNN v8.8.1.3, so i think i have everything installed correctly?

Do note when i used detect.py on a video i was getting 10-20ms inference times, so im guessing it may be a code error?

[glenn-jocher]

@MagicalPotato0001 it looks like your PyTorch installation is properly set up with GPU support and your CUDA and cuDNN versions match the requirements. It's possible that the issue might be related to your code.

One thing I notice in your while loop is that you are calling np.array(pyg.screenshot()) to capture the screen. This might be causing an overhead in processing the image. Instead, you could try using pyg.screenshot().convert('RGB') to capture the screen in RGB format directly.

You could also experiment with different batch sizes when passing images through the model to see if that improves performance. Additionally, you could try using a profiler such as Nvidia's Nsight Systems to identify performance bottlenecks in your code.

Lastly, I see that you are getting good results with detect.py on a video, which indicates that your system is capable of providing better inference times.

I hope these suggestions help you improve the performance of your custom YOLOv5 model. Let me know if you have any further questions!

[MagicalPotato0001]

@glenn-jocher I have installed Nvidia Nsight, as for how to use it I'm a bit lost. I changed the np.array(pyg.screenshot()) to pyg.screenshot().convert('RGB') with no noticeable difference unfortunately. I noticed in Detect.py you use DetectMultiBackend instead of torch.hub.load, is there any reason why, or could that be why mine is slower?

Also im not sure if this is important but i am running this on base Python 3.9, not anaconda or a virtual env.

[MagicalPotato0001]

So i figured something out lol, using time.time() before and after the model inference i found out it isnt the model that is slowing it down, the model is getting roughly 15ms times.


I will update you when i figure out more.

[MagicalPotato0001]

So taking a screenshot adds roughly 45ms of time, that may be the main issue?

[glenn-jocher]

@MagicalPotato0001 thank you for the update! It's great to hear that you were able to pinpoint the overhead in your code when capturing screenshots, which was adding roughly 45ms of time. This was the likely cause of the lower than expected FPS.

To improve performance, you could explore alternative methods to capture frames, such as using a dedicated camera or stream input. Additionally, you could consider optimizing your code to reduce the time it takes to capture and process each frame, especially if the frames are being processed in real-time.

I hope this helps and please let us know if you have any further questions!

[MagicalPotato0001]

@glenn-jocher Do you have any recommendations for modules that will take screenshots faster?

[MagicalPotato0001]

I got it working perfectly, thanks for all your help :)

[glenn-jocher]

@MagicalPotato0001 That's great to hear! I'm glad that you were able to improve the performance of your custom YOLOv5 model. If you have any further questions or run into any issues in the future, please don't hesitate to ask - we are always here to help.

[github-actions]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐