Model does not detect when on GPU

[Joyel1441]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

When I use GPU in Pytorch the model outputs "no detection", but when i convert that model to cpu it detects objects successfully, what is the issue and how to fix it?
but detect.py uses GPU and detects objects successfully, but model is unable to do it

Additional

This is the code:

import cv2
import numpy as np

cap = cv2.VideoCapture(0)

while cap.isOpened():
ret, frame = cap.read()
result = model(frame)
cv2.imshow("YOLO", np.squeeze(result.render()))
if cv2.waitKey(20) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()

---

here when I convert model to cpu it works fine

[github-actions]

👋 Hello @Joyel1441, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python>=3.6.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

$ git clone https://github.com/ultralytics/yolov5
$ cd yolov5
$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@Joyel1441 it appears you may have environment problems. Hub models employ Automatic Mixed Precision (AMP) context managers (as do train.py and val.py) that are absent from detect.py. If you are having AMP problems this likely stems from environment issues, i.e. possibly CUDA, conda, or windows related.

Please ensure you meet all dependency requirements if you are attempting to run YOLOv5 locally. If in doubt, create a new virtual Python 3.8 environment, clone the latest repo (code changes daily), and pip install -r requirements.txt again. We also highly recommend using one of our verified environments below.

Requirements

Python>=3.6.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

$ git clone https://github.com/ultralytics/yolov5
$ cd yolov5
$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[Joyel1441]

@Joyel1441 it appears you may have environment problems. Hub models employ Automatic Mixed Precision (AMP) context managers (as do train.py and val.py) that are absent from detect.py. If you are having AMP problems this likely stems from environment issues, i.e. possibly CUDA, conda, or windows related.

Please ensure you meet all dependency requirements if you are attempting to run YOLOv5 locally. If in doubt, create a new virtual Python 3.8 environment, clone the latest repo (code changes daily), and pip install -r requirements.txt again. We also highly recommend using one of our verified environments below.

Requirements

Python>=3.6.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

$ git clone https://github.com/ultralytics/yolov5
$ cd yolov5
$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

Still didn't fix the issue, I created new env and tried, still getting same problem

[glenn-jocher]

@Joyel1441 if you're having local environment issues you can use one of our verified environments below.

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

[Joyel1441]

@Joyel1441 if you're having local environment issues you can use one of our verified environments below.

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Is there any way I can run it locally?

[glenn-jocher]

@Joyel1441 we don't assist in debugging local environments, but if I were you I would start from the verified environments to see how to set up your environment, and also refer to https://pytorch.org/get-started/locally/

[Zengyf-CVer]

@Joyel1441
I don't quite understand your sentence, please say it in detail and accurately.

but detect.py uses GPU and detects objects successfully, but model is unable to do it

[Joyel1441]

@Joyel1441 we don't assist in debugging local environments, but if I were you I would start from the verified environments to see how to set up your environment, and also refer to https://pytorch.org/get-started/locally/

@Joyel1441 I don't quite understand your sentence, please say it in detail and accurately.

but detect.py uses GPU and detects objects successfully, but model is unable to do it

When I load model using pytorch hub and pass images it doesn't detect any objects, but when I convert that model to cpu it detects objects
OS: windows 11
GPU: gtx 1650ti
Cuda: 11.3
pytorch: 1.10

[Zengyf-CVer]

@Joyel1441
I understand, it's actually very simple. I read your code. You are using OpenCV to read the video frame. Therefore, you must strictly follow the OpenCV reading when using Pytorch Hub, and you need to convert the format of the picture read by OpenCV. For normal use, you can refer to this: https://docs.ultralytics.com/yolov5/tutorials/pytorch_hub_model_loading

import cv2
import torch
from PIL import Image

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')

# Images
for f in ['zidane.jpg', 'bus.jpg']:
    torch.hub.download_url_to_file('https://ultralytics.com/images/' + f, f)  # download 2 images
img1 = Image.open('zidane.jpg')  # PIL image
img2 = cv2.imread('bus.jpg')[..., ::-1]  # OpenCV image (BGR to RGB)
imgs = [img1, img2]  # batch of images

# Inference
results = model(imgs, size=640)  # includes NMS

# Results
results.print()  
results.save()  # or .show()

results.xyxy[0]  # img1 predictions (tensor)
results.pandas().xyxy[0]  # img1 predictions (pandas)
#      xmin    ymin    xmax   ymax  confidence  class    name
# 0  749.50   43.50  1148.0  704.5    0.874023      0  person
# 1  433.50  433.50   517.5  714.5    0.687988     27     tie
# 2  114.75  195.75  1095.0  708.0    0.624512      0  person
# 3  986.00  304.00  1028.0  420.0    0.286865     27     tie

[Zengyf-CVer]

@Joyel1441
In order to use PytorchHub normally, it is recommended that you use the pillow component.

[Joyel1441]

@Joyel1441 In order to use PytorchHub normally, it is recommended that you use the pillow component.

Even this code doesn't work:
results = model("image.jpg")
results.print()

It outputs no detections

But when I do this:
model = model.cpu()
results = model("image.jpg")
results.print()
It gives correct detections

[Zengyf-CVer]

@Joyel1441
First of all, to be clear, there is no problem with the yolov5 pytorch hub program. I have used this for a long time, and your problem did not appear.
Secondly, I suggest you use anaconda to recreate a virtual environment.
Finally, it is also very important that pytorch is installed using the official website installation instructions. Strictly look at the configuration. To a large extent, your pytorch is problematic. https://pytorch.org/get-started/locally/

[Joyel1441]

@Joyel1441 First of all, to be clear, there is no problem with the yolov5 pytorch hub program. I have used this for a long time, and your problem did not appear. Secondly, I suggest you use anaconda to recreate a virtual environment. Finally, it is also very important that pytorch is installed using the official website installation instructions. Strictly look at the configuration. To a large extent, your pytorch is problematic. https://pytorch.org/get-started/locally/

I guess the problem is with my CUDA installation

[glenn-jocher]

@Joyel1441 yes, sometimes PyTorch installations fail to run inference properly on GPU. This seems to be related to CUDA, conda, or Windows, or some combination of those. The failure occurs when models are reduced in precision using .half() or using an AMP context manager. The observable result of the failure is simply a lack of detections.

Definitely follow https://pytorch.org/get-started/locally/, and consider using the Docker image if you have local environment problems. Lastly you can use one of our verified environments like Google Colab if you are having problems locally, that's why we made them.

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

[Joyel1441]

@Joyel1441 yes, sometimes PyTorch installations fail to run inference properly on GPU. This seems to be related to CUDA, conda, or Windows, or some combination of those. The failure occurs when models are reduced in precision using .half() or using an AMP context manager. The observable result of the failure is simply a lack of detections.

Definitely follow https://pytorch.org/get-started/locally/, and consider using the Docker image if you have local environment problems. Lastly you can use one of our verified environments like Google Colab if you are having problems locally, that's why we made them.

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Oh ok, Thank you for your help, I'll try using verified envs

[IvanLauLinTiong]

@Joyel1441 we don't assist in debugging local environments, but if I were you I would start from the verified environments to see how to set up your environment, and also refer to https://pytorch.org/get-started/locally/

@Joyel1441 I don't quite understand your sentence, please say it in detail and accurately.

but detect.py uses GPU and detects objects successfully, but model is unable to do it

When I load model using pytorch hub and pass images it doesn't detect any objects, but when I convert that model to cpu it detects objects OS: windows 11 GPU: gtx 1650ti Cuda: 11.3 pytorch: 1.10

@Joyel1441
Hi, just my 2 cents, try reinstall everything with downgraded Cuda version 10.x. It might help fixing the GPU detection failure.

I was also facing this issue too and our local working environments are roughy similar, my env is

OS: Windows 10
GPU: gtx 1650
Cuda: 11.3
PyTorch: 1.10

After deep diving through Github issues, here what I found other people have similar problems and they suggest to use Cuda version of 10.x as a workaround :
#1910
#2080

So I used Cuda 10.x (specifically 10.2) with PyTorch 1.10.0 via pip installation:
pip3 install torch==1.10.0+cu102 torchvision==0.11.1+cu102 torchaudio===0.10.0+cu102 -f https://download.pytorch.org/whl/cu102/torch_stable.html

and now it working fine on my machine.

Hope this helps.

[Joyel1441]

@Joyel1441 we don't assist in debugging local environments, but if I were you I would start from the verified environments to see how to set up your environment, and also refer to https://pytorch.org/get-started/locally/

@Joyel1441 I don't quite understand your sentence, please say it in detail and accurately.

but detect.py uses GPU and detects objects successfully, but model is unable to do it

When I load model using pytorch hub and pass images it doesn't detect any objects, but when I convert that model to cpu it detects objects OS: windows 11 GPU: gtx 1650ti Cuda: 11.3 pytorch: 1.10

@Joyel1441 Hi, just my 2 cents, try reinstall everything with downgraded Cuda version 10.x. It might help fixing the GPU detection failure.

I was also facing this issue too and our local working environments are roughy similar, my env is

OS: Windows 10 GPU: gtx 1650 Cuda: 11.3 PyTorch: 1.10

After deep diving through Github issues, here what I found other people have similar problems and they suggest to use Cuda version of 10.x as a workaround : #1910 #2080

So I used Cuda 10.x (specifically 10.2) with PyTorch 1.10.0 via pip installation: pip3 install torch==1.10.0+cu102 torchvision==0.11.1+cu102 torchaudio===0.10.0+cu102 -f https://download.pytorch.org/whl/cu102/torch_stable.html

and now it working fine on my machine.

Hope this helps.

Thanks, I'll try

[Joyel1441]

@Joyel1441 we don't assist in debugging local environments, but if I were you I would start from the verified environments to see how to set up your environment, and also refer to https://pytorch.org/get-started/locally/

@Joyel1441 I don't quite understand your sentence, please say it in detail and accurately.

but detect.py uses GPU and detects objects successfully, but model is unable to do it

When I load model using pytorch hub and pass images it doesn't detect any objects, but when I convert that model to cpu it detects objects OS: windows 11 GPU: gtx 1650ti Cuda: 11.3 pytorch: 1.10

@Joyel1441 Hi, just my 2 cents, try reinstall everything with downgraded Cuda version 10.x. It might help fixing the GPU detection failure.

I was also facing this issue too and our local working environments are roughy similar, my env is

OS: Windows 10 GPU: gtx 1650 Cuda: 11.3 PyTorch: 1.10

After deep diving through Github issues, here what I found other people have similar problems and they suggest to use Cuda version of 10.x as a workaround : #1910 #2080

So I used Cuda 10.x (specifically 10.2) with PyTorch 1.10.0 via pip installation: pip3 install torch==1.10.0+cu102 torchvision==0.11.1+cu102 torchaudio===0.10.0+cu102 -f https://download.pytorch.org/whl/cu102/torch_stable.html

and now it working fine on my machine.

Hope this helps.

Thank you so much, downgrading cuda to 10.2 worked

[github-actions]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com/hub
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

[Rohan-Python]

Cuda 10.2 will require lower version of pytorch and torchvision. Pytorch =1.6 and torchvision=0.7 . So will it be okay to run yolov7 on these versions of pytorch and torchvisions?

[Rohan-Python]

@Joyel1441 we don't assist in debugging local environments, but if I were you I would start from the verified environments to see how to set up your environment, and also refer to https://pytorch.org/get-started/locally/

@Joyel1441 I don't quite understand your sentence, please say it in detail and accurately.

but detect.py uses GPU and detects objects successfully, but model is unable to do it

When I load model using pytorch hub and pass images it doesn't detect any objects, but when I convert that model to cpu it detects objects OS: windows 11 GPU: gtx 1650ti Cuda: 11.3 pytorch: 1.10

@Joyel1441 Hi, just my 2 cents, try reinstall everything with downgraded Cuda version 10.x. It might help fixing the GPU detection failure.
I was also facing this issue too and our local working environments are roughy similar, my env is
OS: Windows 10 GPU: gtx 1650 Cuda: 11.3 PyTorch: 1.10
After deep diving through Github issues, here what I found other people have similar problems and they suggest to use Cuda version of 10.x as a workaround : #1910 #2080
So I used Cuda 10.x (specifically 10.2) with PyTorch 1.10.0 via pip installation: pip3 install torch==1.10.0+cu102 torchvision==0.11.1+cu102 torchaudio===0.10.0+cu102 -f https://download.pytorch.org/whl/cu102/torch_stable.html
and now it working fine on my machine.
Hope this helps.

Thank you so much, downgrading cuda to 10.2 worked

Cuda 10.2 will require lower version of pytorch and torchvision. Pytorch =1.6 and torchvision=0.7 . So will it be okay to run yolov7 on these versions of pytorch and torchvisions?... @Joyel1441 which yolo version are you using. and also let me know the pytorch and torchvision version for which you got succesful detections