Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception with NMS when using gpus #1004

Closed
chrisway613 opened this issue Apr 3, 2020 · 4 comments
Closed

Exception with NMS when using gpus #1004

chrisway613 opened this issue Apr 3, 2020 · 4 comments
Labels

Comments

@chrisway613
Copy link

chrisway613 commented Apr 3, 2020

When I run:
nohup python detect.py --cfg cfg/yolov3.cfg --weights weights/yolov3.pt --device 1,3 --save-txt > detect.log 2>&1 &

it got the error as below:

image 1/2 data/samples/bus.jpg: Traceback (most recent call last):
  File "detect.py", line 179, in <module>
    detect()
  File "detect.py", line 91, in detect
    pred = non_max_suppression(pred, opt.conf_thres, opt.iou_thres, classes=opt.classes, agnostic=opt.agnostic_nms)
  File "/root/private/yolov3/utils/utils.py", line 557, in non_max_suppression
    i = torchvision.ops.boxes.nms(boxes, scores, iou_thres)
  File "/usr/local/lib/python3.6/dist-packages/torchvision/ops/boxes.py", line 31, in nms
    return torch.ops.torchvision.nms(boxes, scores, iou_threshold)
RuntimeError: Not compiled with GPU support (nms at /tmp/pip-req-build-dkbjnmj4/torchvision/csrc/nms.h:20)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x33 (0x7fba5fd55813 in /usr/local/lib/python3.6/dist-packages/torch/lib/libc10.so)
frame #1: nms(at::Tensor const&, at::Tensor const&, double) + 0x113 (0x7fba4a0ebc03 in /usr/local/lib/python3.6/dist-packages/torchvision/_C.so)
frame #2: c10::detail::wrap_kernel_functor_boxed<c10::detail::WrapRuntimeKernelFunctor_<at::Tensor (*)(at::Tensor const&, at::Tensor const&, double), at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, at::Tensor const&, double> >, true, void>::call(c10::OperatorKernel*, std::vector<c10::IValue, std::allocator<c10::IValue> >*) + 0x77 (0x7fba4a0ff757 in /usr/local/lib/python3.6/dist-packages/torchvision/_C.so)
frame #3: <unknown function> + 0x31d9729 (0x7fb97c23c729 in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so)
frame #4: <unknown function> + 0x3e16013 (0x7fb97ce79013 in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so)
frame #5: <unknown function> + 0x4fe43c (0x7fba60e4f43c in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #6: <unknown function> + 0x4ccda4 (0x7fba60e1dda4 in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #7: <unknown function> + 0x2110f4 (0x7fba60b620f4 in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #8: python() [0x50abc5]
<omitting python frames>
frame #10: python() [0x509ce8]
frame #11: python() [0x50aa1d]
frame #13: python() [0x5081d5]
frame #14: python() [0x50a020]
frame #15: python() [0x50aa1d]
frame #17: python() [0x5064a4]
frame #18: python() [0x50aa1d]
frame #20: python() [0x5081d5]
frame #22: python() [0x635082]
frame #27: __libc_start_main + 0xe7 (0x7fba65454b97 in /lib/x86_64-linux-gnu/libc.so.6)

maybe I should compile sth for torchvision.ops.boxes.nms?
if so, how to do it?
ps: my pytorch version is 1.3, or is this reason?

@github-actions
Copy link

github-actions bot commented Apr 3, 2020

Hello @chrisway613, thank you for your interest in our work! Please visit our Custom Training Tutorial to get started, and see our Google Colab Notebook, Docker Image, and GCP Quickstart Guide for example environments.

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

@chrisway613
Copy link
Author

chrisway613 commented Apr 3, 2020

Hello @chrisway613, thank you for your interest in our work! Please visit our Custom Training Tutorial to get started, and see our Google Colab Notebook, Docker Image, and GCP Quickstart Guide for example environments.

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

I've read them, they are not the answer to my question.
image

it is something wrong with 'torchvision.ops.boxes.nms', i guess maybe the reason of my pytorch version.

@glenn-jocher
Copy link
Member

glenn-jocher commented Apr 3, 2020

@chrisway613 yes you should definitely upgrade to the latest pytorch and torchvision stable releases. You can access working environments as well which have these installed. I'll attach you our default message for working environments here.

Reproduce Our Environment

To access an up-to-date working environment (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled), consider a:

@github-actions
Copy link

github-actions bot commented May 4, 2020

This issue is stale because it has been open 30 days with no activity. Remove Stale label or comment or this will be closed in 5 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants