run onnx model in opencv

[hrshovon]

❔Question

Hello all,
I trained yolov5 on my own data and created onnx model using the export script. The coreml model failed but onnx model was created successfully. But when I try to import this model in opencv(4.3.0-dev and 4.4.0-dev), I get the following error:
`cv2.error: OpenCV(4.4.0-dev) /tmp/ocv_bld/opencv/modules/dnn/src/onnx/onnx_importer.cpp:562: error: (-2:Unspecified error) in function 'void cv::dnn::dnn4_v20200609::ONNXImporter::populateNet(cv::dnn::dnn4_v20200609::Net)'

Slice layer only supports steps = 1 (expected: 'countNonZero(step_blob != 1) == 0'), where
'countNonZero(step_blob != 1)' is 1
must be equal to
'0' is 0`
How can I solve this?

[github-actions]

Hello @hrshovon, thank you for your interest in our work! Please visit our Custom Training Tutorial to get started, and see our Jupyter Notebook , Docker Image, and Google Cloud Quickstart Guide for example environments.

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom model or data training question, please note Ultralytics does not provide free personal support. As a leader in vision ML and AI, we do offer professional consulting, from simple expert advice up to delivery of fully customized, end-to-end production solutions for our clients, such as:

• Cloud-based AI systems operating on hundreds of HD video streams in realtime.
• Edge AI integrated into custom iOS and Android apps for realtime 30 FPS video inference.
• Custom data training, hyperparameter evolution, and model exportation to any destination.

For more information please visit https://www.ultralytics.com.

[zsy9512]

I also meet this error,have you solved this now ？or, do you have any ohter way to run the yolov5s.onnx model ?thanks

[hrshovon]

I have not received any reply from the developer no unfortunately no. as for running model, I guess now pytorch might be the only option.

[erheng-dawnlight]

Is there any solution? I also encountered this issue. The original code worked well with yolov3. Thanks.

[zsy9512]

Is there any solution? I also encountered this issue. The original code worked well with yolov3. Thanks.

You can try on onnxruntime C++ ，I find that is ok when I use onnx1.6 and opset_version=11 to export the model.May you can refrence this sample https://github.com/microsoft/onnxruntime/blob/master/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests.Capi/CXX_Api_Sample.cpp

[erheng-dawnlight]

Thanks @zsy9512 . We finally served the onnx model using tersorrt.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ihmc3jn09hk]

Same issue. The exported file does not work.
Any solution for OpenCV ?

[hfzarslan]

Same issue any solution please

[glenn-jocher]

@ihmc3jn09hk @hfzarslan for running YOLOv5 in a local python project PyTorch Hub is the recommended method. See PyTorch Hub tutorial for more details:
https://docs.ultralytics.com/yolov5

import torch
from PIL import Image

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True).autoshape()  # for PIL/cv2/np inputs and NMS

# Images
img1 = Image.open('zidane.jpg')
img2 = Image.open('bus.jpg')
imgs = [img1, img2]  # batched list of images

# Inference
prediction = model(imgs, size=640)  # includes NMS

[ihmc3jn09hk]

@ihmc3jn09hk @hfzarslan for running YOLOv5 in a local python project PyTorch Hub is the recommended method. See PyTorch Hub tutorial for more details:
https://docs.ultralytics.com/yolov5

import torch
from PIL import Image

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True).autoshape()  # for PIL/cv2/np inputs and NMS

# Images
img1 = Image.open('zidane.jpg')
img2 = Image.open('bus.jpg')
imgs = [img1, img2]  # batched list of images

# Inference
prediction = model(imgs, size=640)  # includes NMS

Hi @glenn-jocher , thank you for your suggestions. But I think the question of this thread is either

1. the exportation of the yolov5 model into other NN model frameworks supported by opencv or,
2. a Pytorch C++ implementation of yolov5.

[glenn-jocher]

@ihmc3jn09hk hmm ok I see. I don't have much experience with ML models in opencv or C++, but perhaps the torchscript export is the easiest way to get a model into C?

ONNX export should work also, though in its current open-source form it lacks the Detect() layer outputs, and would require additional handling that would have to be provided separately.

[ihmc3jn09hk]

@ihmc3jn09hk hmm ok I see. I don't have much experience with ML models in opencv or C++, but perhaps the torchscript export is the easiest way to get a model into C?

ONNX export should work also, though in its current open-source form it lacks the Detect() layer outputs, and would require additional handling that would have to be provided separately.

@glenn-jocher I guess @hrshovon has tried on the exported ONNX which produced the errors mentioned at the top. I have tried export ONNX too and received same results. When browsing through the issue-list, I can see many others have tried with no success. Some have claimed a work-around is possible but no example is given (#1216 (@InfiniteLife)).
There is one very good suggestion at (#1043 (@MohamedAliRashad)) which I think worth to take deeper look.

[erolgerceker]

same issue. any solution?

[hpc203]

❔Question

Hello all,
I trained yolov5 on my own data and created onnx model using the export script. The coreml model failed but onnx model was created successfully. But when I try to import this model in opencv(4.3.0-dev and 4.4.0-dev), I get the following error:
`cv2.error: OpenCV(4.4.0-dev) /tmp/ocv_bld/opencv/modules/dnn/src/onnx/onnx_importer.cpp:562: error: (-2:Unspecified error) in function 'void cv::dnn::dnn4_v20200609::ONNXImporter::populateNet(cv::dnn::dnn4_v20200609::Net)'

Slice layer only supports steps = 1 (expected: 'countNonZero(step_blob != 1) == 0'), where
'countNonZero(step_blob != 1)' is 1
must be equal to
'0' is 0`
How can I solve this?

I solve the problem, please watch my github code
https://github.com/hpc203/yolov5-dnn-cpp-python

[naruarjun]

Hello @erheng-dawnlight @hpc203 , I wanted to ask what kind of FPS improvement and accuracy drops(if any) did you guys observe after the conversion and running in CPP?

[pooya-mohammadi]

@naruarjun did you see FPS improvements?

[glenn-jocher]

@hrshovon @zsy9512 @erheng-dawnlight @ihmc3jn09hk @hfzarslan @hpc203 @practical-ai @erolgerceker good news 😃! Your original issue may now be fixed ✅ in PR #4833 by @SamFC10. This PR implements architecture updates to allow for ONNX-exported YOLOv5 models to be used with OpenCV DNN.

To receive this update:

• Git – git pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
• PyTorch Hub – Force-reload with model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
• Notebooks – View updated notebooks
• Docker – sudo docker pull ultralytics/yolov5:latest to update your image

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!

[PauloMendes33]

Any sugestion on how to use Yolov5 onnx file with opencv? Last time I checked it result in errors.

[glenn-jocher]

@PauloMendes33 its very simple:

python detect.py --weights yolov5s.onnx

[PauloMendes33]

I understand and tested it and it runs very well. It works fine.

The problem is that I am trying to read the model with OpenCV DNN module with the function
auto net = cv::dnn::readNetFromONNX("best.onnx")
and it returns an error:
error: (-210: Unsupported format or combination of formats) Failed to parse onnx model in function 'ONNXImporter'

[glenn-jocher]

@PauloMendes33 I would raise errors generated by external packages directly with their authors, they are outside of Ultralytics scope and control.