Openvino Usage

[I3orn2FLY]

Does anyone know how to use it in OpenVINO toolkit c++?❔

Additional context

I converted yolov5 to ONNX using export.py, then I cloned the OpenVINO toolkit and installed an inference engine. Then I got the IR files by using mo.py in the cloned OpenVINO toolkit. I managed to run the IR model somehow by using sample codes in OpenVINO. However, now I do not know how to decode the output of the model. Is the procedure the same as in samples of yolov3? Does someone have a script in c++? If yes, I would be more than happy for clues.

[github-actions]

Hello @I3orn2FLY, thank you for your interest in our work! Please visit our Custom Training Tutorial to get started, and see our Jupyter Notebook , Docker Image, and Google Cloud Quickstart Guide for example environments.

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom model or data training question, please note Ultralytics does not provide free personal support. As a leader in vision ML and AI, we do offer professional consulting, from simple expert advice up to delivery of fully customized, end-to-end production solutions for our clients, such as:

• Cloud-based AI systems operating on hundreds of HD video streams in realtime.
• Edge AI integrated into custom iOS and Android apps for realtime 30 FPS video inference.
• Custom data training, hyperparameter evolution, and model exportation to any destination.

For more information please visit https://www.ultralytics.com.

[I3orn2FLY]

Hi guys, I managed to decode the output of the model. First of all, look at the yolo.py, Detect class where the forward function is written, it seems like when the model is converted to ONNX anything which is covered in if statements are not exported, therefore, you will need to rewrite that part of the code in your project as well. Here is the if statement, which you should implement in your project:

yolov5/models/yolo.py

Line 47 in bd5cfff

if not self.training: # inference

. Note that you can rewrite Detect.py so that at least sigmoid is run before this if statement, so it will be exported with sigmoid. There are also some details about yolo decoding (regarding anchor boxes / grid) which you should know if you want to rewrite it in your project.

[ALMerrill]

@I3orn2FLY, do you have a good understanding of what the code is doing inside of that if statement? I'm working on doing this for coreml, but I'm having a hard time understanding what the lines inside the if statement are doing in order to convert them to coreml. Do you have any insight on this?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ZhixiongSun]

Hello, you can get an executable cpp code here: https://github.com/fb029ed/yolov5_cpp_openvino,
this is based on yolov5 v3.1

[glenn-jocher]

@I3orn2FLY @ALMerrill @ZhixiongSun good news 😃! Your original issue may now be fixed ✅ in PR #6057. This PR adds native YOLOv5 OpenVINO export:

python export.py --weights yolov5s.pt --include openvino  # export to OpenVINO

To receive this update:

• Git – git pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
• PyTorch Hub – Force-reload model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
• Notebooks – View updated notebooks
• Docker – sudo docker pull ultralytics/yolov5:latest to update your image

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!