Using export function

[Snitte]

I am perhaps not too bright or have been running in circles for too long

I have been trying to speed up my inference and have found that the export function might be the key. I can however not figure out how to make use of the onnx or torchscript. I have tried installing and running NVIDIA TensorRT but only came up sort as it apparently does not run on windows.
Do any of you have a "for dummies" guide or the likes that would be much appreciated!

[glenn-jocher]

@Snitte I'm not familiar with an 'extract' function. Can you cite a source?

We've written tutorials that cover most of the common YOLO use cases here:

Tutorials

• Train Custom Data  🚀 RECOMMENDED
• Weights & Biases Logging  🌟 NEW
• Multi-GPU Training
• PyTorch Hub  ⭐ NEW
• ONNX and TorchScript Export
• Test-Time Augmentation (TTA)
• Model Ensembling
• Model Pruning/Sparsity
• Hyperparameter Evolution
• Transfer Learning with Frozen Layers  ⭐ NEW
• TensorRT Deployment

[Snitte]

My mistake i ment the export function that exports the weights as an onnx and torchscript. I have folowed the tutorial
and extracted the models however i dont know how to deploy them with onnx or pytorch respectively the most common issue i have run into is that most tutorials require linux and i run windows

[glenn-jocher]

@Snitte export.py should work identically under windows, and export tests are part of the CI tests run on the 3 major operating systems, including windows.

Please ensure you meet all dependency requirements if you are attempting to run YOLOv5 locally. If in doubt, create a new virtual Python 3.8 environment, clone the latest repo (code changes daily), and pip install -r requirements.txt again. We also highly recommend using one of our verified environments below.

Requirements

Python 3.8 or later with all requirements.txt dependencies installed, including torch>=1.7. To install run:

$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are passing. These tests evaluate proper operation of basic YOLOv5 functionality, including training (train.py), testing (test.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu.

[Snitte]

I might have been unclear, i have no problem running yolov5 or the export functon
my problem is using the ONNX or torchscript file i get after i export

[glenn-jocher]

What's your use case, i.e. what are you trying to do with your ONNX or torchscript model?

This is true that we do not extend the export tutorials to inference examples, though in many cases this is not possible to present this concisely as it requires significant follow-on work, i.e. post processing, NMS, etc.

[Snitte]

Im really just trying to speed up my inference

[glenn-jocher]

For speeding up inference in GPU environments you probably want TensorRT. See TensorRT Tutorial:

Tutorials

• Train Custom Data  🚀 RECOMMENDED
• Weights & Biases Logging  🌟 NEW
• Multi-GPU Training
• PyTorch Hub  ⭐ NEW
• ONNX and TorchScript Export
• Test-Time Augmentation (TTA)
• Model Ensembling
• Model Pruning/Sparsity
• Hyperparameter Evolution
• Transfer Learning with Frozen Layers  ⭐ NEW
• TensorRT Deployment

[Snitte]

Thank you!
I will try to write an issue at the tensorrtx page since that is where my problem is as it would seem.