Export YOLOv5 to run on Intel/AMD CPU

Export processing consists of two steps:
1️⃣ Convert Pytorch model weights to MNN model weights.
2️⃣ Run the inference on Intel/AMD CPU.
You can compare inference time of YOLOv5 model on two frameworks MNN and Pytorch in my Google Colab (👍👍👍Open and run on Google Chrome recommended).

Documentation

ℹ️ MNN is a lightweight deep neural network inference engine.
🔎 You can find more information about MNN in here.
🔎 MNN github repository in here.

Requirements

👍 python>=3.6 is required.

Installation

Step 1: Convert Pytorch model weights to MNN model weights

If you don't want to install anything on your system then use this Google Colab (Recommended). (👍👍👍Open and run on Google Chrome recommended).

And if you want to perform the conversion on your system then follow bellow instructions:

📣 I recommend to create a new conda environment (python version 3.6 recommended):

$ conda create -n yolov5_conversion python=3.6
$ conda activate yolov5_conversion

➡️ Then run below commands and replace yolov5s.pt with your own model weights in path weights/pt/ and also change yolov5s.yaml in path models/ accordingly.

$ git clone https://github.com/AnhPC03/yolov5.git
$ cd yolov5
$ pip install -r requirements.txt
$ bash export_mnn.sh yolov5s 640

✅ With yolov5s is model name and 640 is input size of your Pytorch model.
✅ After you run above commands, you will see successfully message. And you can find MNN converted model in path weights/mnn/.
✅ Attention about model quantization: The size of MNN model weight is much smaller than origin Pytorch model weight because of using --weightQuantBits 8 in export_mnn.sh file. This reduced model size while also improving CPU and hardware accelerator latency, with little degradation in model accuracy. If you want to keep fully model accuracy, feel free to delete --weightQuantBits 8 in export_mnn.sh file.
✅ Because of running on Intel/AMD CPU, so I decided to quantize my model.

Step 2: Run the inference on Intel/AMD CPU

⚙️ Setup

If you have created conda environment in conversion step then activated it ($ conda activate yolov5_conversion) and follow below steps.
📣 If you don't want to install conda environment into your system, feel free to skip these below commands.
Otherwise I recommend you creat a conda environment (python version 3.6 recommended):

$ conda create -n cpu_mnn python=3.6
$ conda activate cpu_mnn

➡️ Follow below steps to install minimum required environment for converting Pytorch to MNN model weight file:

$ git clone https://github.com/AnhPC03/yolov5-export-to-cpu-mnn.git
$ cd yolov5-export-to-cpu-mnn
$ pip install -r requirements.txt

✅ Replace content of classes.txt file with your classes when you trained your Pytorch model.

🎉 Run inference
🍻 Result will be saved to results/ folder

$ python inference/run_mnn_detector.py \
            --weights path/to/your/mnn/weight \
            --source 0  # webcam
                     file.jpg  # image 
                     file.mp4  # video
                     path/  # directory
                     'https://youtu.be/NUsoVlDFqZg'  # YouTube video
                     'rtsp://example.com/media.mp4'  # RTSP, RTMP, HTTP stream

✅ With path/to/your/mnn/weight is path to MNN model weight which you just converted in the above step.
✅ You can use --nodisplay to do not display image or webcam while inference. Or --nosave to do not save inference results.
✅ For example: my MNN model weight in yolov5-export-to-cpu-mnn/weights/yolov5s.mnn. Then I run inference on images in inference/images/ folder as below:

$ python inference/run_mnn_detector.py --weights weights/yolov5s.mnn --source inference/images