yolov8-segmentation.Onnx Static Quantization

[shimaamorsy]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

I'm struggling to find the material to help me for solving my task. I'm trying to speed up the performance of YOLOv5-segmentation using static quantization. I have followed the ONNX Runtime official tutorial on how to apply static quantization

However, I encountered an error when i tried to preprocess the library

To reproduce
ONNX Runtime official tutorial

Additional

No response

[github-actions]

👋 Hello @shimaamorsy, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

[glenn-jocher]

Hello there! 🌟

It looks like you're on an adventurous path with static quantization for YOLOv5-segmentation models. For this specific issue regarding preprocessing the library and the error you've encountered, it's key to ensure that all the prerequisites for the ONNX Runtime and model compatibility are met. This often includes making sure the model is properly exported and conforms to ONNX standards expected by the quantization process.

Given that the evolving nature of models and ONNX tools might introduce variable factors, double-checking the versions of both YOLOv5 (ensuring it's up to date) and your ONNX Runtime environment is a good start. Compatibility between these can significantly influence the outcome.

If the problem persists, a detailed error log and the exact steps you've followed (beyond the tutorial reference) would be incredibly helpful for diagnosis. Please include the specific command or code snippet that leads to the error, alongside the error message itself (minus the screenshot, kindly paste the text of the error instead).

Broadly speaking, a common step for ONNX model quantization involves:

1. Exporting the YOLOv5 model to ONNX if not already done.
2. Ensuring the model input and output tensors are correctly set up for quantization.
3. Using the ONNX Runtime tools to apply static quantization.

For detailed steps and updated methods, kindly refer to the ONNX Runtime documentation and ensure all installation requirements are accurately met.

If there's anything more specific or another hurdle arises, feel free to share additional details! Your quest for efficiency is absolutely commendable, and we're here to help navigate through it. 🚀

[shimaamorsy]

Firstly, I'd like to express my gratitude for your response.
Initially, I exported yolov8-seg.onnx.pt to the ONNX format :
import ultralytics model = YOLO('yolov8n-seg.pt') model.export(format='onnx')
Following that, I attempted to preprocess it for static quantization:
!python -m onnxruntime.quantization.preprocess --input yolov8n-seg.onnx --output yolov8n-seg-processed.onnx

However, I encountered the following error:
2024-04-11 21:05:44.0934723 [W:onnxruntime:, graph.cc:108 onnxruntime::MergeShapeInfo] Error merging shape info for output. '/model.22/Split_output_0' source:{1,64,8400} target:{1,72,8400}. Falling back to lenient merge. 2024-04-11 21:05:44.0966858 [W:onnxruntime:, graph.cc:108 onnxruntime::MergeShapeInfo] Error merging shape info for output. '/model.22/Split_output_1' source:{1,80,8400} target:{1,72,8400}. Falling back to lenient merge. 2024-04-11 21:05:44.1253802 [W:onnxruntime:, graph.cc:108 onnxruntime::MergeShapeInfo] Error merging shape info for output. '/model.22/Sigmoid_output_0' source:{1,80,8400} target:{1,72,8400}. Falling back to lenient merge. 2024-04-11 21:05:44.1356836 [W:onnxruntime:, graph.cc:108 onnxruntime::MergeShapeInfo] Error merging shape info for output. 'output0' source:{1,116,8400} target:{1,108,8400}. Falling back to lenient merge.

I have limited experience with ONNX and quantization, and I'm still in the early stages of learning. Therefore, I don't possess extensive knowledge about resolving these errors. I would greatly appreciate your help !

[glenn-jocher]

Hi there! 👋

Thanks for reaching out and detailing your issue with quantizing the YOLOv8-segmentation model to ONNX format. I'm here to help!

The error message you're encountering seems related to shape mismatches during the preprocessing for static quantization. This could suggest that the ONNX export didn't preserve the expected tensor shapes, or that the quantization tool is interpreting them differently.

One potential workaround is ensuring your model and tools (both Ultralytics and ONNX Runtime) are fully up-to-date, as both are actively developed and issues might be resolved in newer versions.

Also, when exporting the model, you've done it correctly. Still, it's essential to ensure that the ONNX model correctly reflects the architecture of your .pt model, especially regarding the dimensions of its layers and outputs.

Given the complexity of the issue and considering your familiarity level with ONNX, my suggestion is to start with simpler models (if possible) to get the hang of the quantization process without the added complexities of segmentation models. This will help you build up your understanding gradually.

For the specific error, unfortunately, without a deeper dive into the precise architecture of your model and the complete stack trace, it's challenging to provide a pinpoint solution. However, addressing discrepancies in layer output sizes, as indicated by the error, may involve reviewing the model architecture or seeking updates/patches that address such mismatches during ONNX export or quantization.

Lastly, don’t hesitate to dive into the ONNX and ONNX Runtime documentation for quantization. It contains a wealth of information that can sometimes offer insights into these very specific problems.

Keep experimenting, and don't get discouraged by setbacks. Your journey is part of the learning experience! If you have more details or other questions come up, feel free to ask. 🚀

[shimaamorsy]

Thank you for replying .

I updated the onnxruntime, but unfortunately, the same error still appears.

These are the models I am trying to quantize:
yolov5n-seg.onnx

yolov8n-seg.onnx

[glenn-jocher]

Hi there!

Thanks for updating the ONNX Runtime and sharing the models you're working with. Given the persistence of the error even after updating, it seems we might be looking at an issue specific to the model’s structure or the quantization process itself.

Considering the error nature and without being able to directly access the models via the links provided, I would suggest a couple of things that might help:

• Verify Model Compatibility: Ensure that the models in question, especially with segmentation tasks, are fully compatible with ONNX Runtime's quantization process. Certain model constructs might not be fully supported yet.
• Consult ONNX Runtime Docs: Given the complexity around quantization, ONNX Runtime offers a variety of nuances that could be crucial. Make sure to consult the ONNX Runtime documentation on quantization for any updates or specific guidance around models like YOLOv5-segmentation.
• Model Simplification: If possible, attempt to quantize a simpler version of the model (e.g., without segmentation tasks) to isolate whether the issue stems from the segmentation specifics or a more general quantization mismatch.

Without having a direct look at the error logs or having more context about the preprocessing steps, these suggestions aim to guide you towards potential areas of resolution.

Keep pushing through, and please, if there are any new insights or errors that come up, don't hesitate to share. Your efforts bring valuable learning opportunities for everyone involved! 🚀

[github-actions]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐