Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New DetectMultiBackend() class #5549

Merged
merged 67 commits into from
Nov 9, 2021
Merged

Conversation

glenn-jocher
Copy link
Member

@glenn-jocher glenn-jocher commented Nov 7, 2021

Consolidates all backends into a single YOLOv5 DetectMultiBackend() class. Supports following model formats:

  • PyTorch *.pt
  • TorchScript *.torchscript.pt
  • CoreML *.mlmodel
  • TensorFlow saved_model
  • TensorFlow *.pb
  • TensorFlow Lite *.tflite
  • TensorFlow.js web_model
  • ONNX Runtime *.onnx
  • OpenCV DNN *.onnx

Usage

# Export
python export.py --weights yolov5s.pt --include tflite

# Inference
python detect.py --weights yolov5s.tflite
python val.py --weights yolov5s.tflite

πŸ› οΈ PR Summary

Made with ❀️ by Ultralytics Actions

🌟 Summary

Refactoring and optimization of YOLOv5 inference and export.

πŸ“Š Key Changes

  • πŸ“₯ Modified the download URL in the coco128.yaml file.
  • 🧹 Code cleanup in detect.py by removing unused imports and refactoring model loading code.
  • πŸ€– Introduced DetectMultiBackend class to models/common.py for flexible backend support during inference.
  • πŸ” Added JSON export option with model metadata in PyTorch export in export.py.
  • 🀏 Half-precision (FP16) support and device selection improvements for inference code.
  • πŸ‘β€πŸ—¨ Enhanced DetectMultiBackend to include visualization features.
  • πŸ”— Major refactoring in detect.py, val.py, and export.py to utilize new backend class and streamline the codebase.
  • ❌ Removed load_classifier function from utils/torch_utils.py as it was deprecated.

🎯 Purpose & Impact

  • πŸš€ The changes aim to improve code clarity, maintainability, and execution efficiency.
  • ✨ These updates lay the groundwork for more versatile model deployment by unifying the inference pipeline across different computing backends.
  • πŸ‘€ Users will benefit from a more streamlined inference process, potentially faster execution times, and easier integration with various deployment environments.

@glenn-jocher glenn-jocher self-assigned this Nov 7, 2021
@glenn-jocher
Copy link
Member Author

Ok this is PR is ready to merge!

@zldrobit it doesn't make sense to include TensorFlow.js in here because this model needs a javascript environment to run right?

@zldrobit
Copy link
Contributor

zldrobit commented Nov 9, 2021

@glenn-jocher I agree with you. It doesn't make sense, since TesnorFlow.js requires a browser or node.js environment to run and can only be called by javascript.

@glenn-jocher glenn-jocher merged commit 3883261 into master Nov 9, 2021
@glenn-jocher glenn-jocher deleted the add/detect_multi_backend branch November 9, 2021 15:45
@glenn-jocher glenn-jocher linked an issue Nov 9, 2021 that may be closed by this pull request
1 task
BjarneKuehl pushed a commit to fhkiel-mlaip/yolov5 that referenced this pull request Aug 26, 2022
* New `DetectMultiBackend()` class

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* pb to pt fix

* Cleanup

* explicit apply_classifier path

* Cleanup2

* Cleanup3

* Cleanup4

* Cleanup5

* Cleanup6

* val.py MultiBackend inference

* warmup fix

* to device fix

* pt fix

* device fix

* Val cleanup

* COCO128 URL to assets

* half fix

* detect fix

* detect fix 2

* remove half from DetectMultiBackend

* training half handling

* training half handling 2

* training half handling 3

* Cleanup

* Fix CI error

* Add torchscript _extra_files

* Add TorchScript

* Add CoreML

* CoreML cleanup

* New `DetectMultiBackend()` class

* pb to pt fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Cleanup

* explicit apply_classifier path

* Cleanup2

* Cleanup3

* Cleanup4

* Cleanup5

* Cleanup6

* val.py MultiBackend inference

* warmup fix

* to device fix

* pt fix

* device fix

* Val cleanup

* COCO128 URL to assets

* half fix

* detect fix

* detect fix 2

* remove half from DetectMultiBackend

* training half handling

* training half handling 2

* training half handling 3

* Cleanup

* Fix CI error

* Add torchscript _extra_files

* Add TorchScript

* Add CoreML

* CoreML cleanup

* revert default to pt

* Add Usage examples

* Cleanup val

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants