-
-
Notifications
You must be signed in to change notification settings - Fork 15.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is there a script to test the mAP of the engine file? #5278
Comments
👋 Hello @shihanyu, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution. If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you. If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available. For business inquiries or professional support requests please visit https://ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com. RequirementsPython>=3.6.0 with all requirements.txt installed including PyTorch>=1.7. To get started: $ git clone https://github.com/ultralytics/yolov5
$ cd yolov5
$ pip install -r requirements.txt EnvironmentsYOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):
StatusIf this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit. |
@shihanyu no there is not. val.py will only produce metrics on PyTorch models currently. Detect.py will run inference on PyTorch, ONNX runtime, ONNX OpenCV DNN, TorchScript, TFLite models, TF pb and TF saved_model formats. |
@shihanyu coco eval api could be used to get the mAP. In order to do so, you have to create the json (in the coco format) that contain the prediction. You can create this json using the output from the engine. |
@shihanyu good news 😃! Your original issue may now be partially addressed ✅ in PR #5549. This PR consolidates all backends into a single YOLOv5
TensorRT is not currently supported, but if you have experience with it please consider submitting a PR to help others run inference with it. Usage# Export
python export.py --weights yolov5s.pt --include tflite
# Inference
python detect.py --weights yolov5s.tflite
python val.py --weights yolov5s.tflite To receive this update:
Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀! |
Hi, I have solved the problem by combine the test.py in yolov5 and the yolov5trt.py in tensorrtx,so that it can validate the .engine file.
And if any of you want the file, I will give my file to you .
从 Windows 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>发送
发件人: Glenn ***@***.***>
发送时间: 2021年11月10日 6:03
收件人: ***@***.***>
抄送: ***@***.***>; ***@***.***>
主题: Re: [ultralytics/yolov5] Is there a script to test the mAP of the engine file? (Issue #5278)
@shihanyu<https://github.com/shihanyu> good news 😃! Your original issue may now be partially addressed ✅ in PR #5549<#5549>. This PR consolidates all backends into a single YOLOv5 DetectMultiBackend() class. Supports following model formats:
* [X] PyTorch *.pt
* [X] TorchScript *.torchscript.pt
* [X] CoreML *.mlmodel
* [X] TensorFlow saved_model
* [X] TensorFlow *.pb
* [X] TensorFlow Lite *.tflite
* [ ] TensorFlow.js web_model
* [X] ONNX Runtime *.onnx
* [X] OpenCV DNN *.onnx
TensorRT is not currently supported, but if you have experience with it please consider submitting a PR to help others run inference with it.
Usage
# Export
python export.py --weights yolov5s.pt --include tflite
# Inference
python detect.py --weights yolov5s.tflite
python val.py --weights yolov5s.tflite
To receive this update:
* Git<https://github.com/ultralytics/yolov5> – git pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
* PyTorch Hub<https://pytorch.org/hub/ultralytics_yolov5/> – Force-reload model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
* Notebooks<https://github.com/ultralytics/yolov5/blob/master/tutorial.ipynb> – View updated notebooks [Open In Colab] <https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb> [Open In Kaggle] <https://www.kaggle.com/ultralytics/yolov5>
* Docker<https://hub.docker.com/r/ultralytics/yolov5> – sudo docker pull ultralytics/yolov5:latest to update your image [Docker Pulls] <https://hub.docker.com/r/ultralytics/yolov5>
Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#5278 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AE546SSRCR722BW3PUD5XHTULGLCBANCNFSM5GNU7Z5Q>.
Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
@shihanyu hmm interesting. Maybe I'll contact the trtx author to see if he'd help us with a PR. |
I will be very glad if I can help. Looking forward to your good news.
从 Windows 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>发送
发件人: Glenn ***@***.***>
发送时间: 2021年11月10日 18:58
收件人: ***@***.***>
抄送: ***@***.***>; ***@***.***>
主题: Re: [ultralytics/yolov5] Is there a script to test the mAP of the engine file? (Issue #5278)
@shihanyu<https://github.com/shihanyu> hmm interesting. Maybe I'll contact the trtx author to see if he'd help us with a PR.
―
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#5278 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AE546SXCS34GCMNHXWKJXKTULJF43ANCNFSM5GNU7Z5Q>.
Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
Hi, I just apply the merge request for the function to validate the tensorrt model file. Hope it helps, let me know if it works or not.
Best Regards
Hanyu
从 Windows 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>发送
发件人: Glenn ***@***.***>
发送时间: 2021年11月10日 18:58
收件人: ***@***.***>
抄送: ***@***.***>; ***@***.***>
主题: Re: [ultralytics/yolov5] Is there a script to test the mAP of the engine file? (Issue #5278)
@shihanyu<https://github.com/shihanyu> hmm interesting. Maybe I'll contact the trtx author to see if he'd help us with a PR.
―
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#5278 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AE546SXCS34GCMNHXWKJXKTULJF43ANCNFSM5GNU7Z5Q>.
Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
@shihanyu great, I will take a look today! |
Hi, does the script work ? Do I need to reform my code? Maybe ajust the structure?
从 Windows 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>发送
发件人: Glenn ***@***.***>
发送时间: 2021年11月12日 19:25
收件人: ***@***.***>
抄送: ***@***.***>; ***@***.***>
主题: Re: [ultralytics/yolov5] Is there a script to test the mAP of the engine file? (Issue #5278)
@shihanyu<https://github.com/shihanyu> great, I will take a look today!
―
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#5278 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AE546SWXOKYIQ6XBMQEERV3ULT2RTANCNFSM5GNU7Z5Q>.
Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
Hi Glenn
My requrest for merge failed because it can not install pycuda …,any advice for that ?
Best regards
hanyu
从 Windows 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>发送
…________________________________
发件人: Glenn Jocher ***@***.***>
发送时间: Friday, November 12, 2021 7:25:13 PM
收件人: ultralytics/yolov5 ***@***.***>
抄送: shihanyu ***@***.***>; Mention ***@***.***>
主题: Re: [ultralytics/yolov5] Is there a script to test the mAP of the engine file? (Issue #5278)
@shihanyu<https://github.com/shihanyu> great, I will take a look today!
―
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#5278 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AE546SWXOKYIQ6XBMQEERV3ULT2RTANCNFSM5GNU7Z5Q>.
Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
@shihanyu yes, all 3rd party dependencies that are not part of requirements.txt can be placed in check_requirements() calls only if/when they are actually used. For example ONNX inference first checks to see if Lines 315 to 319 in e80a09b
|
👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs. Access additional YOLOv5 🚀 resources:
Access additional Ultralytics ⚡ resources:
Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed! Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐! |
🚀 Feature
Motivation
Pitch
Alternatives
Additional context
The text was updated successfully, but these errors were encountered: