-
Notifications
You must be signed in to change notification settings - Fork 481
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
YOLO NAS'S Precision is significantly lower compare to other later YOLO model even when using same dataset ? #2023
Comments
Not an expert at all but how did you train your model ? I think (if you follow most tutorials), there is several data augmentations step performed by default (pretty hidden) on the training dataset. That was messing up my training a lot, especially the mixup one. |
@YGBRS , I use Roboflow to store and data augmentations , to train I use Google Colab notebook from here https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/train-yolo-nas-on-custom-dataset.ipynb#scrollTo=SdQ5JGblbJTk . |
You can see the list of default data transforms done on your training data with |
It depends if you see the COCO dataset the average of the images have a pixel rate, if you are training to train imagen that the size are too different the results could change, so maybe if you resized your dataset would help |
馃挕 Your Question
I was doing a object detection project , which is a drowning detection . Below is the result of YOLO-NAS S with 250 epochs.
Despite having high recall and mAP value , both precision and f1 score is low under (0.1) . When testing the model , it do quite well on detecting class despite having low precision .
So I try to find if other people face the same issue .
![image](https://private-user-images.githubusercontent.com/117511292/342005708-a94300b9-6594-470f-bde0-5b5bcc031ece.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA4MzI5OTIsIm5iZiI6MTcyMDgzMjY5MiwicGF0aCI6Ii8xMTc1MTEyOTIvMzQyMDA1NzA4LWE5NDMwMGI5LTY1OTQtNDcwZi1iZGUwLTViNWJjYzAzMWVjZS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzEzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxM1QwMTA0NTJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0zYzBlZGZmYWFhMjI0OTRhMjgzM2ZlMDJmZmFkMGY2MTM0MmU0MThkZWE3ODg3ODJmYWJkZmYwNTYzNmI5MmE2JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.9NkE08FISuE2HnZ-dMLbgNqPuJSoWeUbojkYq0M8SB8)
1.#1191
2.#1734
3.Performance Analysis of YOLO-NAS SOTA Models on CAL Tool Detection
YoloNASCALPaper-3.pdf
4.ASSESSING THE EFFECTIVENESS OF YOLO ARCHITECTURES FOR SMOKE AND WILDFIRE DETECTION
![image](https://private-user-images.githubusercontent.com/117511292/342005731-0ed9a520-1db3-4611-bc67-ccb15e4a85b1.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA4MzI5OTIsIm5iZiI6MTcyMDgzMjY5MiwicGF0aCI6Ii8xMTc1MTEyOTIvMzQyMDA1NzMxLTBlZDlhNTIwLTFkYjMtNDYxMS1iYzY3LWNjYjE1ZTRhODViMS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzEzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxM1QwMTA0NTJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iNWE1ZDQwYWUyMjkzYmE0Y2QzMzAzNDMzYzIwYmUzNWZlNGRhZGRmYmEyYzk3MDllZjQ3ZDg1MTI3YWNkOTZmJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.xl8fEO1-5eFpw0DABVDn_cT-A1TI2iRFVdQ6xdizBGM)
![image](https://private-user-images.githubusercontent.com/117511292/342005733-042512f7-0266-4752-ab45-c9e9374ede6d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA4MzI5OTIsIm5iZiI6MTcyMDgzMjY5MiwicGF0aCI6Ii8xMTc1MTEyOTIvMzQyMDA1NzMzLTA0MjUxMmY3LTAyNjYtNDc1Mi1hYjQ1LWM5ZTkzNzRlZGU2ZC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzEzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxM1QwMTA0NTJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1kYzhlZmY3YzI5MDRhNTc3NzdlMjRmNzg1MjY4YjdmMTlhMDk3ZWM0NDBkODAxOWYyNjFmYjY0NjA2MDZmNGIwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.LL_3kSBmcfHk8npAbT8_ePpL4ZRjHOTzSpi_f0cdBb8)
Assessing_the_Effectiveness_of_YOLO_Architectures_for_Smoke_and_Wildfire_Detection.pdf
5.COMPARATIVE STUDY OF YOLOV8 AND YOLO-NAS FOR AGRICULTURE APPLICATION
![image](https://private-user-images.githubusercontent.com/117511292/342005739-6f86eaae-19fa-473f-9040-88d108686503.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA4MzI5OTIsIm5iZiI6MTcyMDgzMjY5MiwicGF0aCI6Ii8xMTc1MTEyOTIvMzQyMDA1NzM5LTZmODZlYWFlLTE5ZmEtNDczZi05MDQwLTg4ZDEwODY4NjUwMy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzEzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxM1QwMTA0NTJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT00OWVkMTcyMDE5ZjBhNzM4MzBiMzQ4YmMxNWFmNGYyODhjMmY1NGFkYWM1YWEyMDFmY2E3NmI0YzFmMTA4ZGI0JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.v8KyyNLCgmJvKxVf3Nw3CG1jEDstWL7Af6iH1JkNO_E)
Comparative_study_of_YOLOv8_and_YOLO-NAS_for_agriculture_application.pdf
From the 5 cases above , it seem like having very low precision but at the same time high recall and mAP is quite normal for YOLO-NAS. When compare with other YOLO version with same dataset, the other YOLO model does not face the same problem . Why does this happen , considering that YOLO NAS generally perform better than other lower YOLO model? Is it because how different YOLO-NAS been evaluate ?
Versions
No response
The text was updated successfully, but these errors were encountered: