test_dir (val/test) for classification training

[maxsitt]

Search before asking

• I have searched the YOLOv5 issues and found no similar bug report.

YOLOv5 Component

Training

Bug

Currently the classify/train.py script uses images from the dataset test split for validation during model training if both a test and val folder exist in the dataset folder (see here).

I'm wondering if there is a specific reason for this behaviour, as normally I would expect that the val directory is preferred to load the images for validation if both test and val folders exist in the dataset folder. If the validation is already done on the test folder during model training, subsequent validation on the test folder could lead to misleading results if the user is not aware of this behaviour.

If there is no specific reason to prefer the test folder for validation, it would probably be better to align the behaviour with the detection model training and user expectation: that the val folder is used for validation during training and the test folder can be used for subsequent validation on the test split.

Thanks!

Environment

No response

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[glenn-jocher]

@maxsitt thank you for bringing this up. The reason why test split is used by default for validation during model training if both test and val folders exist in the dataset folder is to align with the behavior of the original YOLOv5 paper. However, we totally agree with you that this could lead to misleading results if the user is not aware of this behavior, and using val instead of test folder for validation would be more intuitive. Therefore, we will update the script to use the val folder for validation during model training if both test and val folders exist in the dataset folder. Thank you again for bringing this to our attention!

[maxsitt]

@glenn-jocher thank you for the explanation and for the upcoming update!

[github-actions]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

[glenn-jocher]

@maxsitt you're welcome! We greatly appreciate your understanding and contribution to the YOLO community. If you have any more questions or suggestions, feel free to reach out. Thank you!