Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add segmentation and classification support for ClearML #10752

Conversation

thepycoder
Copy link
Contributor

@thepycoder thepycoder commented Jan 12, 2023

This PR adds ClearML support to GenericLogger which is used by the instance segmentation and classification training scripts, also solves issue #10710

The feature list proposed in PR #8620 has been loosely followed, depending on what I had access to through the GenericLogger interface:

  • Current YOLO5 segmentation Tensorboard scalar metrics that can be visualized in a graph with ClearML -> so losses and segmentation mAPs (AP50:95, AP50, etc). The ones YOLO5 logs into "results.csv" file
  • opt.yaml, the config of the run
  • If possible, per class mAP metrics (we get those printed in the console when running inference, but not during training sadly)
  • Inference images on the test set (the one we get with YOLO automatically in the experiment folder)
  • Train batch images (the one we get with YOLO automatically in the experiment folder)
  • (this will be probably too much, but prediction masks on the test set in the COCO format)

The 2 features that are not yet included, aren't because I cannot access them via the existing logging interface. @glenn-jocher does it make sense to extend the GenericLogger for this?

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Enhancements to ClearML logging support in Ultralytics YOLOv5.

📊 Key Changes

  • Added logging of label images and plots to ClearML during pretraining.
  • Improved how scalars (like loss, mAP) are logged to ClearML by introducing a new method log_scalars.
  • Enabled ClearML logging at the end of training to log a summary of results, all generated plots, and to differentiate between the best and last models.
  • ClearmlLogger initialization is now more robust, with exceptions caught for failed setup, and the project_name logic was updated to default to 'YOLOv5' when opt.project starts with 'runs/'.
  • Integrated matplotlib with ClearML to report plots directly.
  • Expanded the logging capabilities to include model weights, final summary metrics, and custom plots.
  • Code cleanup and removal of redundant or commented-out code blocks.

🎯 Purpose & Impact

  • Enhanced Monitoring: Users now have a more comprehensive and granular visualization of their models' training progress in ClearML.
  • Simplified Integration: Simplifying the codebase for ClearML integration makes maintenance easier and enhances user experience.
  • Better Experiment Tracking: Improved logging features allow for better tracking and comparison of different training runs, leading to more informed decision-making for model improvements.
  • Increased Compatibility: The upgrade ensures compatibility between YOLOv5 and the latest features offered by ClearML, facilitating a seamless workflow for those relying on this combination for their ML experiments.

@glenn-jocher
Copy link
Member

@thepycoder thanks for the PR! We'll take a look at this.

@AyushExel

@Robotatron
Copy link

@thepycoder I've tested it and everything works great, thank you!

ps . Could be off topic here but would you know if I have to configure clearML or edit your logger to also log the best model under "artefacts"? I am new to ClearML so could be a stupid question :)

@thepycoder thepycoder changed the title Feature/clearml segmentation classification Add segmentation and classification support for ClearML Jan 16, 2023
@pktiuk
Copy link

pktiuk commented Jun 22, 2023

Is there any chance for merging this?

@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2023

👋 Hello there! We wanted to let you know that we've decided to close this pull request due to inactivity. We appreciate the effort you put into contributing to our project, but unfortunately, not all contributions are suitable or aligned with our product roadmap.

We hope you understand our decision, and please don't let it discourage you from contributing to open source projects in the future. We value all of our community members and their contributions, and we encourage you to keep exploring new projects and ways to get involved.

For additional resources and information, please see the links below:

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

@github-actions github-actions bot added the Stale label Oct 3, 2023
@pktiuk
Copy link

pktiuk commented Oct 3, 2023

Will this pr be closed because nobody had time to look at this?

@glenn-jocher
Copy link
Member

@pktiuk thanks for your contribution! We apologize for the delay in reviewing your pull request. Our team is working hard to review and address all incoming requests. We appreciate your patience in this matter. Rest assured, your pull request will be reviewed and evaluated soon. Thanks again for your contribution to YOLOv5!

@github-actions github-actions bot removed the Stale label Oct 4, 2023
@pktiuk
Copy link

pktiuk commented Nov 30, 2023

@glenn-jocher I hope so

@glenn-jocher
Copy link
Member

@pktiuk we greatly appreciate your understanding and patience. Thank you for your valuable contribution to the YOLOv5 repository! Your PR will be reviewed soon.

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
@glenn-jocher glenn-jocher merged commit 151c953 into ultralytics:master Jan 3, 2024
6 checks passed
pleb631 pushed a commit to pleb631/yolov5 that referenced this pull request Jan 6, 2024
…10752)

* Added ClearML instance segmentation and classification support

* Cleaned up ClearML plot output

* typos

* Log results as plots instead of debug samples

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants