Weights & Biases (W&B) Feature Addition

[AyushExel]

What does this PR do?

This is a cleaned-up and updated version of this PR #1108 which adds W&B logging support.
Updates:

• The code no longer typecasts co-ordinates into float before passing them to the logging dictionary.
• Reformatted the code
  Some of the import statement orders didn't pass through the PEP8 linter but these were not related to the changes I have made to I didn't edit them.

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Enhanced YOLOv5 logging with Weights & Biases integration for better experiment tracking and visualization 📈.

📊 Key Changes

• Added Weights & Biases (wandb) logging as a dependency in requirements.txt.
• Included an option in test.py to log images with predictions during testing.
• Updated train.py function signature to accept a wandb parameter for logging.
• Initialized wandb logging in train.py to track experiments and log model performance metrics and hyperparameters.
• Implemented functionality to save wandb run ID in checkpoints to resume logging from a specific point.
• Provided an option in the argument parser of train.py to set the number of images logged to wandb.

🎯 Purpose & Impact

• Enhanced Experiment Tracking: Enabling wandb integration supports researchers and developers in tracking experiments, comparing results, and visualizing model performance more effectively.
• Better Visualization: Logging images with predictions allows for quick and easy qualitative analysis of the model's performance.
• Streamlined Workflows: Saving the wandb run ID in checkpoints allows users to continue logging from a specific point without starting a new logging instance, ensuring continuity in metric tracking.
• Customization: Users can customize the number of images they want to log via the --log-imgs command-line argument, tailoring the verbosity of visual logging to their needs.

In summary, these updates will make it simpler for users to monitor, analyze, and share their YOLOv5 model training and testing processes, promoting more informed and iterative machine learning development. 🚀

[glenn-jocher]

@AyushExel I noticed test.py imports wandb directly without a try statement. I'm surprised CI tests are passing like this, I would have thought this would produce an error. Will investigate.

EDIT: the import is inside an if, perhaps this is the reason.

[AyushExel]

@AyushExel I noticed test.py imports wandb directly without a try statement. I'm surprised CI tests are passing like this, I would have thought this would produce an error. Will investigate.

EDIT: the import is inside an if, perhaps this is the reason.

@glenn-jocher It won't produce an error. If wandb is not installed num_predictions will be set to 0 in train.py, which is also it's default value . And wandb will be imported in test.py only if num_predictions is greater than 0. You can test it on your end as well

[glenn-jocher]

@AyushExel I've done a general reorganization of the code:

• removed seperate wandb logging boolean, now we simply use wandb package treated as a boolean (same handling as tb).
• resume functionality moved to ckpt resume statement, cleaned up and shortened.
• project has been seet to logdir rather than opt.name. This is very important since it automatically includes incremented experiment numbers, and it also includes opt.name if it is found: i.e. logdir=runs/exp0 or runs/exp0_name
• wandb importing logic has been moved inside if opt.global_rank in [-1, 0]: statement to prevent multiple loggers opening up on multi-GPU training.
• default image logging count reduced from 50 images to 10 to reduce added time to about 1.0 seconds from 5 seconds.

[glenn-jocher]

@AyushExel I think I've completed all of my updates. Can you review and verify that I haven't broken anything inadvertently?

This should work in multi-gpu now, but I will test today to verify. CI tests only run on CPU, single-GPU and multi-GPU testing is done by hand at the moment.

[AyushExel]

@glenn-jocher I'm getting another error now:

 File "train.py", line 342, in train
    wandb.log({tag: x})  # W&B
 raise wandb.Error("You must call wandb.init() before {}()".format(name))
wandb.errors.error.Error: You must call wandb.init() before wandb.log()
Internal process exited

Looking into this.

[glenn-jocher]

@glenn-jocher I'm getting another error now:

 File "train.py", line 342, in train
    wandb.log({tag: x})  # W&B
 raise wandb.Error("You must call wandb.init() before {}()".format(name))
wandb.errors.error.Error: You must call wandb.init() before wandb.log()
Internal process exited

Looking into this.

Ah I see. This makes sense actually, as I placed .init() in an if opt.resume statement, so it will only initialize if resuming. I need to take it back out. I'll do that.

[AyushExel]

@glenn-jocher I'm getting another error now:

 File "train.py", line 342, in train
    wandb.log({tag: x})  # W&B
 raise wandb.Error("You must call wandb.init() before {}()".format(name))
wandb.errors.error.Error: You must call wandb.init() before wandb.log()
Internal process exited

Looking into this.

Ah I see. This makes sense actually, as I placed .init() in an if opt.resume statement, so it will only initialize if resuming. I need to take it back out. I'll do that.

Ahhh gotcha. Great catch. Thanks!

[glenn-jocher]

All updates for this PR are complete. CI checks passing. Google colab example is working well. Will merge.

[AyushExel]

All updates for this PR are complete. CI checks passing. Google colab example is working well. Will merge.

Great!

[glenn-jocher]

About Weights & Biases

Think of W&B like GitHub for machine learning models — save everything you need to debug, compare and reproduce your models — architecture, hyperparameters, weights, model predictions, GPU usage, git commits, and even datasets — with a few lines of code.

W&B lightweight integrations work with any Python script, and all you need to do is sign up for a free W&B account to start tracking and visualizing your models.

Used by the likes of OpenAI, Lyft, Github and researchers at top machine learning labs across the world, W&B is part of the new standard of best practices for machine learning.

How W&B can help you optimize your machine learning workflows:

• Debug model performance in real time
• Automatically tracked GPU, CPU usage and other system metrics
• Powerful custom charts
• Share model insights interactively
• Efficient hyperparameter optimization
• Dataset and model pipeline tracking and production model management

W&B is free for individuals, academics and open source projects.

YOLOv5 with W&B

Here's an in depth look at how the integration works: How are your yolov5 models doing?

When training with wandb installed, you get a an interactive bounding box debugging panel as shown here: