ultralytics 8.1.39 add YOLO-World training

[Laughing-q]

Usage:

from ultralytics import YOLOWorld

model = YOLOWorld("yolov8s-worldv2.pt")
model.train(data="coco128.yaml")

TODO:

• LVIS docs
• Verify WorldTrainerFromScratch

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Enhancements in LVIS dataset support, model training updates, and technical refinements across the Ultralytics framework.

📊 Key Changes

• Added support and detailed documentation for the LVIS dataset.
• Updated CLIP model installation to use the Ultralytics repository.
• Introduced refactorings in model training for the YOLO-World architecture, including enhanced CLIP model support.
• Refined data augmentation processes, particularly for multi-modal (image + text) training scenarios.
• Made improvements in dataset management, including better cache file handling and support for additional datasets.
• Adjustments in model and dataset configurations for more accurate training results.

🎯 Purpose & Impact

• Enhanced Dataset Support: Including LVIS enhances the model's ability to train on a wider range of object categories, improving versatility and accuracy.
• Model Training and Evaluation Improvements: Updates in model training, especially with the introduction of YOLO-World-related features, pave the way for more advanced multi-modal learning capabilities.
• Robust Data Handling: Improved dataset caching and configurations streamline the data preparation process, making model training more efficient.
• Technical Refinements: General code enhancements contribute to a more robust, efficient, and flexible machine learning pipeline.

[glenn-jocher]

@Laughing-q hey I looked through this, this is really nice and comprehensive, including added docs and reference section pages.

I added a few missing docstrings and updated the val.py class_map (I think it needs to be zero-indexed), but that's the only changes I've made. Are there are sections you're not sure about, or is this ready to merge to main now?

[Laughing-q]

@glenn-jocher Thanks for reviewing! I updated the class_map to start from index 1 because that's what LVIS data needs when I intended to save json and manually validated mAP by using their API. Then I figured that for user cases they might not really care about whether the index starts from 0 or 1 so I eventually pushed the update. Do you think we should keep the index starting from 0? then probably create a class_map for LVIS just like we did for COCO?

EDIT: This reminds me we probably want do the same thing to LVIS evaluation as well, introduce a is_lvis variable maybe and do evaluation by using LVIS api when it's True.

[Laughing-q]

@glenn-jocher rest of the PR I think is all good, but since you were asking, I feel like it's better to hold it for another several hours and let me check if there're any places to improve, since it's a PR adding more than 2000 lines(1200 lines are from lvis.yaml though..). What do you think?

[Laughing-q]

@glenn-jocher ok I've added lvis api support to evaluate the final results just like we did for coco 9b2ecaf.
Also I add an extra +1 for category_id for lvis dataset while saving predictions in json format so it can be evaluated correctly e76a479.
I think the PR is all set! :)

[glenn-jocher]

@Laughing-q PR merged!!

[ccl-private]

@Laughing-q Hi, can you help me take a look at this issue? #13793