models/yolo-world/

[giscus[bot]]

models/yolo-world/

Discover YOLO-World, a YOLOv8-based framework for real-time open-vocabulary object detection in images. It enhances user interaction, boosts computational efficiency, and adapts across various vision tasks.

https://docs.ultralytics.com/models/yolo-world/

[IamShubhamGupto]

Authors of yolo world released instructions to fine tune their models, would that be supported by ultralytics? if so id like to contribute - https://github.com/AILab-CVC/YOLO-World/blob/master/docs/finetuning.md

[pderrenger]

@IamShubhamGupto hey there! 👋

Absolutely, we're all for community contributions and support for fine-tuning models like YOLO-World. If you're interested in contributing, that's fantastic! 🌟

Just a heads up, while we do support various YOLOv8 tasks and modes, the training mode for YOLO-World models (yolov8*-world.pt) isn't currently supported. However, you can still use the pre-trained weights for inference and validation.

If you have a specific fine-tuning process in mind that's compatible with our existing framework, we'd love to see what you have! Feel free to fork the repo, make your changes, and then submit a pull request. If you need any help or have questions along the way, don't hesitate to ask.

Happy coding! 😊👨‍💻

[woerstcase]

If I get you right you thinking in a use like this?
https://supervision.roboflow.com/develop/notebooks/zero-shot-object-detection-with-yolo-world/#filtering-detectuions-by-area

[pderrenger]

Hey there! 👋

Yes, you're on the right track! The example you've linked from Roboflow demonstrates filtering detections by area, which is indeed supported by Ultralytics YOLOv8. You can achieve similar functionality by manipulating the results object after prediction to filter out detections based on their bounding box area or any other criteria you prefer.

Here's a quick example of how you might do that:

from ultralytics import YOLO

# Load a pretrained YOLOv8 model
model = YOLO('yolov8n.pt')

# Perform object detection on an image
results = model('path/to/image.jpg')

# Filter detections by bounding box area (e.g., area > 1000 pixels)
filtered_results = [det for det in results[0].boxes.xyxy if (det[2] - det[0]) * (det[3] - det[1]) > 1000]

# Do something with filtered_results

Hope this helps! If you have more questions or need further assistance, feel free to ask. Happy coding! 😊🚀

[bdv29]

im having trouble understanding what this is exactly,
does this work like a VLM you ask it a question about where something is?

[pderrenger]

Hey there! 👋

It seems like you're curious about what YOLO-World is and how it works. In a nutshell, YOLO-World is a framework based on YOLOv8 that's designed for real-time object detection in images. It's not exactly like a virtual machine you ask questions to, but rather a powerful tool that can identify objects in images based on descriptions.

Here's a quick example of how you might use it in Python:

from ultralytics import YOLOWorld

# Initialize the model with pre-trained weights
model = YOLOWorld('yolov8s-world.pt')

# Set what you'd like to find in your image
model.set_classes(["red car", "happy face"])

# Run object detection for your custom classes on your image
results = model.predict('path/to/your/image.jpg')

# Display the results
results[0].show()

This code loads a pre-trained YOLO-World model, runs it on an image, and then shows you the detected objects. It's efficient, fast, and doesn't require a ton of computational power. Plus, you can even customize it to look for specific objects by setting prompts!

Hope that clears things up! If you have more questions or need further assistance, feel free to ask. 😊

[aikedaerC]

I guess the "VLM" refered to as "Vision Language Model"

[sandgrowagro]

I am getting different results for YOLOv8l-world compared to what I am trying on Yolo-World hugging face demo page, Any idea why this could be happening?

[woerstcase]

Maybe you using other parameters in inference result?
There are a lot you can choose:

for example:https: results = model(image, save_conf=True, conf=0.02, iou=0.1) and many more

https://docs.ultralytics.com/modes/predict/#inference-arguments

[pderrenger]

Hey there! 👋 It's possible that the difference in results between your YOLOv8l-world model and the Hugging Face demo could be due to a variety of factors. Here are a few things to consider:

1. Parameter Settings: Ensure that the inference parameters like confidence threshold (conf) and Intersection Over Union (iou) are consistent across both platforms. Small changes in these values can lead to different detection results.

2. Model Version: Double-check that you're using the same model version and weights in both cases. Sometimes, updates or different versions can cause discrepancies.

3. Input Preprocessing: The way images are preprocessed before being fed into the model might differ. Look into normalization, resizing, or any other preprocessing steps.

4. Environment: Differences in the software environment, such as the version of PyTorch, CUDA, or other dependencies, might affect the results.

If you're looking to match the settings, here's a quick example of how to set your inference parameters in Python:

from ultralytics import YOLO

# Load your model
model = YOLO('yolov8l-world.pt')

# Set inference parameters
results = model.predict('path/to/image.jpg', conf=0.25, iou=0.45)

For more detailed guidance on inference arguments, check out the docs here. If you continue to see discrepancies, feel free to share more details, and we can dive deeper into this together! 🕵️‍♂️💡

[KishoreElvicto]

i tried this code to detect car and it's number plate. but i am not getting any results for number plate. car is working fine. what to do?
'
from ultralytics import YOLOWorld
model = YOLOWorld('yolov8l-world.pt')
model.set_classes(["car","number plate"])
results = model.predict('smile.jpg')
results[0].show()
'
also tried with happy face. didn't work. it works with only coco classes? i want to detect number plates. help

[glenn-jocher]

@KishoreElvicto hey there! 👋 It looks like you're trying to detect cars and their number plates using YOLO-World, but you're not getting results for the number plates. Since YOLO-World is designed for open-vocabulary detection, it should be able to detect objects beyond the COCO classes.

Here's a couple of things you could try:

1. Ensure that your image ('smile.jpg') contains clear and visible number plates. The model might struggle if the number plates are too small, blurry, or obstructed.
2. Try using the set_classes() method with more descriptive prompts, like "vehicle number plate" instead of just "number plate", to see if it helps the model identify the plates better.

If you're still facing issues, it could be helpful to check if the number plates are part of the model's offline vocabulary. If they're not, you might need to use a custom dataset that includes number plates for retraining or find a pre-trained model specifically for number plate detection.

Here's a quick example of how you might adjust your code:

from ultralytics import YOLOWorld

# Initialize a YOLO-World model
model = YOLOWorld('yolov8l-world.pt')

# Define custom classes with more descriptive prompts
model.set_classes(["car", "vehicle number plate"])

# Execute prediction for specified categories on an image
results = model.predict('smile.jpg')

# Show results
results[0].show()

Give these suggestions a try, and if you're still stuck, feel free to share more details or reach out again! 😊

[qihuijia]

Nice job. I am very glad that ultralytics have already support yolo-world. And I found that you are already support the coreml export.

This works.

from ultralytics import YOLOWorld

model = YOLOWorld('yolov8s-world.pt')
model.export(format='coreml')

But when I want to set the classes, it does not work.

from ultralytics import YOLOWorld

model = YOLOWorld('yolov8s-world.pt')
model.set_classes(["colorchecker", "ball", "object", "painting", "flower", "vase", "lavander", "rabbit"])
model.export(format='coreml')

The error is:

Ultralytics YOLOv8.1.17 🚀 Python-3.10.12 torch-2.1.0+cu121 CPU (Intel Xeon 2.00GHz)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-67-a013544c0706> in <cell line: 1>()
----> 1 model.export(format='coreml')

8 frames
/usr/local/lib/python3.10/dist-packages/torch/_tensor.py in __deepcopy__(self, memo)
     84             return handle_torch_function(Tensor.__deepcopy__, (self,), self, memo)
     85         if not self.is_leaf:
---> 86             raise RuntimeError(
     87                 "Only Tensors created explicitly by the user "
     88                 "(graph leaves) support the deepcopy protocol at the moment.  "

RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment.  If you were attempting to deepcopy a module, this may be because of a torch.nn.utils.weight_norm usage, see https://github.com/pytorch/pytorch/pull/103001

[pderrenger]

Yes, you're absolutely right! The screenshot you've shared is from the CoreML model's class labels. You can customize your own class settings there to match the specific classes you're interested in detecting. Just replace the existing labels with your desired ones like "colorchecker", "ball", etc., and you should be good to go! If you need any further assistance, feel free to reach out. Happy coding! 😊👍

[pierre1618]

Hi
I encountered a similar issue when attempting to export a YOLO model with custom classes. I managed to resolve it by saving the modified model and then reloading it. However, I faced another error related to gradient tracking, which I resolved by deactivating it using torch.no_grad().

import torch
from ultralytics import YOLO  # Ensure Ultralytics is installed

# To handle potential CUDA out of memory errors, disable gradient tracking with torch.no_grad()
with torch.no_grad():
    # Initialize a YOLO model with pretrained weights
    model = YOLO('yolov8s-world.pt')  # You can also choose yolov8m/l-world.pt based on your needs

    # Define custom classes specific to your application
    custom_classes = ["car", "vehicle number plate"]
    model.set_classes(custom_classes)

    # Save the model with the custom classes defined (modified code)
    model.save("custom_yolov8s.pt")  # This saves extra metadata required for CoreML conversion

    # Load the saved model with custom classes
    model = YOLO("custom_yolov8s.pt")

    # Export the model to CoreML format with non-maximum suppression enabled
    model.export(format="coreml", nms=True)

This approach allowed me to successfully export the model with custom classes to CoreML format. Hope this could help!

[pderrenger]

@pierre1618 hey there! 👋

Great to hear you've found a workaround for exporting a YOLO model with custom classes and resolved the gradient tracking issue with torch.no_grad(). Your approach to saving and reloading the model is a smart move! 🧠

For anyone else encountering similar issues, this user-shared snippet could be a helpful reference. Just remember to adjust the model and custom_classes to fit your specific use case.

Keep up the good work, and if you have any more insights or questions, feel free to drop them here. Happy coding! 🚀

[qihuijia]

Thanks everyone, now since lastest version(8.1.19) 2024.2.27 launched, I can convert the yolo world model in M1/M2 chip macOS, thanks very much.

[pderrenger]

@qihuijia hey there! 🎉 It's fantastic to hear that the latest version (8.1.19) as of 2024.2.27 has enabled you to successfully convert the YOLO-World model for use on M1/M2 chip macOS. We're thrilled to see our updates making a positive impact on your projects.

If you're diving into customizing classes or further exploring YOLO-World's capabilities, remember that the flexibility and efficiency of YOLO-World make it a powerful tool for a wide range of vision tasks. Whether you're fine-tuning for specific objects with custom prompts or leveraging the model's real-time detection prowess, YOLO-World is designed to adapt to your needs.

Should you have any more questions or if there's anything else we can help you with, don't hesitate to reach out. Happy detecting! 🚀

[bdv29]

is there a away to fine tune the model if you manage to get fairly close to accurate to what you want to box bound but just need to focused on your particular application once you've set the model?

[pderrenger]

Absolutely! Fine-tuning a YOLOv8 model for your specific application is a great way to enhance its performance on the tasks most relevant to you. Here's a quick guide on how to do it:

1. Prepare Your Dataset: Make sure your dataset is annotated with the classes of interest. For fine-tuning, it's crucial to have a dataset that closely represents the scenarios where you'll deploy the model.

2. Set Custom Classes: If you're focusing on specific classes, you can use the set_classes method to tailor the model. This is especially useful with YOLO-World models, as they allow for dynamic class specification. Here's how you might do it:

   from ultralytics import YOLO
   
   # Load your YOLO-World model
   model = YOLO('yolov8s-world.pt')  # Choose the appropriate model size
   
   # Define your custom classes
   model.set_classes(["your_class_1", "your_class_2"])
   
   # Now your model is focused on these classes

3. Train (Fine-Tune) Your Model: Use the train mode to fine-tune your model on your dataset. Remember to adjust the training parameters according to your dataset size and the computational resources available.

   yolo detect train data=your_dataset.yaml model=custom_yolov8s.pt epochs=50

4. Evaluate and Iterate: After training, evaluate your model's performance using the val mode. It might take a few iterations of training with adjusted parameters to achieve the desired accuracy.

Fine-tuning allows you to leverage the powerful YOLOv8 architecture while focusing the model's learning on your specific use case. This process can significantly improve detection performance for your classes of interest. Happy fine-tuning! 😊

[bdv29]

i get the error NotImplementedError: WARNING 'YOLOWorld' model does not support 'train' mode for 'detect' task yet.
are there plans for adding it anytime soon?

[pderrenger]

Hey there! 😊 Thanks for reaching out. Currently, the YOLO-World models, including yolov8s-world.pt, are optimized for inference and validation tasks, and they don't support the 'train' mode for the 'detect' task just yet.

We're always working on enhancing the capabilities of our models, and training support for YOLO-World is definitely on our radar. While I can't provide a specific timeline right now, keep an eye on our GitHub repo and docs for updates!

In the meantime, if you're looking to fine-tune a model for detection, you might consider using the standard YOLOv8 models which fully support training. Here's a quick example on how to start training:

yolo detect train data=your_custom_data.yaml model=yolov8n.pt epochs=100

Make sure your your_custom_data.yaml is set up with your dataset. We appreciate your patience and interest in pushing the boundaries of what's possible with YOLO! 🚀

[AdadAlShabab]

How can we run 'videos' with custom classes defined?

[glenn-jocher]

Hey there! Running videos with custom classes using YOLO-World is straightforward. You can leverage the set_classes() method to specify your custom classes before running predictions. Here's a quick example:

from ultralytics import YOLO

# Load your YOLO-World model
model = YOLO('yolov8s-world.pt')  # Adjust the model as needed

# Define your custom classes (e.g., 'cat', 'dog')
custom_classes = ['cat', 'dog']
model.set_classes(custom_classes)

# Now you can run predictions on your video
results = model.predict('path/to/your/video.mp4')

# Display the results
results[0].show()

This way, the model will focus on detecting only the classes you've defined. Hope this helps! Let me know if you have any more questions. 😊

[AdadAlShabab]

Thanks a lot!! @glenn-jocher

[glenn-jocher]

@AdadAlShabab hey there! 🌟 To run videos with custom classes using YOLO-World, you can leverage the set_classes() method to specify your desired classes. Here's a quick example to get you started:

from ultralytics import YOLO

# Load your YOLO-World model
model = YOLO('yolov8s-world.pt')  # Adjust based on your model choice

# Define your custom classes (e.g., 'cat', 'dog')
custom_classes = ['cat', 'dog']
model.set_classes(custom_classes)

# Run predictions on your video
results = model.predict('path/to/your/video.mp4')

# Display the results
results[0].show()

This way, the model will focus on detecting only the classes you've specified. Hope this helps! Let me know if you have any more questions. 😊

[qihuijia]

I think you can try if you set the classes more than 8. It will not work.

[glenn-jocher]

Hey there! Thanks for reaching out. 😊

If you're encountering issues with setting more than 8 classes in YOLO-World, it might be due to how the classes are being specified. Here's a quick example on how to correctly set custom classes for your model:

from ultralytics import YOLO

# Load your YOLO-World model
model = YOLO('yolov8s-world.pt')  # Adjust based on your model choice

# Define your custom classes
custom_classes = ['class1', 'class2', 'class3', 'class4', 'class5', 'class6', 'class7', 'class8', 'class9']
model.set_classes(custom_classes)

# Now you can run predictions on your video or images
results = model.predict('path/to/your/video.mp4')

Make sure your class names match exactly with those in the model's training data. If you're still facing issues, could you provide more details or error messages you're getting? That way, I can help you more effectively! 🚀

[qihuijia]

when the yolo world support "Training"?

[glenn-jocher]

Hey there! 🌟 Currently, the YOLO-World models are optimized for inference and validation tasks, with training and export modes not supported at the moment (as indicated by the ❌ in the documentation). However, we're always working on enhancing the capabilities of our models. Stay tuned for future updates where training support might be introduced! If you're looking to train a model for open-vocabulary detection, I'd recommend exploring the existing YOLOv8 models and their training functionalities. Feel free to share any specific requirements or use cases you have in mind, and we might be able to suggest an alternative approach. Happy coding! 🚀

[bdv29]

are there plans to implement it to the export method?
tried exporting to tensorRT but it doesn't seem it is implmented

[pderrenger]

@bdv29 hey there! 😊 Awesome to see your interest in exporting YOLO models to TensorRT! The current support for exporting directly to TensorRT is indeed a bit limited. However, there's a workaround: you can first export your model to the ONNX format and then convert the ONNX model to TensorRT. Here's a quick guide:

1. Export to ONNX:

yolo export model=yolov8s-worldv2.pt format=onnx

2. Convert ONNX to TensorRT (assuming you have TensorRT installed):

trtexec --onnx=yolov8s-worldv2.onnx --saveEngine=yolov8s-worldv2.engine

Keep an eye on our GitHub repo or docs for updates on direct TensorRT support. Your feedback is essential to us, and we're continually working to improve! 🚀

[AnandSingh-0619]

Can we run inference with multiple images or batch of images for reducing overall computation time?

[glenn-jocher]

Absolutely, running inference with a batch of images can be an effective way to reduce overall computation time. You can achieve this by using the predict function and passing your batch of images as a list. For example:

from ultralytics import YOLO

# Initialize model
model = YOLO('yolov8n.pt')  # Example for detection

# List of image paths
images = ['path/to/image1.jpg', 'path/to/image2.jpg']

# Run inference on batch of images
results = model(images)

# Process the results as needed
for result in results:
    result.show()  # Display

This way, you leverage batch processing, which could significantly speed up the inference time, especially on GPUs. 🚀

[Dragosjosan]

The model performance of the Yolo World Hugging Face model (https://huggingface.co/spaces/stevengrove/YOLO-World) is better for my purpose than the standard inference one "YOLOv8x-worldv2" for example.

Because my lack of domain knowledge regarding this topics I can't make the HF model work on my Mac (not an M processor) so I would like to be able to use the model weights with the abstraction from the ultralytics library.

Is there any way to use the author's provided weights together with the inference library?
For example:
model = YOLOWorld('yolow-v8_l_clipv2_frozen_t2iv2_bn_o365_goldg_pretrain.pth')

From the author's instruction on how to run the model on an image one needs the configuration file and the weights: https://github.com/AILab-CVC/YOLO-World

The conf file I want to use and the weights are available here:
https://huggingface.co/spaces/stevengrove/YOLO-World/blob/main/configs/pretrain/yolo_world_l_t2i_bn_2e-4_100e_4x8gpus_obj365v1_goldg_train_lvis_minival.py
https://huggingface.co/spaces/stevengrove/YOLO-World/blob/main/yolow-v8_l_clipv2_frozen_t2iv2_bn_o365_goldg_pretrain.pth

Thank you in advance :)

[junnnnchi]

Hey @Dragosjosan I am also interested in this topic, did you figure out how to convert .pth to .pt?

[Dragosjosan]

Hey @junnnnchi,

no, sorry.
How do you plan on tackling this problem?
Maybe we can figure it out together.

[tpsvoets]

@Dragosjosan ah too bad, it's a shame we cannot get any network file to work with the code. Right now it remains unknown for me how the two demos can differ quite so much. Do we know if the architecture is the same, so if the .pth to .pt conversion works that it will result in something that actually works?

[junnnnchi]

@Dragosjosan I used the roboflow's implementation for YOLO-World. This version gives better results for my project. You might consider this.

[Dragosjosan]

@junnnnchi for my application the model from Hugging Face works best...
But good that you found something that works.
May I ask what your App do? :)

[Nishantg7]

Hello There,
I am having trouble using YOLO-WORLD when I convert the results into voice output. The yolov8 model works fine with the same voice input method please help me out with this problem.

CODE:
import cv2
import tkinter as tk
from PIL import Image, ImageTk
from ultralytics import YOLOWorld
import pyttsx3

class WebcamApp:
def init(self, window, window_title):
self.window = window
self.window.title(window_title)

    # Open the webcam
    self.cap = cv2.VideoCapture(0)

    # Create a label for displaying the video feed
    self.label = tk.Label(window)
    self.label.pack()

    # Button to capture image
    self.btn_capture = tk.Button(window, text="Capture Image", width=50, command=self.capture_image)
    self.btn_capture.pack()

    # Initialize YOLO-World model
    self.model = YOLOWorld("yolov8s-world.pt")

    # Initialize text-to-speech engine
    self.engine = pyttsx3.init('sapi5')
    voices = self.engine.getProperty('voices')
    self.engine.setProperty('voice', voices[0].id)

    self.window.protocol("WM_DELETE_WINDOW", self.on_closing)
    self.window.mainloop()

def capture_image(self):
    # Read frame from webcam
    ret, frame = self.cap.read()
    if ret:
        # Convert the frame to RGB color space
        frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        # Convert the frame to PIL format
        img = Image.fromarray(frame_rgb)
        # Save the captured image
        img.save("captured_image.jpg")
        # Process the captured image with YOLO-World model
        results = self.model.predict("captured_image.jpg")
        # Extract class names from results
        objects = [res['name'] for res in results]
        # Convert detected objects list to speech
        speech_text = f"I see {', '.join(objects)}."
        # Speak the detected objects
        self.speak(speech_text)
    else:
        print("Error capturing image.")


def speak(self, audio):
    self.engine.say(audio)
    self.engine.runAndWait()

def on_closing(self):
    # Release the webcam
    self.cap.release()
    # Close the application window
    self.window.destroy()

Create a Tkinter window

root = tk.Tk()

Create the WebcamApp object

app = WebcamApp(root, "Webcam Application")

ERRORS:
image 1/1 d:\CODING\YOLO WORLD\captured_image.jpg: 480x640 1 bottle, 1 clock, 427.0ms
Speed: 3.0ms preprocess, 427.0ms inference, 1703.1ms postprocess per image at shape (1, 3, 480, 640)
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Users\gawde\AppData\Local\Programs\Python\Python310\lib\tkinter_init_.py", line 1921, in call
return self.func(*args)
File "d:\CODING\YOLO WORLD\test3.py", line 47, in capture_image
objects = [res['name'] for res in results]
File "d:\CODING\YOLO WORLD\test3.py", line 47, in
objects = [res['name'] for res in results]
File "C:\Users\gawde\AppData\Local\Programs\Python\Python310\lib\site-packages\ultralytics\engine\results.py", line 126, in getitem
return self._apply("getitem", idx)
File "C:\Users\gawde\AppData\Local\Programs\Python\Python310\lib\site-packages\ultralytics\engine\results.py", line 163, in _apply
setattr(r, k, getattr(v, fn)(*args, **kwargs))
File "C:\Users\gawde\AppData\Local\Programs\Python\Python310\lib\site-packages\ultralytics\engine\results.py", line 63, in getitem
return self.class(self.data[idx], self.orig_shape)
IndexError: too many indices for tensor of dimension 2

[pderrenger]

Hello there! 👋 It looks like you're encountering issues with using YOLO-World and converting the results to voice output. From the logs, the error seems to stem from how the results are being parsed and accessed. Specifically, the error indicates a possible mismatch in the expected structure or dimensions of the results object.

To address the issue, we need to ensure we're accessing the results properly. With YOLO-World's output format, you might need to adjust how you're iterating through the results. Here’s a small tweak to how you process the results:

# Process the captured image with YOLO-World model
results = self.model.predict("captured_image.jpg")
# Assuming results contain detections
if results.detections:
    objects = [detection.name for detection in results.detections[0]]
    ...

This ensures you're directly accessing the detections attribute, which holds the detected object info. Each detection provides the .name property among others like .confidence, etc.

Please give this a try and check if it resolves the error. If the issue persists or if there’s anything more specific you’re facing, feel free to share more details! 🤞

[Nishantg7]

Thank you so much worked for me.

[pderrenger]

Hey there! 🎉 I'm thrilled to hear that the solution worked for you!

For anyone venturing into integrating YOLO-World's detection capabilities with voice output, it's key to correctly parse the results object. For YOLO-World specifically, the setup might differ slightly from YOLOv8 due to its unique output format tailored for open-vocabulary tasks. 🤖✨

If you're tweaking the code for YOLO-World and bump into any snags, make sure you're accessing the detection results as intended by the model's output structure. The Ultralytics docs provide great insights on handling outputs effectively.

Happy coding, and if further questions surface, feel free to drop them here!

[illink7]

How can I combine YOLO-World with SAHI to process satellite images?
I suppose it has to be interesting

[pderrenger]

Combining YOLO-World with SAHI for processing satellite images sounds like a fascinating prospect!🌍✨ You can leverage the strong open-vocabulary detection capabilities of YOLO-World to identify a wide range of objects in satellite imagery. Here’s how you can get started:

1. Install YOLO-World and SAHI: Make sure both libraries are installed in your environment.
2. Load YOLO-World Model: Choose a suitable YOLO-World model based on your need for accuracy or speed (e.g., yolov8s-worldv2.pt for a balance between the two).
3. Use SAHI for Inference: Utilize SAHI's slicing and prediction merging capabilities to handle large satellite images.

from ultralytics import YOLO
from sahi.predict import predict

# Load the YOLO-World model
model = YOLO('yolov8s-worldv2.pt')  # Adjust the model file as needed

# Predict using SAHI with a large satellite image
result = predict(
    model_type='yolov5',  # Use 'yolov5' type as a placeholder
    model=model,
    source='path/to/satellite/image.jpg'
)

This example provides a starting point. You might need to customize it further based on SAHI's current API and how it interfaces with models. Remember to keep an eye on both YOLO-World and SAHI documentation for updates or changes in the integration process.

Happy detecting! 🛰️💡 Feel free to share your findings or ask more questions as you dive into this interesting integration.

[Kunal8020]

I finetuned a yolo world model for a different classes which was not detectable using the original weights Now I have a weights for for detection of that classes. I also get another weight for class which are present in the original weight

from ultralytics import YOLOWorld
model = YOLOWorld("yolov8x-worldv2.pt")
model.set_classes(["baseball bat","ball"])
model.save("custom_yolo.pt")

How can I combine both weights so that i can detect all required classes??

[Kunal8020]

To be simple, How can I combine two custom weights??

[glenn-jocher]

Currently, combining two different sets of weights directly isn't supported in the YOLO architecture. Each set of weights is trained under specific conditions and merging them could lead to conflicts or degraded performance.

A practical approach would be to combine the datasets for both classes and retrain the model from scratch or fine-tune from one of the pre-trained weights. This way, the model learns to detect all the required classes simultaneously, ensuring optimal performance and compatibility.

If you need further guidance on how to proceed with training on a combined dataset, feel free to ask! 🚀

[Kunal8020]

But i do not have dataset for the classes present in the YOLO world. That is why i am using set classes of required objects. I finetuned a model with my dataset.
Also, when i finetume the model using below code

from ultralytics import YOLOWorld
model=YOLOWorld('weights.pt')
results = model.train(data='config.yaml', epochs=100, imgsz=640)

all previous classes are not detectable with new weights. How can I train a model such that it will detect the all previous classes from the previous weights and also finetuned classes ??

[glenn-jocher]

Thank you for your detailed explanation! Combining weights from two different models directly isn't supported, as it can lead to conflicts and degraded performance. However, you can achieve your goal by merging the datasets and retraining the model. Here’s a suggested approach:

1. Combine Datasets: Merge your dataset with the dataset containing the classes from the original YOLO-World weights.
2. Retrain the Model: Use the combined dataset to retrain the model. This way, the model learns to detect all the required classes simultaneously.

Here’s an example of how you can proceed with training:

from ultralytics import YOLOWorld

# Load a pretrained YOLO-World model
model = YOLOWorld("yolov8x-worldv2.pt")

# Train the model on the combined dataset
results = model.train(data="combined_dataset.yaml", epochs=100, imgsz=640)

If you need further assistance with dataset preparation or training, feel free to ask! 😊

[Kunal8020]

I am trying to get the tracking id from image using below code
from ultralytics import YOLO
model=YOLO('path/yolov8x-worldv2.pt')

img=cv2.imread('WA0004_444.jpg')
results=model.track(img)

when i check the results of tracking it is showing false
results[0].boxes

id: None
is_track: False

[Kunal8020]

when i am using the finetune weight on custom datset then it is working
model_yoloworld = YOLOWorld('best.pt')
results=model_yoloworld.track(img)

results[0].boxes

id: tensor([1., 2.])
is_track: True

[pderrenger]

@Kunal8020 hello!

Thank you for reaching out and providing detailed information about your issue. It seems like you're encountering a problem with tracking IDs when using the yolov8x-worldv2.pt model. Let's troubleshoot this together.

Firstly, ensure that you are using the latest versions of torch and ultralytics packages. You can update them using the following commands:

pip install --upgrade torch ultralytics

If the issue persists, please provide a minimum reproducible example so we can investigate further. This helps us understand the context and reproduce the bug on our end. You can refer to our guide on creating a minimum reproducible example here: Minimum Reproducible Example.

In the meantime, here's a refined version of your code to ensure everything is set up correctly:

import cv2
from ultralytics import YOLO

# Load the YOLO-World model
model = YOLO('path/yolov8x-worldv2.pt')

# Read the image
img = cv2.imread('WA0004_444.jpg')

# Perform tracking
results = model.track(img)

# Check the results
if results[0].boxes.id is not None:
    print(f"Tracking IDs: {results[0].boxes.id}")
else:
    print("No tracking IDs found.")

If you still encounter issues, it might be helpful to verify if the model is correctly loaded and if the image is being processed as expected. Additionally, ensure that the image path and model path are correct.

Feel free to share any additional details or errors you encounter, and we'll be happy to assist further. 😊

[pderrenger]

Hello!

Thank you for reaching out and providing detailed information about your issue. It seems like you're encountering a problem with tracking IDs when using the yolov8x-worldv2.pt model. Let's troubleshoot this together.

Firstly, ensure that you are using the latest versions of torch and ultralytics packages. You can update them using the following commands:

pip install --upgrade torch ultralytics

Next, let's verify that the model and the image are being processed correctly. Here is a minimum reproducible example to help you debug:

from ultralytics import YOLO
import cv2

# Load the YOLO-World model
model = YOLO('path/yolov8x-worldv2.pt')

# Read the image
img = cv2.imread('WA0004_444.jpg')

# Perform tracking
results = model.track(img)

# Check the tracking results
print(results[0].boxes)

If the id is still None and is_track is False, it might be due to the confidence threshold or other tracking parameters. You can try adjusting the confidence threshold and other parameters to see if it resolves the issue:

results = model.track(img, conf=0.25, iou=0.45)

Additionally, if you have a custom-trained model (best.pt) that works correctly, it might be beneficial to compare the configurations and training parameters used for both models. Sometimes, fine-tuning on a specific dataset can significantly impact the tracking performance.

If the issue persists, please provide more details or a minimum reproducible example as outlined in our Minimum Reproducible Example Guide. This will help us investigate the problem more effectively.

Feel free to reach out if you have any further questions or need additional assistance. We're here to help! 😊

[Kunal8020]

thank you for the response. It worked when i use the below code
results = model.track(img, conf=0.25,tracker="bytetrack.yaml")

where i reduced the track_thresh parametrers

[pderrenger]

Hi @Kunal8020,

Thank you for the update! I'm glad to hear that adjusting the track_thresh parameter and using the bytetrack.yaml tracker resolved the issue. 🎉

For anyone else encountering similar issues, adjusting the conf and track_thresh parameters can significantly impact the tracking performance. Here's a quick example:

results = model.track(img, conf=0.25, tracker="bytetrack.yaml")

This approach ensures that the model is more sensitive to detections, which can help in scenarios where tracking IDs are not being generated as expected.

If you have any further questions or need additional assistance, feel free to ask. We're here to help!

[chickencoins]

Is the pre-trained weight file of YOLO-world currently provided by Ultralytics trained with Objects365v1, GQA, and Flickr30k, or was a different dataset used? Despite no difference in hyperparameters, why is there a performance difference between the versions from Hugging Face and Ultralytics?

[pderrenger]

@chickencoins hi there,

Thank you for your question! The pre-trained weights for the YOLO-World models provided by Ultralytics are indeed trained on the Objects365v1, GQA, and Flickr30k datasets. These datasets are used to enhance the model's open-vocabulary detection capabilities, allowing it to identify a wide range of objects based on descriptive texts.

Regarding the performance differences between the versions from Hugging Face and Ultralytics, there could be several factors at play. Even with identical hyperparameters, variations in performance can arise due to differences in the training environment, data preprocessing steps, or even minor updates in the underlying libraries. I recommend ensuring that you are using the latest versions of the models and dependencies from both sources to minimize discrepancies.

If you continue to experience significant performance differences, providing a minimum reproducible example would be very helpful for us to diagnose the issue more effectively. You can find guidance on creating a reproducible example here: Minimum Reproducible Example.

Feel free to reach out if you have any further questions or need additional assistance!

[spacewalk01]

Doesn't work for a simple class that is not included in coco classes. Original model works for all vocabulary.

[pderrenger]

@spacewalk01 thank you for your feedback! It sounds like you're encountering issues with detecting a class not included in the COCO dataset using the YOLO-World model. To help us better understand and resolve the issue, could you please provide a minimum reproducible example? This will allow us to replicate the problem and offer a more precise solution. You can find guidance on creating a reproducible example here: Minimum Reproducible Example.

Additionally, please ensure you are using the latest version of the YOLO-World model and the Ultralytics package. Sometimes, updates include important fixes and improvements that might resolve your issue.

For custom classes, you can dynamically specify detection classes through custom prompts without retraining the model. Here’s a quick example:

from ultralytics import YOLOWorld

# Initialize a YOLO-World model
model = YOLOWorld("yolov8s-world.pt")  # or choose yolov8m/l-world.pt

# Define custom classes
model.set_classes(["custom_class"])

# Execute prediction for specified categories on an image
results = model.predict("path/to/image.jpg")

# Show results
results[0].show()

If you continue to face issues, please share more details about your setup and the specific steps you are taking. This will help us provide more targeted assistance. Thank you! 😊

For more information, you can also refer to the YOLO-World documentation.

[jasukej]

Hello! I'm getting this error only when running model.set_classes. model.predict without fine-tuning the prompts work fine.

Error occurred: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1000)>

[pderrenger]

@jasukej hello! Thank you for reaching out. It seems like you're encountering an SSL certificate verification issue when using model.set_classes. This error typically arises due to the system's inability to verify the SSL certificate chain, often because of a self-signed certificate.

Here are a few steps you can take to resolve this:

1. Update Packages: Ensure you are using the latest versions of all relevant packages, including ultralytics and any dependencies. This can often resolve SSL-related issues.

   pip install --upgrade ultralytics

2. Set Environment Variables: You can bypass SSL verification by setting the SSL_CERT_FILE environment variable to a trusted certificate file. However, this is generally not recommended for production environments due to security risks.

3. Custom SSL Context: If you are in a controlled environment and understand the risks, you can create a custom SSL context to ignore certificate verification:

   import ssl
   import urllib.request
   
   ssl._create_default_https_context = ssl._create_unverified_context
   
   from ultralytics import YOLOWorld
   
   # Initialize a YOLO-World model
   model = YOLOWorld("yolov8s-world.pt")
   
   # Define custom classes
   model.set_classes(["person", "bus"])
   
   # Execute prediction for specified categories on an image
   results = model.predict("path/to/image.jpg")
   results[0].show()

4. Reproducible Example: If the issue persists, please provide a minimum reproducible example to help us diagnose the problem more effectively. You can find guidelines for creating one here.

5. Network Configuration: Sometimes, network configurations or proxies can interfere with SSL verification. Ensure your network settings allow for proper SSL certificate validation.

Please try these steps and let us know if the issue persists. We're here to help! 😊

For more details on using YOLO-World, you can refer to the YOLO-World documentation.

[meidachen]

Thanks for the great work! I'm wondering which training data has been used for yolov8x-worldv2 model?
The model on the official site has "YOLO-Worldv2-X" trained on O365+GoldG+CC3M-Lite, are the training data the same as from here?

[glenn-jocher]

@meidachen thank you for your kind words! 😊

The YOLOv8x-worldv2 model has indeed been trained on a combination of datasets, specifically Objects365, GoldG, and CC3M-Lite. This diverse training data helps the model excel in open-vocabulary detection tasks by leveraging a wide variety of object categories and contexts.

If you have any further questions or need additional details, feel free to ask. For more information on the YOLO-World model, you can also visit our YOLO-World documentation.

Happy experimenting! 🚀