Effectively Identifying Unknown or Ambiguous Items while Facilitating Addition/Removal of Support Classes without Costly Retraining

[ENNURSILA]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

Hello,

Do you have an idea or advice how to determine effectively which items are unknown, ambigious whilst still being able to add/remove support classes without expensive retraining ?

Thank you.

Additional

No response

[glenn-jocher]

@ENNURSILA hello!

Great question! To handle unknown or ambiguous items, you can consider using an "unknown" class during training, where you include examples of objects that do not belong to any of your known classes. This can help the model learn to identify objects that it hasn't been explicitly trained on.

For adding or removing classes without full retraining, you might look into few-shot learning techniques or incremental learning, where the model is updated with new information without forgetting the previous knowledge. This is an active area of research and might require some custom implementation.

Remember to keep an eye on the model's performance metrics when making such changes, as they can help you understand the impact of the modifications on your model's ability to generalize.

For more detailed guidance, please refer to our documentation at https://docs.ultralytics.com/yolov5/.

Best of luck with your project! 😊🚀

[ENNURSILA]

@glenn-jocher Hello,
Great answer, Thank you very much.

[glenn-jocher]

You're welcome, @ENNURSILA! If you have any more questions or need further assistance as you work on your project, feel free to reach out. Happy coding and best of luck with your YOLOv5 endeavors! 😄👍

[github-actions]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

[ENNURSILA]

Hello @glenn-jocher ,
I will ask you one more thing.
I will give a example.
Classes in my model: Chicken soup, Tomata soup, Pasta, Rice, Fish , Salad, Milk, Tea, Orange juice.
I want to limit it according to the menu of the day.
The menu of the day: Tomata soup, Pasta, Tea.
The model should not be able to recognize dishes that are not on the menu of the day. We want to limit the menu by days.
And the next day the menu will change. Must only recognize the meals on the menu of the day. So I want to add and remove classes according to the menu.
But retraining every day is costly. I hope I could explain.
Do you have any advice?

Best wishes,

[glenn-jocher]

@ENNURSILA hello!

Certainly, dynamically adjusting the recognized classes without retraining the model each day is a clever approach. One straightforward method to achieve this without modifying your model is to filter the model's predictions based on your daily menu.

Here's a simplified pseudocode example:

# Assume 'predictions' is a list of tuples from your model output, 
# where each tuple contains ('class_name', confidence_score)

daily_menu = ['Tomata soup', 'Pasta', 'Tea']  # Your menu of the day

# Filter predictions
filtered_predictions = [pred for pred in predictions if pred[0] in daily_menu]

# Now, 'filtered_predictions' will only contain items from your daily menu

This way, you can adjust daily_menu every day as needed without the need for retraining. The model still recognizes all its trained classes, but you programmatically limit the focus to only those relevant for a given day.

Hope this helps! If you have any further questions, feel free to ask. Happy coding! 😊

[ENNURSILA]

Hello @glenn-jocher , I have one more question.

What if tomorrow one should recognize a new meal instead of another? E.g. this magical model would detect 100 different items today, but tomorrow should 99 of the old but 1 new. Also, how should the model react if by accident the dessert from yesterday is still sold, so 101 items may appear under the camera?

Thank you very much

[glenn-jocher]

Hello!

For dynamically adjusting recognized classes including adding a new item while potentially still recognizing yesterday's items, consider an approach that allows for flexibility without retraining your model daily:

1. Model Capacity: Ensure your model is initially trained on a superset of items that could potentially appear. In your case, beyond the 100 items, include any additional items that might come into rotation.

2. Daily Item Filtering: Use a script to filter model predictions based on the daily menu, similar to the previous suggestion. This can be easily updated for additions or changes.

3. Handling Overlapping Days: To accommodate items from yesterday that might still appear, you could include a "margin" in your daily menu filters. For instance, include a small set of likely carry-over items each day in your script's filter list.

# Your extended daily menu might look something like this
daily_menu = ['New Soup', 'Pasta', ...]  # Today's main items
carry_over_items = ['Yesterday Dessert']  # Potential carry-over items

# Combine lists for filtering
full_menu = daily_menu + carry_over_items

# Filter predictions similarly as shown in the earlier example

This way, your model remains static, but its application dynamically adjusts to your daily needs without the overhead of retraining.

Hope this helps! Feel free to reach out if you have more questions. 😊

[github-actions]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐