How can I train one or several new classes on top of an existing training set.

[SpongeBab]

❔Question

How can I train one or several new classes on top of an existing training set.

Additional context

For example,If I have a new dataset, and I want to add the new class on the trained COCO weights.Um.....What I should do or is there any guide?

[yannclaes]

I think this tutorial should do it. All you need to do is to adapt the data/.yaml file for your specific dataset !

[SpongeBab]

@yannclaes No, you don't understand what I'm saying! What I mean is ,simply,I want to train a new class on the base of the model that has been trained.At the same time, keep the classes that have been trained.I need to get 80+x classes.
@glenn-jocher This is a meaningless reply.You shouldn't give him 👍.
☹

[glenn-jocher]

@SpongeBab by definition training a model modifies all of the weights and biases in it to minimize the loss on your new labels.

There is no such thing as training a model on new classes while retaining existing weights and bias values anymore than I can drink a glass of water without affecting the water in the glass.

[yannclaes]

@SpongeBab I think my answer still applies, but you have to provide COCO images too. When you modify your data.yaml, the number of classes is modified (as you noted, 80 + x) thus the number of outputs is modified too (it becomes nb_anchors * (80 + x + 5)). When loading your model with pre-trained weights, you'll fall into this:

yolov5/train.py

Line 91 in 7b36e38

state_dict = intersect_dicts(state_dict, model.state_dict(), exclude=exclude) # intersect

Your Detect() module will be initialized with new weights corresponding to this new nc because shapes don't match anymore. So the bottom line is that to train a model on COCO + x classes, you need to provide data (images + labels) for all classes you want to train on, including COCO classes.

Edit: found this after a quick search, conclusions were already identical.

[SpongeBab]

@glenn-jocher @yannclaes Thank you for your kind answers.
@glenn-jocher Yeah,I know. Thank you for your patient answer😆.I found some information about this:https://zhuanlan.zhihu.com/p/73162940. Emmm,What do you think about it?I'm trying to run the program. But there are some problems that I haven't solved yet.
I think this is a very very great and significant technology.And I'm really looking forward to your V5 paper.The Scaled-yolo, which have published a paper, its code is similar to your code. You've achieved a significant improvement👍.
Edit:Something great, the V5 can train at dozens of times the speed of the V4, or even more. And, of course, about three to five times faster than Scaled-YOLO.Even retraining is acceptable.

[SpongeBab]

@yannclaes Hi.
If I understand you correctly. You mean to add the new images and categories to the COCO data set and retrain again, right? It's like training class 83 instead of 80+3.
What I want to do is keep 80 classes of COCO and then train a new dataset, assuming it contains only two classes, then I use the weights of COCO already trained and train these two classes on the new dataset and finally I am able to detect 83(80+3).
And I've seen that so far it doesn't seem to work......
Some information: https://zhuanlan.zhihu.com/p/73162940.

[yannclaes]

@yannclaes Hi.
If I understand you correctly. You mean to add the new images and categories to the COCO data set and retrain again, right? It's like training class 83 instead of 80+3.
What I want to do is keep 80 classes of COCO and then train a new dataset, assuming it contains only two classes, then I use the weights of COCO already trained and train these two classes on the new dataset and finally I am able to detect 83(80+3).
And I've seen that so far it doesn't seem to work......
Some information: https://zhuanlan.zhihu.com/p/73162940.

That's exactly it ! I guess your goal is to save training time... However I think you have no other choice than full re-training from COCO weights, as explained in the tutorial. Nevertheless, preparing your data to include your 3 new classes should not be too hard as you would simply need to move your splits to the appropriate COCO split folders and do the same for your annotation files.

I didn't know about GroupSoftmax but it seems rather aimed towards dealing with class imbalance in the learning set, so I don't see how it could help here but I might miss it ;)

[SpongeBab]

@yannclaes ahah，yeah. Everyone is working to improve AP.Thank you again.

[glenn-jocher]

@SpongeBab @yannclaes yes that is correct. You can train multiple datasets simultaneously by adding them to the train and val fields of your dataset.yaml as a list. All the datasets in the list must share common class naming convention though, so for example if you want to train classes in addition to coco you can add your custom dataset to the train/val lists, making sure it's class indices start at 80. GlobalWheat2020.yaml is a good example of grouping multiple datasets togethor:

yolov5/data/GlobalWheat2020.yaml

Lines 9 to 27 in 0e2f2cb

	# train and val data as 1) directory: path/images/, 2) file: path/images.txt, or 3) list: [path1/images/, path2/images/]
	train: # 3422 images
	- ../datasets/GlobalWheat2020/images/arvalis_1
	- ../datasets/GlobalWheat2020/images/arvalis_2
	- ../datasets/GlobalWheat2020/images/arvalis_3
	- ../datasets/GlobalWheat2020/images/ethz_1
	- ../datasets/GlobalWheat2020/images/rres_1
	- ../datasets/GlobalWheat2020/images/inrae_1
	- ../datasets/GlobalWheat2020/images/usask_1
	val: # 748 images (WARNING: train set contains ethz_1)
	- ../datasets/GlobalWheat2020/images/ethz_1
	test: # 1276 images
	- ../datasets/GlobalWheat2020/images/utokyo_1
	- ../datasets/GlobalWheat2020/images/utokyo_2
	- ../datasets/GlobalWheat2020/images/nau_1
	- ../datasets/GlobalWheat2020/images/uq_1

[github-actions]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

[rizi1122]

@SpongeBab hello !
hope so you are fine!
i am solve these type of problem but still not solve .kindly share with me your best idea and notebook ?
thanks in advance.