Loading two models about license plate detection and license plate character detection.

[jayer95]

Hi, I am trying to use YOLOv5 to implement a license plate detection and license plate character detection system. The license plate detection model will be trained with YOLOv5, and the license plate character detection model will also be trained with YOLOv5. I already have a data set.

I would like to ask you, does the YOLOv5 framework load two models at the same time when detact.py is executed once?
This process will first use the license plate detection model to detect the license plate and then perform the crop, then the cropped license plate will pass through the second-level license plate character detection model, and finally the license plate number will be obtained, as shown in the following reference website.

https://github.com/stephanecharette/DarkPlate

[glenn-jocher]

@jayer95 no, detect.py has provisional structure in place for second stage classification.

In general second stage classification may be useful for reducing FPs. See TowerScout for a deployed example of YOLOv5 paired with a second stage EfficientNet model.
https://groups.ischool.berkeley.edu/TowerScout/

[jayer95]

@glenn-jocher
May I ask the "detect.py" you mentioned has provisional structure in place for second stage classification, how to use it?
Do you refer to the webpage you mentioned? Or are there parameters that can be used on "detect.py"?

[jayer95]

I saw this!
Can I directly load the second yolov5.pt here?

if classify: # second-stage classifier modelc = load_classifier(name='resnet50', n=2) # initialize modelc.load_state_dict(torch.load('resnet50.pt', map_location=device)['model']).to(device).eval()

[glenn-jocher]

@jayer95 yes that's the idea! If you have a classification model with the same classes as your detection model you can pass it here :)

[jayer95]

@glenn-jocher Currently, I use your reserved classify function to achieve the following effect.
But the classes of the two models are not the same, the class names are different, and the number of classes is also different.
The first one: License plate detection model (YOLOv5s, 640x640), the number of categories is 1 (license plate)
The second one: Character detection model (LPRNet, 94x27), the number of categories is 36 (0 ~ 9, A ~ Z)

[glenn-jocher]

@jayer95 wow very cool! It looks like it's working very well.

[jayer95]

The tracking algorithm of Deep Sort is currently being integrated. I have added two classification models of "car color" and "car make and model", which are under development.
Good news will be reported to you, thank you for your immediate reply.
The YOLOv5 you developed has many functions, everything is perfect.

I would like to ask you, if both layers use YOLOv5 object detection for the same development goal, from which piece of code should I start to modify or can you provide a new update?
I think this is a good new feature, which will pass the ROI detected by the first-layer model object to the second-layer model object detection.

I have a dataset (U.S.) of license plate and a dataset (U.S.) of license plate characters (0 ~ 9, A ~ Z), both of which are marked in YOLO format. The license plate dataset has 500,000 images and license plate characters dataset has 600,000 images.

I think that using YOLOv5's object detection on both layers will achieve better results, because American license plates have special conditions.
Such as, "M" "C" "L"

LPRNet can only scan from left to right and pass through the network.
Hope to get your help. The reason why I use the classification model (LPRNet) in the second layer first is because the code you reserve allows me to develop quickly, and I need to develop a simple Proof of Concept (POC) in a short time.

[glenn-jocher]

@jayer95 looks really good! I think the two stage approach is correct as you can detect license plates at low resolution and then use a higher resolution ROI for the character recognition.

[jayer95]

@glenn-jocher
Thank you for your reply. If you plan to develop a new update, please let me know about the two-stage YOLOv5 object detection. Maybe I can help.

It is indeed divided into two stages to increase the accuracy of license plate character detection. It can even solve the problem of stack character.

https://github.com/bharatsubedi/ALPR-Yolov5

This is the reference I found about using YOLOv5 object detection on two stages.

Please let me know if you have any ideas.

[github-actions]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com/hub
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

[jayer95]

@glenn-jocher
Hi, I recently integrated the ideas mentioned earlier.
I used 6 models,
Contains 2 object detection models,
Model with 1 object tracking,
And 3 object classification models,
Object detection models all use YOLOv5,
Object tracking uses deep learning algorithms such as Deep Sort,
Object classification uses TensorFlow classification model,
It is worth noting that there is an order in the inference of models.
The car must be detected before the ROI of the car is crop to the two classification models of the car for inference (color, make, etc...),
The ROI cropping of the car will also make inferences for the license plate object detection model.
This can solve the problem that the license plate pixels are too small to be correctly detected, etc.
Finally, the ROI of the license plate is crop to the classification of license plate characters.
However, the LPRNet we used is not ideal and is expected to be replaced with YOLOv5n.
The above layer and layer approach, I suggest that the author can adopt it into the next development project,
Develop new functions for "detect.py" of YOLOv5, that is, to execute 2 YOLOv5,
This is very useful and interesting.

If you have any ideas, please let me know and discuss with me, thank you.

In addition, I would like to ask you, "detect.py" is there a way to increase the speed when inferring videos? For example, increase "batch".

[glenn-jocher]

@jayer95 very impressive! Nice example of cascaded detection and overall integration of various tools into a single result.

[github-actions]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com/hub
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

[jelena2712]

@glenn-jocher Hi, I recently integrated the ideas mentioned earlier. I used 6 models, Contains 2 object detection models, Model with 1 object tracking, And 3 object classification models, Object detection models all use YOLOv5, Object tracking uses deep learning algorithms such as Deep Sort, Object classification uses TensorFlow classification model, It is worth noting that there is an order in the inference of models. The car must be detected before the ROI of the car is crop to the two classification models of the car for inference (color, make, etc...), The ROI cropping of the car will also make inferences for the license plate object detection model. This can solve the problem that the license plate pixels are too small to be correctly detected, etc. Finally, the ROI of the license plate is crop to the classification of license plate characters. However, the LPRNet we used is not ideal and is expected to be replaced with YOLOv5n. The above layer and layer approach, I suggest that the author can adopt it into the next development project, Develop new functions for "detect.py" of YOLOv5, that is, to execute 2 YOLOv5, This is very useful and interesting.

If you have any ideas, please let me know and discuss with me, thank you.

In addition, I would like to ask you, "detect.py" is there a way to increase the speed when inferring videos? For example, increase "batch".

@glenn-jocher Hi, I recently integrated the ideas mentioned earlier. I used 6 models, Contains 2 object detection models, Model with 1 object tracking, And 3 object classification models, Object detection models all use YOLOv5, Object tracking uses deep learning algorithms such as Deep Sort, Object classification uses TensorFlow classification model, It is worth noting that there is an order in the inference of models. The car must be detected before the ROI of the car is crop to the two classification models of the car for inference (color, make, etc...), The ROI cropping of the car will also make inferences for the license plate object detection model. This can solve the problem that the license plate pixels are too small to be correctly detected, etc. Finally, the ROI of the license plate is crop to the classification of license plate characters. However, the LPRNet we used is not ideal and is expected to be replaced with YOLOv5n. The above layer and layer approach, I suggest that the author can adopt it into the next development project, Develop new functions for "detect.py" of YOLOv5, that is, to execute 2 YOLOv5, This is very useful and interesting.

If you have any ideas, please let me know and discuss with me, thank you.

In addition, I would like to ask you, "detect.py" is there a way to increase the speed when inferring videos? For example, increase "batch".

Hello, this is very interesting and the results are very good. Character detection model (LPRNet, 94x27), did You implement it on Your own, or what did You use? Thank You in advance for Your answer!

[glenn-jocher]

@jelena2712 hello! It's great to see your sophisticated use of YOLOv5 in creating such an integrated system. Your suggestions for modifications to detect.py to enable multi-stage detection are definitely worth considering for those needing similar use-cases. As for increasing the inference speed on videos, utilizing larger batch sizes can help if your GPU memory allows. Another approach is to optimize model inference by using model quantization or pruning techniques which may reduce the computational load without severely impacting the detection performance.

Keep up the great work! If you decide to move forward with a commercial application, remember to explore the Ultralytics Enterprise License since all implementations using YOLOv5, whether for community or internal use, need to either fully open-source under AGPL 3.0 or have a proper commercial license. 🚀

[jayer95]

@jelena2712
LPRNet is very good, but its performance on American license plates is not ideal because some characters on American license plates are very small and overlapped.

In fact, what model should be used in layer 2 depends on your application and the Dataset you have.