Suitability for integration with YOLOv5?

[glenn-jocher]

Hi, very interesting repo! We've had some requests for confusion matrices in YOLOv5, i.e.:
ultralytics/yolov5#1402 (comment)

Do you think this work might be suitable for integration? Would you be interested in perhaps helping contribute a PR? We have a file here that might make a good destination for integration, where we currently calculate our TP/FP vectors and mAP scores.
https://github.com/ultralytics/yolov5/blob/master/utils/metrics.py

[kaanakan]

Hi,

thank you for your interest. I wrote this code for ultralytics yolov3 repository but I do not have the code right now. It can be integrated to yolov5 as well. However, I do not have time to implement a PR currently.

I can help you if you wish to implement it. I will leave it open for further questions.

[glenn-jocher]

@kaanakan oh great, thanks for the feedback. The Ultralytics v3 and v5 repos share a lot in common, so if it worked for v3 we should be able to do it for v5. I will look over it and perhaps try a PR myself.

[glenn-jocher]

@kaanakan I've started an integration here:
ultralytics/yolov5#1474

It needs a little more work but everything seems to be working well. I need to figure out how to scale the image from low class counts to high appropriately.

[glenn-jocher]

Here I have YOLOv5s on VOC. What's the interpretation of the last row and column. Last column seems to be class mistaken for background (FN), last row seems to be background mistaken for class (FP). Is that correct?

[glenn-jocher]

@kaanakan I've updated the axes labels a bit with my FP FN interpretation:

[kaanakan]

I think you are right. To be sure, the last row is for the detections that doesn't match with any ground truth label, so, background FP is correct, I think. The last column is for the ground truth objects that cannot be detected by the mode, so, background FN is correct.

I think the image you posted looks like correct. If you have any other questions, please let me know.

[glenn-jocher]

@kaanakan great, thanks!