How to use missing data in test set

[Blyuger]

I have a question about missing data for OpenImage Dataset v4. I am formating my data to OpenImage Dataset v4, but i have missing data (no objects for detection in a image) in my test dataset and I didn't find in the github an example of annotations when the image has no objects to detect: review_object_detection_metrics/data/database/gts/openimages_format/

Could you help me with this?

I think that is commom don't having annotations in some of the imagens in the test set and the model do not recognize objects in it, but I didn't find how to label cases like this in the example.

How can I fill an example like this in the layout with no objects for detection in a image:

ImageID: example.jpg
LabelName: ?
XMin: ?
XMax: ?
YMin: ?
Ymax: ?

The other columns I know that can be empty.

Tks,
Bruna Barreira.

[rafaelpadilla]

Hi @Blyuger,

Yes, cases with no ground-truth bounding box in a given image might happen.

For the evaluation tool, in such cases you just do not include this image in the csv file. So, in the given example, your csv file will not have any line whose column "ImageID" is "example.jpg".

Best regards,
Rafael

[Blyuger]

Thanks @rafaelpadilla !

But we could have a problem in these cases if the algorithm detects something in the imagens that dont have anything to detect, couldn't we?

The false positive case I mean.

How the tool could identify these cases?

[rafaelpadilla]

Hi @Blyuger ,

If you take a look at these lines, you will note that all detected and ground-truth reported bounding boxes are listed.

In your example, if an image does not have any ground-truth boxes, do not include the ground-truth file related to this image. In this case, any detection reported for that particular image will be considered false positive as I will show you next.

Consider the following practical case:
Your img_1.png does not have any ground-truth bounding box, but your system detected a bounding box in it. Therefore, your groundtruth folder will not have the img_1.txt file, but the folder with detections will have the img_1.txt file.

In this case, when this line is reached, it will loop through all detections, including the ones in the img_1.txt file.

But, the gt object here will be an empty list, because there is no ground-truth related to img_1. Therefore, the else statement in line 181 will be executed, and the detection is set as false positive.

Got it?

[Blyuger]

Got it Rafael!!!

It works :)

Thank you for the explanation.