val.py metrics. (what is used?)

[felix-knopp]

Hi,
Is there any official description on how the metrics in val.py are calculated?
ap_per_class in metrics.py mentions Object-Detection-Metrics as source.
However, I get slightly different results when using their updated version of Object-Detection-Metrics to validate my models.

I see that pycocotools is used for map and map50 but only if save_json=True is set.
This leads to unexpected behaviour as the printed results by val.py will differ from the return values when using save_json (assuming the evaluation in pycocotools has different results than metricy.py ).

I assume that P and R is computed for IOU >= 0.5 at maximum F1-Score.
The PR-Curve seems to be interpolated over all points (or over 101 points?). So I assume that AP values are calculated as area under the interpolated PR-Curves?

Edit: I know the general definition of the metrics used in val.py. I'm just curious about the details in the implementation. Maybe an 'official' method is used that is described somewhere? (like coco metrics, pascal VOC metrics, ...)

[glenn-jocher]

@felix-knopp yes that all sounds right. There is an open issue of pycocotools reporting higher mAP than our own mAP code in #2258.

All of the metrics code is in metrics.py:
https://github.com/ultralytics/yolov5/blob/master/utils/metrics.py