[OpenPifPaf] ONNX Evaluation Pipeline

[dbogunowicz]

Feature Preview:

python3 -m openpifpaf.export_onnx --input-width 641 --input-height 641
deepsparse.pose_estimation.eval --model-path openpifpaf-resnet50.onnx  --dataset cocokp --image_size 641

Results in:

...
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets= 20 ] = 0.502
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets= 20 ] = 0.732
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets= 20 ] = 0.523
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.429
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.605
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 20 ] = 0.534
 Average Recall     (AR) @[ IoU=0.50      | area=   all | maxDets= 20 ] = 0.744
 Average Recall     (AR) @[ IoU=0.75      | area=   all | maxDets= 20 ] = 0.554
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.457
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.643
...

For more information please refer to the updated README.md

Manual Testing

Compare the output above to the original openpifpaf eval:

python3 -m openpifpaf.eval --dataset cocokp --checkpoint shufflenetv2k16

Outputs:

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets= 20 ] = 0.501
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets= 20 ] = 0.732
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets= 20 ] = 0.515
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.428
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.604
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 20 ] = 0.532
 Average Recall     (AR) @[ IoU=0.50      | area=   all | maxDets= 20 ] = 0.742
 Average Recall     (AR) @[ IoU=0.75      | area=   all | maxDets= 20 ] = 0.549
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.454
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.643

[corey-nm]

📈