-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Segmentation model vs object detetion model on MOT metrics
Mike edited this page Dec 21, 2022
·
6 revisions
HOTA: exp333-pedestrian HOTA DetA AssA DetRe DetPr AssRe AssPr LocA OWTA HOTA(0) LocA(0) HOTALocA(0)
MOT17-04-FRCNN 48.026 41.227 56.245 44.678 76.993 60.018 83.968 84.253 50.09 62.296 76.817 47.854
MOT17-05-FRCNN 40.099 39.414 41.327 45.198 63.145 50.738 62.782 76.957 43.138 59.225 66.514 39.393
MOT17-09-FRCNN 49.958 54.709 45.795 61.094 75.552 51.28 75.088 83.485 52.886 64.615 78.253 50.563
MOT17-10-FRCNN 46.113 46.385 46.021 50.54 71.788 51.213 73.212 79.139 48.219 63.915 72.6 46.402
MOT17-11-FRCNN 54.829 49.251 61.381 60.395 67.211 67.402 82.495 84.224 60.857 70.917 76.47 54.23
MOT17-13-FRCNN 41.167 34.433 49.517 36.534 74.319 54.577 74.772 80.885 42.492 53.459 75.688 40.462
COMBINED 47.479 42.552 53.446 47.023 73.331 58.48 79.954 82.554 50.056 62.434 75.29 47.007
CLEAR: exp333-pedestrian MOTA MOTP MODA CLR_Re CLR_Pr MTR PTR MLR sMOTA CLR_TP CLR_FN CLR_FP IDSW MT PT ML Frag
MOT17-04-FRCNN 40.213 83.836 40.371 49.2 84.785 20.482 45.783 33.735 32.26 23398 24159 4199 75 17 38 28 432
MOT17-05-FRCNN 32.861 74.732 34.22 52.899 73.904 17.293 63.158 19.549 19.495 3659 3258 1292 94 23 84 26 226
MOT17-09-FRCNN 61.446 81.816 62.347 71.606 88.551 46.154 50 3.8462 48.425 3813 1512 493 48 12 13 1 65
MOT17-10-FRCNN 53.766 75.986 54.42 62.411 88.649 31.579 50.877 17.544 38.779 8013 4826 1026 84 18 29 10 334
MOT17-11-FRCNN 42.9 83.879 43.122 66.49 73.995 33.333 26.667 40 32.181 6274 3162 2205 21 25 20 30 63
MOT17-13-FRCNN 40.947 77.701 41.651 45.405 92.364 23.636 36.364 40 30.822 5286 6356 437 82 26 40 44 214
COMBINED 43.095 81.139 43.526 53.825 83.939 25 46.281 28.719 32.943 50443 43273 9652 404 121 224 139 1334
Identity: exp333-pedestrian IDF1 IDR IDP IDTP IDFN IDFP
MOT17-04-FRCNN 56.412 44.574 76.813 21198 26359 6399
MOT17-05-FRCNN 51.55 44.224 61.785 3059 3858 1892
MOT17-09-FRCNN 60.409 54.629 67.557 2909 2416 1397
MOT17-10-FRCNN 60.819 51.819 73.603 6653 6186 2386
MOT17-11-FRCNN 62.238 59.082 65.751 5575 3861 2904
MOT17-13-FRCNN 54.466 40.62 82.631 4729 6913 994
COMBINED 57.373 47.082 73.422 44123 49593 15972
Count: exp333-pedestrian Dets GT_Dets IDs GT_IDs
MOT17-04-FRCNN 27597 47557 148 83
MOT17-05-FRCNN 4951 6917 164 133
MOT17-09-FRCNN 4306 5325 77 26
MOT17-10-FRCNN 9039 12839 122 57
MOT17-11-FRCNN 8479 9436 151 75
MOT17-13-FRCNN 5723 11642 127 110
COMBINED 60095 93716 789 484
HOTA: exp334-pedestrian HOTA DetA AssA DetRe DetPr AssRe AssPr LocA OWTA HOTA(0) LocA(0) HOTALocA(0)
MOT17-04-FRCNN 42.684 40.487 45.264 43.756 77.042 52.906 64.577 84.061 44.466 55.523 76.212 42.315
MOT17-05-FRCNN 33.745 39.342 29.444 45.853 61.503 47.476 43.938 76.73 36.633 49.82 66.034 32.898
MOT17-09-FRCNN 45.458 53.836 38.558 60.645 74.307 50.176 57.253 83.197 48.344 59.416 77.381 45.976
MOT17-10-FRCNN 38.952 44.903 33.998 49.422 70.332 45.939 48.433 78.888 40.968 53.613 71.707 38.444
MOT17-11-FRCNN 47.463 49.058 46.369 59.666 67.784 62.357 61.86 84.195 52.523 60.833 76.163 46.332
MOT17-13-FRCNN 35.429 32.463 38.945 34.402 74.486 49.667 52.393 81.229 36.552 45.686 75.553 34.517
COMBINED 41.572 41.677 41.937 46.087 72.935 52.77 59.31 82.411 43.866 54.584 74.69 40.769
CLEAR: exp334-pedestrian MOTA MOTP MODA CLR_Re CLR_Pr MTR PTR MLR sMOTA CLR_TP CLR_FN CLR_FP IDSW MT PT ML Frag
MOT17-04-FRCNN 39.441 83.793 39.725 48.26 84.972 20.482 40.964 38.554 31.62 22951 24606 4059 135 17 34 32 502
MOT17-05-FRCNN 31.054 74.491 33.092 53.824 72.193 19.549 63.91 16.541 17.324 3723 3194 1434 141 26 85 22 232
MOT17-09-FRCNN 61.296 81.575 62.085 71.85 88.035 50 46.154 3.8462 48.057 3826 1499 520 42 13 12 1 63
MOT17-10-FRCNN 50.627 75.889 52.045 61.157 87.032 28.07 50.877 21.053 35.881 7852 4987 1170 182 16 29 12 381
MOT17-11-FRCNN 43.822 83.888 44.15 66.087 75.078 33.333 29.333 37.333 33.174 6236 3200 2070 31 25 22 28 55
MOT17-13-FRCNN 37.528 78.254 38.816 42.501 92.022 21.818 33.636 44.545 28.286 4948 6694 429 150 24 37 49 251
COMBINED 41.8 81.128 42.526 52.858 83.65 25 45.248 29.752 31.825 49536 44180 9682 681 121 219 144 1484
Identity: exp334-pedestrian IDF1 IDR IDP IDTP IDFN IDFP
MOT17-04-FRCNN 47.305 37.086 65.298 17637 29920 9373
MOT17-05-FRCNN 41.743 36.432 48.866 2520 4397 2637
MOT17-09-FRCNN 54.017 49.052 60.101 2612 2713 1734
MOT17-10-FRCNN 49.668 42.285 60.175 5429 7410 3593
MOT17-11-FRCNN 52.858 49.693 56.453 4689 4747 3617
MOT17-13-FRCNN 40.719 29.763 64.441 3465 8177 1912
COMBINED 47.539 38.79 61.387 36352 57364 22866
Count: exp334-pedestrian Dets GT_Dets IDs GT_IDs
MOT17-04-FRCNN 27010 47557 92 83
MOT17-05-FRCNN 5157 6917 85 133
MOT17-09-FRCNN 4346 5325 40 26
MOT17-10-FRCNN 9022 12839 47 57
MOT17-11-FRCNN 8306 9436 73 75
MOT17-13-FRCNN 5377 11642 60 110
COMBINED 59218 93716 397 484
HOTA -5.907 MOTA -1.295 IDF1 -9.834
All metric take large hits. Only use segmentation models if segmentations when tracking are absolutely needed.