Question about predition mat generation

[ahangchen]

When we do Multi person Estimation and generate the prediction mat

% Set predicted x_pred, y_pred coordinates and prediction score for each body joint
pred(imgidx).annorect(ridx).annopoints.point(pidx).x = x_pred;
pred(imgidx).annorect(ridx).annopoints.point(pidx).y = y_pred;
pred(imgidx).annorect(ridx).annopoints.point(pidx).score = score;

Should ridx be in rectidxs_multi_test (extracted by getMultiPersonGroups(groups,RELEASE,false))? Or it's just a iterator over the people in an image according to my preditions result?

[anibali]

I have no idea about multi-person estimation, sorry. I've only ever attempted the single person task.

[ahangchen]

OK, same question in single person way:
When we generate pred_keypoints_mpii.mat for single person evaluation, what's ridx referring to?

pred = annolist_test;
pred(imgidx).annorect(ridx).annopoints.point(pidx).x = x_pred;
pred(imgidx).annorect(ridx).annopoints.point(pidx).y = y_pred;
pred(imgidx).annorect(ridx).annopoints.point(pidx).id = id;

For example, for img 1, we predict 4 persons. But in the ground truth, single person field infers that img 1 only contains 2 single person. In this condition, what's the range of ridx when we prepare the mat? 1~4 or 1~2? Should we filter out some results or sort them according to the single person field?

[ahangchen]

In multi person evaluation, the problem may be more serious. If we predict a single person that doesn't belong to any group. It will be treated as false prediction, but actually, there is indeed one corresponding person in the image. This makes multi person evaluation weired.

[anibali]

Single person evaluation

Each example has a unique combination of imgidx and ridx. In the .h5 annotation files (which are derived from the original .m annotations), all of the examples are flattened, and can be indexed linearly by an example index. You can look up the original ridx and imgidx values for a specific example from the .h5 by using the example index to look up values from /person and /index respectively. Here's an example of looking up ridx from my code for preparing test results:

eval-mpii-pose/prepareTestResults.m

Line 79 in 2ae9913

pred(idx).annorect(test_persons(i)).annopoints = point;

In this instance, i is the example index, idx is the value of imgidx for the example, and test_persons is used to look up the ridx value for the example.

So if there are only examples for two people in image 1, that's no problem---just look up what their ridx values are from the annotations, and fill in the joint predictions appropriately.

Multi person evaluation

I agree that the multi person evaluation protocol looks confusing. Like I said, I have not worked with it myself. You should probably contact the actual creators of the dataset to try and get things clarified. If you do work it out, please post your findings here to help other people in the future.