CVAT keypoint annotation like COCO dataset

[hagonata]

Actions before raising this issue

• I searched the existing issues and did not find anything similar.
• I read/searched the docs

Steps to Reproduce

1. Make Skeleton of keypoints
2. Label
3. Exportt with COCO keypoints v 1.0

Expected Behavior

I want exact the same jsons like in COCO dataset.

"annotations": [{"segmentation": [[125.12,539.69,140.94,522.43,100.67,496.54,84.85,469.21,73.35,450.52,104.99,342.65,168.27,290.88,179.78,288,189.84,286.56,191.28,260.67,202.79,240.54,221.48,237.66,248.81,243.42,257.44,256.36,253.12,262.11,253.12,275.06,299.15,233.35,329.35,207.46,355.24,206.02,363.87,206.02,365.3,210.34,373.93,221.84,363.87,226.16,363.87,237.66,350.92,237.66,332.22,234.79,314.97,249.17,271.82,313.89,253.12,326.83,227.24,352.72,214.29,357.03,212.85,372.85,208.54,395.87,228.67,414.56,245.93,421.75,266.07,424.63,276.13,437.57,266.07,450.52,284.76,464.9,286.2,479.28,291.96,489.35,310.65,512.36,284.76,549.75,244.49,522.43,215.73,546.88,199.91,558.38,204.22,565.57,189.84,568.45,184.09,575.64,172.58,578.52,145.26,567.01,117.93,551.19,133.75,532.49]],"num_keypoints": 10,"area": 47803.27955,"iscrowd": 0,"keypoints": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,142,309,1,177,320,2,191,398,2,237,317,2,233,426,2,306,233,2,92,452,2,123,468,2,0,0,0,251,469,2,0,0,0,162,551,2],"image_id": 425226,"bbox": [73.35,206.02,300.58,372.5],"category_id": 1,"id": 183126},

but from CVAT I got smth like this:
"annotations": [{"id": 1, "image_id": 1, "category_id": 15, "segmentation": [], "area": 7147.334000000006, "bbox": [684.64, 84.72, 53.45, 133.72], "iscrowd": 0, "attributes": {"occluded": false, "track_id": 0, "keyframe": true}, "keypoints": [738.09, 84.72, 2, 700.15, 145.59, 0, 754.48, 150.75, 0, 706.86, 218.44, 2, 684.64, 190.14, 2], "num_keypoints": 3},

I just don't have segmenation part.

Possible Solution

No response

Context

Environment

I use web version of CVAT.

[zhiltsov-max]

Hi. Getting exactly the same annotations will require some extra manual actions from your side. Normally, CVAT will only export skeleton annotations in COCO Keypoints. However, there is a way to control the segmentation and bbox fields in the export.

You will need to define 2 labels: a skeleton label and a regular label. Then you can draw and attach a custom polygon or a bbox to a skeleton using the Group Annotations tool:

For example, here I have 1 skeleton with 4 points ("1", "2", "3", "4"), and 1 polygon annotation, merged in a group:

You can use the group highlight mode to see grouped annotations:

Once you have such grouped annotations ready, you can export in COCO Keypoints normally and get something like this:

"categories": [
    {
      "id": 1,
      "name": "skele",
      "supercategory": "",
      "keypoints": ["1", "2", "3", "4"],
      "skeleton": []
    },
    {
      "id": 6,
      "name": "skele-poly",
      "supercategory": "",
      "keypoints": [],
      "skeleton": []
    }
  ],
...
"annotations": [
    {
      "id": 1,
      "image_id": 1,
      "category_id": 6,
      "segmentation": [
        [
          675.93, 366.0, 756.19, 413.09, 743.35, 481.58, 834.31, 429.14, 857.86,
          521.18, 745.49, 581.11, 777.59, 659.23, 702.68, 714.88, 664.15,
          605.72, 591.38, 622.85, 627.77, 537.23, 519.68, 581.11, 505.77,
          482.65, 680.21, 491.21, 583.89, 436.63, 523.96, 370.28, 637.4, 349.95
        ]
      ],
      "area": 61515.0,
      "bbox": [505.77, 349.95, 352.09, 364.93],
      "iscrowd": 0,
      "attributes": { "occluded": false },
      "keypoints": [
        598.2, 406.67, 2, 786.15, 472.79, 2, 737.33, 649.6, 2, 568.91, 540.35, 2
      ],
      "num_keypoints": 4
    }
  ]

It's close, but here we have inconsistent / invalid category ids for the exported annotations. You can fix it manually by removing the polygon category (6 in the example), and search-replacing category_id with the skeleton category id (6 to 1 in the example).

Another option is to fix it automatically with Datumaro. Install it using pip: pip install 'pip install 'git+https://github.com/cvat-ai/datumaro.git@8a14a99fe17f19d98595a2a4a74ab459051cc23b''

Then run the command in the terminal (replace paths, skele-poly and skele with your values).

datum transform -t remap_labels -o "./remapped-coco" "./job-coco-keypoints-1.0/:coco_person_keypoints" -- -l 'skele-poly:skele'

If you want to control visibility of separate keypoints in a skeleton, set the Outside (for COCO visibility 0) and Occluded (for COCO visibility 1) properties of these keypoints.

[hagonata]

Thank you very much for the very detailed response. I will try it out!