data.md

Data

Please refer to the README in foler data_preparation/ for details about how we preprocess data.

Preprocessed Data

Name	Size	Details	Required for training?
mp3d_planercnn_json.zip	160 MB	Jsons that contain the dataset information.	Yes
rgb.zip	21 GB	Habitat generated images.	Yes
observations.zip	64 GB	Depth and semantic labels.	Yes
id2semantic.zip	728 KB	Instance id to semantic name.	No
planes_ply_mp3dcoord_refined.zip	28 GB	Plane annotations.	No
cameras.zip	4.4 MB	Camera poses.	No

If the above Dropbox link does not work, alternatively, you can download from Google Drive.

Dataset Json Files

We write a custom dataloader in Detectron2 and it loads jsons that contain the dataset information. mp3d_planercnn_json.zip contains jsons for train/val/test split. Each json file stores images pairs and their annotations.

# json file data structure
"info": "...",
"categories":  [{'id': 0, 'name': 'plane'}],
"data": [
    "0": {                                      # image A
        "file_name": /path/to/image_id.png,
        "image_id": image_id,
        "height": 480,
        "width": 640,
        "camera": {                             # camera pose in the asset
            "position": [x,y,z],                
            "rotation": [w, xi, yi, zi],        # quaternion
        }
        "annotations": [                        # list of planes, with detectron2 annotations format.
            {                                   
                "id":                           
                "image_id":
                "category_id":
                "iscrowd":
                "area": 
                "bbox":
                "bbox_mode":
                "width":
                "height":
                "segmentation":
                "plane":                        # plane parameters
            },
        ]
    },
    "1": {...},                                 # image B
    "gt_corrs": [...],                          # List of pairs of corresponding plane indices
    'rel_pose': {                               # A's pose in B's coordinate frame.
        'position':
        'rotation':
    },
    ...
]