Training Mask R-CNN using RLE bitmasks error

[alono88]

I am following the Mask R-CNN tutorial and changed the dataset_dict to support segmentation maps in bitmap format using RLE instead of polygons. I confirmed the data is processed properly using detectron2 visualization tool. When trying to train the network, I'm getting an error regarding polygons.

File "/home/ubuntu/detectron2/detectron2/data/detection_utils.py", line 149, in transform_instance_annotations polygons = [np.asarray(p).reshape(-1, 2) for p in annotation["segmentation"]] File "/home/ubuntu/detectron2/detectron2/data/detection_utils.py", line 149, in <listcomp> polygons = [np.asarray(p).reshape(-1, 2) for p in annotation["segmentation"]] ValueError: cannot reshape array of size 1 into shape (2)

It seems there is no support for RLE format during training although the visualization works. Is there any way to train using bitmasks?

[ppwwyyxx]

The model does support training with bitmasks, if instances.gt_masks = BitMasks(..). See the BitMasks class at https://detectron2.readthedocs.io/modules/structures.html#detectron2.structures.BitMasks

But the data loader does not recognize RLE, and therefore does not convert RLE to bitmasks.

[ppwwyyxx]

You can use a different dataloader following https://detectron2.readthedocs.io/tutorials/data_loading.html

[alono88]

Thanks!

[adriaciurana]

I'm with the same problem. To solve it I have edited the following files.

In structures/masks.py BitMasks class add the following method:

@staticmethod
    def from_rle(
        rles
    ) -> "BitMasks":
        """
        Args:
            rles
            height, width (int)
        """
        masks = [mask_utils.decode(rle).astype(np.bool) for rle in rles]
        return BitMasks(torch.stack([torch.from_numpy(x) for x in masks]))

In data/detection_utils.py change:

if len(annos) and "segmentation" in annos[0]:
        polygons = [obj["segmentation"] for obj in annos]
        if mask_format == "polygon":
            masks = PolygonMasks(polygons)
        else:
            assert mask_format == "bitmask", mask_format
            masks = BitMasks.from_polygon_masks(polygons, *image_size)
        target.gt_masks = masks

by:

if len(annos) and "segmentation" in annos[0]:
        polygons = [obj["segmentation"] for obj in annos]
        if mask_format == "polygon":
            masks = PolygonMasks(polygons)
        elif mask_format == "bitmask":
            masks = BitMasks.from_polygon_masks(polygons, *image_size)
        elif mask_format == "rle":
            masks = BitMasks.from_rle(polygons)

        else:
            assert mask_format
        target.gt_masks = masks

In also dataset_mapper.py change:

# USER: Implement additional transformations if you have other types of data
            annos = [
                utils.transform_instance_annotations(
                    obj, transforms, image_shape, keypoint_hflip_indices=self.keypoint_hflip_indices
                )
                for obj in dataset_dict.pop("annotations")
                if obj.get("iscrowd", 0) == 0
            ]

by:

annos = [
                obj
                for obj in dataset_dict.pop("annotations")
                if obj.get("iscrowd", 0) == 0
            ]

In your cfg change cfg.INPUT.MASK_FORMAT = 'rle'

I think I have not had to apply more changes.

[ppwwyyxx]

Now the default dataloader can work with RLE formats inside your dataset.
All you need is:

1. use RLE format (documented at https://detectron2.readthedocs.io/tutorials/datasets.html#standard-dataset-dicts) in your dataset
2. set INPUT.MASK_FORMAT='bitmask'.

[pvti]

Hi @ppwwyyxx
I'm trying to train a instance segmentaion model with 1 class X using detectron2's tutorial, but with a binary mask dataset.
With your instructions, I use RLE format (documented at https://detectron2.readthedocs.io/tutorials/datasets.html#standard-dataset-dicts).
In each "annotations", my "segmention" is a dict "represents the per-pixel segmentation mask in COCO’s RLE format. The dict should have keys “size” and “counts”. You can convert a uint8 segmentation mask of 0s and 1s into RLE format by pycocotools.mask.encode(np.asarray(mask, order="F"))." (I verify that bit-mask is successfully converted to RLE format).
So in this case, can you explain some of my questions, please:

1. What exactly is the "segmentation"?
   For example:
   
   left: original binary mask of 0s and 1s
   center: binary mask of the 1st instance
   right: binary mask of the 2nd instance
   => "segmentation" = pycocotools.mask.encode(np.asarray(left, order="F")) OR if I'm not wrong, in each instance "segmentation" = pycocotools.mask.encode(np.asarray(center/right , order="F")), right?
2. I trained the model with the boilerplate code in your getting started collab notebook, every thing is ok. But when I try to evaluate the model's performance using AP metric implemented in COCO API as in your example, I got the error:
   WARNING [03/04 10:32:52 d2.evaluation.coco_evaluation]: json_file was not found in MetaDataCatalog for 'hand10_val'. Trying to convert it to COCO format ... [03/04 10:32:52 d2.data.datasets.coco]: Converting dataset annotations in 'hand10_val' to COCO format ...) [03/04 10:32:52 d2.data.datasets.coco]: Converting dataset dicts into COCO format Traceback (most recent call last): File "/home/n/detectron2/datasets/EgteaGaze+/hand10/test.py", line 132, in <module> evaluator = COCOEvaluator("hand10_val", cfg, False, output_dir="./output/") File "/home/n/detectron2/detectron2/evaluation/coco_evaluation.py", line 71, in __init__ convert_to_coco_json(dataset_name, cache_path) File "/home/n/detectron2/detectron2/data/datasets/coco.py", line 416, in convert_to_coco_json coco_dict = convert_to_coco_dict(dataset_name) File "/home/n/detectron2/detectron2/data/datasets/coco.py", line 334, in convert_to_coco_dict polygons = PolygonMasks([segmentation]) File "/home/n/detectron2/detectron2/structures/masks.py", line 271, in __init__ process_polygons(polygons_per_instance) for polygons_per_instance in polygons File "/home/n/detectron2/detectron2/structures/masks.py", line 271, in <listcomp> process_polygons(polygons_per_instance) for polygons_per_instance in polygons File "/home/n/detectron2/detectron2/structures/masks.py", line 262, in process_polygons "Got '{}' instead.".format(type(polygons_per_instance)) AssertionError: Cannot create polygons: Expect a list of polygons per instance. Got '<class 'dict'>' instead.

---

I guess that it must have to set INPUT.MASK_FORMAT='bitmask' (this is in the case of training config), so in the case of evaluating, what need to be set?
I also tried to print("cfg.dict= ", cfg.dict) but only got
type(cfg)= <class 'detectron2.config.config.CfgNode'> cfg.__dict__= {'__immutable__': False, '__deprecated_keys__': set(), '__renamed_keys__': {}, '__new_allowed__': False}
How can I see all the atribute of the cfg?
3. Can you note all the things relate in case of training with binary mask, is there anything need to be noticed/different with the polygon-training.
Many thanks for your great framework and your enthusiastic supports!

[ppwwyyxx]

Evaluation of a generic (not COCO format) dataset with RLE is not yet supported. There is a TODO in

detectron2/detectron2/data/datasets/coco.py

Lines 335 to 337 in 5e2a1ec

	# TODO: check segmentation type: RLE, BinaryMask or Polygon
	polygons = PolygonMasks([segmentation])
	area = polygons.area()[0].item()

(cc @botcs )

[ppwwyyxx]

The above issue about evaluation is hopefully fixed in 6901cc7

[siyuanfeng-tri]

The above issue about evaluation is hopefully fixed in 6901cc7

I think I need to do a bit more to get this actually working.

1. I had to cast this into int explicitly, otherwise json dump complains about TypeError: Object of type 'uint32' is not JSON serializable
   https://github.com/facebookresearch/detectron2/blob/master/detectron2/data/datasets/coco.py#L339
2. I had to decode segmentation["counts"] into an ascii string otherwise json dump complains about not being able to serialize bytes. TypeError: Object of type 'bytes' is not JSON serializable

[ppwwyyxx]

There are a few different RLE formats cocoapi recognizes and the 'counts' in coco's original json is actually a list of int.
Should figure out how to convert the bytes to that format.

[siyuanfeng-tri]

Thanks for the reply! I am not that familiar with different flavors of coco TBH. The only thing I really cared about here is to preserve the RLE annotation (which comes from pycocotools.mask.encode()) through json conversion and back through detectron2's wrappers, since I have a custom dataset, where everything is an image.

So I looked briefly around https://github.com/facebookresearch/detectron2/blob/master/detectron2/data/datasets/coco.py#L160 and looked up where imgToAnns comes from in https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/coco.py
I think my workaround is probably fine for what I care about? As long as json is serializing and deserializing that bytes array the same way I should be fine. I did something like segmentation["counts"] = segmentation["counts"].decode(''ascii") before calling json dump.

[kharyal]

Now the default dataloader can work with RLE formats inside your dataset.
All you need is:

1. use RLE format (documented at https://detectron2.readthedocs.io/tutorials/datasets.html#standard-dataset-dicts) in your dataset
2. set INPUT.MASK_FORMAT='bitmask'.

Still not working for me. I am not getting any error but the network is not learning anything. It just runs for 500 and doesn't segment anything. The Epoch error is also dropping to 0.

[Paragjain10]

There are a few different RLE formats cocoapi recognizes and the 'counts' in coco's original json is actually a list of int.
Should figure out how to convert the bytes to that format.

Hello @ppwwyyxx,

I am trying to implement CenterMask2. My annotations are in coco rle format, I have set cfg.INPUT.MASK_FORMAT='bitmask'
But unfortunately, this error is being raised:

/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/detectron2/data/detection_utils.py:414: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /opt/conda/conda-bld/pytorch_1607370172916/work/torch/csrc/utils/tensor_numpy.cpp:141.)
torch.stack([torch.from_numpy(np.ascontiguousarray(x)) for x in masks])
/home/student2/Desktop/Parag_Centremask2/centermask2/centermask/modeling/fcos/fcos_outputs.py:402: UserWarning: This overload of nonzero is deprecated:
nonzero()
Consider using one of the following signatures instead:
nonzero(*, bool as_tuple) (Triggered internally at /opt/conda/conda-bld/pytorch_1607370172916/work/torch/csrc/utils/python_arg_parser.cpp:882.)
per_candidate_nonzeros = per_candidate_inds.nonzero()
Traceback (most recent call last):
File "/home/student2/Desktop/Parag_Centremask2/centermask2/train_net.py", line 294, in
launch(
File "/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/detectron2/engine/launch.py", line 62, in launch
main_func(*args)
File "/home/student2/Desktop/Parag_Centremask2/centermask2/train_net.py", line 288, in main
return trainer.train()
File "/home/student2/Desktop/Parag_Centremask2/centermask2/train_net.py", line 165, in train
self.train_loop(self.start_iter, self.max_iter)
File "/home/student2/Desktop/Parag_Centremask2/centermask2/train_net.py", line 154, in train_loop
self.run_step()
File "/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/detectron2/engine/defaults.py", line 423, in run_step
self._trainer.run_step()
File "/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/detectron2/engine/train_loop.py", line 228, in run_step
loss_dict = self.model(data)
File "/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 166, in forward
_, detector_losses = self.roi_heads(images, features, proposals, gt_instances)
File "/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/student2/Desktop/Parag_Centremask2/centermask2/centermask/modeling/centermask/center_heads.py", line 401, in forward
losses, mask_features, selected_mask, labels, maskiou_targets = self._forward_mask(features, proposals)
File "/home/student2/Desktop/Parag_Centremask2/centermask2/centermask/modeling/centermask/center_heads.py", line 476, in _forward_mask
loss, selected_mask, labels, maskiou_targets = mask_rcnn_loss(mask_logits, proposals, self.maskiou_on)
File "/home/student2/Desktop/Parag_Centremask2/centermask2/centermask/modeling/centermask/mask_head.py", line 80, in mask_rcnn_loss
cropped_mask = crop(instances_per_image.gt_masks.polygons, instances_per_image.proposal_boxes.tensor)
AttributeError: 'BitMasks' object has no attribute 'polygons'

Process finished with exit code 1

[DavidNguyen95]

w to convert the bytes to that

Did you find the solution? when I try CenterMask.I also got the same error 'BitMasks' object has no attribute 'polygons' like you.

Here is my process:
Follow detectron2 document: I convert a bitmask dataset to RLE format using pycocotools.mask.encode(np.asarray(mask, order="F")) . Then I set cfg.INPUT.MASK_FORMAT='bitmask'

`

[sipie800]

cfg.INPUT.MASK_FORMAT = "bitmask" doesn't work. I do use compressed rle in json which other api like torch or mmdetection can read.

[koreandrum97]

cfg.INPUT.MASK_FORMAT = "bitmask" doesn't work. I do use compressed rle in json which other api like torch or mmdetection can read.

Is it working now? I'd like to semantic segmentation task with detectron2. But I wonder which types of annotation(RLE or Polygons) Please reply!! @ppwwyyxx

[leiluoray1]

check the model output here, say if you want to save the first instance segmentation mask, which is pred_masks[0]. you can

mask0 = outputs["instances"].pred_masks[0].cpu().numpy()*255.0
mask0 = mask0.astype(int)
cv2_imshow(mask0)

[hamzagorgulu]

@koreandrum97 have you found a solution? I train the model with rle masks and assuming the task is converted to semantic segmentation, but I only get masks for instance segmentation. In the [docs](url), it says the output dict should contain "sem_seg" key but I dont get the prediction probabilities.