SipMask-VIS mAP is not stable

[qihao-huang]

I reproduced the SipMask-VIS experiments (the default one, not the ms-train) 4 times, and got the reuslts:

116 | 0.3129611757 | S_1.zip | 07/07/2021 08:02:18 | Finished|
117 | 0.3296189271 | S_2.zip | 07/07/2021 08:02:43 | Finished| 
118 | 0.3118949771 | S_3.zip | 07/07/2021 08:05:01 | Finished| 
119 | 0.3301602886 | S_4.zip | 07/07/2021 08:05:56 | Finished| 

using torch==1.1.0; torchvision==0.3.0; mmcv==0.2.12; with 12 epochs

which are below and beyond 32.5 mAP reported in the paper.

Are there any solutions to make it stabe as the paper?

Also, some relevant issues in repo MaskTrackRCNN reported the mAP number is not stable as the paper one.

Thanks!

[JialeCao001]

@qihao-huang Thanks for interest. How many GPUs are you using?

[qihao-huang]

@qihao-huang Thanks for interest. How many GPUs are you using?

4 GPUs using V100 with 32 GB