PSPNet final conv

[nklein23]

Hi qubvel,

are you sure that the final conv layer of your PSPNet implementation is correct?

model = PSPNet(backbone_name = 'resnet18',
               final_interpolation = 'bilinear',
               encoder_weights = 'imagenet',
               freeze_encoder = False,
               input_shape = (384, 384, 3),
               classes = 1,
               activation = 'sigmoid',
               use_batchnorm = True,
               downsample_factor = 8, 
               psp_pooling_type = 'avg',
               psp_conv_filters = 256,
               dropout = 0.5)

yields for the last couple of layers:

---

concatenate_1 (Concatenate) (None, 48, 48, 1152) 0 stage3_unit1_relu1[0][0]
resize_image_1[0][0]
resize_image_2[0][0]
resize_image_3[0][0]
resize_image_4[0][0]

---

conv_block_conv (Conv2D) (None, 48, 48, 512) 589824 concatenate_1[0][0]

---

conv_block_bn (BatchNormalizati (None, 48, 48, 512) 2048 conv_block_conv[0][0]

---

conv_block_relu (Activation) (None, 48, 48, 512) 0 conv_block_bn[0][0]

---

spatial_dropout2d_1 (SpatialDro (None, 48, 48, 512) 0 conv_block_relu[0][0]

---

final_conv (Conv2D) (None, 48, 48, 1) 4609 spatial_dropout2d_1[0][0]

---

resize_image_5 (ResizeImage) (None, 384, 384, 1) 0 final_conv[0][0]

---

sigmoid (Activation) (None, 384, 384, 1) 0 resize_image_5[0][0]

---

conv_block_conv has 589.824 params, I guess these originate from 512 1 * 1 * 1152 convolutions.
And final_conv (Conv2D) has 4609 params, likely constructed from 1 3 * 3 * 512 conv.
Shouldnt it be the other way around? conv_block_conv having 3 * 3 convs and the final layer having 1 * 1 convs? Sorry if I am mistaken here.

[qubvel]

Hi @NiklasDL
The most implementations od PSPNet I saw have (3,3) -> (1,1) final convolutions (like you describe), however (3,3) convolution have 9 times more params (500K -> 4,5M), so I decide to switch them and make something like bottleneck in ResNet models.

[qubvel]

P.S. If you would test both cases, please, let me know your results. Thanks. 😄

[nklein23]

Hi @qubvel

will take some time because my model trains for almost two weeks but I will let you know :)

[nklein23]

Hi @qubvel, the difference between both cases is slightly noticeable. Please find the results for both models for the same random subset of 200 test images (binary segmentation problem, IoU-threshold optimized results). The training data contains roughly 600k images.

"Wrong" final conv:

"Corrected" final conv:

[qubvel]

Hi @NiklasDL,
Thanks a lot for reporting results of your experiments! If difference is not noticeable I will leave it as it is.
Btw, I have mention your work in release note in CHANGELOG.md, thanks!