Cannot download backbone weights

[xpngzhng]

backbone weights listed at the end of this file cannot be download
https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/blob/master/efficientnet/utils.py

[zylo117]

the original author changed the url.

https://github.com/lukemelas/EfficientNet-PyTorch/releases

[xpngzhng]

there are keyword mismatches between the pretrained model weights and the param name in the network definition.
Run the following code

p = r'D:\ExternProjects\efficientnet-b2-8bb594d6.pth'
state_dict = torch.load(p)
for key in state_dict.keys():
    print(key)

I get

_conv_stem.weight
_bn0.weight
_bn0.bias
_bn0.running_mean
_bn0.running_var
_bn0.num_batches_tracked
_blocks.0._depthwise_conv.weight
_blocks.0._bn1.weight
_blocks.0._bn1.bias
_blocks.0._bn1.running_mean
_blocks.0._bn1.running_var
_blocks.0._bn1.num_batches_tracked
...

However, the param names inside the model by the following code
are different

model = effnet_model.EfficientNet.from_name('efficientnet-b2')
for k, v in model.named_parameters():
    print(k)

_conv_stem.conv.weight
_bn0.weight
_bn0.bias
_blocks.0._depthwise_conv.conv.weight
_blocks.0._bn1.weight
_blocks.0._bn1.bias
...

we can see an additional conv. inside the conv param name

the pytorch version I use is 1.3.1
I am not sure whether it is the reason

[zylo117]

I use different conv now, so loading it directly is not possible.

[xpngzhng]

I use different conv now, so loading it directly is not possible.

@zylo117 you mean now we now have to train without effcientnet weights?

[pfeatherstone]

As a work around, you can just load the final weights, then save the backbone weights

[pfeatherstone]

Alternatively, use https://github.com/rwightman/pytorch-image-models. That repo has every single imagenet classifier known to man, including every single efficientnet model you can imagine, in different flavours, trained with different repositories, including NoisyStudent from tensorflow ported to pytorch. All the weights are provided. You just need to do a bit of hacking for it to work if you want to use the pretrained weights for efficientdet in this repo. If you don't care about using the provided weights and just train efficientdet from scratch on coco, then there is little hacking required.

[xpngzhng]

As a work around, you can just load the final weights, then save the backbone weights

Actually I want to train efficientdet from initial efficientnet weights.
I have solved the problem by modifying the keys (removing the conv.) in the state dict before loading

[lockeregg]

As a work around, you can just load the final weights, then save the backbone weights

Actually I want to train efficientdet from initial efficientnet weights.
I have solved the problem by modifying the keys (removing the conv.) in the state dict before loading

Hi,
How's the results?
I tried to train without efficientnet weights, got really bad result.

Thank you.

[pfeatherstone]

I've used https://github.com/rwightman/pytorch-image-models for the backbone.
To create the backbone you just need:
backbone = timm.create_model("efficientnet-b{}".format(coeff))
You need to modify some glue code in a couple of places. Then you can remove a lot of the code related to EfficientNet.
Of course this won't work if you want to load the pretrained weights, but if you want to train from scratch, which I did, it works fine.

[lockeregg]

I've used https://github.com/rwightman/pytorch-image-models for the backbone.
To create the backbone you just need:
backbone = timm.create_model("efficientnet-b{}".format(coeff))
You need to modify some glue code in a couple of places. Then you can remove a lot of the code related to EfficientNet.
Of course this won't work if you want to load the pretrained weights, but if you want to train from scratch, which I did, it works fine.

I'll try! Thank you so much!

[xpngzhng]

I put the backbone and bifpn of the repo into mmdetection for training, arch is efficientdet d2
Now 50 epochs have passed, the mAP is around 0.275
It really takes a long time, 450 epochs left

[pfeatherstone]

Yeah EffcientDet is super slow to train. I'm running some experiments with CIOU loss. That seems to help. But there is some hyper parameter tuning required, namely loss gains.

[xpngzhng]

https://github.com/rwightman/pytorch-image-models has a better efficientnet implementation, we can obtain the feature layers that should be fed to bifpn much more easily

efficientdet paper mislead us by using p3 to p7 features from efficientnet, that's one reason why many third party implementation can only obtain poor mAP

[zylo117]

are you serious? it really is p3 to p7.

[pfeatherstone]

I'm using https://github.com/rwightman/pytorch-image-models to minimise code and to experiment with training. With regards to using output features from the backbone I'm mimicking what this repo does

[xpngzhng]

are you serious? it really is p3 to p7.

I am serious

See the feature map size of efficienet b2 in https://github.com/rwightman/pytorch-image-models
tf_efficientnet_b2
stem: torch.Size([1, 3, 224, 224]) -> torch.Size([1, 32, 112, 112])
stage 0: torch.Size([1, 32, 112, 112]) -> torch.Size([1, 16, 112, 112])
stage 1: torch.Size([1, 16, 112, 112]) -> torch.Size([1, 24, 56, 56])
stage 2: torch.Size([1, 24, 56, 56]) -> torch.Size([1, 48, 28, 28])
stage 3: torch.Size([1, 48, 28, 28]) -> torch.Size([1, 88, 14, 14])
stage 4: torch.Size([1, 88, 14, 14]) -> torch.Size([1, 120, 14, 14])
stage 5: torch.Size([1, 120, 14, 14]) -> torch.Size([1, 208, 7, 7])
stage 6: torch.Size([1, 208, 7, 7]) -> torch.Size([1, 352, 7, 7])
head: torch.Size([1, 352, 7, 7]) -> torch.Size([1, 1408, 7, 7])

actually, only stage 2 4 and 6 are used to build bifpn,
the additionally two layers of the first stack in bifpn are resized from stage 6, similiar to what the traditional fpn does

see https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/blob/master/efficientdet/model.py#L55
there is a first_time param to control the building of bifpn

[zylo117]

I know that, I wrote that part in bifpn.

That part of the paper seems misleading alright.

But in the official impl, they actually modify the efficientnet before bifpn so that there really are p6 and p7 in efficientnet.

[xpngzhng]

google does evil this time, in the latest revision of this paper, the author has not clarified the correct way of building p3 to p7.