Dropout

[khornlund]

Hi Luke,

Thanks for sharing this implementation - it's getting a lot of use on Kaggle!

Changes

This PR makes some minor changes to the final layers of the network, in order to allow for a simpler and more flexible implementation of forward():

1. Pooling and Dropout are stored as attributes on the model with the same name as the Tensorflow implementation, and use nn.Module instead of the functional version. This makes the final layers of the network easier to modify, as users can simply override the values of these attributes.
2. Reshaping of the data inside forward() has been refactored to use view() instead of multiple calls to squeeze(). This allows users to replace the pooling module without having to conform to dimension requirements.

Notes

Dropout training/eval

The current training check for dropout is no longer necessary:

if self._dropout:
    x = F.dropout(x, p=self._dropout, training=self.training)

When you call net.train() or net.eval(), it recursively sets the training attribute on all child modules. This is then used by the nn.Dropout instance in its call to F.dropout( ... ). Source here.

I've added some tests to make it easy for you to check this.

Modifying the final layers

I've added some tests to show how users can now replace the final layers of the network, without having to subclass EfficientNet and override the forward() method.

Running the tests

pip install pytest
pytest tests/

These tests are provided as a convenience - it's of course up to you whether you want to merge any of them in. Let me know what you think!

Kind Regards,
Karl