Add FedPer

[WilliamLindskog]

@jafermarq I think I've set the PR to ready for review now. Let me know if something is missing.

Issue

"Ready for review implementation for FedPer (Federated Learning using Personalization Layers) for summer of reproducibility.

Description

Federated Learning with Personalization Layers has been implemented using ResNet34 and MobileNet-v1 for CIFAR10 and FLICKR-AES. The basics of this model is to only transfer local body layers to the server and not the head layers (basically the n layers at the end of a neural network).

Related issues/PRs

Proposal

I suggest a model manager and model split class that can manage the split between head and body layers, and process data accordingly. Moreover, an AggregateBodyPipeline serves FedPer accordingly and only aggregates body layers.

Explanation

The models.py file includes classes (1) ModelManager and (2) ModelSplit. These are called when creating new models in the folder /implemented_models. Since client states are destroyed after fit and evaluate functions, I store the client head layer values to reach them in the next round so that these are not initialized randomly every time.

Checklist

• Implement proposed change
• Write tests
• Update documentation
• Update changelog
• Make CI checks pass
• Ping maintainers on Slack (channel #contributions)

Any other comments?

[WilliamLindskog]

Hi @jafermarq ,

Just did some minor updates to the code (mainly README from your comments). Please proceed as you pleases.

Best regards,
W

[jafermarq]

Hi @WilliamLindskog,

I left some minor comments about the readme and pyproject.toml, also some suggestions for your run_figures.sh script.

I also took a closer look at the results for CIFAR-10. I think some revisiting to the training loop (including dataset augmentation and hyperparameters used) is needed because some results are some ~10% or more off from those in the paper. Is there some parts of your code you are less certain about ? or maybe some decisions you had to take because something wasn't fully described in the paper.

Happy to iterate further. Ping me if you want to discuss things.

[jafermarq]

Hi @WilliamLindskog ,

Just a minor comment. I spotted an issue in your run_figures.sh script when running the Flickr experiments.

[jafermarq]

@WilliamLindskog, this looks ready to be merged! Amazing work!