Implement Krum and MultiKrum

[edogab33]

Reference Issues/PRs

Fixes #1482 and fixes #1483

What does this implement/fix? Explain your changes.

Implement the Krum + MultiKrum strategy as explained in the issue.
Added the strategy class with a new (optional) parameter named num_malicious_clients. Krum, in fact, assumes that the number of malicious clients is known a priori.
Added the aggregation function and tests.

Instead of creating two separate classes and aggregation functions for Krum and MultiKrum I decided to include the latter in the same class of the former by including a parameter num_clients_to_keep in the strategy class which defaults to 0. If an integer != 0 is passed, parameters are aggregated according to MultiKrum (i.e. evaluate parameters vectors assigning a score to each of them using Krum, then average the num_clients_to_keep smallest ones).

[edogab33]

@danieljanes there's a problem with the tests.

format.sh spaces list slices on colon like that lst[i : n] and then flake8 raises the error E203 because there shouldn't be any space before a colon. If I do not put any space around the colon, another test fails telling me that the file is not properly formatted. Therefore if you either put a space or not, the merging is blocked.

I had to add # noqa: E203 as a workaround.

[viktorvaladi]

I've done some tests with this and also compared it to the specification in the Krum paper and it looks good to me. Tests work as expected, lower impact poisoning clients slip through from time to time while if you increase the amount of poisoning a client does they will never be selected for aggregation.

[danieljanes]

Thanks for the PR @edogab33 & thanks for the review @viktorvaladi 🙌

[danieljanes]

@edogab33 we can merge is like that, just one last question: is multikrum_test.py supposed to be a part of this PR or of another one?

[edogab33]

@danieljanes Yes! This PR implement both Krum and MultiKrum, so it is supposed to be here. I'll edit the title.

Thanks @viktorvaladi. I am researching on a similar topic. What poisoning attack did you test?

[viktorvaladi]

Fun topic to work with, thanks for your commit @edogab33 , nice when there's already code for what you need :) I've done tests with gradient ascent attacks with scaling the norms and backdoor label swapping attacks. From what I can see mkrum gets into issues when attacks slip in even if they are not very strong attacks. Since they disregard half of the number of clients then when malicious clients slip in they will be a larger portion of the total clients.

[edogab33]

@viktorvaladi Interesting, I've only worked on model poisoning attacks up to now. I confirm that MultiKrum can be a kind of "double-edge blade": if there are a lot of malicious clients and you select too many clients each round then it will be hard to defend the global model. According to me Krum and MultiKrum have the unrealistic assumption that the server knows how many malicious clients there are in the system, so there are better alternatives. Though it's almost impossible to defend against "low impact" attacks... Maybe the best defence is to have a massive amount of clients partecipating in the FL system and hope that the attacker doesn't have a botnet as large as 1/2 of the client pool :D

However I hope to add even more strategies to the official repository in the near future. They are already implemented, but I have to find the time.

[danieljanes]

Great work, @edogab33!