Support Mean in DDP Sync

[justusschock]

What does this PR do?

Add support for mean in DDP Sync

Fixes # (issue)

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?
• Did you verify new and existing tests pass locally with your changes?
• If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[pep8speaks]

Hello @justusschock! Thanks for updating this PR.

• In the file tests/metrics/test_converters.py:

Line 138:44: E203 whitespace before ','
Line 148:44: E203 whitespace before ','
Line 194:44: E203 whitespace before ','
Line 234:44: E203 whitespace before ','

Comment last updated at 2020-08-04 15:57:36 UTC

[codecov]

Codecov Report

Merging #2568 into master will decrease coverage by 3%.
The diff coverage is 0%.

@@           Coverage Diff           @@
##           master   #2568    +/-   ##
=======================================
- Coverage      88%     86%    -3%     
=======================================
  Files          78      78            
  Lines        7069    6921   -148     
=======================================
- Hits         6250    5932   -318     
- Misses        819     989   +170     

[justusschock]

@Borda are TPU tests still to be neglected?

Edit: Also the failing GPU Test seems unrelated to the metrics (which is the only thing I touched here)

[Borda]

@Borda are TPU tests still to be neglected?

TPU shall be fine...

EDIT: ok we have some problem with TPU tests :{

[awaelchli]

nice @justusschock, this will be very useful

[Borda]

it seems that the test acc dropped significantly...

>       assert result['test_acc'] > 0.8
--
1102 | E       assert 0.25 > 0.8

[justusschock]

@Borda @williamFalcon @SkafteNicki can we get this done?

[justusschock]

@SkafteNicki you're right. This is fixed now.

[SkafteNicki]

LGTM