Option to provide seed to random generators to ensure reproducibility

[kumuji]

I added small function in utilities which imports torch, numpy, python
random and sets seed for all of the libraries to ensure reproducibility
of results.

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?
• If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?

Partially fixes #1565 (issue).

This code makes sure to fix seed for all pseudo-random generators, to ensure the same results for different runs. But probably doesn't work fully with ddp .

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[codecov]

Codecov Report

Merging #1572 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #1572   +/-   ##
======================================
  Coverage      88%     88%           
======================================
  Files          70      70           
  Lines        4346    4346           
======================================
  Hits         3828    3828           
  Misses        518     518           

[jeremyjordan]

this is a great feature! might i suggest a reorg moving this to pytorch_lightning/seed.py? i think we are eventually trying to get rid of the utilities module in favor of more direct names.

[awaelchli]

Nice! Would it maybe be good to document for which libraries the seed is set (numpy, torch, random, ...)? This could maybe go to trainer/__init__.py where the Trainer args are explained in more detail. Also to make it clear it's not set by default.

[Borda]

@kumuji mind check all examples (probably also elsewhere) and make then use its seeding

[Borda]

i think we are eventually trying to get rid of the utilities module in favor of more direct names.

@jamesjjcondon do you have any suggestions? 🐰 me personally not much fun of having packages with just one module (and fund inside)

[kumuji]

Also, I am sorry, I rebased manually on master, but seems like it was not a good idea since there is autorebase in github actions. Should I fix it?

[williamFalcon]

For seeding to work well with ddp the order needs to be:

seed_everything()

# seed should be the first thing done because in ddp, model init needs to \\
# be the same on every process
model()
fit()

[kumuji]

For seeding to work well with ddp the order needs to be:

seed_everything()

# seed should be the first thing done because in ddp, model init needs to \\
# be the same on every process
model()
fit()

In the last commit fixed it in the benchmark code - model was initialized before trainer, where seeding happens

[Borda]

Also, I am sorry, I rebased manually on master, but seems like it was not a good idea since there is autorebase in github actions. Should I fix it?

it is completely fine, in fact the rebase from action is nothing now.. and also cannot resolve conflicts...

[awaelchli]

@kumuji We can't really see your changes here on Github, could you rebase onto master, please?

[Borda]

@williamFalcon done ^^

[williamFalcon]

@kumuji amazing!