Enable val/test loop disabling + datamodule tests

[nateraw]

What does this PR do?

This updates MisconfigurationExceptions to be warnings instead within trainer when you have dataloaders/defined steps for val/test that aren't configured correctly. Just because you have a validation dataloader doesn't mean you want to use it, and just because your model defines a val_step doesn't mean you should need to provide a validation dataset.

CC: @awaelchli we discussed something about this the other day 😄

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?
• Did you verify new and existing tests pass locally with your changes?
• If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[rohitgr7]

Nice! It solves the problem of commenting out the validation_step or val_dataloader / test code whenever it's not needed. 😅

[nateraw]

Nice! It solves the problem of commenting out the validation_step or val_dataloader / test code whenever it's not needed. 😅

Exactly!! I have it a bit wrong so far here though. Test step should still raise misconfiguration but only if self.testing.

[pep8speaks]

Hello @nateraw! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-07-25 16:43:11 UTC

[williamFalcon]

Features:

• enables skipping val loop when not needed
• enables skipping test loop when not needed
• refactors some of the trainer
• adds datamodule tests
• adds config validator tests

future

• sets up more refactors for the trainer

[awaelchli]

it looks like val dataloaders get reloaded every epoch no matter the setting of self.reload_dataloaders_every_epoch. Isn't this a bug?

[williamFalcon]

that’s how it was before no? i think that was intended this way

[williamFalcon]

is there an issue about it?

[awaelchli]

if it's intended, ok. Just the name of the flag is a bit misleading then :)

is there an issue about it?

There does not seem to be one.