Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disabling validation with val_percent_check=0.0 not working #1248

Closed
thegyro opened this issue Mar 26, 2020 · 3 comments · Fixed by #1251
Closed

Disabling validation with val_percent_check=0.0 not working #1248

thegyro opened this issue Mar 26, 2020 · 3 comments · Fixed by #1251
Labels
bug Something isn't working help wanted Open to be worked on

Comments

@thegyro
Copy link

thegyro commented Mar 26, 2020

🐛 Bug

I am trying to train on a small subset of data by settting train_percent_check and val_percent_check flags. I am not able to disable validation by setting ver_percent_check=0.0. Despite it being set to 0, validation_epoch_end is executed.

To Reproduce

Steps to reproduce the behavior: Running the MNIST model on the Google Colab link

Screen Shot 2020-03-26 at 11 06 16 AM

Code sample

mnist_model = MNISTModel()

# most basic trainer, uses good defaults (1 gpu)
trainer = pl.Trainer(gpus=1, val_percent_check=0.0, train_percent_check=0.1)    
trainer.fit(mnist_model)

Expected behavior

I expected the validation_epoch_end to not be invoked but it is.

Environment

Running on Google Colab with the sample MNIST tutorial

@thegyro thegyro added bug Something isn't working help wanted Open to be worked on labels Mar 26, 2020
@github-actions
Copy link
Contributor

Hi! thanks for your contribution!, great first issue!

@awaelchli
Copy link
Member

I can confirm there is an issue. I tested it with the pl_example gpu template on master. I get a different error:

...
  File "C:\Users\aeduw\Documents\Repositories\pytorch-lightning\pytorch_lightning\trainer\training_loop.py", line 437, in run_training_epoch
    self.run_evaluation(test_mode=self.testing)
  File "C:\Users\aeduw\Documents\Repositories\pytorch-lightning\pytorch_lightning\trainer\evaluation_loop.py", line 368, in run_evaluation
    eval_results = self.evaluate(self.model, dataloaders, max_batches, test_mode)
  File "C:\Users\aeduw\Documents\Repositories\pytorch-lightning\pytorch_lightning\trainer\evaluation_loop.py", line 312, in evaluate
    eval_results = model.validation_epoch_end(outputs)
  File "C:\Users\aeduw\Documents\Repositories\pytorch-lightning\pl_examples\basic_examples\lightning_module_template.py", line 190, in validation_epoch_end
    val_loss_mean /= len(outputs)
ZeroDivisionError: division by zero

I think I know how to fix it and will submit a PR soon + unit test.

@awaelchli
Copy link
Member

@thegyro This should now be fixed. I tested it in MNIST colab by installing PL from master branch:
! pip install git+https://github.com/PytorchLightning/pytorch-lightning.git@master --upgrade
Feel free to reopen if there are unresolved issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Open to be worked on
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants