Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't use None (anymore) in checkpoint_callback #2547

Closed
s-rog opened this issue Jul 8, 2020 · 4 comments · Fixed by #2565 or #2832
Closed

Can't use None (anymore) in checkpoint_callback #2547

s-rog opened this issue Jul 8, 2020 · 4 comments · Fixed by #2565 or #2832
Labels
bug Something isn't working help wanted Open to be worked on

Comments

@s-rog
Copy link
Contributor

s-rog commented Jul 8, 2020

🐛 Bug

using None in checkpoint_callback now errors out

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 20, in _wrap
    fn(i, *args)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/distrib_data_parallel.py", line 562, in ddp_train
    q.put(self.checkpoint_callback.best_model_path)
AttributeError: 'NoneType' object has no attribute 'best_model_path'

To Reproduce

trainer = Trainer(checkpoint_callback=None)

Ran into this issue from upgrading to masters, was using masters from a few commits ago before

Edit: False casuses the same error as well

@s-rog s-rog added bug Something isn't working help wanted Open to be worked on labels Jul 8, 2020
@s-rog
Copy link
Contributor Author

s-rog commented Jul 10, 2020

@williamFalcon I saw that this issue was mentioned and supposedly fixed in the merge, but I just tested with master and I'm still getting the same error

@s-rog
Copy link
Contributor Author

s-rog commented Aug 5, 2020

  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 1018, in fit
    results = self.accelerator_backend.teardown(model)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/accelerator_backends/ddp_spawn_backend.py", line 52, in teardown
    self.trainer.checkpoint_callback.best_model_path = best_path
AttributeError: 'NoneType' object has no attribute 'best_model_path'

This is the error I get when I set checkpoint_callback to false on master (0.9.0rc6)

Edit: master...s-rog:patch-1 simple enough change I suppose... unless I'm missing something?

@Borda Borda reopened this Aug 5, 2020
@awaelchli
Copy link
Member

@s-rog I think your proposed fix is good. In case user sets checkpoint_callback to None, we should not try to set a path there.

@awaelchli
Copy link
Member

Would you like to submit this in a PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Open to be worked on
Projects
None yet
3 participants