Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Images not being logged after using auto_lr_find #1919

Closed
SubodhDahal opened this issue May 21, 2020 · 4 comments
Closed

Images not being logged after using auto_lr_find #1919

SubodhDahal opened this issue May 21, 2020 · 4 comments
Labels
help wanted Open to be worked on won't fix This will not be worked on

Comments

@SubodhDahal
Copy link
Contributor

🐛 Bug

When I use the auto LR finder, the images that I logged are no longer being displayed in Tensorboard. The logging works normally when setting it to False or when not specifying it.

To Reproduce

Steps to reproduce the behavior:

  1. Set auto_lr_find = True for the trainer
  2. Log image during test/validation step as self.logger.experiment.add_image(title, image, step)
  3. Load the Tensorboard WebUI and see that the image is not being logged

Expected behavior

The image that I logged should be visible in Tensorboard, even when using auto LR finder.

Environment

* CUDA:
        - GPU:
                - GeForce GTX 1050 Ti with Max-Q Design
        - available:         True
        - version:           10.2
* Packages:
        - numpy:             1.18.3
        - pyTorch_debug:     False
        - pyTorch_version:   1.5.0
        - pytorch-lightning: 0.7.6
        - tensorboard:       2.2.1
        - tqdm:              4.46.0
* System:
        - OS:                Linux
        - architecture:
                - 64bit
                - ELF
        - processor:         x86_64
        - python:            3.8.2
        - version:           #41~1586790036~18.04~600aeb5-Ubuntu SMP Mon Apr 13 17:47:15 UTC
  • How you installed PyTorch (conda, pip, source): pip

Additional context

Not sure if relevant, but I couldn't figure out how to pass the params to the Trainer as I'm using from_argparse_args(hparams), where hparams contains the keys and values obtained from command-line ArgumentParser. So I set hparams.auto_lr_finder = True before instantiating the Trainer. The LR finder work's, so I assume there's no issue with that.

@SubodhDahal SubodhDahal added the help wanted Open to be worked on label May 21, 2020
@github-actions
Copy link
Contributor

Hi! thanks for your contribution!, great first issue!

@danielhomola
Copy link

I can second this. Same goes for the graph of the net.. Spent almost a full day trying to figure out what am I doing wrong, but once auto_lr_find=False, everything works fine..

@SkafteNicki
Copy link
Member

@SubodhDahal and @danielhomola could one of you try out the fix that is on master now, and report back if it worked. If so please close the issue :)

@stale
Copy link

stale bot commented Jul 24, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the won't fix This will not be worked on label Jul 24, 2020
@stale stale bot closed this as completed Aug 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Open to be worked on won't fix This will not be worked on
Projects
None yet
Development

No branches or pull requests

3 participants