default logger is now tensorboard

[williamFalcon]

Sets Tensorboard logger as default instead of test-tube.

Also removes the test-tube dep from the package and thus the tensorflow dep

[neggert]

I was hoping to have a few people try it out before we make it the default...

[williamFalcon]

sure. let's keep the PR open for a few days while we try it out.
This was the last main thing for this release.

@tullie

[Borda]

replace tensorboard longer also in tests as default logger

[Borda]

add test-tube to test/requirements.txt

[Borda]

The logger is broken:

tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_tensorboard_logger0')
    def test_tensorboard_logger(tmpdir):
        """Verify that basic functionality of Tensorboard logger works."""
    
        hparams = tutils.get_hparams()
        model = LightningTestModel(hparams)
    
        logger = TensorboardLogger(save_dir=tmpdir, name="tensorboard_logger_test")
    
        trainer_options = dict(max_epochs=1, train_percent_check=0.01, logger=logger)
    
        trainer = Trainer(**trainer_options)
>       result = trainer.fit(model)
tests/test_logging.py:207: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pytorch_lightning/trainer/trainer.py:403: in fit
    self.run_pretrain_routine(model)
pytorch_lightning/trainer/trainer.py:455: in run_pretrain_routine
    self.logger.save()
pytorch_lightning/logging/base.py:14: in wrapped_fn
    fn(self, *args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <pytorch_lightning.logging.tensorboard.TensorboardLogger object at 0x7fd30a81e8d0>
    @rank_zero_only
    def save(self):
>       self.experiment.flush()
E       AttributeError: 'SummaryWriter' object has no attribute 'flush'
pytorch_lightning/logging/tensorboard.py:78: AttributeError

Lets fix the CI - Travis, Appveyor and CircleCI so we do not merge code which fails...

[Borda]

@williamFalcon rebase master, pls...

[snie2012]

Any update on this?

[CarloLucibello]

I get the following error due to a call to SummaryWriter.add_hparams:

{'batch_size': 128, 'epochs': 10, 'lr': 0.1, 'weight_decay': 0.0005, 'save_model': False, 'load_model': '', 'droplr': 5, 'opt': 'nesterov', 'loss': 'nll', 'model': 'mlp_100_100', 'dataset': 'fashion', 'datapath': '~/data/', 'log_interval': 2, 'train_frac': 1.0, 'test_frac': 1.0, 'preprocess': False, 'gpus': None, 'use_16bit': False, 'seed': 872846804}
ERROR:LightDNN:Failed after 0:00:00!
Traceback (most recent calls WITHOUT Sacred internals):
  File "scripts/light_dnn.py", line 220, in main
    trainer.fit(model)
  File "/home/carlo/Git/pytorch-lightning/pytorch_lightning/trainer/trainer.py", line 403, in fit
    self.run_pretrain_routine(model)
  File "/home/carlo/Git/pytorch-lightning/pytorch_lightning/trainer/trainer.py", line 453, in run_pretrain_routine
    self.logger.log_hyperparams(ref_model.hparams)
  File "/home/carlo/Git/pytorch-lightning/pytorch_lightning/logging/base.py", line 14, in wrapped_fn
    fn(self, *args, **kwargs)
  File "/home/carlo/Git/pytorch-lightning/pytorch_lightning/logging/tensorboard.py", line 76, in log_hyperparams
    self.experiment.add_hparams(hparam_dict=dict(params), metric_dict={})
  File "/home/carlo/miniconda3/envs/sacred37/lib/python3.7/site-packages/torch/utils/tensorboard/writer.py", line 292, in add_hparams
    exp, ssi, sei = hparams(hparam_dict, metric_dict)
  File "/home/carlo/miniconda3/envs/sacred37/lib/python3.7/site-packages/torch/utils/tensorboard/summary.py", line 156, in hparams
    raise ValueError('value should be one of int, float, str, bool, or torch.Tensor')
ValueError: value should be one of int, float, str, bool, or torch.Tensor

This is because I have a NoneType value for the gpus params:

hparams = { 'gpus': None}

It can also be a problem, for list of gpus indexes, since lists are not supported as well.
I suggest we cast the gpus field (or any other non-supported field type) to string.

[Borda]

This is because I have a NoneType value for the gpus params:

hparams = { 'gpus': None}

It can also be a problem, for list of gpus indexes, since lists are not supported as well.
I suggest we cast the gpus field (or any other non-supported field type) to string.

good point, for list I would assume string with some common separator and of None I would drop such option from hparams...

[Borda]

This is because I have a NoneType value for the gpus params:

hparams = { 'gpus': None}

It can also be a problem, for list of gpus indexes, since lists are not supported as well.
I suggest we cast the gpus field (or any other non-supported field type) to string.

good point, for list I would assume string with some common separator and of None I would drop such option from hparams...

[williamFalcon]

@neggert @Borda is this ready to merge?

[neggert]

It looks like somehow the test-tube dependency got added back in with the merge commit.

[neggert]

Oh, wait. I was just doing weird things with the merge commit summary. LGTM.

[snie2012]

Quick question, why is test-tube still in requirements.txt?

[Borda]

we are fixing it in #609 together with some other minor issues... :]

[snie2012]

Do you mean #687 ? @Borda

[shubhamagarwal92]

This is because I have a NoneType value for the gpus params:

```python
hparams = { 'gpus': None}

It can also be a problem, for list of gpus indexes, since lists are not supported as well.
I suggest we cast the gpus field (or any other non-supported field type) to string.

@williamFalcon @Borda I am still facing this issue for pytorch-lightning==0.7.1.

Basically, I tried to pass a list of gpus in the NER example in the transformers repo here and was facing the same value error raise ValueError('value should be one of int, float, str, bool, or torch.Tensor'). I can see a simple fix for this. Tensorboard limits passing Nonetype or list or even dictionary values here

Instead of passing all the hparams to tensorboard via pl here, we put a small check to pass int, float, str, bool, or torch.Tensor for logging.

        from six import string_types
        from torch.utils.tensorboard.summary import hparams
        tensorboard_params = {}
        for k, v in params.items():
            if isinstance(v, int) or isinstance(v, float) or isinstance(v, string_types) or isinstance(
                    v, bool) or isinstance(v, torch.Tensor):
                tensorboard_params[k] = v
        exp, ssi, sei = hparams(tensorboard_params, {})

IMO, there could be cases when the user wants to parse a list or maybe nonetype objects through argparse but doesnt want them to be logged via tensorboard.

Please let me know if this makes sense and I should raise a PR!

[Borda]

@shubhamagarwal92 hs for this elaboration, sure PR is welcome!