Inconsistent behavior with paths when using loggers and ModelCheckpoint together

[kyoungrok0517]

🐛 Bug

Something's quite wrong with the path handling in Logger and ModelCheckpoint. I'm having trouble dealing with situation when I try to set the following settings work together. The effect is unpredictable and hard to find patterns. Sometimes I see the folders like default_default as a logging dir when I expect default, or sometimes I see ModelCheckpoint()'s filepath doesn't have any effect. I think there should be more intuitive way to harmonize the path settings of multiple loggers and ModelCheckpoint()'.

root_dir = Path(__file__).parent.absolute()
exp_name = "my_exp"

# loggers
tb_logger = TensorBoardLogger("tb_logs", name=exp_name)
neptune_logger = NeptuneLogger(experiment_name=exp_name)
logger_list = [neptune_logger, tb_logger]

# checkpoints
checkpoint_dir = root_dir / "checkpoints"
checkpoint_callback = ModelCheckpoint(filepath=checkpoint_dir)

trainer = Trainer(
        logger=logger_list,
        default_root_dir=root_dir,
        checkpoint_callback=checkpoint_callback
    )

Environment

PyTorch version: 1.4.0
Is debug build: No
CUDA used to build PyTorch: 10.1

OS: Ubuntu 18.04.4 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: Could not collect

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: 10.1.243
GPU models and configuration:
GPU 0: TITAN V
GPU 1: TITAN V

Nvidia driver version: 440.82
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.4

Versions of relevant libraries:
[pip] numpy==1.18.1
[pip] pytorch-lightning==0.7.6rc1
[pip] pytorch-ranger==0.1.1
[pip] torch==1.4.0
[pip] torch-optimizer==0.0.1a12
[pip] torchtext==0.5.0
[pip] torchvision==0.5.0
[conda] blas                      1.0                         mkl
[conda] mkl                       2020.0                      166
[conda] mkl-service               2.3.0            py37he904b0f_0
[conda] mkl_fft                   1.0.15           py37ha843d7b_0
[conda] mkl_random                1.1.0            py37hd6b4f25_0
[conda] pytorch                   1.4.0           py3.7_cuda10.1.243_cudnn7.6.3_0    pytorch
[conda] pytorch-lightning         0.7.6rc1                 pypi_0    pypi
[conda] pytorch-ranger            0.1.1                    pypi_0    pypi
[conda] torch-optimizer           0.0.1a12                 pypi_0    pypi
[conda] torchtext                 0.5.0                    pypi_0    pypi
[conda] torchvision               0.5.0                py37_cu101    pytorch

[Borda]

@kyoungrok0517 mind send a PR with the test case? rabbit
https://pytorch-lightning.readthedocs.io/en/latest/CONTRIBUTING.html#bug-fixes